Method for developing a computer tool and a computer tool for remote diagnosis of chemical element content in plants

Abstract

The subject of the invention is a method for developing a computer tool that utilises learning in a feedforward neural network FNN for the remote diagnosis of chemical element content in plants. The method includes collecting a training dataset, consisting of hyperspectral data from cultivated fields and measurement data on element content in plant leaves. The hyperspectral data is normalised and converted into input vectors, after which the dataset is divided into a training set and a test set. The training of the FNN involves an input layer with 64 neurons, a hidden layer with 32 neurons, and an output layer with a single neuron performing a regression function. The network is trained to match the predicted element content values to actual measurement data. The trained network is validated using the test dataset to ensure diagnostic accuracy. The method is applicable in precision agriculture for analysing the chemical element content in plants. The invention also concerns a computer-implemented method for the remote diagnosis of chemical element content in plants using neural network learning.

Description

[0001] METHOD FOR DEVELOPING A COMPUTER TOOL AND A COMPUTER TOOL

[0002] FOR REMOTE DIAGNOSIS OF CHEMICAL ELEMENT CONTENT IN PLANTS

[0003] The subject of the invention is a method for developing a computer tool and a computer tool for remote diagnosis of chemical element content in plants.

[0004] The method and computer tool are applicable in precision agriculture for analysing the chemical element content in plants, particularly in the context of optimising fertilisation strategies.

[0005] In the prior art, a method for determining nitrogen content in wheat leaves using a neural network is known, as well as a computer tool using machine learning in a neural network to enhance the resolution of satellite data representing the quantities of chemical elements and compounds.

[0006] CN112613338A discloses a method for estimating nitrogen content in wheat leaves based on RGB image fusion features. This method involves estimating the nitrogen content in wheat leaves based on RGB image data captured by a drone-mounted camera, representing wheat fields, and nitrogen content data from wheat leaf samples collected for experimental analysis. Next, the RGB image data of wheat is processed, which requires additional steps, including lens distortion elimination and geometric correction based on previously recorded drone data on tilt angle, roll angle, yaw angle, and altitude. In the next step, the average pixel brightness values from the R, G, and B channels of the RGB image are extracted and normalised, followed by the calculation of a visible light vegetation index related to the estimation of nitrogen content in leaves. A trained convolutional neural network is used to extract deep features from the Wheatfield RGB image.

[0007] CN1185372220A discloses a method for reconstructing GOSAT XCO2 data, combining the bicubic interpolation method and the Swin Transformer model. This method uses bicubic interpolation to enhance the resolution of XCO2 satellite data and then applies the Swin Transformer to reconstruct the XCO2 satellite data, addressing the issue of low accuracy after interpolation. This method involves collecting three remote sensing monitoring datasets for the studied area, including GOSAT XCO2, OCO-2XCO2, TCCONCO2, followed by preprocessing, in which the original greyscale image is divided by the maximum pixel value for data normalisation. Next, bicubic interpolation is applied to enhance the resolution of GOSAT XCO2 data, and a super-resolution algorithm dataset is built, which is then used to train the Swin Transformerbased super-resolution network. Preferably, the resolution of GOSAT XCO2 data is enhanced using a 16-point bicubic spline interpolation algorithm, resulting in high-precision GOSAT XCO2 data. The multiscale transformer architecture used in this method is designed for efficient processing of high-resolution images, increasing the effectiveness of carbon source studies.

[0008] The US11521073B2 document discloses a method for hyperspectral assessment of phosphorus content in rubber tree leaves. The method includes acquiring hyperspectral data from the detected leaves of the rubber tree, extracting key wavelengths of the rubber tree leaves based on the hyperspectral data and a predefined wavelength extraction model, where the key wavelengths are associated with phosphorus content in the leaves. The predefined wavelength extraction model is obtained by training on sample hyperspectral data and corresponding phosphorus content data pairs stored in a predefined sample database, using the Competitive Adaptive Reweighted Sampling (CARS) algorithm and the Successive Projection Algorithm (SPA). The key wavelengths are then fed into a predefined phosphorus content prediction model to calculate the phosphorus content in the detected rubber tree leaves. Additionally, the CARS and SPA algorithms are jointly applied to extract key wavelengths closely related to phosphorus content in rubber tree leaves.The invention is defined in patent claim 1.

[0009] The essence of the invention is a method for developing a computer tool for remote diagnosis of chemical element content in plants using neural network training, comprising steps in which: a training dataset is collected, consisting of hyperspectral data with multiple electromagnetic wavelength ranges representing cultivated fields and measurement data representing the corresponding element content in plant leaves from cultivated fields. The hyperspectral data is then normalised to a common value range, after which the training data is converted into an input vector, with a length corresponding to the number of hyperspectral data ranges, and the dataset is split into a training set and a test set. The converted training data is fed into a feedforward neural network (FNN) consisting of three layers: an input layer with 64 neurons with a ReLU activation function, each receiving the input vector; a hidden layer with a ReLU activation function, consisting of 32 neurons, fully connected to the input layer; and an output layer with a single neuron performing a regression function. The FNN neural network is trained so that the output data, representing the predicted chemical element content in plant leaves, corresponds to the actual chemical element content in plant leaves; the trained FNN neural network is validated for accuracy in diagnosing chemical element content using the test dataset.

[0010] Preferably, the method according to the invention is characterised in that the chemical element determined in the leaves of cultivated plants is phosphorus P or nitrogen N.

[0011] Preferably, the method according to the invention is characterised in that the plants for which the chemical element content is determined are cereals.

[0012] Preferably, the method according to the invention is characterised in that the plants for which the chemical element content is determined include maize, wheat, rye, oats, or rice.

[0013] Preferably, the method according to the invention is characterised in that the hyperspectral data collection representing cultivated fields occurs in the wavelength ranges of 500-550 nm and 650-1000 nm.

[0014] Preferably, the method according to the invention is characterised in that the diagnosis of chemical element content is performed during the following plant growth stages: BBCH 10-16 for maize or BBCH 15-32 for other plants.

[0015] Preferably, the method according to the invention is characterised in that the feedforward neural network (FNN) is trained using the backpropagation algorithm to minimise the mean squared error (MSE) between the predicted and actual element content values, along with the Adam optimiser.

[0016] The essence of the invention also includes a computer-implemented method for the remote diagnosis of chemical element content in plants using neural network learning, which comprises steps in which: hyperspectral data is collected, consisting of multiple electromagnetic wavelength ranges representing cultivated fields, after which the hyperspectral data is normalised to a common value range; and subsequently the normalised hyperspectral data is converted into an input vector with a length corresponding to the number of hyperspectral data ranges; the input vector is fed into a neural network (FNN). Next, the output block receives a value representing the chemical element content in the leaves of cultivated plants, obtained from the processing of hyperspectral data by the neural network, where the hyperspectral data consists of multiple electromagnetic wavelength ranges representing cultivated fields.

[0017] Preferably, the method according to the invention is characterised in that the hyperspectral data is obtained from aerial or satellite imagery.Preferably, the method according to the invention is characterised in that the hyperspectral data as images is automatically retrieved from a hyperspectral image database, with the data being selected based on the geographic coordinates of the cultivated field.

[0018] Preferably, the invention also includes a computer programme, which, when executed on a computer, performs the steps of the method according to the invention.

[0019] The advantage of the method and the computer tool is the ability to remotely determine the chemical element content in plants, allowing the detection of element deficiencies at all plant growth stages, particularly in the early growth stages. This enables the identification of emerging deficiencies and the application of precise fertilisation, thereby increasing yields. The increased yield of cultivated plants, in turn, helps to meet the nutritional needs of a growing population, while precision fertilisation reduces the negative impact of agriculture on the planet's ecosystem.

[0020] The subject of the invention is further illustrated in a preferred embodiment through the following Figure, in which:

[0021] Figure 1 a a block diagram of the method according to the invention;

[0022] Figure 1b a block diagram of the computational tool according to the invention;

[0023] Figure 2 structure of the neural network used in the invention;

[0024] Figure 3 contribution of drone-based spectral bands using Sentinel-2 bands in the phosphorus level prediction model for winter rye;

[0025] Figure 4 scatter plot of the predicted and measured phosphorus content in winter rye using drone-based hyperspectral data;

[0026] Figure 5 scatter plot of the predicted and measured phosphorus content in winter rye using drone-based hyperspectral data corresponding to Sentinel-2 bands.

[0027] The objective of the invention is to recommend the timing of intervention fertilisation. It concerns remote observation and analysis of nutrient deficiencies during critical plant growth stages. A key aspect of this solution is the collection of hyperspectral data within precisely defined electromagnetic wavelength ranges, specifically within intervals indicating water content in the plant and its structural properties, as well as chlorophyll absorption and photosynthetic activity. These data facilitate the determination of chemical element content in plant leaves at a given growth stage.

[0028] Figure 1a presents a block diagram representing the process of creating the computational tool using deep neural network training, applied to the remote diagnosis of phosphorus content in the leaves of winter rye of the Dahkowskie Zlote variety.

[0029] The cultivated field 100 was regularly photographed from the air using drones 10 and satellites 11. The photographic data was stored in a database 12, from which it could be retrieved by entering geographic coordinates 14. The input module 50 for image processing performs the standardisation process; for example, each sample represents the result of a hyperspectral measurement at a single point in the field (numerical response values for different wavelengths at a single image point). Standardisation consists of subtracting the mean value for each channel and scaling the data to unit variance. Additionally, the phosphorus (P) content in plants from the corresponding section of the field is measured. The measurement data is input into module 60, and at the output of module 60, an input vector is obtained. The set of inputvectors for different points in the field forms a training dataset. After standardisation, each sample in the training dataset consists of a numerical vector with a length corresponding to the number of spectral channels and the measured phosphorus content value. Module 70 schematically represents the training module of the FNN network, which is described in detail below with reference to Figure 2. In module 70, the steps related to training the neural network, as described in reference to Figure 2, are carried out. At the output of the process, a trained FNN network model, designated as 80, is obtained, which responds to an input vector and returns the predicted phosphorus content in the leaves, in this case, of winter rye of the Dahkowskie Zlote variety.

[0030] Figure 1b presents a block diagram of the computational tool according to the invention, utilising deep neural network training for the remote diagnosis of phosphorus content in winter rye leaves of the Dahkowskie Zlote variety. In Figure 1b, the same numerical designations are used to indicate identical elements. The computational tool presented in Figure 1b receives images from drones 11 or satellites 11, either directly or through a database collecting image datasets for a given area subjected to aerial or orbital monitoring. The photographic data may be stored in database 12, from which it can be retrieved after entering geographic coordinates 14. The input module 50 for image processing performs the standardisation process. For example, each sample represents the result of a hyperspectral measurement at a single point in the field (numerical response values for different wavelengths at a single image point). Standardisation consists of subtracting the mean value for each channel and scaling the data to unit variance. At the output of module 50, an input vector is obtained, consisting of a numerical vector with a length corresponding to the number of spectral channels in which images were taken for a single point in the cultivated field. In module 90, the input vector is fed into the neural network 80, which was obtained through the process presented in Figure 1a. In response, the predicted phosphorus (P) content in the leaves of the photographed plants is generated. It is essential that the standardisation process, the number of hyperspectral channels, and the wavelengths in these channels are closely matched as possible to those used in the training of the FNN 80 during the tool development stage.

[0031] Figure 2 presents an example architecture of the FNN neural network used in the solution according to the invention. This is a feedforward neural network (FNN) built using Keras in TensorFlow, consisting of three layers: an input layer with 64 neurons using a ReLU activation function, each receiving an input vector; a hidden layer with 32 neurons using a ReLU activation function, fully connected to the input layer; and an output layer with a single neuron performing a regression function.

[0032] The first step in this method involves collecting hyperspectral data related to light reflectance from Sentinel-2 spectral channels representing cultivated fields in Redlo near Polczyn-Zdroj and in Kuklowka Zarzeczna, along with measurement data representing phosphorus content in the leaves of winter rye (Dahkowskie Zlote variety). These data were obtained from two field experiments, conducted between the sowing period on 23 September 2022, at a planting density of 550 plants / m2, and the harvest period on 2 August 2023, in the cultivated fields in Redlo near Polczyn-Zdroj and in Kuklowka Zarzeczna.

[0033] The field experiments were conducted at three locations: the Experimental Station of the Institute of Agriculture, SGGW in Warsaw, the experimental fields in Redlo near Polczyn-Zdroj, and the experimental fields in Kuklowka Zarzeczna. The first experiment involved a four-field crop rotation with potatoes, spring barley, winter rye, and oats, in which the following fertilisation combinations were applied: control without fertilisation (0), NK, NP, PK, and NPK. The experiment was conducted with and without manure application, in three replications. The second experiment utilised a 100-year-old monoculture of rye and included the following fertilisation combinations: control without fertilisation (Ca only), as well as CaNP, CaKN, CaPK, and CaNPK. This experiment was also conducted in three replications. In both experiments, fertilisation was applied at the following rates: 90 kg N ha1- CO(NH2)2, 26 kg P ha-1 - Ca(H2PO4)2, 91 kg K ha1- KCI,and FYM (manure) at 20 Mg ha1every four years. Plant material samples were collected from an area of 0.25 m2at the time when rye was in the late tillering stage (BBCH 24), followed by another sampling when rye was in the stem elongation stage (BBCH 31 and BBCH 37, respectively). The plant material was then dried at a temperature of 50 °C and ground using a Retsch mill (Katowice, Poland) at a speed of 5000 revolutions per minute. The phosphorus content was determined using the molybdenum-vanadium method (BN-81 0520-15) after prior mineralisation in a mixture of HNO3 and HCIO4. The availability of phosphorus in the soil in its assimilable forms was determined using the Egner-Riehm method (PN-R-04023). Measurements were carried out during the same growth stage of winter rye.

[0034] Figure 3 presents the contribution of drone-based spectral bands using Sentinel-2 bands in the phosphorus level prediction model for winter rye. Based on the measurements, a comparative analysis of light reflectance across different spectral bands was conducted, and the bands B2 - 490 nm, B4 - 665 nm, and B1 - 443 nm were selected as input data. These bands also appear in the spectral bands of satellite imagery from the Sentinel-2 mission, allowing the integration of data from different sources. In other preferred embodiments, hyperspectral data representing cultivated fields is collected within the spectral ranges of 500-550 nm and 650-1000 nm.

[0035] In the next step, the hyperspectral data is normalised to a common value range, using, for example, the StandardScaler functionality from the scikit-learn library.

[0036] The training data for the neural network consists of a set of samples. Each sample represents the result of a hyperspectral measurement at a single point in the field (numerical response values for different wavelengths at a single image point) along with the measured phosphorus content in plants from the corresponding section of the field. Before being fed into the neural network, the hyperspectral measurement values are standardised - for each spectral channel, the mean value is subtracted, and the data is scaled to unit variance. After standardisation, each sample in the training dataset consists of a numerical vector with a length corresponding to the number of spectral channels and the measured phosphorus content value.

[0037] The training data is then converted into an input vector with a length corresponding to the number of hyperspectral data ranges, after which the dataset is split into a training set and a test set in an 80:20 ratio.

[0038] The converted training data is fed into a feedforward neural network built using Keras in TensorFlow, consisting of three layers. The input layer receives a vector of size n, corresponding to the number of hyperspectral measurement channels, with each value being processed by each neuron in the input layer. The input data, derived from hyperspectral measurements, is thus represented as a vector with n values, each corresponding to signal intensity for a specific wavelength after dimensionality reduction. The input layer consists of 64 neurons, each fully connected to every element of the input vector. This means that each of the n input values is transmitted to every neuron in the input layer, and the output of each neuron is calculated as a combination of the input values, incorporating weights and a bias term.

[0039] The feedforward neural network FNN, consisting of an input layer with 64 neurons using a ReLU activation function, each receiving an input vector; a hidden layer with 32 neurons, fully connected to the input layer and using a ReLU activation function; and an output layer with a single neuron performing a regression function, is trained such that the output data, representing the predicted chemical element content in plant leaves, corresponds to the actual chemical element content in plant leaves. The trained FNN network is then validated for its accuracy in diagnosing phosphorus content using the test dataset.The training process lasts for 10,000 epochs, in batches of 32 samples, optimising the network parameters using the backpropagation algorithm to minimise the mean squared error (MSE) between the predicted and actual phosphorus content values. The Adam optimiser was used to ensure optimal convergence. During the training process, progress was monitored using validation data.

[0040] Upon completion of neural network training, the results generated by the network are evaluated on the test dataset to assess prediction accuracy. The MSE and the coefficient of determination (R2) are calculated, and the data is visualised in a scatter plot, indicating predicted phosphorus deficiencies.

[0041] Figure 4 presents a scatter plot comparing predicted vs. actual phosphorus content, with a reference line and a prediction accuracy percentage for values below 4 g / kg, which is identified as the threshold for phosphorus deficiency.

[0042] The position of a point in the scatter plot in Figure 4 represents the result of comparing the predicted phosphorus values, as determined by the neural network, with actual in situ measurements obtained under laboratory conditions. The scatter plot of the predicted and measured phosphorus content exhibits a high accuracy in phosphorus deficiency assessment (approximately 64%) with a Root Mean Squared Error (RMSE) of 0.84 g / kg. Foran ideal example of the functioning of a neural network, I. e., a case in which all predicted values would correspond to the measured values, the points in Figure 4 chart would lie on the diagonal reference line marked with a dashed line.

[0043] Figure 5 shows the result of comparing the predicted phosphorus values, as determined by the neural network, with actual in situ measurements obtained under laboratory conditions. The coefficient of determination (R2) for the FNN model was 0.57, indicating a moderate correlation between the predicted and actual phosphorus values. The RMSE was 0.69 g / kg, reflecting the average magnitude of model prediction errors. Additionally, the model achieved 58% accuracy in predicting phosphorus content below the 4 g / kg threshold.

[0044] The first embodiment of the invention, concerning the method for developing a computational tool for remote diagnosis of nutrient content in plants using machine learning in a neural network, has been presented above in the context of phosphorus content measurement. In another preferred embodiment, the element whose content is determined in plant leaves is nitrogen.

[0045] In a preferred embodiment of the invention, the cultivated plant used for neural network training is winter rye of the Dahkowskie Zlote variety. In other preferred embodiments, element content is determined for other cereal species, particularly maize, wheat, other rye varieties, oats, or rice.

[0046] Preferably, the method according to the invention is implemented for the diagnosis of nutrient content during the following plant growth stages: BBCH 10-16 for maize, BBCH 15-32 for other crops.

[0047] For the diagnosis of nutrient content, a computational method is applied, enabling the remote determination of nutrient content in plants using learning in a neural network according to the invention. The computational tool implements a method comprising steps in which: hyperspectral data is collected, consisting of multiple electromagnetic wavelength ranges representing cultivated fields 100, the hyperspectral data is normalised to a common value range, the normalised hyperspectral data is converted into an input vector with a length corresponding to the number of hyperspectral data ranges, the input vector is fed into a neural network (FNN) developed according to the invention, the output block returns the predicted chemical element content in the leaves of cultivated plants, based on the processing of hyperspectral data consisting of multiple electromagnetic wavelength ranges representing cultivated fields 100.In preferred embodiments, the hyperspectral data is obtained from aerial or satellite imagery, depending on availability. Furthermore, in other preferred embodiments, the hyperspectral data in the form of images is automatically retrieved from a hyperspectral image database, with data being selected based on the geographic coordinates of the cultivated field.

Claims

Patent claims1. A method for developing a computer tool for the remote diagnosis of chemical element content in plants using neural network learning, comprising steps in which:a training dataset is collected, consisting of hyperspectral data with multiple electromagnetic wavelength ranges representing cultivated fields (100) and measurement data representing element content in plant leaves from cultivated fields (100);the hyperspectral data is normalised to a common value range;the training data is converted into an input vector with a length corresponding to the number of hyperspectral data ranges;the dataset is split into a training set and a test set;the converted training data is fed into a feedforward neural network FNN consisting of three layers:an input layer with 64 neurons with a ReLU activation function, each receiving the input vector,a hidden layer with a ReLU activation function, with 32 neurons, fully connected to the input layer andan output layer with a single neuron performing a regression function, wherein the FNN neural network is trained so that the output data, representing the predicted chemical element content in plant leaves, corresponds to the actual chemical element content in plant leaves,the trained neural network FNN is validated for accuracy in diagnosing chemical element content using the test dataset.

2. The method according to claim 1 characterised in that the chemical element determined in the leaves of cultivated plants is phosphorus (P) or nitrogen (N).

3. The method according to claim 1 or 2 characterised in that the plants for which the chemical element content is determined are cereals.

4. The method according to any of claims 1 to 3 characterised in that the plants for which the chemical element content is determined include maize, wheat, rye, oats, or rice.

5. The method according to any of claims 1 to 4 characterised in that the hyperspectral data collection representing cultivated fields occurs in the wavelength ranges of 500-550 nm and 650-1000 nm.

6. The method according to any of claims 1 to 5 characterised in that the diagnosis of chemical element content is performed during the following plant growth stages:- BBCH 10-16 for maize, or- BBCH 15-32 for other plants.

7. The method according to any of claims 1 to 6, characterised in that the feedforward neural network (FNN) is trained using the backpropagation algorithm to minimise the mean squared error (MSE) between the predicted and actual element content values, along with the Adam optimiser.

8. A computer-implemented method for the remote diagnosis of chemical element content in plants using neural network learning, comprising steps in which:hyperspectral data is collected, consisting of multiple electromagnetic wavelength ranges representing cultivated fields (100), and thenthe hyperspectral data is normalised to a common value range;the normalised hyperspectral data is converted into an input vector with a length corresponding to the number of hyperspectral data ranges;the input vector is fed into a feedforward neural network (FNN) developed according to claims 1 to 7;the output block receives a value representing the chemical element content in the leaves of cultivated plants, obtained from the processing of hyperspectral data by the neural network, where the hyperspectral data consists of multiple electromagnetic wavelength ranges representing cultivated fields (100).

9. The method according to claim 8 characterised in that the hyperspectral data is obtained from aerial or satellite imagery.

10. The method according to claim 8 or 9, characterised in that the hyperspectral data in the form of images is automatically retrieved from a hyperspectral image database, with data being selected based on the geographic coordinates of the cultivated field.

11. A computer programme, which, when executed on a computer, performs the steps of the method according to any of claims 8 to 10.

Images