Nitrogen fertilizer decision-making method and device for crops, electronic equipment and storage medium

Abstract

The invention relates to the technical field of intelligent agriculture, in particular to a nitrogen fertilizer decision-making method and device for crops, electronic equipment and a storage medium, and the method comprises the steps: obtaining the basic information, decision-making daily precipitation and a canopy image of a to-be-observed crop; constructing an environment body model based on the basic information, obtaining a crop growth simulation state of the to-be-observed crop by using the environment body model, and inputting the canopy image into a preset estimation model to obtain a basic growth state of the to-be-observed crop; and based on a preset assimilation model, fusing the crop growth simulation state and the basic growth state to obtain an updated crop growth state, and based on a preset agent model, generating fertilization decision time and a fertilization decision value according to the updated crop growth state and the daily precipitation of the decision. Therefore, the problem of nitrogen fertilizer decision errors caused by deviation between the environment body and the application environment is avoided, the method has the advantages of being rapid, lossless and high in precision, and real-time farmland crop nitrogen fertilizer decision can be carried out.

Description

Technical Field

[0001] The present application relates to the field of smart agricultural technology, and in particular to a nitrogen fertilizer decision-making method, device, electronic device and storage medium for crops. Background Art

[0002] Existing farmland nitrogen fertilizer management relies on traditional experience. During the crop growth process, the time and amount of nitrogen fertilizer application are relatively fixed. The same management model is usually adopted in a region (county or township), ignoring the temporal and spatial differences in crop growth, and unable to ensure the synchronization of fertilization and crop nitrogen demand. Crop nitrogen fertilizer decisions need to determine the optimal time and amount of fertilization to maximize crop yields or economic benefits. Faced with the long sequence of crop growth cycles and the large nitrogen application action space, the computational complexity of the dynamic programming method is very high, so the current nitrogen fertilizer optimization management still relies on a fixed schedule for decision-making.

[0003] The reinforcement learning method combines Monte Carlo sampling and dynamic programming algorithms, uses random walks to explore the environment, and can provide different decision-making solutions based on the state of the external environment. Deep reinforcement learning addresses the problem that traditional reinforcement learning is difficult to handle with a large number of states and actions. It introduces deep neural networks to perceive more complex environmental states and establish more complex strategies, thereby improving the capabilities of reinforcement learning algorithms.

[0004] However, in related technologies, most deep reinforcement learning environments are a single crop model or its alternative model, which does not fully consider the possible deviation between crop simulation and actual growth. The uncertainty of the decision state will be directly transmitted to the decision model, which can easily lead to decision errors. In addition, the rewards of deep reinforcement learning in agricultural decision-making mainly focus on yield or economic benefits, and lack consideration of environmental pollution caused by nitrogen fertilizer emissions, and cannot comprehensively consider planting income and environmental benefits. Finally, the cost of acquiring the state variables of the decision is high, and destructive sampling is usually used, which is time-consuming and labor-intensive, and cannot achieve low-cost and rapid real-time decision-making on crop nitrogen fertilizers. Summary of the invention

[0005] The present application provides a nitrogen fertilizer decision-making method, device, electronic device and storage medium for crops to solve the problems that the relevant technologies do not consider the deviation between crop simulation and actual growth, which easily leads to decision-making errors, fail to comprehensively consider planting benefits and environmental benefits, and the efficiency of state variable acquisition is low. Data assimilation can be used to reduce decision errors, and images are used to estimate crop growth status to efficiently obtain state variables, and crop yield, economic benefits and environmental benefits are comprehensively considered.

[0006] The first embodiment of the present application provides a method for nitrogen fertilizer decision-making for crops, comprising the following steps:

[0007] Obtain basic information of the crops to be observed, decision-making day precipitation and canopy images;

[0008] An environment model is constructed based on the basic information, and the environment model is used to obtain a simulated growth state of the crop to be observed, and the canopy image is input into a preset estimation model to obtain a basic growth state of the crop to be observed;

[0009] Based on a preset assimilation model, the crop growth simulation state and the basic growth state are integrated to obtain an updated crop growth state, and based on a preset intelligent agent model, a fertilization decision time and a fertilization decision value are generated according to the updated crop growth state and the precipitation on the decision day.

[0010] Optionally, in some embodiments, after generating the fertilization decision time and the fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on the preset agent model, the method further comprises:

[0011] Based on the environment model, generating an agent reward according to the fertilization decision time and the fertilization decision value;

[0012] Based on the preset agent model, the fertilization decision time and the fertilization decision value are updated according to the agent reward.

[0013] Optionally, in some embodiments, the agent reward is:

[0014] r t =w1×2.72×Y tarvest -w2×5.53×N t -w3×150×Labor t -w4×P t ;

[0015] Among them, r t is the reward at time t, Y harvest is the output, N t is the nitrogen application rate at time t, lABOR t is the labor cost at time t, P t is the harmful cost of nitrogen fertilizer loss to the environment, and w1, w2, w3 and w4 are all reward coefficients.

[0016] Optionally, in some embodiments, before generating the fertilization decision time and the fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on the preset agent model, the method includes:

[0017] The preset agent model is trained by the SAC value function and the SAC state function, wherein the SAC value function is:

[0018]

[0019] The SAC state function is:

[0020]

[0021] Among them, Q(s t ,a t ) is the state s at time t t Take action a t The state-action value function, r(s t ,a t ) is in state s t Take action a t The reward is , γ is the discount factor, For the next state s t+1 The expectation of the state function, p is the transition probability, π is the strategy, logπ(a t |s t ) is the strategy π in state s t Next select action a t The logarithm of the probability, π new is a new strategy, π′ is a new strategy to be optimized, D KL is the KL divergence, is the normalization constant, For the old strategy π old Next, from state s t The starting value function, For strategy π to all possible actions a t The expectation of the state-action value function.

[0022] Optionally, in some embodiments, the preset assimilation model is:

[0023]

[0024] in, is the updated state matrix, is the state matrix of the model running to time t, K t is the Kalman gain at time t, d obs,t is the perturbed observation value, H t is the operator that converts the state matrix into the observation matrix at time t.

[0025] Optionally, in some embodiments, the crop growth status includes: crop growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield.

[0026] The second aspect of the present application provides a nitrogen fertilizer decision-making device for crops, comprising:

[0027] The acquisition module is used to obtain basic information of the crop to be observed, the precipitation on the decision day, and the canopy image;

[0028] A simulation module, used to construct an environment model based on the basic information, and use the environment model to obtain a simulated growth state of the crop to be observed, and input the canopy image into a preset estimation model to obtain a basic growth state of the crop to be observed;

[0029] The decision-making module is used to obtain an updated crop growth state by fusing the crop growth simulation state and the basic growth state based on a preset assimilation model, and to generate a fertilization decision time and a fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on a preset intelligent agent model.

[0030] Optionally, in some embodiments, after generating the fertilization decision time and the fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on the preset agent model, the decision module includes:

[0031] A reward unit, used to generate an intelligent agent reward based on the environment body model and according to the fertilization decision time and the fertilization decision value;

[0032] An updating unit is used to update the fertilization decision time and the fertilization decision value based on the preset agent model and the agent reward.

[0033] Optionally, in some embodiments, the agent reward is:

[0034] r t =w1×2.72×Y harvest -w2×5.53×N t -w3×150×Labor t -w4×P t ;

[0035] Among them, r t is the reward at time t, Y harvest is the output, N t is the nitrogen application amount at time t, Labor t is the labor cost at time t, P t is the harmful cost of nitrogen fertilizer loss to the environment, and w1, w2, w3 and w4 are all reward coefficients.

[0036] Optionally, in some embodiments, before generating the fertilization decision time and the fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on the preset agent model, the method includes:

[0037] The preset agent model is trained by the SAC value function and the SAC state function, wherein the SAC value function is:

[0038]

[0039] The SAC state function is:

[0040]

[0041] Among them, Q(s t ,a t ) is the state s at time t t Take action a t The state-action value function, r(s t ,a t ) is in state s t Take action a t The reward is , γ is the discount factor, For the next state s t+1 The expectation of the state function, p is the transition probability, π is the strategy, logπ(a t |s t ) is the strategy π in state s t Next select action a t The logarithm of the probability, π new is a new strategy, π′ is a new strategy to be optimized, D KL is the KL divergence, is the normalization constant, For the old strategy π old Next, from state s t The starting value function, For strategy π to all possible actions a t The expectation of the state-action value function.

[0042] Optionally, in some embodiments, the preset assimilation model is:

[0043]

[0044] in, is the updated state matrix, is the state matrix of the model running to time t, K t is the Kalman gain at time t, d obs,t is the perturbed observation value, H t is the operator that converts the state matrix into the observation matrix at time t.

[0045] Optionally, in some embodiments, the crop growth status includes: crop growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield.

[0046] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the nitrogen fertilizer decision-making method for crops as described in the above embodiment.

[0047] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the nitrogen fertilizer decision-making method for crops as described in the above embodiments.

[0048] Therefore, this application has at least the following beneficial effects:

[0049] (1) The embodiment of the present application uses a data assimilation algorithm to reduce the decision error of the deep reinforcement learning model, and achieves accurate decision-making on crop nitrogen fertilizer by integrating the uncertainty of observation and environmental simulation.

[0050] (2) The embodiment of the present application uses a mobile phone to take photos of crop canopies, and estimates various growth states of crops based on mobile phone images, which has the advantages of being fast and lossless, and assimilates the observations of mobile phone images into a deep reinforcement learning environment, enabling real-time crop nitrogen fertilizer decisions.

[0051] (3) The embodiments of the present application comprehensively consider crop yield, economic benefits and environmental benefits, and can make crop nitrogen fertilizer decisions according to the development needs of different regions, so that the deep reinforcement learning model can adapt to different application scenarios.

[0052] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0054] Figure 1 A flowchart of a nitrogen fertilizer decision-making method for crops provided according to an embodiment of the present application;

[0055] Figure 2 A schematic diagram of the principle of a nitrogen fertilizer decision-making method for crops provided according to an embodiment of the present application;

[0056] Figure 3 A schematic diagram of a SAC algorithm training result provided according to an embodiment of the present application, wherein: Figure 3(a1) is a schematic diagram of the nitrogen application rate change process of the SAC algorithm training 50,000 steps. Figure 3 (a2) is a schematic diagram of the rice yield change process after 50,000 steps of SAC algorithm training. Figure 3 (b1) is a schematic diagram of the nitrogen application rate change process during the 100,000 steps of SAC algorithm training. Figure 3 (b2) is a schematic diagram of the rice yield change process after 100,000 steps of SAC algorithm training. Figure 3 (c1) is a schematic diagram of the nitrogen application rate change process of the SAC algorithm training 200000 steps. Figure 3 (c2) is a schematic diagram of the rice yield change process after 200,000 steps of SAC algorithm training;

[0057] Figure 4 This is a schematic diagram of a crop canopy image taken by a mobile phone according to an embodiment of the present application, wherein: Figure 4 (a) is a schematic diagram of the tillering stage. Figure 4 (b) is a schematic diagram of the jointing stage. Figure 4 (c) is a schematic diagram of the grouting period;

[0058] Figure 5 This is a schematic diagram of a crop growth status estimation result based on a mobile phone image according to an embodiment of the present application, wherein: Figure 5 (a) is a schematic diagram of the estimated results of the growth period. Figure 5 (b) is a schematic diagram of the vegetation area index estimation results. Figure 5 (c) is a schematic diagram of the aboveground biomass estimation results. Figure 5 (d) is a schematic diagram of the leaf nitrogen content estimation results;

[0059] Figure 6 This is a schematic diagram of the simulation results of assimilating the crop state estimated by mobile phone images into the rice growth state in the gym-ORYZA environment according to one embodiment of the present application, wherein: Figure 6 (a1) is a schematic diagram of the observed values ​​of the growth period of the gym-ORYZA environment simulation results. Figure 6 (a2) is a schematic diagram of the observed values ​​of the growth period of the ORYZA-EnKF simulation results. Figure 6 (b1) is a schematic diagram of the observed values ​​of the vegetation area index of the gym-ORYZA environment simulation results. Figure 6 (b2) is a schematic diagram of the observed values ​​of the vegetation area index of the ORYZA-EnKF simulation results. Figure 6 (c1) is a schematic diagram of the aboveground biomass observations of the gym-ORYZA environment simulation results. Figure 6(c2) is a schematic diagram of the aboveground biomass observations of the ORYZA-EnKF simulation results. Figure 6 (d1) is a schematic diagram of the observed leaf nitrogen content of the gym-ORYZA environment simulation results. Figure 6 (d2) is a schematic diagram of the observed values ​​of leaf nitrogen content from the ORYZA-EnKF simulation results;

[0060] Figure 7 The schematic diagram is a schematic diagram of the application results of crop nitrogen fertilizer decision-making based on mobile phone images and deep reinforcement learning in different state estimation scenarios according to one embodiment of the present application, wherein: Figure 7 (a) is a schematic diagram of rice nitrogen application decision based on the SAC model under the state-free scenario. Figure 7 (b) is a schematic diagram of rice nitrogen application decision based on the SAC model under the overestimation scenario. Figure 7 (c) is a schematic diagram of rice nitrogen application decision based on the SAC model under the state underestimation scenario;

[0061] Figure 8 A block diagram of a nitrogen fertilizer decision-making device for crops provided according to an embodiment of the present application;

[0062] Fig. 9 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0064] The following describes the nitrogen fertilizer decision method, device, electronic device and storage medium of the crop of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that the related art mentioned in the above background technology does not take into account the deviation between crop simulation and actual growth, which easily leads to decision-making errors, fails to comprehensively consider planting benefits and environmental benefits, and the efficiency of state variable acquisition is low, the present application provides a nitrogen fertilizer decision method for crops, in which the basic information of the crop to be observed, the precipitation on the decision day and the canopy image are obtained, and an environmental body model is constructed based on the basic information, and the crop growth simulation state of the crop to be observed is obtained by using the environmental body model, and the canopy image is input into the preset estimation model to obtain the basic growth state of the crop to be observed; based on the preset assimilation model, the crop growth simulation state and the basic growth state are fused to obtain the updated crop growth state, and based on the preset intelligent body model, the fertilization decision time and fertilization decision value are generated according to the updated crop growth state and the precipitation on the decision day. As a result, the problems that related technologies fail to consider the deviation between crop simulation and actual growth, which can easily lead to decision-making errors, fail to comprehensively consider planting benefits and environmental benefits, and have low efficiency in acquiring state variables are solved. Data assimilation can be used to reduce decision-making errors, and images can be used to estimate crop growth status to efficiently acquire state variables, while comprehensively considering crop yield, economic benefits and environmental benefits.

[0065] Specifically, Figure 1 A schematic diagram of a process flow of a nitrogen fertilizer decision-making method for crops provided in an embodiment of the present application.

[0066] like Figure 1 As shown, the nitrogen fertilizer decision-making method for this crop includes the following steps:

[0067] In step S101, basic information of the crop to be observed, precipitation on the decision day, and canopy images are obtained.

[0068] It should be noted that the embodiments of the present application can obtain basic information of the crops to be observed through historical observation data of the crops to be observed, wherein the basic information includes growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield, etc. The precipitation on the decision day can be estimated through weather forecast information, and the canopy image can be obtained by taking canopy photos of the crops to be observed by mobile phone.

[0069] In the actual implementation process, the embodiment of the present application collects crop canopy images through a mobile phone, and uses a deep convolutional neural network model to estimate the basic growth status of the crop, including the growth period, leaf area index, aboveground biomass and leaf nitrogen content. The deep convolutional neural network model framework can use the lightweight MobileNetV3, which has the characteristics of small number of parameters and fast operation speed. It should be noted that the model input for estimating the growth period, leaf area index and aboveground biomass is a long-range photo of the crop canopy, which was taken at a distance of about 1.5m from the ground; the model input for estimating the leaf nitrogen content is a close-up photo of the crop canopy, which was taken at a distance of about 20-30cm from the crop canopy.

[0070] In step S102, an environment model is constructed based on the basic information, and the environment model is used to obtain the crop growth simulation state of the crop to be observed, and the canopy image is input into a preset estimation model to obtain the basic growth state of the crop to be observed.

[0071] Among them, crop growth status includes: crop growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield.

[0072] It should be noted that the embodiment of the present application can develop a deep reinforcement learning environment model gym-ORYZA based on the classic mechanism crop model, which includes two core modules, "reset" and "step", for simulating the growth and development process of crops and their response to the application of nitrogen fertilizer. Among them, the "reset" module initializes the parameters of the environment body, and the "step" is a module that can adjust the amount of fertilizer and calculate the growth status of crops.

[0073] Specifically, the embodiments of the present application can calibrate the crop model through historical observation data, and the calibrated crop model is used to construct an environment model, wherein the environment model can be a deep reinforcement learning environment gym-ORYZA, which is used to simulate the growth and development process of crops, obtain the simulated growth state of the crops to be observed, and take photos of the crop canopy with a mobile phone, and use a deep learning model to calculate the growth period, leaf area index, aboveground biomass and leaf nitrogen content of the crops, so as to achieve fast and non-destructive high-precision observation of the crop status and obtain the basic growth state of the crops to be observed.

[0074] In step S103, based on the preset assimilation model, the crop growth simulation state and the basic growth state are integrated to obtain an updated crop growth state, and based on the preset intelligent agent model, the fertilization decision time and fertilization decision value are generated according to the updated crop growth state and the precipitation on the decision day.

[0075] It is understandable that before executing this step, the embodiment of the present application needs to construct a preset assimilation model and a preset agent model.

[0076] Among them, the preset assimilation model is the ORYZA-EnKF data assimilation model that needs to assimilate crop multivariate observation information. The EnKF algorithm calculation formula is:

[0077]

[0078] Among them, y t is the state matrix of the model at time t, with dimension N y =N m +N u +N d . m t T is the model parameter matrix, u t T is composed of model state variables, d t T is the matrix of observations.

[0079] Observations of the model can be transformed using an observation operator:

[0080] y t =Hy t ;

[0081] H = [0, I];

[0082] Among them, H refers to the observation operator, 0 is an N d ×(N m +N u )-dimensional matrix, all elements in the matrix are 0, I is an identity matrix, and the dimension is N d ×N d .

[0083] When the model runs to time t, after the state matrix is ​​calculated, it can be updated using the EnKF algorithm:

[0084]

[0085] in, represents the updated state matrix, Represents the state matrix of the model running to time t, d obs,t represents the perturbed observation, K t is the Kalman gain at time t, where the Kalman gain can be calculated by the covariance matrix of the sample set:

[0086]

[0087] in, Refers to the prior covariance matrix of the state vector simulated by the model at time t, is the matrix of the state matrix at time t, i is the sample number, is the observation error.

[0088] The embodiment of the present application can use the deep reinforcement learning algorithm SAC to train the preset intelligent agent model, wherein the SAC algorithm value function and the SAC algorithm state strategy function are:

[0089]

[0090] Among them, -logπ(a t |s t ) indicates that the strategy π is in state s t The strategy is improved by minimizing the KL divergence of the action distribution. The improved strategy is closer to the exponential action distribution of the current value function, thereby improving the performance of the overall strategy.

[0091] The goal of the SAC algorithm is to maximize the expectation of the sum of reward and policy entropy:

[0092]

[0093] Among them, H(π(·|s t )) is the policy π in state s t By regularizing the entropy, the exploration degree of the reinforcement learning algorithm can be increased. The larger the value of α, the stronger the exploration, which can reduce the possibility of the strategy falling into the local optimum.

[0094] Specifically, the embodiment of the present application assimilates the basic growth status information observed by mobile phone images into the gym-ORYZA environment body through the ensemble Kalman filter EnKF algorithm, constructs a preset assimilation model ORYZA-EnKF, and reduces the simulation deviation caused by the uncertainty of the crop model; the crop status updated by the preset assimilation model ORYZA-EnKF is input into the trained deep reinforcement learning model SAC to obtain real-time crop nitrogen fertilizer decision results.

[0095] In the actual implementation process, the deep reinforcement learning algorithm SAC samples through the interaction between the intelligent agent and the environment during the training process. Considering the uncertainty of the model structure, parameters and input, the environment has a large deviation in the simulation process. The ORYZA-EnKF data assimilation system reduces the simulation deviation caused by the uncertainty of the crop model by integrating the uncertainty from mobile phone image observations and gym-ORYZA simulations. Since mobile phone image shooting is low-cost and fast, the updated state of the ORYZA-EnKF system is input into the trained SAC model to complete accurate real-time decision-making on crop nitrogen fertilizer.

[0096] Optionally, in some embodiments, after generating the fertilization decision time and the fertilization decision value according to the updated crop growth status and the precipitation on the decision day based on the preset intelligent agent model, it includes: generating an intelligent agent reward according to the fertilization decision time and the fertilization decision value based on the environmental agent model; and updating the fertilization decision time and the fertilization decision value according to the intelligent agent reward based on the preset intelligent agent model.

[0097] It should be noted that the embodiment of the present application can construct a reward function for reinforcement learning, which is composed of crop yield, fertilizer application amount and environmental nitrogen emissions, and comprehensively considers the crop growth, economic and environmental benefits of nitrogen fertilizer application. Among them, the reward function includes crop yield, fertilizer application amount and environmental nitrogen emissions, and the calculation formula is:

[0098] r t =w1×2.72×Y harvest -w2×5.53×N t -w3×150×Labor t -w4×P t ;

[0099]

[0100] Among them, rt is the reward at time t (unit: yuan / hectare). w1, w2, w3 and w4 refer to the corresponding reward coefficients, which are 1 by default and can be adjusted according to different situations. Yharvest is the crop yield at harvest (unit: kg / hectare). There will be a yield reward only when the crops are harvested. The rest of the time is 0, which is determined to be 2.72 yuan / kg based on the average market price. Nt represents the amount of nitrogen applied at time t (unit: kg nitrogen / hectare), which is determined to be 5.53 yuan / kg based on the average market price of nitrogen fertilizer. Labort is the labor cost at time t (unit: yuan / hectare), which is determined to be 150 yuan / time / hectare based on cost research; if there is no fertilization at time t, it is 0. Pt refers to the harmful cost of nitrogen fertilizer loss to the environment. and They are the emission losses of NH3 and N2O and the leaching losses of NO3- (unit: kg / hectare). According to the survey, the environmental costs are 39.3 yuan / kg, 83.5 yuan / kg and 9.6 yuan / kg respectively.

[0101] The elements contained in the decision space of the preset intelligent agent model are all variables with very low acquisition cost and easy to obtain. The precipitation on the decision day can be estimated through weather forecast information. The growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield in the crop growth status are the basic information of crops. No more complex observations are required. The state space is generated through the environmental body model gym-ORYZA, and the preset intelligent agent model is reinforced to obtain the state and give decision actions. After receiving the decision actions, the environmental body model gym-ORYZA will feedback the corresponding rewards to the intelligent agent to guide the optimization direction of the intelligent agent.

[0102] Therefore, the embodiment of the present application adopts the deep reinforcement learning algorithm SAC to train the intelligent agent, and through interactive sampling between the intelligent agent and the environment, the nitrogen application time and nitrogen application amount are optimized according to the reward function until the optimal nitrogen application strategy is generated; the introduction of entropy regularization and softening strategy update can allow the intelligent agent to better explore unknown states and improve learning efficiency.

[0103] In order to enable relevant technical personnel in the field to further understand the nitrogen fertilizer decision-making method for crops in the embodiment of the present application, it is described in detail below in conjunction with specific embodiments.

[0104] Combination Figure 2 As shown, Figure 2 The figure is a schematic diagram of the principle of the nitrogen fertilizer decision-making method for crops provided according to an embodiment of the present application.

[0105] Step 1. In order to minimize the error between the model simulation results and the observed values, the embodiment of the present application is based on the calibrated crop model and uses Python language to develop the deep reinforcement learning environment body gym-ORYZA, which includes two core modules: "reset" and "step". The "reset" module initializes the parameters of the environment body, and the "step" module can adjust the amount of fertilizer and calculate the crop growth status. Through the environment body gym-ORYZA, the simulation of the crop growth and development process and its response to the application of nitrogen fertilizer can be realized.

[0106] Step 2, build a deep reinforcement learning agent, which can interact with gym-ORYZA. Construct a state space for reinforcement learning decisions. The elements contained in the decision space in the embodiment of the present application are all variables with low acquisition cost and convenient acquisition. The rainfall on the decision day can be estimated by weather forecast information. The growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield in the crop growth state are the basic information of the crop, without more complex observations. The agent obtains the rainfall and crop growth status, and gives the time and value of the fertilization decision; after gym-ORYZA receives the decision action, it will feedback the corresponding reward to the agent, thereby guiding the optimization direction of the agent.

[0107] Step 3, construct the reward for deep reinforcement learning, which includes three components: crop yield, fertilizer application, and environmental nitrogen emissions, and the importance of different components can be adjusted according to different development needs. When the three weights are considered equal, the reward calculation formula is given in the form of profit monetization:

[0108] r t =2.72×Y harvest -5.53×N t -150×Labor t -P t ;

[0109]

[0110] Among them, r t Y is the reward at time t (unit: yuan / hectare). harvest It is the crop yield at harvest (unit: kg / hectare). Yield bonus is only given when the crop is harvested, and it is 0 at other times. It is determined to be 2.72 yuan / kg based on the average market price of rice. t Represents the nitrogen application amount at time t (unit: kg nitrogen / hectare), which is determined to be 5.53 yuan / kg based on the average market price of nitrogen fertilizer. t is the labor cost at time t (unit: yuan / hectare), which is determined to be 150 yuan / time / hectare based on cost research; if there is no fertilization at time t, it is 0. t refers to the harmful cost of nitrogen fertilizer loss to the environment, and They are the emission losses of NH3 and N2O and NO3 - The leaching losses (unit: kg / hectare) were 39.3 yuan / kg, 83.5 yuan / kg and 9.6 yuan / kg respectively.

[0111] Step 4: Use the deep reinforcement learning algorithm SAC to train the reinforcement learning agent. The value function and state strategy function of the SAC algorithm are:

[0112]

[0113]

[0114] Among them, -logπ(a t |s t ) indicates that the strategy π is in state s t The strategy is improved by minimizing the KL divergence of the action distribution. The improved strategy is closer to the exponential action distribution of the current value function, thereby improving the performance of the overall strategy.

[0115] The goal of the SAC algorithm is to maximize the expectation of the sum of reward and policy entropy:

[0116]

[0117] Among them, (π(·|s t )) is the policy π in state s t The entropy under α can be increased by regularizing the entropy. The larger the value of α, the stronger the exploration, which can reduce the possibility of the strategy falling into the local optimum.

[0118] like Figure 3 As shown in the figure, in the early stage of SAC model training, the training trajectory jumps a lot, and gradually stabilizes after about 1,000 rounds of training, and finally converges continuously. After training, the SAC model can make decisions on crop nitrogen fertilizer, and the nitrogen application amount and rice yield obtained are in line with the normal agricultural production practice.

[0119] Step 5: Use a mobile phone to take a canopy image of the crop, such as Figure 4 As shown. The canopy images of crops are collected by mobile phones, and the basic growth status of crops, including growth period, leaf area index, aboveground biomass and leaf nitrogen content, is estimated using a deep convolutional neural network model. The model input for estimating growth period, leaf area index and aboveground biomass is a long-range photo of the crop canopy, which is taken about 1.5m from the ground; the model input for estimating leaf nitrogen content is a close-up photo of the crop canopy, which is taken about 20-30cm from the crop canopy. The shooting angles are usually 0° and 60° from the horizontal. The estimation model framework uses the lightweight MobileNetV3, which has the characteristics of small number of parameters and fast computing speed.

[0120] like Figure 5 As shown, the crop growth status estimated by the mobile phone images in the embodiment of the present application is highly accurate, with an estimated RRMSE between 16% and 30%, indicating that fast, lossless and high-precision crop status monitoring can be achieved using mobile phone images.

[0121] Step 6: Build an ORYZA-EnKF data assimilation system that can assimilate multivariate observations of crops, including crop growth period, leaf area index, aboveground biomass, and leaf nitrogen content. Use the Ensemble Kalman Filter EnKF algorithm to assimilate mobile phone image observations into the gym-ORYZA environment volume:

[0122]

[0123] in, represents the updated state matrix; Represents the state matrix of the model running to time t; d obs,trepresents the perturbed observation value; K t is the Kalman gain at time t. The Kalman gain can be calculated by the covariance matrix of the sample set:

[0124]

[0125]

[0126] in, Refers to the prior covariance matrix of the state vector simulated by the model at time t; is the matrix of the state matrix at time t; i is the sample number; is the observation error.

[0127] like Figure 6 As shown in the figure, by fusing the observations of mobile phone images into the gym-ORYZA environment body, the simulation accuracy of crop growth status of the environment body is further improved, and the RRMSE is reduced to 9%-29%.

[0128] The embodiment of the present application continues to provide the updated state of the ORYZA-EnKF data assimilation system to the trained SAC deep reinforcement learning model, such as Figure 7 shown. Figure 7 (a) is the nitrogen application decision made by the SAC model based on the calibrated gym-ORYZA environment when the state estimation is unbiased. Figure 7 In (b), the open-loop simulation based on overestimation of growth status applied nitrogen three times, concentrated in the early stage of rice growth, with a total nitrogen application of 188 kg N ha -1 The final yield was 6574kg ha -1 . As the rice leaf area index, aboveground biomass, and leaf nitrogen content in the overestimated scenario are all high, rice does not seem to need much nitrogen application, which leads to an underestimation of rice's nitrogen demand in this overestimated state. Despite the uncertainty, the EnKF algorithm can improve the simulation of rice growth status, so a more accurate nitrogen application decision process can be obtained, and the final nitrogen application rate is 258 kg N ha -1 The final yield is 6980kg ha -1 From the perspective of rice yield and nitrogen fertilizer economic benefits, the benefit of the decision made by Open-loop simulation under the overestimation state is 16,392 yuan / ha, while the benefit of nitrogen application decision made based on ORYZA-EnKF is 17,109 yuan / ha, avoiding a loss of 717 yuan / ha. Figure 7 In (c), due to underestimation of the rice growth status, the number of nitrogen applications based on the open-loop simulation was 5, and the total nitrogen application rate was very high, reaching 456 kg Nha. -1, which can be understood as the poor growth of rice and the need for more fertilizer, but this is not the case in reality, leading to an overestimation of the nitrogen application rate. Similarly, the nitrogen application rate based on ORYZA-EnKF is more reasonable, with 3 applications in the early growth stage and 1 supplement at the booting stage, for a total nitrogen application of 293 kg N ha -1 , the yield is 7002kg ha -1 It can be found that in the underestimated state, the decision of the Open-loop simulation increased by 136 kg N ha -1 The yield was only slightly increased (31 kg ha -1 ), the total economic benefit is 15,893 yuan / ha; while the benefit of nitrogen application decision based on ORYZA-EnKF is 16,825 yuan / ha, which is 932 yuan / ha more than that of Open-loop simulation.

[0129] In summary, the embodiment of the present application develops a deep reinforcement learning environment based on the calibrated crop model; builds a reinforcement learning agent for crop nitrogen fertilizer decision-making, and constructs the state space and reward function of reinforcement learning decision-making; trains the agent through the deep reinforcement learning algorithm SAC, introduces entropy regularization and softening strategy update; based on the crop canopy photos taken by the mobile phone, the deep learning model is used to calculate the growth period, leaf area index, aboveground biomass and leaf nitrogen content of the crop; the ensemble Kalman filter algorithm EnKF is used to assimilate the crop growth state observed by the mobile phone image into the gym-ORYZA environment, and the ORYZA-EnKF data assimilation system is constructed; the updated crop state is input into the trained deep reinforcement learning model SAC to obtain the crop nitrogen fertilizer decision result. Thus, the problem of nitrogen fertilizer decision errors caused by the deviation between the environment and the application environment is avoided, and it has the advantages of being fast, lossless and highly accurate, and real-time farmland crop nitrogen fertilizer decisions can be made.

[0130] According to the nitrogen fertilizer decision method for crops proposed in the embodiment of the present application, by obtaining the basic information of the crop to be observed, the precipitation on the decision day and the canopy image, and constructing an environmental body model based on the basic information, the crop growth simulation state of the crop to be observed is obtained by using the environmental body model, and the canopy image is input into the preset estimation model to obtain the basic growth state of the crop to be observed; based on the preset assimilation model, the crop growth simulation state and the basic growth state are integrated to obtain the updated crop growth state, and based on the preset intelligent body model, the fertilization decision time and fertilization decision value are generated according to the updated crop growth state and the precipitation on the decision day. Thus, the problem that the deviation between crop simulation and actual growth is not considered in the related technology, which is easy to lead to decision-making errors, fails to comprehensively consider the planting benefits and environmental benefits, and the efficiency of state variable acquisition is low, data assimilation can be used to reduce decision errors, and the crop growth state is estimated by using images to efficiently obtain state variables, and crop yield, economic benefits and environmental benefits are comprehensively considered.

[0131] Next, the nitrogen fertilizer decision-making device for crops proposed in accordance with the embodiment of the present application will be described with reference to the accompanying drawings.

[0132] Figure 8 It is a block diagram of a nitrogen fertilizer decision-making device for crops according to an embodiment of the present application.

[0133] like Figure 8 As shown, the crop nitrogen fertilizer decision-making device 10 includes: an acquisition module 100, a simulation module 200 and a decision module 300.

[0134] Among them, the acquisition module is used to obtain the basic information of the crop to be observed, the precipitation on the decision day and the canopy image;

[0135] A simulation module is used to construct an environmental model based on basic information, and use the environmental model to obtain a simulated growth state of the crop to be observed, and input the canopy image into a preset estimation model to obtain a basic growth state of the crop to be observed;

[0136] The decision-making module is used to obtain an updated crop growth state by integrating the crop growth simulation state and the basic growth state based on a preset assimilation model, and to generate a fertilization decision time and fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on a preset intelligent agent model.

[0137] Optionally, in some embodiments, after generating the fertilization decision time and fertilization decision value according to the updated crop growth status and the precipitation on the decision day based on the preset agent model, the decision module includes:

[0138] A reward unit, which is used to generate agent rewards based on the environment model and the fertilization decision time and fertilization decision value;

[0139] The updating unit is used to update the fertilization decision time and fertilization decision value according to the agent reward based on the preset agent model.

[0140] Optionally, in some embodiments, the agent reward is:

[0141] r t =w1×2.72×Y harvest -w2×5.53×N t -w3×150×Labor t -w4×P t ;

[0142] Among them, r t is the reward at time t, Y harvest is the output, N t is the nitrogen application amount at time t, Labort t is the labor cost at time t, P t is the harmful cost of nitrogen fertilizer loss to the environment, and w1, w2, w3 and w4 are all reward coefficients.

[0143] Optionally, in some embodiments, before generating the fertilization decision time and the fertilization decision value according to the updated crop growth status and the precipitation on the decision day based on the preset intelligent agent model, the method includes:

[0144] The preset agent model is trained by the SAC value function and the SAC state function, where the SAC value function is:

[0145]

[0146]

[0147] The SAC state function is:

[0148]

[0149] Among them, Q(s t ,a t ) is the state s at time t t Take action a t The state-action value function, r(s t ,a t ) is in state s t Take action a t The reward is , γ is the discount factor, For the next state s t+1 The expectation of the state function, p is the transition probability, π is the strategy, logπ(a t |s t ) is the strategy π in state s t Next select action at The logarithm of the probability, π new is a new strategy, π′ is a new strategy to be optimized, D KL is the KL divergence, is the normalization constant, For the old strategy π old Next, from state s t The starting value function, For strategy π to all possible actions a t The expectation of the state-action value function.

[0150] Optionally, in some embodiments, the preset assimilation model is:

[0151]

[0152] in, is the updated state matrix, is the state matrix of the model running to time t, K t is the Kalman gain at time t, d obs,t is the perturbed observation value, H t is the operator that converts the state matrix into the observation matrix at time t.

[0153] Optionally, in some embodiments, the crop growth status includes: crop growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield.

[0154] It should be noted that the above explanation of the embodiment of the nitrogen fertilizer decision-making method for crops is also applicable to the nitrogen fertilizer decision-making device for crops in this embodiment, and will not be repeated here.

[0155] According to the nitrogen fertilizer decision-making device for crops proposed in the embodiment of the present application, by obtaining the basic information of the crop to be observed, the precipitation on the decision day and the canopy image, and constructing an environmental body model based on the basic information, the crop growth simulation state of the crop to be observed is obtained by using the environmental body model, and the canopy image is input into the preset estimation model to obtain the basic growth state of the crop to be observed; based on the preset assimilation model, the crop growth simulation state and the basic growth state are integrated to obtain the updated crop growth state, and based on the preset intelligent body model, the fertilization decision time and fertilization decision value are generated according to the updated crop growth state and the precipitation on the decision day. Thus, the problem that the deviation between crop simulation and actual growth is not considered in the related technology, which is easy to lead to decision-making errors, fails to comprehensively consider the planting benefits and environmental benefits, and the efficiency of state variable acquisition is low is solved, and data assimilation can be used to reduce decision errors, and the crop growth state is estimated by using images to efficiently obtain state variables, and crop yield, economic benefits and environmental benefits are comprehensively considered.

[0156] Fig. 9A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0157] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0158] When the processor 902 executes the program, the nitrogen fertilizer decision-making method for crops provided in the above embodiment is implemented.

[0159] Furthermore, the electronic device further comprises:

[0160] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0161] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0162] The memory 901 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0163] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0164] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0165] The processor 902 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0166] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned nitrogen fertilizer decision-making method for crops.

[0167] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0168] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0169] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0170] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0171] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0172] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for nitrogen fertilizer decision-making for crops, characterized in that: The following steps are involved: Obtain basic information of the crops to be observed, decision-making day precipitation and canopy images; An environment model is constructed based on the basic information, and the environment model is used to obtain a simulated growth state of the crop to be observed, and the canopy image is input into a preset estimation model to obtain a basic growth state of the crop to be observed; Based on a preset assimilation model, the crop growth simulation state and the basic growth state are integrated to obtain an updated crop growth state, and based on a preset agent model, A fertilization decision time and a fertilization decision value are generated according to the updated crop growth status and the precipitation on the decision day.

2. The method according to claim 1, characterized in that After generating the fertilization decision time and the fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on the preset intelligent agent model, the method includes: Based on the environment model, generating an agent reward according to the fertilization decision time and the fertilization decision value; Based on the preset agent model, the fertilization decision time and the fertilization decision value are updated according to the agent reward.

3. The method according to claim 2, characterized in that The agent reward is: r t =w1×2.72×Y harvest -w2×5.53×N t -w3×150×Labor t -w4×P t ; Among them, r t is the reward at time t, Y harvest is the output, N t is the nitrogen application amount at time t, Labor t is the labor cost at time t, P t is the harmful cost of nitrogen fertilizer loss to the environment, and w1, w2, w3 and w4 are all reward coefficients.

4. The method according to claim 1, characterized in that Before generating the fertilization decision time and the fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on the preset intelligent agent model, it includes: The preset agent model is trained by the SAC value function and the SAC state function, wherein the SAC value function is: The SAC state function is: Among them, Q(s t ,a t ) is the state s at time t t Take action a t The state-action value function, r(s t ,a t ) is in state s t Take action a t The reward is , γ is the discount factor, For the next state s t+1 The expectation of the state function, p is the transition probability, π is the strategy, logπ(a t |s t ) is the strategy π in state s t Next select action a t The logarithm of the probability, π new is a new strategy, π′ is a new strategy to be optimized, D KL is the KL divergence, is the normalization constant, For the old strategy π old Next, from state s t The starting value function, For strategy π to all possible actions a t The expectation of the state-action value function.

5. The method according to claim 1, characterized in that: The preset assimilation model is: in, is the updated state matrix, is the state matrix of the model running to time t, K t is the Kalman gain at time t, d obs,t is the perturbed observation value, H t is the operator that converts the state matrix into the observation matrix at time t.

6. The method according to claim 1, characterized in that The crop growth status includes: crop growth period, leaf area index, aboveground biomass, leaf nitrogen content and yield.

7. A nitrogen fertilizer decision-making device for crops, characterized in that: include: The acquisition module is used to obtain basic information of the crop to be observed, the precipitation on the decision day, and the canopy image; A simulation module, used to construct an environment model based on the basic information, and use the environment model to obtain a simulated growth state of the crop to be observed, and input the canopy image into a preset estimation model to obtain a basic growth state of the crop to be observed; The decision-making module is used to obtain an updated crop growth state by fusing the crop growth simulation state and the basic growth state based on a preset assimilation model, and to generate a fertilization decision time and a fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on a preset intelligent agent model.

8. The device according to claim 7, characterized in that After generating the fertilization decision time and the fertilization decision value according to the updated crop growth state and the precipitation on the decision day based on the preset agent model, the decision module includes: A reward unit, used to generate an intelligent agent reward based on the environment body model and according to the fertilization decision time and the fertilization decision value; An updating unit is used to update the fertilization decision time and the fertilization decision value based on the preset agent model and the agent reward.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the nitrogen fertilizer decision-making method for crops as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the nitrogen fertilizer decision-making method for crops as described in any one of claims 1 to 6.

Images