An intelligent management system based on large-scale planting of crops

Abstract

The application discloses an intelligent management system based on large-scale planting of crops, and relates to the field of large-scale planting of crops, and comprises a construction unit of a historical state space of an RL model; a historical action space construction unit used for acquiring planting parameters of a historical time node t and constructing a historical action space of the RL model; a reward function construction unit used for acquiring a reward item of a delayed time node t+k and constructing a reward function of the RL model according to the reward item; a model updating unit used for updating the RL model according to delayed reward feedback; and an optimal planting strategy output unit used for inputting environmental parameters of a current time node into the updated RL model and obtaining an optimal planting strategy of the current time node. Through the combination of the reinforcement learning model, historical planting data and environmental data are acquired, a state-action-reward cycle is established, and systematic decision-making is carried out based on feedback of the historical time node t and the delayed time node t+k, so that an optimized strategy for the future is finally generated.

Description

Technical Field

[0001] This invention relates to the field of large-scale crop cultivation, specifically to an intelligent management system based on large-scale crop cultivation. Background Technology

[0002] With the development of modern agriculture, large-scale crop cultivation has become an important means to improve agricultural production efficiency. However, traditional large-scale cultivation management methods rely on human experience and manual operation, which have significant shortcomings in resource utilization and decision-making accuracy.

[0003] Chinese patent document CN118297365A discloses a smart agricultural management system and method based on ecological agriculture. This patent acquires historical data of the land to be managed, analyzes its soil composition, calculates soil erosion and land degradation, and then selects suitable plant varieties to achieve intelligent land management. While this method utilizes historical data to optimize planting strategies, it relies on the analysis of independent historical and real-time data, failing to incorporate the delayed feedback mechanism between the two. Therefore, it cannot comprehensively predict and optimize future planting strategies based on the delayed effect of data.

[0004] In summary, while existing technologies utilize historical data, they lack the ability to systematically combine historical and real-time data and fail to leverage delayed feedback mechanisms to optimize future planting strategies. Therefore, a new technology capable of intelligent optimization based on historical data feedback is needed to address these issues. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an intelligent management system based on large-scale crop cultivation. By combining a reinforcement learning model, it acquires historical planting data and environmental data, establishes a state-action-reward loop, and makes systematic decisions based on feedback from historical time nodes t and delayed time nodes t+k, ultimately generating optimization strategies for the future. This solves the problem of lacking dynamic delayed feedback and long-term optimization in existing technologies.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A smart management system for large-scale crop cultivation, the management system comprising:

[0008] The historical state space construction unit is used to obtain the environmental parameters of historical time node t and construct the historical state space of the RL model.

[0009] The historical action space construction unit is used to obtain the planting parameters at the historical time node t and construct the historical action space of the RL model.

[0010] The reward function construction unit is used to obtain the reward items at historical time node t+k and construct the reward function of the RL model based on the reward items; the reward function is used to characterize the delayed reward feedback of the action at historical time node t to historical time node t+k.

[0011] A model update unit is used to update the RL model based on the delayed reward feedback;

[0012] The optimal planting strategy output unit is used to input the environmental parameters of the current time node into the updated RL model to obtain the optimal planting strategy for the current time node.

[0013] In some embodiments, the construction steps of the historical state space construction unit include:

[0014] S1-1: At each historical time point t, obtain several environmental parameters of crops in the planting environment;

[0015] S1-2, several environmental parameters are used as state features to construct the instantaneous environmental state at historical time node t;

[0016] The expression for constructing the instantaneous environment state is: s t ={H t N t W t G t};

[0017] Among them, s t H represents the current environmental state at historical time point t. t N represents the soil moisture at historical time point t. t W represents the soil nutrients at historical time point t. t G represents the climate conditions at historical time point t. t This represents the crop growth status at historical time point t;

[0018] S1-3, in the time series, obtain the instantaneous environmental state of n historical time nodes and construct it as the historical state space in the RL model;

[0019] The expression for constructing the historical state space is:

[0020] Where S represents the historical state space, t1, t2, ... t n It is a historical time series;

[0021] In some embodiments, the construction steps of the historical action space construction unit include:

[0022] S2-1: At each historical time point t, obtain several planting parameters of crops in the planting environment;

[0023] S2-2, several planting parameters are used as planting features to construct the actions of historical time node t;

[0024] The expression for constructing the action is: a t ={I t F t R t};

[0025] Among them, a t I represents the action at historical time point t. t F represents the irrigation amount at historical time point t. t R represents the amount of fertilizer applied at historical time point t. t This indicates the water-fertilizer ratio at historical time point t;

[0026] S2-3, in the time series, obtain the actions of n historical time nodes and construct them as the historical action space in the RL model;

[0027] The expression for constructing the historical action space is:

[0028] Where A represents the historical action space, t1, t2, ... t n It is a time series;

[0029] In some embodiments, the construction steps of the reward function construction unit include:

[0030] S3-1, at the historical time node t+k, obtain the yield per unit area, water and fertilizer consumption, and pest and disease control costs of the planting environment.

[0031] S3-2 uses the unit area yield, water and fertilizer consumption, and pest and disease control costs at the historical time node t+k as reward items to construct the reward function in the RL model.

[0032] The expression for constructing the reward function is: R t =α·Y t+k -β·E t+k -γ·C t+k ;

[0033] Among them, R t It is the reward feedback at historical time point t, Y t+k E represents the yield per unit area at historical time point t+k. t+k C represents the water and fertilizer consumption at historical time point t+k. t+kThis represents the cost of pest and disease control at historical time point t+k; α, β, and γ are the weight coefficients of different reward items, respectively; k represents the time step of delay.

[0034] In some embodiments, the update step of the model update unit includes:

[0035] S4-1, Obtain the corresponding reward item for each historical time node; Substitute the corresponding reward item into the reward function to determine the delayed reward feedback for each historical time node;

[0036] S4-2, based on the delayed reward feedback at each historical time point, update the Q value until the Q value is updated to the convergence condition.

[0037] In some embodiments, a corresponding reward item is obtained based on each historical time node; the corresponding reward item is substituted into a reward function to determine the delayed reward feedback for each historical time node, including:

[0038] S4-1-1, retrieve each historical time point;

[0039] S4-1-2: Obtain corresponding reward items based on each historical time node;

[0040] S4-1-3, Substitute the corresponding reward item into the reward function to determine the delayed reward feedback for each historical time node.

[0041] In some embodiments, the Q-value is updated based on the delayed reward feedback at each historical time point until the Q-value is updated to the convergence condition, including:

[0042] S4-2-1, Quantify the error in the Q-value update using a loss function;

[0043] The expression for the loss function is: L(θ)=[R t+k +γmax a Q(s t+k , a; θ - )-Q(s t a t ;θ)] 2

[0044] Where L(θ) represents the Q-value delay error calculated by the loss function, and θ represents the adjustable weight parameters within the model. - R represents a fixed weight parameter. t+k This represents the reward feedback based on the yield per unit area, water and fertilizer consumption, and pest and disease control costs obtained at historical time node t+k, and characterizes the crop state at historical time node t+k in response to action a at historical time node t. tThe delayed reward feedback; γ represents the discount factor, used to weigh the reward feedback at historical time point t+k, Q(s t a t ) represents a historical time node t, in state s t Take action a t Q value, max a Q(s t+k , a;) represents the historical time node t+k, in state s t+k The Q-value of the optimal action taken in the past represents the historical delayed reward brought about by the historical optimal action;

[0045] S4-2-2, Based on the error calculated by the loss function, the Q-value is updated using gradient descent;

[0046] The expression for updating the Q value is:

[0047] Q(s t a t )←Q(s t a t )+α[R t+k +γmax a Q(s t+k ,a)-Q(s t a t )];

[0048] Where α is the learning rate, which represents the speed at which the Q-value is updated;

[0049] S4-2-3, When the change in Q value between two updates is less than a preset threshold, the Q value update reaches the convergence condition and the Q value update process stops;

[0050] The convergence condition is expressed as: |Q(s) t a t )-Q′(s t a t )|<∈;

[0051] Where Q(s) t a t ) represents a historical time node t, in state s t Take action a t The Q value, Q′(s) t a t ) represents the Q value updated last time, and ∈ is the preset threshold.

[0052] S4-2-4, During the Q-value update process, the optimal planting strategy at each historical time node is updated;

[0053] The update expression for the optimal planting strategy is: π(s t) = argmax a Q(s t a t );

[0054] Wherein, π(s) t ) represents the state s t Choose the action a that maximizes the Q value. t .

[0055] S4-2-5, Based on the mapping rules of the Q value, historical time nodes and corresponding optimal planting strategies, establish a planting strategy table.

[0056] In some embodiments, the output step of the optimal planting strategy output unit includes:

[0057] S5-1. Obtain several environmental parameters at the current time point;

[0058] S5-2, using several environmental parameters at the current time point as indexes, query the optimal planting strategy in the planting strategy table;

[0059] S5-3, Output and execute the optimal planting strategy.

[0060] This invention provides an intelligent management system based on large-scale crop cultivation, which has the following beneficial effects:

[0061] By combining historical data with reinforcement learning models, this invention effectively solves the problems of uneven resource utilization and neglect of long-term effects in existing technologies, ultimately achieving an optimal planting strategy based on a delayed feedback mechanism. This intelligent management system not only increases yield per unit area and optimizes resource use and pest and disease control costs, but also dynamically adjusts strategies according to the crop growth cycle, significantly improving the level of intelligent management in agricultural production. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating the management steps of an intelligent management system based on large-scale crop cultivation, as described in this invention.

[0063] Figure 2 This is a structural block diagram of an intelligent management system based on large-scale crop cultivation according to the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] First, the prior art and related concepts involved in the embodiments of the present invention will be described:

[0066] RL (Reinforcement Learning) models are a type of machine learning model that focuses on learning action policies through interaction with the environment to maximize long-term rewards. Unlike supervised and unsupervised learning, in reinforcement learning, the model learns autonomously through trial and error to find a policy that achieves optimal results in a specific context.

[0067] Q-value: In Q-learning algorithms, Q-value represents the value of a state-action pair; it indicates the total reward an agent can expect to obtain after performing a certain action in a specific state. The Q-value is repeatedly updated to obtain an optimal value, allowing the agent to choose the optimal action in different states. In this embodiment, the intelligent machine is a system for executing the optimal planting strategy or obtaining the aforementioned parameters.

[0068] Time step: In continuous-time simulation, time is divided into small intervals or periods, each interval representing a computational step in the simulation.

[0069] refer to Figures 1 to 2 This invention provides an intelligent management system based on large-scale crop cultivation.

[0070] Example 1: This invention provides an intelligent management system based on large-scale crop cultivation, comprising the following steps:

[0071] in, Figure 1 This is a flowchart illustrating the management steps of an intelligent management system based on large-scale crop cultivation, according to the present invention. The process includes:

[0072] Step S1: Obtain the environmental parameters of the historical time node t and construct the historical state space of the RL model.

[0073] Specifically, step S1 is:

[0074] S1-1: At each historical time point t, obtain several environmental parameters of crops in the planting environment;

[0075] S1-2, several environmental parameters are used as state features to construct the instantaneous environmental state at historical time node t;

[0076] The expression for constructing the immediate environment state is: s t ={H t N t W t G t};

[0077] Among them, s t H represents the current environmental state at historical time point t. t N represents the soil moisture at historical time point t. t W represents the soil nutrients at historical time point t. t G represents the climate conditions at historical time point t. t This represents the crop growth status at historical time point t;

[0078] Specifically, the immediate environmental state s at this historical time point t t It is a composite state plant constructed from several environmental parameters at the historical time node t, and this composite state plant can describe the composite state of the planting environment at the historical time node t.

[0079] For example, soil moisture can be a numerical value representing the water content in the soil, which can be expressed as a percentage or volume ratio. Soil nutrients can also be numerical values ​​representing the content of nutrients such as nitrogen, phosphorus, and potassium in the soil, usually expressed as concentration or proportion. Climatic conditions can include several sub-parameters such as temperature, humidity, rainfall, and sunlight exposure in the current planting environment, each of which can be a numerical value to collectively describe the climatic conditions. Crop growth status can be constructed as a comprehensive index, which may include crop height, number of leaves (averaged across several plants), number of fruits, etc. This comprehensive index can be a continuous numerical value or a categorical index (such as growth stage, early stage, middle stage, and maturity stage). Therefore, when constructing the real-time environmental state, the above parameters are combined to form a parameter set or vector representation describing the environmental state. Even if some parameters are not suitable for numerical representation, they can be encoded and then converted into numerical values ​​for calculation and model training.

[0080] S1-3, in the time series, obtain the instantaneous environmental state of n historical time nodes and construct it as the historical state space in the RL model;

[0081] The expression for constructing the historical state space is:

[0082] Where S represents the historical state space, t1, t2, ... t n It is a historical time series.

[0083] Step S2: Obtain the planting parameters at the historical time node t and construct the historical action space of the RL model.

[0084] Specifically, step S2 is as follows:

[0085] S2-1: At each historical time point t, obtain several planting parameters of crops in the planting environment;

[0086] Planting parameters are mathematical expressions of the planting operations used in the large-scale planting of crops. Specifically, they can be numerical parameters such as fertilizer application, irrigation, and water-fertilizer ratio.

[0087] S2-2, several planting parameters are used as planting features to construct the actions of historical time node t;

[0088] The expression for constructing an action is: a t ={I t F t R t};

[0089] Among them, a t I represents the action at historical time point t. t F represents the irrigation amount at historical time point t. t R represents the amount of fertilizer applied at historical time point t. t This indicates the water-fertilizer ratio at historical time point t;

[0090] S2-3, in the time series, obtain the actions of n historical time nodes and construct them as the historical action space in the RL model;

[0091] The expression for constructing the historical action space is:

[0092] Where A represents the historical action space, t1, t2, ... t n It is a time series;

[0093] Specifically, the actions in this embodiment are specific operational parameters based on historical planting data at historical time points, including irrigation volume, fertilization volume, and water-fertilizer ratio. For example, at a certain historical time point, the historical planting data records the corresponding historical irrigation volume, historical fertilization volume, and historical water-fertilizer ratio, which are then used to construct actions based on that historical time point. Then, through time series analysis, actions are collected in the sequence, forming a historical action space based on the time series. Therefore, this historical action space actually reflects the agricultural operational parameters at specific time points. Agricultural operational parameters based on the time series allow the model to fully utilize historical experience and learn the long-term effects of different operational strategies on crops within the time series.

[0094] Step S3: Obtain the reward items for historical time node t+k, and construct the reward function of the RL model based on the reward items.

[0095] The reward function is used to characterize the delayed reward feedback of the action at historical time node t to historical time node t+k.

[0096] Specifically, step S3 is as follows:

[0097] S3-1, at the historical time node t+k, obtain the yield per unit area, water and fertilizer consumption, and pest and disease control costs of the planting environment.

[0098] S3-2 uses the unit area yield, water and fertilizer consumption, and pest and disease control costs at the historical time node t+k as reward items to construct the reward function in the RL model.

[0099] The expression for constructing the reward function is: R t =α·Y t+k -β·E t+k -γ·C t+k ;

[0100] Among them, R t It is the reward feedback at historical time point t, Y t+k E represents the yield per unit area at historical time point t+k. t+k C represents the water and fertilizer consumption at historical time point t+k. t+k This represents the cost of pest and disease control at historical time point t+k; α, β, and γ are the weight coefficients of different reward items, respectively; k represents the time step of delay.

[0101] Specifically, in large-scale crop cultivation scenarios, the effects of water and fertilizer management at the current time are usually not immediately reflected in the crop's growth status. Changes in yield per unit area, water and fertilizer consumption, and pest and disease control costs may only be observed after several time steps. Therefore, in the RL model of this embodiment, the reward function is defined as a function that provides delayed feedback on actions at historical time points t, allowing the RL model to consider the long-term impact of water and fertilizer management.

[0102] Furthermore, the reward function is based on historical reward feedback, which allows the RL model to learn from existing data. Compared to predictions based on future time points, such as training a general model based on historical water and fertilizer data to obtain a linear regression prediction model for predicting the future, reward feedback based on historical data is more deterministic and reduces the risks of uncertainty and prediction errors introduced in pre-trained models.

[0103] Step S4: Update the RL model based on the delayed reward feedback.

[0104] Specifically, step S4 is as follows:

[0105] S4-1: Obtain the corresponding reward item for each historical time node; substitute the corresponding reward item into the reward function to determine the delayed reward feedback for each historical time node.

[0106] Furthermore, step S4-1 also includes:

[0107] S4-1-1, retrieve each historical time point;

[0108] S4-1-2: Obtain corresponding reward items based on each historical time node;

[0109] S4-1-3, substitute the corresponding reward items into the reward function to determine the delayed reward feedback for each historical time node.

[0110] S4-2, based on the delayed reward feedback at each historical time point, update the Q value until the Q value is updated to the convergence condition.

[0111] Furthermore, step S4-2 also includes:

[0112] S4-2-1, quantifying the error in Q-value updates using a loss function;

[0113] The expression for the loss function is: L(θ) = [R] t+k +γmax a Q(s t+k , a; θ - )-Q(s t a t ;θ)] 2

[0114] Where L(θ) represents the Q-value delay error calculated by the loss function, used to evaluate the action a at historical time point t. t The estimation error of the delay effect (i.e., historical operations); θ represents the adjustable weight parameter within the model, θ - R represents a fixed weight parameter (e.g., the weight coefficients of different reward items in a reward function). t+k This represents the reward feedback based on the yield per unit area, water and fertilizer consumption, and pest and disease control costs obtained at historical time node t+k, and characterizes the crop state at historical time node t+k in relation to action a at historical time node t. t The delayed reward feedback; γ represents the discount factor, used to weigh the reward feedback at historical time point t+k, Q(s t a t ) represents a historical time node t, in state s t Take action a t The Q-value (i.e., expected reward), max a Q(s t+k, a;) represents the historical time node t+k, in state s t+k The Q-value of the optimal action taken represents the historical delayed reward brought about by the historical optimal action;

[0115] In this embodiment, historical time node t refers to a specific point in time within historical data. At this point, the model acquires the environmental conditions (such as soil moisture, climate conditions, crop growth status, etc.) and planting operations (such as irrigation amount, fertilizer amount, water-fertilizer ratio, etc.) at that time node, and considers these operations as the "actions" at that time node.

[0116] The historical time point t+k refers to a historical time point after time point t. This time point is used to assess the delayed impact of planting operations at time point t, and the data obtained includes yield per unit area, water and fertilizer consumption, and pest and disease control costs. This data reflects the subsequent impact of previous operations at historical time point t on crop growth.

[0117] Specifically, in RL models, the concept of time typically includes the current time point t and a future time point t+k. The current time point t represents the environmental state observed by the model at that time. Based on the current state, the model makes a decision and selects action a. t The future time point t+k refers to the point in time after which the model will adjust its state based on the current state s. t+k This is used to evaluate whether previous actions were effective. In reinforcement learning, this is used to evaluate the results of previous decisions and is part of the future reward. In other words, the concept of "future time" in the RL model is actually the concept of "future in history" in this invention. Therefore, this embodiment essentially evaluates delayed rewards based on historical time nodes t and t+k, rather than predicting the future.

[0118] Therefore, in the embodiments of this invention, historical data and the RL model are combined, using the yield per unit area and water and fertilizer consumption at historical time node t+k as delayed reward feedback for the action at historical time node t, which is then used to calculate the reward function. In this way, although the data is historical, the logic of reinforcement learning can optimize decision-making through delayed evaluation.

[0119] In this way, historical data and reinforcement learning are naturally combined. The system can simulate the state-action-reward cycle of reinforcement learning based on historical data, thereby optimizing future planting strategies. This method not only finds the historical optimal balance in water and fertilizer management but also takes into account the impact of crop growth delays, effectively improving the level of intelligent management in agricultural production.

[0120] S4-2-2, based on the error calculated from the loss function, the Q-value is updated using gradient descent;

[0121] The expression for updating the Q value is:

[0122] Q(s t a t )←Q(s t a t )+α[R t+k +γmax a Q(s t+k ,a)-Q(s t a t )];

[0123] Where α is the learning rate, which represents the speed at which the Q-value is updated;

[0124] S4-2-3, When the change in Q value between two updates is less than a preset threshold, the Q value update reaches the convergence condition and the Q value update process stops;

[0125] The convergence condition is expressed as: |Q(s) t a t )-Q′(s t a t )|<∈;

[0126] Where Q(s) t a t ) represents a historical time node t, in state s t Take action a t The Q value, Q′(s) t a t ) represents the Q value updated last time, and ∈ is the preset threshold.

[0127] ∈ is a very small positive number used to characterize the magnitude of change in the Q value. When the magnitude of change is less than this preset threshold, it is considered that the update of the Q value has reached the convergence condition, indicating that the RL model has found the optimal decision strategy in the historical data.

[0128] S4-2-4 updates the optimal planting strategy at each historical time point during the Q-value update process;

[0129] The update expression for the optimal planting strategy is: π(s) t ) = argmax a Q(s t a t );

[0130] Wherein, π(s) t ) represents the state s t Choose the action a that maximizes the Q value. t .

[0131] It is important to note that the updated optimal planting strategy is a management strategy built using reinforcement learning methods, tailored to the specific planting environment. This strategy is trained on historical data and can guide planting decisions under different conditions, maximizing yield per unit area and optimizing water and fertilizer use and pest and disease control costs.

[0132] S4-2-5, Based on the mapping rules of Q value, historical time nodes and corresponding optimal planting strategies, establish a planting strategy table.

[0133] Example 2, Reference Figures 1 to 2 The technical solution of this embodiment 2 differs from that of embodiment 1 in that:

[0134] Step S5: Input the environmental parameters of the current time node into the updated RL model to obtain the optimal planting strategy for the current time node.

[0135] Specifically, step S5 is as follows:

[0136] S5-1. Obtain several environmental parameters at the current time point;

[0137] S5-2, using several environmental parameters at the current time point as indexes, query the optimal planting strategy in the planting strategy table;

[0138] S5-3 outputs the optimal planting strategy and executes it.

[0139] In other words, after the model update is completed, a planting strategy table based on historical experience is constructed according to the mapping relationship between the updated Q value (which already contains the corresponding historical time nodes) and the optimal planting strategy. This planting strategy table contains the aforementioned environmental parameters, planting parameters, and several states and actions constructed based on the above parameters.

[0140] Therefore, once the environmental parameters for the current time point are obtained, the optimized RL model can execute the optimal planting strategy according to the planting strategy table.

[0141] Of course, the environmental parameters at the current time node in step S11 can be used as historical data for updating the RL model. During the use of this method, the passage of time will replace the current environmental parameters with historical environmental parameters. Through the continuous input of historical environmental parameters, the RL model can have the effect of iterative optimization and dynamic updating.

[0142] In the management steps of the aforementioned management system, large-scale crop cultivation is combined with reinforcement learning to output the optimal planting strategy. Specifically, the optimal planting strategy aims to obtain the best planting return reward at future points in time; it should be viewed as an optimization strategy for the future.

[0143] The specific reason is:

[0144] The core objective of reinforcement learning is to enable an agent to continuously interact with the environment and gradually learn the optimal decision-making strategy in order to maximize cumulative rewards in the long run. In the intelligent management system of this invention, the agent is the system that performs the above steps; the rewards include increased yield per unit area, optimized water and fertilizer consumption, and reduced pest and disease control costs, all of which are long-term goals in the planting process.

[0145] The essence of an optimal planting strategy is that, through training with historical data and updating the Q-value, the final output of the optimal planting strategy aims to make the best decisions under current environmental conditions. However, the effects of these decisions are usually not immediately apparent. For example, the impact of current irrigation and fertilization operations on crop growth and yield may not be seen until several time steps later. Therefore, the formulation of an optimal planting strategy is essentially an optimization of future returns.

[0146] From another perspective, the optimal planting strategy can be seen as an optimization for the future, because its purpose is not only to improve the current outcome, but more importantly, to maximize the planting returns at future time points by optimizing current decisions. The delayed reward mechanism in reinforcement learning means that current decisions often affect returns at multiple future time steps. By constructing a reward function that uses historical time points t+k (future time points) as delayed feedback, the model can adjust its strategy to ensure optimal future returns.

[0147] Furthermore, in reinforcement learning, the model not only focuses on immediate rewards but also emphasizes long-term cumulative rewards. In the intelligent management system of this invention, although the optimal planting strategy is generated based on current environmental parameters, its goal is to maximize returns at multiple future time points. Therefore, this optimal planting strategy can be viewed as a long-term optimization decision, which continuously adjusts planting operations at different time points to ultimately achieve the long-term goals of maximizing yield, minimizing resource consumption, and optimizing pest and disease control costs.

[0148] In summary, the optimal planting strategy ultimately output is indeed an optimization strategy for future returns. It not only makes decisions based on current environmental parameters, but more importantly, it incorporates the long-term effects of current decisions through the delayed reward mechanism in the reinforcement learning model, thereby achieving optimal planting returns in the future. This optimization strategy enables the system to learn from historical experience, continuously improve current decisions, and provide long-term optimized solutions for future returns.

[0149] This invention provides an intelligent management system for large-scale crop cultivation. This system is used to implement the above embodiments, and details already described will not be repeated. The terms "module," "unit," and "subunit," etc., used below refer to combinations of software and / or hardware that perform predetermined functions. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0150] like Figure 2 As shown, Figure 2 This is a structural block diagram of an intelligent management system based on large-scale crop cultivation according to the present invention. The system includes:

[0151] The historical state space construction unit is used to obtain the environmental parameters of the historical time node t and construct the historical state space of the RL model.

[0152] The historical action space construction unit is used to obtain the planting parameters at the historical time node t and construct the historical action space of the RL model.

[0153] The reward function construction unit is used to obtain the reward items at historical time node t+k and construct the reward function of the RL model based on the reward items; the reward function is used to characterize the delayed reward feedback of the action at historical time node t to historical time node t+k.

[0154] A model update unit is used to update the RL model based on the delayed reward feedback.

[0155] The optimal planting strategy output unit is used to input the environmental parameters of the current time node into the updated RL model to obtain the optimal planting strategy for the current time node.

[0156] This invention acquires environmental parameters (water and fertilizer use, soil nutrients, crop growth status, etc.) at historical time points and combines them with historical planting operation data (irrigation, fertilization, water and fertilizer ratio, etc.). The system can intelligently adjust resource input to avoid excessive or insufficient resource utilization, thereby achieving higher resource utilization rate.

[0157] By employing delayed feedback mechanisms in reinforcement learning models, the long-term impact of planting operations can be effectively captured. For example, the effects of water and fertilizer management are not immediately apparent. By delaying the feedback of yield per unit area and other costs at specific time steps, the model can learn the long-term patterns of crop growth and optimize the long-term returns of current decisions.

[0158] Through the Q-value update mechanism in the reinforcement learning model, the system can continuously optimize the planting strategy. The decision basis at each historical time point is updated based on actual feedback until the Q-value converges. This mechanism ensures the real-time nature and adaptability of the optimal planting strategy.

[0159] The optimal planting strategy output not only considers current environmental parameters but also focuses on maximizing long-term returns. For example, the impact of current irrigation and fertilization operations on crop growth may not be visible for several time steps. The system ensures that planting returns at future time points are optimal through a delayed reward mechanism.

[0160] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0161] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, units, and processes described above can be referred to the corresponding processes in the foregoing embodiments, and will not be repeated here.

[0162] In the embodiments provided in this application, it should be understood that the disclosed systems, methods, and approaches can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division of a waterway underwater topography change analysis system and method. In actual implementation, there may be other division methods. For example, several units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0163] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across several network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0165] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0166] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A smart management system based on large-scale crop cultivation, characterized in that: The management system includes: The historical state space construction unit is used to obtain the environmental parameters of historical time node t and construct the historical state space of the RL model. The historical action space construction unit is used to obtain the planting parameters at the historical time node t and construct the historical action space of the RL model. The reward function construction unit is used to obtain the reward items at historical time node t+k and construct the reward function of the RL model based on the reward items; the reward function is used to characterize the delayed reward feedback of the action at historical time node t to historical time node t+k. A model update unit is used to update the RL model based on the delayed reward feedback; The optimal planting strategy output unit is used to input the environmental parameters of the current time node into the updated RL model to obtain the optimal planting strategy for the current time node.

2. The intelligent management system based on large-scale crop planting according to claim 1, characterized in that: The construction steps of the historical state space building block include: S1-1: At each historical time point t, obtain several environmental parameters of crops in the planting environment; S1-2, several environmental parameters are used as state features to construct the instantaneous environmental state at historical time node t; The expression for constructing the instantaneous environment state is: s t ={H t N t W t G t }; Among them, s t H represents the current environmental state at historical time point t. t N represents the soil moisture at historical time point t. t W represents the soil nutrients at historical time point t. t G represents the climate conditions at historical time point t. t This represents the crop growth status at historical time point t; S1-3, in the time series, obtain the instantaneous environmental state of n historical time nodes and construct it as the historical state space in the RL model; The expression for constructing the historical state space is: Where S represents the historical state space, t1, t2, ... t n It is a historical time series.

3. The intelligent management system based on large-scale crop planting according to claim 1, characterized in that: The construction steps of the historical action space construction unit include: S2-1: At each historical time point t, obtain several planting parameters of crops in the planting environment; S2-2, several planting parameters are used as planting features to construct the actions of historical time node t; The expression for constructing the action is: a t ={I t F t R t }; Among them, a t I represents the action at historical time point t. t F represents the irrigation amount at historical time point t. t R represents the amount of fertilizer applied at historical time point t. t This indicates the water-fertilizer ratio at historical time point t; S2-3, in the time series, obtain the actions of n historical time nodes and construct them as the historical action space in the RL model; The expression for constructing the historical action space is: Where A represents the historical action space, t1, t2, ... t n It is a time series.

4. The intelligent management system based on large-scale crop planting according to claim 1, characterized in that: The construction steps of the reward function construction unit include: S3-1, at the historical time node t+k, obtain the yield per unit area, water and fertilizer consumption, and pest and disease control costs of the planting environment. S3-2 uses the unit area yield, water and fertilizer consumption, and pest and disease control costs at the historical time node t+k as reward items to construct the reward function in the RL model. The expression for constructing the reward function is: R t =α·Y t+k -β·E t+k -γ·C t+k ; Among them, R t It is the reward feedback at historical time point t, Y t+k E represents the yield per unit area at historical time point t+k. t+k C represents the water and fertilizer consumption at historical time point t+k. t+k This represents the cost of pest and disease control at historical time point t+k; α, β, and γ are the weight coefficients of different reward items, respectively; k represents the time step of delay.

5. The intelligent management system based on large-scale crop planting according to claim 1, characterized in that: The update steps for the model update unit include: S4-1, Obtain the corresponding reward item for each historical time node; Substitute the corresponding reward item into the reward function to determine the delayed reward feedback for each historical time node; S4-2, based on the delayed reward feedback at each historical time point, update the Q value until the Q value is updated to the convergence condition.

6. The intelligent management system based on large-scale crop planting according to claim 1, characterized in that: Obtain the corresponding reward item based on each historical time point; Substitute the corresponding reward items into the reward function to determine the delayed reward feedback for each historical time point, including: S4-1-1, retrieve each historical time point; S4-1-2: Obtain corresponding reward items based on each historical time node; S4-1-3, Substitute the corresponding reward item into the reward function to determine the delayed reward feedback for each historical time node.

7. The intelligent management system based on large-scale crop planting according to claim 1, characterized in that: Based on the delayed reward feedback at each historical time point, the Q-value is updated until it meets the convergence condition, including: S4-2-1, Quantify the error in the Q-value update using a loss function; The expression for the loss function is: L(θ)=[R t+k +γmax a Q(s t+k , a; θ - )-Q(s t a t ;θ)] 2 Where L(θ) represents the Q-value delay error calculated by the loss function, θ represents the adjustable weight parameters within the model, θ- represents the fixed weight parameters, and R... t+k This represents the reward feedback based on the yield per unit area, water and fertilizer consumption, and pest and disease control costs obtained at historical time node t+k, and characterizes the crop state at historical time node t+k in relation to action a at historical time node t. t The delayed reward feedback; γ represents the discount factor, used to weigh the reward feedback at historical time point t+k, Q(s t a t ) represents a historical time node t, in state s t Take action a t Q value, max a Q(s t+k , a;) represents the historical time node t+k, in state s t+k The Q-value of the optimal action taken represents the historical delayed reward brought about by the historical optimal action; S4-2-2, Based on the error calculated by the loss function, the Q-value is updated using gradient descent; The expression for updating the Q value is: Q(s t ,a t )←Q(s t ,a t )+α[R t+k +γmax a Q(s t+k ,a)-Q(s t ,a t )]; Where α is the learning rate, which represents the speed at which the Q-value is updated; S4-2-3, When the change in Q value between two updates is less than a preset threshold, the Q value update reaches the convergence condition and the Q value update process stops; The convergence condition is expressed as: |Q(s) t a t )-Q′(s t a t )|<∈; Where Q(s) t a t ) represents a historical time node t, in state s t Take action a t The Q value, Q′(s) t a t ) represents the Q value updated last time, and ∈ is the preset threshold. S4-2-4, During the Q-value update process, the optimal planting strategy at each historical time node is updated; The update expression for the optimal planting strategy is: π(s t ) = argmax a Q(s t a t ); Wherein, π(s) t ) represents the state s t Choose the action a that maximizes the Q value. t . S4-2-5, Based on the mapping rules of the Q value, historical time nodes and corresponding optimal planting strategies, establish a planting strategy table.

8. The intelligent management system based on large-scale crop planting according to claim 1, characterized in that: The output steps of the optimal planting strategy output unit include: S5-1. Obtain several environmental parameters at the current time point; S5-2, using several environmental parameters at the current time point as indexes, query the optimal planting strategy in the planting strategy table; S5-3, Output and execute the optimal planting strategy.

Images