Non-point source pollution control and ecological agriculture decision-making method based on reinforcement learning

Abstract

The invention discloses a non-point source pollution control and ecological agriculture decision-making method based on reinforcement learning, and relates to the technical field of new-generation information. The method comprises the steps that meteorological parameters, soil parameters and crop growth parameters are collected through a farmland Internet of Things system, and historical management data in the crop growth process and water quality monitoring information of surface runoff are collected to serve as original data; preprocessing the original data to construct a multi-modal data set; constructing an ecological agriculture multi-modal reinforcement learning model based on the multi-modal data set; integrating the ecological agriculture multi-modal reinforcement learning model to an agricultural intelligent decision-making platform, so that the ecological agriculture multi-modal reinforcement learning model is coupled with the agricultural intelligent decision-making platform; and an agricultural intelligent decision-making platform is utilized to issue operation instructions to irrigation, fertilization and pesticide application equipment so as to realize non-point source pollution management and control and ecological agriculture decision-making. Through a machine learning algorithm and real-time data analysis, many limitations of traditional ecological agriculture are overcome.

Description

Technical Field

[0001] The present application relates to the field of new generation information technology, and in particular to a method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning. Background Art

[0002] Traditional agricultural production models typically rely on resource-intensive fertilizer, pesticide, and water inputs. While this extensive resource utilization model may increase yields in the short term, it leads to a series of long-term ecological problems, including soil degradation, water pollution, biodiversity loss, and excessive chemical residues in agricultural products. These issues not only threaten the sustainable development of agriculture but also pose serious risks to human health and ecosystem stability. Against this backdrop, ecological agriculture has emerged as a new agricultural model. Unlike traditional agriculture, ecological agriculture emphasizes optimizing resource allocation while protecting the ecological environment to enhance the economic and environmental benefits of agricultural production. By scientifically balancing the input of nutrients such as fertilizers and trace minerals with biological control technologies, ecological agriculture inherits the ecological wisdom of agricultural civilization while breaking through the limitations of traditional agriculture. By leveraging technological innovation, scientific management methods, and effective practices from traditional agriculture, it achieves ecological, efficient, and sustainable agricultural production.

[0003] While ecological agriculture demonstrates significant advantages in sustainable development, its practical application still faces certain limitations. Traditional ecological agricultural management strategies often rely on farmers' experience and historical data, lacking the ability to respond quickly to real-time environmental changes. This limitation can lead to inefficient resource utilization, making precise regulation difficult, especially in the face of sudden outbreaks of pests and diseases or climate change. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a non-point source pollution control and ecological agriculture decision-making method based on reinforcement learning to optimize agricultural water and fertilizer management, reduce non-point source pollution, and improve crop quality.

[0005] To achieve the above objectives, one aspect of the embodiments of the present application proposes a method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning, the method comprising the following steps:

[0006] The farmland IoT system collects meteorological parameters, soil parameters, and crop growth parameters, as well as historical management data during crop growth and surface runoff water quality monitoring information as raw data;

[0007] Preprocessing the raw data to construct a multimodal dataset;

[0008] Constructing an eco-agricultural multimodal reinforcement learning model based on the multimodal dataset;

[0009] Integrating the eco-agriculture multimodal reinforcement learning model into the agricultural intelligent decision-making platform, so that the eco-agriculture multimodal reinforcement learning model is coupled with the agricultural intelligent decision-making platform;

[0010] The agricultural intelligent decision-making platform is used to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making.

[0011] In some embodiments, the method of collecting meteorological parameters, soil parameters, and crop growth parameters through the farmland IoT system and acquiring historical management data of the crop growth process and water quality monitoring information of surface runoff as raw data includes the following steps:

[0012] Collecting meteorological parameters, soil parameters, and crop growth parameters through a weather station, a multimodal sensor, and an imaging device that achieves a target resolution, and collecting historical management data during crop growth and water quality monitoring information of surface runoff as the raw data;

[0013] The farmland IoT system includes the weather station, a multimodal sensor, and an imaging device that achieves the target resolution.

[0014] Said meteorological parameters include air temperature, wind speed, light intensity and atmospheric humidity;

[0015] The soil parameters include soil volumetric moisture content, soil electrical conductivity, pH value and soil pesticide residue concentration;

[0016] The crop growth parameters include crop growth stage codes, crop surface temperature, crop height, crop fruit size, and crop phenotypic feature vectors extracted based on convolutional neural networks;

[0017] The historical management data includes irrigation water volume, fertilizer volume, and pesticide usage volume;

[0018] The surface runoff water quality monitoring information includes nitrogen, phosphorus, organic pollutants and heavy metal contents in the water.

[0019] In some embodiments, preprocessing the raw data to construct a multimodal dataset comprises the following steps:

[0020] Arrange the meteorological parameters, the soil parameters, the crop growth parameters, and the historical management data in chronological order to construct a time series data set;

[0021] Perform data cleaning on the data features in the time series dataset, identify and remove mutation points and abnormal sequences during the data collection process, and use data filling strategies to fill in missing values; wherein the data filling strategies include mean filling or nearest neighbor interpolation;

[0022] The numerical features in the time series dataset are Z-score standardized, and the categorical features in the time series dataset are one-hot encoded, thereby unifying the data format of the data features in the time series dataset to construct the multimodal dataset.

[0023] In some embodiments, constructing an eco-agriculture multimodal reinforcement learning model based on the multimodal dataset comprises the following steps:

[0024] Extracting data features based on the multimodal dataset includes the following steps:

[0025] For crop image data, convolutional neural networks are used to extract image features for crop and weed identification, pest and disease detection, and nutritional status monitoring;

[0026] Determine numerical data as feature vector input; wherein the numerical data includes the meteorological parameters and the soil parameters;

[0027] Based on the extracted multimodal features, a multidimensional state vector is constructed to define the state space at each time point in the agricultural environment and obtain time series data.

[0028] Processing the time series data using a long short-term memory network to construct a time series state vector;

[0029] Designing a decision action space, which includes irrigation water quantity regulation, fertilizer quantity regulation, and pesticide usage quantity regulation, and dynamically limiting the regulation range according to environmental conditions and corresponding standard limits;

[0030] A reward function is constructed based on crop growth indicators, resource utilization efficiency, and environmental protection requirements. Multi-objective optimization is achieved through dynamic weight allocation, and a real-time feedback mechanism is used to guide the ecological agriculture multimodal reinforcement learning model to learn the optimal strategy. When the management measures taken enable crop growth to reach a preset state and resource utilization efficiency and environmental impact are both at preset levels, positive rewards are given through the reward function. When crop growth is hindered, resources are wasted, or environmental pollution occurs, negative rewards are given through the reward function.

[0031] The deep deterministic policy gradient algorithm is selected to train the ecological agriculture multimodal reinforcement learning model, including the following steps:

[0032] Based on the historical agricultural operation data set, supervised pre-training is performed. The input of the Actor network is set as the agricultural environment state, and the output is the corresponding agricultural management decision. The mean square error loss function is used for training. After the training is completed, the obtained agricultural management strategy is used as the initial strategy for reinforcement learning. The MSE loss function expression is as follows:

[0033]

[0034] in, is the output of the Actor network, a t is the real historical operation value;

[0035] Unsupervised reinforcement learning based on the initial strategy obtained from supervised pre-training includes the following steps:

[0036] Define the Actor network and the Critic network. Initialize the Actor network using a supervised pre-trained Actor network and set the scope of agricultural management actions based on practical constraints. In the Critic network, perform feature fusion of the agricultural environment state vector and the management decision action under the corresponding state as input, and calculate the output value of the management action based on the environment state vector and the management action.

[0037] An experience replay pool is defined to store experience data during the interaction process, including the current state of the agricultural environment, the agricultural management measures taken, the reward value calculated according to the reward function, the next state of the agricultural environment after the agricultural management measures are taken, and the end state identifier;

[0038] Random sampling from the experience replay pool is performed to update the parameters of the ecological agriculture multimodal reinforcement learning model, including the following steps:

[0039] The target Q value is calculated using the target network, and the mean square error loss function is constructed to update the critic network parameters using the gradient descent method;

[0040] By using the Q-value gradient provided by the Critic network, the policy gradient method is used to optimize the Actor network parameters, so that the agricultural management actions output by the Actor network can obtain a higher Q-value under the evaluation of the Critic network;

[0041] Regularly use soft updates to update the parameters of the target network to improve the stability and convergence of the model. The soft update expression is as follows:

[0042] θ′=τθ+(1-τ)θ′;

[0043] Among them, θ ′ are the parameters of the target network, θ are the parameters of the current network, and τ are the soft update coefficients;

[0044] Bayesian optimization is used to perform integrated optimization of model hyperparameters and decision action parameters, including the following steps:

[0045] Determine the hyperparameter combination and decision action parameter combination to be optimized and define the corresponding search space;

[0046] Optimize the hyperparameter combination and build a Gaussian process regression agent model. Take the hyperparameter combination as input, maximize the average cumulative reward as the goal, and select the optimal hyperparameter combination through the expected improvement acquisition function.

[0047] Optimize the decision action parameter combination, define the objective function according to plant growth status indicators and environmental indicators, build a Gaussian process regression agent model based on historical environmental status data, historical agricultural management data and the corresponding objective function value, and select the optimal decision action parameter combination through the expected improvement acquisition function;

[0048] The ecological agriculture multimodal reinforcement learning model is continuously trained based on the above steps until the training termination condition is met.

[0049] In some embodiments, the use of the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making includes the following steps:

[0050] Preprocessing the multimodal dataset and converting it into an input format for the eco-agriculture multimodal reinforcement learning model;

[0051] Based on the meteorological parameters, soil parameters, and crop growth parameters in the input format, the eco-agricultural multimodal reinforcement learning model is used to reason and output optimal agricultural management decisions; wherein the agricultural management decisions include operating parameters for irrigation water volume, fertilizer application volume, and pesticide application volume;

[0052] The agricultural management decision is transmitted to the agricultural intelligent decision-making platform and converted into an instruction format recognizable by the agricultural intelligent decision-making platform, and then the real-time docking and issuance of the operation instructions are achieved with the irrigation, fertilization and pesticide application equipment through the preset communication interface and protocol.

[0053] In some embodiments, the method further comprises the following steps:

[0054] After the irrigation, fertilization, and pesticide application equipment executes the operation instructions, continuously collecting environmental data and crop growth data;

[0055] The environmental data and the crop growth data are fed back to the eco-agriculture multimodal reinforcement learning model to form a closed-loop control and optimize the operation instructions.

[0056] In some embodiments, feeding back the environmental data and the crop growth data to the eco-agriculture multimodal reinforcement learning model to form a closed-loop control and optimize the operation instructions includes the following steps:

[0057] Preprocessing the environmental data and the crop growth data as feedback data, including data cleaning, standardization, and feature extraction, to improve data quality and consistency;

[0058] Inputting the preprocessed data into the ecological agriculture multimodal reinforcement learning model, thereby reasoning and evaluating the effect of the current agricultural management decision and calculating the corresponding reward value;

[0059] According to the reward value and the new state, the feedback data is stored in the experience replay pool, and training samples are extracted from the experience replay pool based on the priority sampling mechanism. The deep deterministic policy gradient algorithm is used to update the policy network and value network parameters, so that the ecological agricultural multimodal reinforcement learning model can dynamically adapt to environmental changes and agricultural management needs.

[0060] To achieve the above objectives, another aspect of the present invention provides a non-point source pollution control and eco-agriculture decision-making device based on reinforcement learning, the device comprising:

[0061] A data acquisition unit is used to collect meteorological parameters, soil parameters, and crop growth parameters through the farmland Internet of Things system, and to collect historical management data during crop growth and water quality monitoring information of surface runoff as raw data;

[0062] A data set construction unit, configured to preprocess the raw data to construct a multimodal data set;

[0063] A model building unit, configured to build an eco-agricultural multimodal reinforcement learning model based on the multimodal dataset;

[0064] A model integration unit, configured to integrate the eco-agriculture multimodal reinforcement learning model into the agricultural intelligent decision-making platform, so that the eco-agriculture multimodal reinforcement learning model is coupled with the agricultural intelligent decision-making platform;

[0065] The decision-making unit is used to use the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making.

[0066] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0067] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above-mentioned method when executed by a processor.

[0068] The embodiments of the present application include at least the following beneficial effects:

[0069] This application can collect meteorological parameters, soil parameters and crop growth parameters through the farmland Internet of Things system and collect historical management data during the crop growth process and water quality monitoring information of surface runoff as raw data; preprocess the raw data to construct a multimodal data set; construct an ecological agricultural multimodal reinforcement learning model based on the multimodal data set; integrate the ecological agricultural multimodal reinforcement learning model into the agricultural intelligent decision-making platform, so that the ecological agricultural multimodal reinforcement learning model and the agricultural intelligent decision-making platform are coupled; use the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide equipment to achieve non-point source pollution control and ecological agricultural decision-making. This application overcomes many limitations of traditional ecological agriculture through machine learning algorithms and real-time data analysis. It can monitor multi-source data such as soil, climate, pests and diseases, and crop growth in real time, and combine reinforcement learning algorithms to autonomously optimize agricultural management strategies, achieve dynamic regulation, and thus be able to quickly respond to environmental changes; at the same time, this application creates a more suitable growth environment for crops by optimizing management strategies, which can reduce the input of agricultural resources, reduce production costs, and reduce agricultural non-point source pollution. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0071] Figure 1 A flowchart of a reinforcement learning-based non-point source pollution control and ecological agriculture decision-making method provided in an embodiment of the present application;

[0072] Figure 2 This is an example flow chart of a reinforcement learning-based non-point source pollution control and ecological agriculture decision-making method provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of the structure of a non-point source pollution control and ecological agriculture decision-making device based on reinforcement learning provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0076] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0077] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0078] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0079] Reference Figure 1 The embodiment of the present application provides a method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning. The method may include but is not limited to S100 to S140, as follows:

[0080] S100: The farmland IoT system collects meteorological parameters, soil parameters, and crop growth parameters, as well as historical management data during crop growth and surface runoff water quality monitoring information as raw data;

[0081] S110: Preprocessing the raw data to construct a multimodal dataset;

[0082] S120: Constructing an eco-agriculture multimodal reinforcement learning model based on the multimodal dataset;

[0083] S130: Integrating the eco-agriculture multimodal reinforcement learning model into the agricultural intelligent decision-making platform, so that the eco-agriculture multimodal reinforcement learning model is coupled with the agricultural intelligent decision-making platform;

[0084] S140: Using the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agriculture decision-making.

[0085] Optionally, the method of collecting meteorological parameters, soil parameters, and crop growth parameters through the farmland IoT system and acquiring historical management data of the crop growth process and water quality monitoring information of surface runoff as raw data includes the following steps:

[0086] Collecting meteorological parameters, soil parameters, and crop growth parameters through a weather station, a multimodal sensor, and an imaging device that achieves a target resolution, and collecting historical management data during crop growth and water quality monitoring information of surface runoff as the raw data;

[0087] The farmland IoT system includes the weather station, a multimodal sensor, and an imaging device that achieves the target resolution.

[0088] Said meteorological parameters include air temperature, wind speed, light intensity and atmospheric humidity;

[0089] The soil parameters include soil volumetric moisture content, soil electrical conductivity, pH value and soil pesticide residue concentration;

[0090] The crop growth parameters include crop growth stage codes, crop surface temperature, crop height, crop fruit size, and crop phenotypic feature vectors extracted based on convolutional neural networks;

[0091] The historical management data includes irrigation water volume, fertilizer volume, and pesticide usage volume;

[0092] The surface runoff water quality monitoring information includes nitrogen, phosphorus, organic pollutants and heavy metal contents in the water.

[0093] Optionally, the preprocessing of the raw data to construct a multimodal dataset comprises the following steps:

[0094] Arrange the meteorological parameters, the soil parameters, the crop growth parameters, and the historical management data in chronological order to construct a time series data set;

[0095] Perform data cleaning on the data features in the time series dataset, identify and remove mutation points and abnormal sequences during the data collection process, and use data filling strategies to fill in missing values; wherein the data filling strategies include mean filling or nearest neighbor interpolation;

[0096] The numerical features in the time series dataset are Z-score standardized, and the categorical features in the time series dataset are one-hot encoded, thereby unifying the data format of the data features in the time series dataset to construct the multimodal dataset.

[0097] Optionally, constructing an eco-agriculture multimodal reinforcement learning model based on the multimodal dataset comprises the following steps:

[0098] Extracting data features based on the multimodal dataset includes the following steps:

[0099] For crop image data, convolutional neural networks are used to extract image features for crop and weed identification, pest and disease detection, and nutritional status monitoring;

[0100] Determine numerical data as feature vector input; wherein the numerical data includes the meteorological parameters and the soil parameters;

[0101] Based on the extracted multimodal features, a multidimensional state vector is constructed to define the state space at each time point in the agricultural environment and obtain time series data.

[0102] Processing the time series data using a long short-term memory network to construct a time series state vector;

[0103] Designing a decision action space, which includes irrigation water quantity regulation, fertilizer quantity regulation, and pesticide usage quantity regulation, and dynamically limiting the regulation range according to environmental conditions and corresponding standard limits;

[0104] A reward function is constructed based on crop growth indicators, resource utilization efficiency, and environmental protection requirements. Multi-objective optimization is achieved through dynamic weight allocation, and a real-time feedback mechanism is used to guide the ecological agriculture multimodal reinforcement learning model to learn the optimal strategy. When the management measures taken enable crop growth to reach a preset state and resource utilization efficiency and environmental impact are both at preset levels, positive rewards are given through the reward function. When crop growth is hindered, resources are wasted, or environmental pollution occurs, negative rewards are given through the reward function.

[0105] The deep deterministic policy gradient algorithm is selected to train the ecological agriculture multimodal reinforcement learning model, including the following steps:

[0106] Based on the historical agricultural operation data set, supervised pre-training is performed. The input of the Actor network is set as the agricultural environment state, and the output is the corresponding agricultural management decision. The mean square error loss function is used for training. After the training is completed, the obtained agricultural management strategy is used as the initial strategy for reinforcement learning. The MSE loss function expression is as follows:

[0107]

[0108] in, is the output of the Actor network, a t is the real historical operation value;

[0109] Unsupervised reinforcement learning based on the initial strategy obtained from supervised pre-training includes the following steps:

[0110] Define the Actor network and the Critic network. Initialize the Actor network using a supervised pre-trained Actor network and set the scope of agricultural management actions based on practical constraints. In the Critic network, perform feature fusion of the agricultural environment state vector and the management decision action under the corresponding state as input, and calculate the output value of the management action based on the environment state vector and the management action.

[0111] An experience replay pool is defined to store experience data during the interaction process, including the current state of the agricultural environment, the agricultural management measures taken, the reward value calculated according to the reward function, the next state of the agricultural environment after the agricultural management measures are taken, and the end state identifier;

[0112] Random sampling from the experience replay pool is performed to update the parameters of the ecological agriculture multimodal reinforcement learning model, including the following steps:

[0113] The target Q value is calculated using the target network, and the mean square error loss function is constructed to update the critic network parameters using the gradient descent method;

[0114] By using the Q-value gradient provided by the Critic network, the policy gradient method is used to optimize the Actor network parameters, so that the agricultural management actions output by the Actor network can obtain a higher Q-value under the evaluation of the Critic network;

[0115] Regularly use soft updates to update the parameters of the target network to improve the stability and convergence of the model. The soft update expression is as follows:

[0116] θ′=τθ+(1-τ)θ′;

[0117] Among them, θ ′ are the parameters of the target network, θ are the parameters of the current network, and τ are the soft update coefficients;

[0118] Bayesian optimization is used to perform integrated optimization of model hyperparameters and decision action parameters, including the following steps:

[0119] Determine the hyperparameter combination and decision action parameter combination to be optimized and define the corresponding search space;

[0120] Optimize the hyperparameter combination and build a Gaussian process regression agent model. Take the hyperparameter combination as input, maximize the average cumulative reward as the goal, and select the optimal hyperparameter combination through the expected improvement acquisition function.

[0121] Optimize the decision action parameter combination, define the objective function according to plant growth status indicators and environmental indicators, build a Gaussian process regression agent model based on historical environmental status data, historical agricultural management data and the corresponding objective function value, and select the optimal decision action parameter combination through the expected improvement acquisition function;

[0122] The ecological agriculture multimodal reinforcement learning model is continuously trained based on the above steps until the training termination condition is met.

[0123] Optionally, the method of using the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making includes the following steps:

[0124] Preprocessing the multimodal dataset and converting it into an input format for the eco-agriculture multimodal reinforcement learning model;

[0125] Based on the meteorological parameters, soil parameters, and crop growth parameters in the input format, the eco-agricultural multimodal reinforcement learning model is used to reason and output optimal agricultural management decisions; wherein the agricultural management decisions include operating parameters for irrigation water volume, fertilizer application volume, and pesticide application volume;

[0126] The agricultural management decision is transmitted to the agricultural intelligent decision-making platform and converted into an instruction format recognizable by the agricultural intelligent decision-making platform, and then the real-time docking and issuance of the operation instructions are achieved with the irrigation, fertilization and pesticide application equipment through the preset communication interface and protocol.

[0127] Optionally, the method further comprises the following steps:

[0128] After the irrigation, fertilization, and pesticide application equipment executes the operation instructions, continuously collecting environmental data and crop growth data;

[0129] The environmental data and the crop growth data are fed back to the eco-agriculture multimodal reinforcement learning model to form a closed-loop control and optimize the operation instructions.

[0130] Optionally, feeding back the environmental data and the crop growth data to the eco-agriculture multimodal reinforcement learning model to form a closed-loop control and optimize the operation instructions comprises the following steps:

[0131] Preprocessing the environmental data and the crop growth data as feedback data, including data cleaning, standardization, and feature extraction, to improve data quality and consistency;

[0132] Inputting the preprocessed data into the ecological agriculture multimodal reinforcement learning model, thereby reasoning and evaluating the effect of the current agricultural management decision and calculating the corresponding reward value;

[0133] According to the reward value and the new state, the feedback data is stored in the experience replay pool, and training samples are extracted from the experience replay pool based on the priority sampling mechanism. The deep deterministic policy gradient algorithm is used to update the policy network and value network parameters, so that the ecological agricultural multimodal reinforcement learning model can dynamically adapt to environmental changes and agricultural management needs.

[0134] Next, the solution of the embodiment of the present application will be introduced and explained in detail with reference to specific application examples.

[0135] Reference Figure 2 This embodiment provides an example implementation of a non-point source pollution control and ecological agriculture decision-making method based on reinforcement learning. The specific solution is as follows:

[0136] According to the actual situation of agricultural production, weather stations are configured to monitor meteorological parameters of the agricultural environment, and corresponding multimodal sensors (such as soil moisture sensors, conductivity meters, and soil pH meters) and high-resolution imaging equipment are deployed. Through the Internet of Things system composed of weather stations, multimodal sensors, and high-resolution imaging equipment, data acquisition terminals are configured at the same time to stably transmit the data collected by the Internet of Things system to the central database via wireless or wired means to collect meteorological parameters, soil parameters, and crop growth parameters. At the same time, historical management data of the production process is collected to form the original data set; among them, meteorological parameters include air temperature, wind speed, light intensity, and atmospheric humidity; soil parameters include soil volume moisture content, soil conductivity, soil pH value, and soil pesticide residue concentration; crop growth parameters include crop growth stage code, crop surface temperature, crop height, crop fruit size, and crop phenotypic feature vectors extracted based on convolutional neural networks;

[0137] The original dataset consisting of meteorological parameters, soil parameters, crop growth parameters, and historical management data is preprocessed to construct a multimodal dataset. The specific steps include:

[0138] The meteorological parameters, soil parameters, crop growth parameters and historical management data of the production process are sorted and organized according to the chronological order of agricultural production to construct a time series dataset;

[0139] Data cleaning is performed on each feature in the time series data set. The Isolation Forest algorithm is used to identify and remove mutation points and abnormal sequences during the data collection process. In order to avoid real emergencies in the agricultural production process being judged as abnormal data and eliminated, a time series before and after discrimination mechanism is introduced. If the detected abnormal data is only a short-term isolated mutation (such as a feature that only undergoes an abnormal mutation at a certain sampling moment) and has no continuous evolution trend, it is regarded as a monitoring anomaly and eliminated; on the contrary, if it shows continuous changes and is consistent with the actual changes in the agricultural environment, the data is retained for subsequent model training to enhance the model's adaptability to sudden events. At the same time, according to the characteristics of each feature data, an appropriate data filling strategy (such as mean filling or nearest neighbor interpolation, etc.) is selected to fill in the missing values;

[0140] We perform Z-score normalization on numerical features and one-hot encoding on categorical features such as crop growth stage codes, unify the data format, and construct a multimodal dataset. The Z-score normalization expression is as follows:

[0141]

[0142] Among them, x ′ is the standardized feature, x is the mean of the feature, and σ is the standard deviation.

[0143] Based on the multimodal dataset, an ecological agriculture multimodal reinforcement learning model is constructed, which includes the following steps:

[0144] Based on the multimodal dataset, data features are extracted, including the following steps:

[0145] For crop image data, a deep convolutional neural network (CNN) is used to extract image features. High-resolution imaging equipment such as multispectral cameras and thermal imaging cameras are deployed to capture images of crops during their growth process. The raw images are then subjected to standardized preprocessing, including image resizing, noise filtering, illumination normalization, and enhancement, to meet model input requirements and accelerate model training. The output is a high-dimensional feature vector representing the crop status, such as leaf area, leaf number, leaf edge characteristics, leaf color distribution, and leaf vein structure. This is used for crop and weed identification, pest and disease detection, and nutritional status monitoring.

[0146] For numerical data such as meteorological parameters and soil parameters, they are directly input as feature vectors.

[0147] By extracting features from multimodal data, a multidimensional state vector is constructed, defining the state space at each time point in the agricultural environment, including meteorological parameters, soil parameters, and crop growth parameters. An LSTM network is introduced to process these state vectors, capturing dynamic changes and long-term dependencies in the time series and generating a time series state vector that includes historical evolution trends.

[0148] Design the decision action space, including irrigation water regulation, fertilizer regulation, and pesticide usage regulation, and ensure that the decision action space matches the objective constraints and ecological goals of agricultural production practices. For irrigation management, the action space includes a continuous adjustment range for irrigation water volume. Irrigation volume is dynamically adjusted according to weather forecasts, soil moisture content, and crop water requirements to ensure that water supply matches crop growth needs while avoiding water resource waste. For fertilization management, the action space covers the application adjustment range for different types of fertilizers. According to soil nutrient content, crop growth stage, and environmental conditions, fertilizer application is dynamically adjusted to ensure adequate and not excessive nutrient supply and reduce environmental pollution. For pest and disease control, the action space includes a continuous adjustment range for pesticide usage. According to pest and disease monitoring data and crop growth stage, pesticide usage is dynamically adjusted to ensure effective control of pests and diseases while reducing agricultural residues;

[0149] Taking into account crop growth indicators (such as leaf size, fruit quantity and size, crop height, etc.), resource utilization efficiency (such as soil moisture changes, soil nutrient changes, etc.), and environmental protection requirements (such as the proportion of pesticide use to the prescribed limit, etc.), a reward function R is constructed. Multi-objective optimization is achieved through dynamic weight allocation, and a real-time feedback mechanism is used to guide the model to learn the optimal strategy. When the management measures taken achieve ideal crop growth, high resource utilization efficiency, and reasonable environmental impact, positive rewards are given; conversely, when crop growth is hindered, resources are wasted, or environmental pollution occurs, negative rewards are given.

[0150] The Deep Deterministic Policy Gradient (DDPG) algorithm is selected to train the model, which includes the following steps:

[0151] Based on a historical agricultural operation dataset, supervised pre-training is performed. The input of the Actor network is set as the agricultural environment state, and the output is the corresponding agricultural management decision. The mean square error (MSE) loss function is used for training. After training, the agricultural management strategy obtained is used as the initial strategy for reinforcement learning. The MSE loss function is expressed as follows:

[0152]

[0153] in, is the output of the Actor network, a t is the real historical operation value;

[0154] Unsupervised reinforcement learning based on the initial strategy obtained from supervised pre-training includes the following steps:

[0155] Define the Actor network and the Critic network. In the Actor network, use the supervised pre-trained Actor network for initialization, and set reasonable value ranges for agricultural management actions (such as irrigation water volume, fertilizer application, and pesticide usage) based on actual constraints. In the Critic network, the agricultural environment state vector and the management decision action under that state are fused as features and used as network input. The value of the output management action is calculated through multiple fully connected layers.

[0156] Define an experience replay pool to store the experience data from the interaction process. Each piece of experience data includes the current state of the agricultural environment (including meteorological data, soil parameters, and crop growth parameters), the agricultural management measures taken (irrigation water regulation, fertilizer regulation, pesticide use regulation), the reward value calculated according to the reward function, the next state of the agricultural environment after the agricultural management measures are taken, and the end state identifier;

[0157] Randomly sampling from the experience replay pool and updating the model parameters includes the following steps:

[0158] The target Q value is calculated using the target network, and the mean square error loss function is constructed based on the actual reward and the predicted value, and the critic network parameters are updated using the gradient descent method;

[0159] By using the Q-value gradient provided by the Critic network, the policy gradient method is used to optimize the Actor network parameters, so that the agricultural management actions output by the Actor network can obtain a higher Q-value under the evaluation of the Critic network;

[0160] Regularly use soft updates to update the parameters of the target network to improve the stability and convergence of the model. The soft update expression is as follows:

[0161] θ′=τθ+(1-τ)θ′;

[0162] Among them, θ ′ are the parameters of the target network, θ are the parameters of the current network, and τ are the soft update coefficients;

[0163] Bayesian optimization is used to perform integrated optimization of model hyperparameters and decision action parameters, including the following steps:

[0164] Determine the hyperparameter combination to be optimized (soft update coefficient, exploration noise variance and discount factor) and decision action parameter combination (irrigation water amount, fertilizer amount, pesticide use amount) and define the corresponding search space;

[0165] Optimize the hyperparameter combination and build a Gaussian process regression agent model. Take the hyperparameter combination as input, maximize the average cumulative reward as the goal, and automatically search for the optimal hyperparameter combination through the expected improvement acquisition function.

[0166] Optimize the decision action parameter combination, define the objective function according to plant growth status indicators and environmental indicators, build a Gaussian process regression agent model based on historical environmental status data, historical agricultural management data and the corresponding objective function value, and select the optimal decision action parameter combination through the expected improvement acquisition function;

[0167] Based on the above steps, unsupervised reinforcement learning training is continued until the reinforcement learning model satisfies the requirement that the average reward change rate is less than 0.5% within 100 consecutive rounds, thereby obtaining the optimal management strategy that can adapt to changes in the agricultural environment.

[0168] Integrate the multimodal reinforcement learning model of ecological agriculture into the agricultural intelligent decision-making platform and deeply couple it with the agricultural control system to achieve automated and precise operation instructions for irrigation, fertilization, and pesticide application equipment. The following steps are included:

[0169] The multimodal reinforcement learning model for ecological agriculture is packaged into a deployable service using containerization technology and deployed to the agricultural intelligent decision-making platform. An IoT system consisting of weather stations, multimodal sensors, and high-resolution imaging equipment is used to collect multimodal data on the agricultural environment in real time. Preprocessing steps such as data cleaning, normalization, and feature extraction are then used to convert the raw data into a model input format.

[0170] Based on real-time collected meteorological parameters, soil parameters, and crop growth parameters, the model is used to reason and output optimal agricultural management decisions. These decisions include operational parameters such as irrigation water volume, fertilizer application volume, and pesticide application volume.

[0171] The agricultural management decisions obtained by model reasoning are transmitted to the agricultural control system and converted into an instruction format recognizable by the agricultural control system. Through the preset communication interface and protocol, real-time docking and instruction issuance with irrigation, fertilization, and pesticide application equipment are achieved, such as adjusting the amount of irrigation, fertilization, and pesticide spraying.

[0172] Build a real-time monitoring and feedback mechanism to continuously collect environmental and crop growth data after the equipment executes instructions, and feed it back to the ecological agriculture multimodal reinforcement learning model to form a closed-loop optimization, including the following steps:

[0173] The IoT system continuously collects multimodal data of the agricultural environment, including meteorological parameters, soil parameters, and crop growth parameters, and transmits them to the agricultural intelligent decision-making platform in real time;

[0174] The collected data was preprocessed, using the Z-score standardization method to normalize the numerical data and perform one-hot encoding on the categorical data to unify the data format. At the same time, feature extraction technology was used to convert the original multimodal data into a time series state vector that matched the input of the ecological agriculture multimodal reinforcement learning model.

[0175] The preprocessed data is input into the ecological agriculture multimodal reinforcement learning model. The model reasoning is used to evaluate the effectiveness of current agricultural management decisions. Based on the crop growth status, environmental changes, and the preset reward function, the corresponding immediate reward value is calculated to reflect the positive and negative impact of the current decision on the overall benefits of the agricultural system.

[0176] Based on the reward value and the new state, feedback data from the interaction process is stored in an experience replay pool, forming a complete experience record that includes the agricultural production environment state, agricultural management actions, management action rewards, and the next agricultural environment state after the agricultural management action is implemented. Training samples are extracted from the experience replay pool based on a priority sampling mechanism, and a deep deterministic policy gradient algorithm is used to update the policy network and value network parameters based on the sampled data. Through continuous feedback data and model updates, a closed-loop optimization system is formed, allowing the model's decision-making strategy to be continuously revised and improved, thereby achieving automated and precise optimization of agricultural management.

[0177] Reference Figure 3 The embodiment of the present application further provides a non-point source pollution control and eco-agriculture decision-making device based on reinforcement learning, which can implement the above-mentioned non-point source pollution control and eco-agriculture decision-making method based on reinforcement learning. The device includes:

[0178] A data acquisition unit is used to collect meteorological parameters, soil parameters, and crop growth parameters through the farmland Internet of Things system, and to collect historical management data during crop growth and water quality monitoring information of surface runoff as raw data;

[0179] A data set construction unit, configured to preprocess the raw data to construct a multimodal data set;

[0180] A model building unit, configured to build an eco-agricultural multimodal reinforcement learning model based on the multimodal dataset;

[0181] A model integration unit, configured to integrate the eco-agriculture multimodal reinforcement learning model into the agricultural intelligent decision-making platform, so that the eco-agriculture multimodal reinforcement learning model is coupled with the agricultural intelligent decision-making platform;

[0182] The decision-making unit is used to use the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making.

[0183] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0184] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method of the present application. The electronic device can be any smart terminal, such as a tablet computer or an in-vehicle computer.

[0185] It can be understood that the contents of the above method embodiments are all applicable to the embodiments of the present device, the functions specifically implemented by the embodiments of the present device are the same as those of the method of the present application, and the beneficial effects achieved are also the same as those achieved by the method of the present application.

[0186] See also Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0187] The processor 401 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0188] The memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called by the processor 401 to execute the methods of the embodiments of this application.

[0189] Input / output interface 403, used to implement information input and output;

[0190] Communication interface 404, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0191] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0192] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via a bus 405 .

[0193] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method of the present application is implemented.

[0194] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0195] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0196] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0197] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0198] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0199] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0200] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0201] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0202] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0203] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0205] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0206] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A non-point source pollution control and ecological agriculture decision-making method based on reinforcement learning, characterized by: The method comprises the following steps: The farmland IoT system collects meteorological parameters, soil parameters, and crop growth parameters, as well as historical management data during crop growth and surface runoff water quality monitoring information as raw data; Preprocessing the raw data to construct a multimodal dataset; Constructing an eco-agricultural multimodal reinforcement learning model based on the multimodal dataset; Integrating the eco-agriculture multimodal reinforcement learning model into the agricultural intelligent decision-making platform, so that the eco-agriculture multimodal reinforcement learning model is coupled with the agricultural intelligent decision-making platform; The agricultural intelligent decision-making platform is used to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making.

2. The method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning according to claim 1, characterized in that: The method of collecting meteorological parameters, soil parameters, and crop growth parameters through the farmland IoT system and collecting historical management data during crop growth and water quality monitoring information of surface runoff as raw data includes the following steps: Collecting meteorological parameters, soil parameters, and crop growth parameters through a weather station, a multimodal sensor, and an imaging device that achieves a target resolution, and collecting historical management data during crop growth and water quality monitoring information of surface runoff as the raw data; The farmland IoT system includes the weather station, a multimodal sensor, and an imaging device that achieves the target resolution. Said meteorological parameters include air temperature, wind speed, light intensity and atmospheric humidity; The soil parameters include soil volumetric moisture content, soil electrical conductivity, pH value and soil pesticide residue concentration; The crop growth parameters include crop growth stage codes, crop surface temperature, crop height, crop fruit size, and crop phenotypic feature vectors extracted based on convolutional neural networks; The historical management data includes irrigation water volume, fertilizer volume, and pesticide usage volume; The surface runoff water quality monitoring information includes nitrogen, phosphorus, organic pollutants and heavy metal contents in the water.

3. The method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning according to claim 1 is characterized in that: The preprocessing of the raw data to construct a multimodal dataset comprises the following steps: Arrange the meteorological parameters, the soil parameters, the crop growth parameters, and the historical management data in chronological order to construct a time series data set; Perform data cleaning on the data features in the time series dataset, identify and remove mutation points and abnormal sequences during the data collection process, and use data filling strategies to fill in missing values; wherein the data filling strategies include mean filling or nearest neighbor interpolation; The numerical features in the time series dataset are Z-score standardized, and the categorical features in the time series dataset are one-hot encoded, thereby unifying the data format of the data features in the time series dataset to construct the multimodal dataset.

4. The method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning according to claim 1, characterized in that: The step of constructing an eco-agriculture multimodal reinforcement learning model based on the multimodal dataset comprises the following steps: Extracting data features based on the multimodal dataset includes the following steps: For crop image data, convolutional neural networks are used to extract image features for crop and weed identification, pest and disease detection, and nutritional status monitoring; Determine numerical data as feature vector input; wherein the numerical data includes the meteorological parameters and the soil parameters; Based on the extracted multimodal features, a multidimensional state vector is constructed to define the state space at each time point in the agricultural environment and obtain time series data. Processing the time series data using a long short-term memory network to construct a time series state vector; Designing a decision action space, which includes irrigation water quantity regulation, fertilizer quantity regulation, and pesticide usage quantity regulation, and dynamically limiting the regulation range according to environmental conditions and corresponding standard limits; A reward function is constructed based on crop growth indicators, resource utilization efficiency, and environmental protection requirements. Multi-objective optimization is achieved through dynamic weight allocation, and a real-time feedback mechanism is used to guide the ecological agriculture multimodal reinforcement learning model to learn the optimal strategy. When the management measures taken enable crop growth to reach a preset state and resource utilization efficiency and environmental impact are both at preset levels, positive rewards are given through the reward function. When crop growth is hindered, resources are wasted, or environmental pollution occurs, negative rewards are given through the reward function. The deep deterministic policy gradient algorithm is selected to train the ecological agriculture multimodal reinforcement learning model, including the following steps: Based on the historical agricultural operation data set, supervised pre-training is performed. The input of the Actor network is set as the agricultural environment state, and the output is the corresponding agricultural management decision. The mean square error loss function is used for training. After the training is completed, the obtained agricultural management strategy is used as the initial strategy for reinforcement learning. The MSE loss function expression is as follows: in, is the output of the Actor network, a t is the real historical operation value; Unsupervised reinforcement learning based on the initial strategy obtained from supervised pre-training includes the following steps: Define the Actor network and the Critic network. Initialize the Actor network using a supervised pre-trained Actor network and set the scope of agricultural management actions based on practical constraints. In the Critic network, perform feature fusion of the agricultural environment state vector and the management decision action under the corresponding state as input, and calculate the output value of the management action based on the environment state vector and the management action. An experience replay pool is defined to store experience data during the interaction process, including the current state of the agricultural environment, the agricultural management measures taken, the reward value calculated according to the reward function, the next state of the agricultural environment after the agricultural management measures are taken, and the end state identifier; Random sampling from the experience replay pool is performed to update the parameters of the ecological agriculture multimodal reinforcement learning model, including the following steps: The target Q value is calculated using the target network, and the mean square error loss function is constructed to update the critic network parameters using the gradient descent method; By using the Q-value gradient provided by the Critic network, the policy gradient method is used to optimize the Actor network parameters, so that the agricultural management actions output by the Actor network can obtain a higher Q-value under the evaluation of the Critic network; Regularly use soft updates to update the parameters of the target network to improve the stability and convergence of the model. The soft update expression is as follows: θ′=τθ+(1-τ)θ′; Among them, θ′ is the parameter of the target network, θ is the parameter of the current network, and τ is the soft update coefficient; Bayesian optimization is used to perform integrated optimization of model hyperparameters and decision action parameters, including the following steps: Determine the hyperparameter combination and decision action parameter combination to be optimized and define the corresponding search space; Optimize the hyperparameter combination and build a Gaussian process regression agent model. Take the hyperparameter combination as input, maximize the average cumulative reward as the goal, and select the optimal hyperparameter combination through the expected improvement acquisition function. Optimize the decision action parameter combination, define the objective function according to plant growth status indicators and environmental indicators, build a Gaussian process regression agent model based on historical environmental status data, historical agricultural management data and the corresponding objective function value, and select the optimal decision action parameter combination through the expected improvement acquisition function; The ecological agriculture multimodal reinforcement learning model is continuously trained based on the above steps until the training termination condition is met.

5. The method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning according to claim 1, characterized in that: The method of using the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making includes the following steps: Preprocessing the multimodal dataset and converting it into an input format for the eco-agriculture multimodal reinforcement learning model; Based on the meteorological parameters, soil parameters, and crop growth parameters in the input format, the eco-agricultural multimodal reinforcement learning model is used to reason and output optimal agricultural management decisions; wherein the agricultural management decisions include operating parameters for irrigation water volume, fertilizer application volume, and pesticide application volume; The agricultural management decision is transmitted to the agricultural intelligent decision-making platform and converted into an instruction format recognizable by the agricultural intelligent decision-making platform, and then the real-time docking and issuance of the operation instructions are achieved with the irrigation, fertilization and pesticide application equipment through the preset communication interface and protocol.

6. A method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning according to any one of claims 1 to 5, characterized in that: The method further comprises the following steps: After the irrigation, fertilization, and pesticide application equipment executes the operation instructions, continuously collecting environmental data and crop growth data; The environmental data and the crop growth data are fed back to the eco-agriculture multimodal reinforcement learning model to form a closed-loop control and optimize the operation instructions.

7. The method for non-point source pollution control and ecological agriculture decision-making based on reinforcement learning according to claim 6 is characterized in that: Feeding back the environmental data and the crop growth data to the eco-agriculture multimodal reinforcement learning model to form a closed-loop control and optimize the operation instructions includes the following steps: Preprocessing the environmental data and the crop growth data as feedback data, including data cleaning, standardization, and feature extraction, to improve data quality and consistency; Inputting the preprocessed data into the ecological agriculture multimodal reinforcement learning model, thereby reasoning and evaluating the effect of the current agricultural management decision and calculating the corresponding reward value; According to the reward value and the new state, the feedback data is stored in the experience replay pool, and training samples are extracted from the experience replay pool based on the priority sampling mechanism. The deep deterministic policy gradient algorithm is used to update the policy network and value network parameters, so that the ecological agricultural multimodal reinforcement learning model can dynamically adapt to environmental changes and agricultural management needs.

8. A non-point source pollution control and ecological agriculture decision-making device based on reinforcement learning, characterized in that: The device comprises: A data acquisition unit is used to collect meteorological parameters, soil parameters, and crop growth parameters through the farmland Internet of Things system, and to collect historical management data during crop growth and water quality monitoring information of surface runoff as raw data; A data set construction unit, configured to preprocess the raw data to construct a multimodal data set; A model building unit, configured to build an eco-agricultural multimodal reinforcement learning model based on the multimodal dataset; A model integration unit, configured to integrate the eco-agriculture multimodal reinforcement learning model into the agricultural intelligent decision-making platform, so that the eco-agriculture multimodal reinforcement learning model is coupled with the agricultural intelligent decision-making platform; The decision-making unit is used to use the agricultural intelligent decision-making platform to issue operating instructions to irrigation, fertilization, and pesticide application equipment to achieve non-point source pollution control and ecological agricultural decision-making.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images