Multi-agent reinforcement learning based agricultural production multi-objective decision optimization method and system

Abstract

The application relates to the technical field of wisdom agricultural decision-making, and particularly discloses an agricultural production multi-target decision optimization method and system based on multi-agent reinforcement learning, which comprises the following steps: each agent executes a decision and stores experience in a local cache pool; a concept drift quantitative index is calculated by comparing recent and historical data, and a collaborative adaptive group is dynamically clustered; a group consensus gradient is generated by federated learning aggregation in the group to update the strategy of each agent; a long-term ecological simulator is used to evaluate the long-term ecological value deficit of the strategy, and a meta-strategy correction signal is generated and fed back to the gradient aggregation process to guide the strategy to avoid long-term risks; and the system comprises four functionally corresponding modules. The application solves the problems of continuous degradation of the decision model in a non-stationary agricultural environment and possible deviation of the adaptive strategy from long-term sustainability, and realizes robust collaborative optimization of short-term benefits and long-term ecological health under dynamic conditions.

Description

Technical Field

[0001] This invention relates to the field of smart agricultural decision-making technology, specifically to a method and system for optimizing multi-objective decisions in agricultural production based on multi-agent reinforcement learning. Background Technology

[0002] With the development of precision agriculture and artificial intelligence technologies, utilizing multi-agent reinforcement learning (MARL) for collaborative decision-making optimization in agricultural production has become an important research direction. However, existing technologies still face several unresolved technical problems when dealing with complex real-world agricultural systems: Silent model degradation due to non-stationary environments: Existing MARL applications often assume that the training and deployment environments are stationary. However, agricultural production systems are affected by slow variables such as climate pattern changes and the gradual evolution of soil properties, making them inherently strong non-stationary environments. This leads to a silent decay in the decision-making effectiveness of a trained, fixed model after long-term deployment due to conceptual drift caused by environmental dynamics. Existing periodic retraining or federated learning frameworks struggle to effectively capture and adapt to this distributed, asynchronous local drift.

[0003] The neglect of socio-economic feedback mechanisms such as market interactions: Existing research largely focuses on the synergistic optimization of physical resources such as water and fertilizer, generally neglecting the virtual interactions between multiple production entities through market supply and demand and price mechanisms. This leads to the potential fallacy of composition in optimal solutions, where rational individual decisions to increase production trigger price declines at the collective level, resulting in a systemic market risk of bumper harvests without increased income. There is a lack of endogenous modeling and avoidance mechanisms for such socio-economic feedback loops.

[0004] Lack of proactive assessment and constraint of long-term ecological risks: Current methods often limit their objective functions to short-term measurable indicators such as yield and resource efficiency, making it difficult to internalize the slowly accumulating ecological threshold effects such as soil degradation and groundwater over-extraction. The strategies that models evolve to adapt to short-term environmental changes may systematically damage long-term ecological health, creating a sustainable development trap where adaptation undermines the foundation.

[0005] Therefore, there is an urgent need for a new paradigm of collaborative decision-making among multiple agricultural agents that can continuously adapt to non-stationary environments, internalize socio-economic feedback, and proactively avoid long-term ecological risks, in order to support the true sustainable optimization of agricultural systems. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for multi-objective decision optimization in agricultural production based on multi-agent reinforcement learning, so as to solve the problems of continuous degradation of decision models and potential deviation of adaptive strategies from long-term sustainability in non-stationary agricultural environments in the prior art.

[0007] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution: A multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production includes the following steps: S1. Each local agent executes a decision and stores the experience in a local recurrent experience cache pool; S2. Compare the recent and historical data of each of the local recurrent experience cache pools to generate a concept drift quantification index, and cluster them to form a co-adaptation group; S3. Within the cooperative adaptation group, a group consensus gradient is generated through federated learning to update the policy network of each agent; S4. Based on the updated policy network, perform long-term ecological simulation, calculate the long-term ecological value deficit, and generate a meta-policy correction signal to guide the calculation of subsequent group consensus gradients and avoid long-term ecological risks.

[0008] As a preferred embodiment of the present invention, S1 specifically includes: S11. In the multi-agent system, a local decision-making agent is independently deployed for each geographically dispersed agricultural production unit; each local decision-making agent includes a policy network for generating operational instructions and a local value network for evaluating state values. S12. In each decision cycle, each agent collects the current production environment status of its corresponding unit and inputs the status into its own policy network. The policy network outputs a specific agricultural operation instruction. S13. Each agent executes the agricultural operation instructions it generates. After obtaining the next state and immediate reward from the environmental feedback, it encapsulates the current state, the operation executed, the reward obtained, and the next state into a piece of experience data. S14. The encapsulated experience data is stored in the local loop experience cache pool corresponding to the agent; the local loop experience cache pool manages the data according to the first-in-first-out rule and only retains the experience data within the most recent decision cycles.

[0009] As a preferred embodiment of the present invention, S2 specifically includes: S21. Each agent randomly samples a certain amount of the latest experience data from its local recurrent experience cache pool to form a recent experience dataset; at the same time, it randomly samples an equal amount of historical experience data from a long-term stored historical benchmark experience pool to form a benchmark experience dataset. S22. Input the state and operation pairs in the recent experience dataset and the benchmark experience dataset into a drift evaluation network shared by all agents. The drift evaluation network outputs the latent feature representation of the state and operation pairs and their corresponding expected value evaluation. S23. Calculate the statistical difference between the latent feature distributions of the recent experience dataset and the benchmark experience dataset, and calculate the mean deviation of their expected value assessments; combine the difference and mean deviation to generate a concept drift quantification index for the agent. S24. The central coordinator collects the concept drift quantification indicators of all agents and performs cluster analysis based on the value and trend of the indicators. Agents with similar indicators and similar trends are grouped into the same set, and each set forms a cooperative adaptation group.

[0010] As a preferred embodiment of the present invention, S24 specifically includes: S241. The central coordinator collects concept drift quantification metrics uploaded by all agents; S242. The central coordinator standardizes all collected indicators and extracts the trend characteristics of each indicator within a recent continuous time window. S243. The central coordinator uses a clustering algorithm to group all agents based on the standardized index values ​​and their changing trend characteristics; agents whose index values ​​are close in distance in the feature space and whose changing trend characteristics are similar are grouped into the same cluster; S244. The central coordinator formally defines and constructs each cluster generated by clustering into an independent co-adaptation group, and assigns a unique identifier to each group to coordinate the subsequent intra-group federated learning process.

[0011] As a preferred embodiment of the present invention, S3 specifically includes: S31. For each co-adaptation group, the central coordinator initiates a round of federated learning for each group; S32. Each agent in the group uses the recent data in its local recurrent experience cache pool as the training set, performs multiple local training iterations on its own policy network and local value network, and calculates the parameter update gradient of its respective network. S33. Each agent uploads the calculated parameter update gradient to the central coordinator. The central coordinator selects gradient components that are consistent in direction and change significantly among most agents in the group, and aggregates them to generate a unified group consensus gradient. S34. The central coordinator distributes the group consensus gradient to all agents within the cooperative adaptation group. Each agent uses the received group consensus gradient to synchronously update the parameters of its local policy network and local value network.

[0012] As a preferred embodiment of the present invention, S33 specifically includes: S331. The central coordinator receives local policy gradients and local value gradients uploaded by all agents within the same cooperative adaptation group, and temporarily stores the gradients according to the agent's source. S332. For each network parameter position, the coordinator analyzes the gradient values ​​of all agents in the group on the gradient component; calculates the directional consistency measure of the gradient component in the group; filters out gradient components whose directional consistency is higher than a preset threshold and whose absolute gradient magnitude is also higher than the corresponding threshold, and marks them as significant gradient components. S333. Calculate an aggregate weight for each labeled significant gradient component. The weight is positively correlated with the direction of the gradient component within the group and with its average magnitude. Using the calculated weight, perform a weighted average of the values ​​of the significant gradient components in each agent within the group. The result is the group consensus gradient component at the parameter position. S334. Traverse all network parameter positions, calculate consensus values ​​for the selected significant gradient components, and set the consensus values ​​of non-significant gradient components that fail the screening to zero or a very small default value; assemble the consensus values ​​of all positions according to the original network parameter structure to finally form a complete group consensus strategy gradient and group consensus value gradient.

[0013] As a preferred embodiment of the present invention, S4 specifically includes: S41. The system maintains a regional ecological health status vector to record slow-moving variables, including changes in soil organic matter, groundwater balance, and biodiversity; the vector is updated periodically by aggregating low-frequency monitoring data from various production units. S42. Periodically put the updated policy network of all current agents into a long-term ecological simulator; the long-term ecological simulator starts from the current environmental state, simulates and runs multiple future planning cycles, and gradually increases the penalty for the deterioration of the regional ecological health state vector in the later stage of the simulation. S43. Through simulation, calculate the discounted cumulative return under the current strategy after long-term ecological constraints, and compare it with the discounted cumulative value of short-term training returns to obtain the long-term ecological value deficit; if the deficit exceeds the preset threshold, it is determined that there is a long-term ecological risk. S44. When a risk is determined to exist, the system analyzes the key decision-making patterns that lead to ecological degradation in the simulation and generates a meta-policy correction signal. The meta-policy correction is injected into the gradient aggregation stage to adjust the gradient components related to the harmful patterns.

[0014] As a preferred embodiment of the present invention, S42 specifically includes: S421. The long-term ecological simulator obtains the latest environmental state snapshot of all agricultural production units at the current moment, and loads the regional ecological health state vector as the initial environmental state for inference; S422. The simulator loads and maps the policy network of all agents in the current system after step S3 to the corresponding virtual production units in the simulation environment, forming a policy set to be evaluated. S423. The simulator starts from the initial environmental state and runs multiple consecutive planning cycles. In each cycle, each virtual unit generates operation instructions according to the strategy network it loads. Based on the preset soil, hydrology, crop growth and market interaction dynamics model, the simulator calculates the environmental state changes after the instructions are executed, the immediate economic returns, and the cumulative impact on each indicator of the regional ecological health state vector. S424. In the early stages of the simulation, the simulator calculates rewards in the same way as in short-term training, focusing on immediate goals such as output and resource efficiency. In the middle and later stages of the simulation, the simulator gradually introduces and increases the penalty term for any deterioration in any dimension of the regional ecological health state vector. The rewards and penalties after discounting are accumulated for all cycles, and finally the long-term ecological constraint discounted cumulative return is obtained.

[0015] As a preferred embodiment of the present invention, S44 specifically includes: S441. When the long-term ecological value deficit exceeds the threshold, the system analyzes the deduction process of the long-term ecological simulator, identifies one or more high-frequency, highly correlated agricultural operation decision sequences that cause a significant deterioration of at least one key indicator in the regional ecological health state vector, and defines them as harmful decision patterns. S442. Based on the identified harmful decision-making patterns, the system back-maps them to the parameter space or decision feature space of the agent policy network to generate a meta-policy correction signal. The meta-policy correction signal is specifically represented as a correction coefficient vector with the same dimension as the network parameters, wherein the parameter positions that are highly correlated with the harmful decision-making patterns are given inhibitory correction coefficients, and the remaining positions are given neutral coefficients. S443. When executing step S33 in the next training cycle, the central coordinator injects the meta-policy correction signal into the generation process of the group consensus gradient; the specific method is: before calculating the group consensus gradient, the correction coefficient vector is multiplied element-wise with the local policy gradient uploaded by each agent to adjust the gradient in advance, thereby suppressing the parameter update direction and intensity related to harmful patterns. S444. Based on the adjusted local policy gradients of each agent, the coordinator executes the original sparsification and differential weighted aggregation algorithm to generate and distribute a group consensus gradient that implicitly contains a long-term ecological sustainability orientation, so as to guide the policy network of all agents in the group to update synchronously and gradually move away from harmful decision-making patterns.

[0016] A multi-agent reinforcement learning-based multi-objective decision optimization system for agricultural production is provided to implement a multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production, including: The environmental interaction and experience caching module is deployed in each agricultural production unit. It is used to generate and execute agricultural operation instructions based on the local production environment status, and store the experience data generated by the decision in a local circulating experience caching pool. The concept drift collaborative detection module is connected to each of the aforementioned environment interaction and experience caching modules. It is used to periodically sample data from each cache pool, calculate the concept drift quantification index of each unit through a shared drift evaluation network, and perform cluster analysis based on this to dynamically form collaborative adaptation groups. The federated policy collaborative update module, connected to the concept drift collaborative detection module, is used to initiate a federated learning process within each collaborative adaptation group, aggregate the model gradients of each unit within the group, generate a group consensus gradient through sparsification and differential weighting algorithms, and synchronously update the decision models of all agents within the group accordingly. The long-term ecological assessment and meta-correction module, connected to the federated strategy collaborative update module, is used to maintain the regional ecological health state vector, use a long-term ecological simulator to extrapolate the updated decision model, calculate the long-term ecological value deficit, and generate a meta-strategy correction signal when ecological risks are identified, which is fed back to the federated strategy collaborative update module to guide the direction of strategy updates.

[0017] Compared with the prior art, the present invention has the following advantages: 1. By dynamically clustering to form collaborative adaptation groups and establishing a consensus gradient evolution mechanism within the groups, precise collaborative optimization of large-scale heterogeneous production units is achieved. This design overcomes the shortcomings of traditional methods that are either one-size-fits-all or lack collaboration, ensuring overall optimal allocation of global resources while respecting the individual differences of units.

[0018] 2. Through long-term ecological simulation and a meta-strategy correction signal feedback mechanism, the long-term sustainability of decision-making is intrinsically guaranteed. This scheme can proactively quantify the implicit ecological risks of short-term strategies and guide the evolutionary direction of the system by intervening in the learning process, fundamentally avoiding the trap of damaging the ecological foundation in order to adapt to the environment.

[0019] 3. A complete autonomous closed loop of perception, adaptation, auditing, and correction has been constructed, endowing the system with strategic adaptive capabilities to cope with non-stationary environments. The system can not only passively respond to changes, but also assess the long-term consequences of its collective behavior and revise its learning rules, thereby achieving continuous and robust optimization and significantly improving overall robustness. Attached Figure Description

[0020] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the method described in Embodiment 1 of the present invention.

[0022] Figure 2 This is a framework diagram of the system described in Embodiment 2 of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The concepts involved in this application will first be described with reference to the accompanying drawings. It should be noted that the following descriptions of various concepts are only for the purpose of making the content of this application easier to understand and do not constitute a limitation on the scope of protection of this application; furthermore, the embodiments and features in the embodiments of this application can be combined with each other unless otherwise specified. This application will now be described in detail with reference to the accompanying drawings and embodiments. Example 1

[0025] like Figure 1 As shown, this invention provides a multi-objective decision optimization method for agricultural production based on multi-agent reinforcement learning, comprising the following steps: S1. Each local agent executes a decision and stores the experience in a local recurrent experience cache pool; specifically including: S11. Distributed deployment and architecture configuration of local decision-making agents, specifically: S111. In a multi-agent system consisting of N geographically dispersed agricultural production units, a local decision-making agent is independently deployed for each geographically dispersed agricultural production unit. This deployment architecture embodies a distributed computing paradigm. The agricultural production units are physically distributed across different geographical locations, covering various agricultural formats such as plains, terraced fields, greenhouse facilities, or intensive pastures. The units are sparsely connected through wide-area IoT or edge computing networks, avoiding a continuous and strong dependence on centralized cloud servers.

[0026] S112. The hardware carrier for each local decision-making agent is an edge computing node deployed in the corresponding agricultural production unit. This node integrates a data preprocessing module, a model inference acceleration unit such as a GPU or NPU, and local storage media, enabling it to independently complete decision-making cycles in weak network or network outage environments. Logically, each agent is equal in status and there is no master-slave hierarchy, but they achieve periodic coordination through a central coordinator in subsequent step S3.

[0027] S113. At the software architecture level, each local decision-making agent contains two core functional network modules: a. Policy Network Module: This module is responsible for generating agricultural operation instructions. Its network topology can employ an Actor network under the Deep Deterministic Policy Gradient (DDPG) framework, a policy function approximation network under the Proximal Policy Optimization (PPO) framework, or a sequential decision model based on an attention mechanism. The network input is a local production environment state vector processed by feature engineering, with its dimensions dynamically adjustable based on sensor configuration. The network output layer is designed to adapt to the action space type. If the action space is continuous, such as precise irrigation water volume values, the output layer uses a fully connected layer to directly generate the action parameter vector. If the action space is discrete, such as agricultural machinery operation mode selection, the output layer uses a Softmax function to generate the probability distribution of each candidate action. The parameters of the policy network can be initialized during the initialization phase by loading pre-trained weights from the agricultural domain through transfer learning, or randomly initialized when prior knowledge is lacking.

[0028] b. Local Value Network Module: This module is responsible for state value evaluation, estimating the expected cumulative discounted reward obtained by following the current policy in a given state. Its network architecture maintains symmetry with the policy network or adopts a design that shares a low-level feature extraction layer to reduce the number of parameters and improve training stability. In the Actor-Critic architecture-based implementation, the local value network corresponds to the Critic network, and its output is a scalar value estimate; in the Q-learning framework-based implementation, the local value network corresponds to the Q network, and its output is an action value estimate for each state-action pair. The evaluation results of the local value network are not directly used to generate operation instructions, but rather serve as the gradient signal source for updating the policy network parameters. During local training iterations, it is calculated through temporal difference (TD) error or the advantage function, driving the policy to optimize towards higher rewards.

[0029] S114. The policy network and the local value network are functionally decoupled but coupled in training. They achieve co-evolution through sharing experience data and an alternating update mechanism. Both network modules of each agent are stored in persistent storage on the local edge computing node, ensuring that historical model parameters can still be loaded for inference and decision-making in the event of network interruption. The version management of the network weight file follows a rolling update strategy, retaining the model version after the most recent federated aggregations to support model rollback operations when concept drift detection fails.

[0030] During the system initialization phase, the central coordinator distributes a basic configuration package to each agricultural production unit. This package contains a network architecture definition file, an initial hyperparameter set, and an encrypted communication key. Each unit instantiates its local decision-making agent based on the configuration package and registers its geographic coordinates, soil type, main crop varieties, and other metadata with the central coordinator for use in the subsequent collaborative adaptation group clustering analysis in step S2. After registration, each agent enters an independent operating state and begins executing the state acquisition and decision-making loop of step S12.

[0031] S12. Policy reasoning and instruction generation based on local state, specifically: At each pre-defined decision-making cycle, such as hourly, daily, or at key nodes in each crop growth stage, each local decision-making agent first collects the current production environment status through a sensor network deployed within its agricultural production unit, including soil moisture sensors, meteorological monitoring stations, and crop growth visual acquisition devices. This status is a multi-dimensional vector encompassing soil physicochemical parameters, such as nitrogen, phosphorus, and potassium content, pH value, and moisture content; crop physiological indicators, such as leaf area index and NDVI vegetation index; meteorological conditions, such as temperature, humidity, and light radiation; and multi-source heterogeneous data such as historical agricultural operation records.

[0032] The collected status is preprocessed and then input into the policy network of the agent. The policy network calculates and outputs a specific farming operation instruction through forward propagation. This instruction is executable, and its content may include irrigation water volume and duration, fertilizer spraying type and dosage, agricultural machinery operation path planning, greenhouse environment control parameters, etc. The instruction format needs to be compatible with the control protocol of the agricultural Internet of Things devices supporting this unit.

[0033] S13. Encapsulation and collection of empirical data, specifically: S131. After each agent executes the farming operation instruction generated by the policy network, the production environment of its agricultural production unit undergoes a state transition under the domination of physical laws and generates corresponding environmental feedback. This feedback consists of two parts: one is the next state, that is, the state vector of the production environment re-collected after a certain time interval after executing the instruction; the other is the immediate reward, which is calculated by a multi-dimensional reward function. This reward function comprehensively considers factors such as crop yield expectation, resource consumption cost, labor efficiency, and short-term ecological impact, and quantifies the environmental feedback into a scalar value.

[0034] S132. After obtaining the above feedback, each agent encapsulates the complete information of this decision into an empirical data. The data structure of this empirical data strictly follows the formal definition of the Markov decision process and contains a quadruple: the current state, the executed action, the obtained reward, and the next state. In some implementation variants, it can also be extended to include auxiliary information such as action value estimation and policy log probability to support subsequent off-policy optimization or importance sampling correction.

[0035] S14. Storage management mechanism for the cyclic experience cache, specifically: The encapsulated empirical data is stored in the local cyclic experience cache pool corresponding to this agent. This cache pool, as a local storage structure with a finite capacity, adopts the physical implementation of a circular queue and follows the data management rule of first in, first out. The cache pool sets a maximum capacity threshold and only retains the empirical data within the most recent M decision cycles, where M is a hyperparameter pre-configured according to local storage resources, data timeliness requirements, and computational overhead. When new empirical data arrives, if the cache pool is not full, it is directly appended for storage; if the cache pool has reached the capacity limit, the earliest stored historical empirical data is removed from the cache pool to make room, and then the new data is stored. This cyclic overwrite mechanism ensures that the cache pool always maintains a data distribution reflecting the recent local environment interaction history of the agent, providing a data basis for subsequent local training iterations and concept drift detection. At the same time, each agent also maintains an independent historical benchmark experience pool, which stores historical empirical data from earlier cycles and has a lower update frequency than the cyclic experience cache pool, and is used as a benchmark reference for distribution comparison in the drift detection in step S2.

[0036] S2. Compare recent and historical data from each local recurrent experience cache pool to generate a concept drift quantification index, and cluster them into co-adaptation groups; specifically including: S21. Dual-track sampling of recent empirical data and baseline empirical data, specifically: S211. When each agent initiates the drift detection process, it first performs a random sampling operation from its local recurrent experience cache to construct a recent experience dataset. As described in step S14, the local recurrent experience cache retains only experience data from the most recent M decision cycles using a first-in, first-out (FIFO) rule. Therefore, this sampling operation actually acquires the latest trajectory fragment of the agent's interaction with the environment within the current time window. The sampling strategy can employ uniform random sampling to cover diverse state-action combinations, or a priority sampling strategy focusing on key experiences with high reward amplitudes or high TD errors. The number of samples is pre-set according to the frequency requirements of drift detection and computational resource constraints, ensuring statistical significance while controlling processing latency.

[0037] S212. Simultaneously, each agent randomly samples an equal amount of historical experience data from its historical baseline experience pool to form a baseline experience dataset. The historical baseline experience pool operates independently of the recurrent experience cache pool. Its data source is historical data transferred from the recurrent experience cache pool in previous cycles. The storage mechanism employs a lower-frequency update strategy, such as weekly or monthly additions, to ensure that the data in the pool reflects a relatively stable early environmental distribution. The capacity of this pool is typically much larger than M, capable of storing months or even years of experience data, providing a long-term stable reference benchmark for concept drift detection. The equal sampling control ensures the consistency of subsequent statistical comparisons, avoiding distribution measurement bias introduced by differences in sample size.

[0038] S22. Feature extraction and value assessment based on shared drift evaluation networks, specifically: S221. The drift evaluation network, as the core computational engine of the concept drift detection process in step S2, is designed to establish a unified feature representation space and value evaluation scale to support the comparison of experience data distribution across agents and time windows. This network differs from the private policy networks and local value networks of each agent in step S11. Its parameters are centrally maintained and updated by a central coordinator, and are obtained through periodic global aggregation or offline pre-training, ensuring consistent behavioral characteristics across all agents during local inference and eliminating detection bias caused by network heterogeneity.

[0039] The drift evaluation network employs a dual-branch parallel architecture, comprising a latent feature encoding branch and an expected value evaluation branch. The two branches share the weights of the first few layers of the underlying feature extraction layer, achieving synergy in representation learning and optimization of parameter efficiency. Simultaneously, they are separated at the top task layer, specializing in distributed feature extraction and scalar value estimation, respectively. The overall network parameter count is kept lightweight to accommodate the inference latency constraints of edge computing nodes and the distribution bandwidth limitations of the central coordinator.

[0040] S222. The data units input to the drift evaluation network are state-operation pairs, constructed by integrating the production environment state vector generated in step S12 and the agricultural operation instructions recorded in step S13. Specifically, after multi-source sensor acquisition and feature extraction in step S12, the state vector has formed a dense vector representation with a fixed dimension; if the operation instruction is a continuous parameter vector, it is directly quantified and concatenated to the end of the state vector; if it is a discrete categorical variable, it is mapped to a dense embedding vector compatible with the dimension of the state vector through a learnable embedding matrix, and then concatenated. The final joint input vector dimension is usually the sum of the state dimension and the operation dimension. After input distribution stabilization processing through a batch normalization layer, it is fed into the backbone computation flow of the drift evaluation network.

[0041] The input construction process remains consistent for all samples in the recent empirical dataset and the benchmark empirical dataset, ensuring that the two sets of data are processed under the same representation framework, and avoiding the interference of distribution differences introduced by preprocessing on the effectiveness of subsequent drift detection.

[0042] S223. The latent feature encoding branch is responsible for mapping high-dimensional state-operation pairs to a low-dimensional latent feature space. Its network topology adopts an encoder structure. Specifically, it can be implemented using stacked fully connected layers combined with nonlinear activation functions such as ReLU or GELU to progressively reduce the feature dimension and extract hierarchical abstract representations; or a variational autoencoder architecture can be used to model the probability distribution of latent variables while compressing representations, with the output latent feature representations being distribution parameters, and specific representation vectors obtained through posterior sampling; or an attention-based encoder can be used to capture the interaction relationships between various environmental factors within the state vector through self-attention layers, and to explicitly model the conditional dependencies between state features and operation instructions through cross-attention layers.

[0043] The output dimension of the latent feature representation is much lower than the input dimension, typically tens to hundreds of dimensions. This compressed representation aims to retain the information most relevant to predicting the decision consequences, while suppressing local noise and unit-specific interference. For the recent experience dataset and the benchmark experience dataset, the latent feature encoding branch performs batch forward inference, outputting two sets of latent feature representations, denoted as the recent latent feature set and the benchmark latent feature set. These two sets will be used for the distribution difference metric calculation in step S23.

[0044] S224. The expected value assessment branch, based on the latent feature representation output by the latent feature encoding branch, further calculates the expected cumulative reward estimate that the state-action pair can obtain under the current environmental dynamics. The network structure of this branch is a lightweight value estimation head, typically consisting of one or two fully connected layers, with a single neuron as the output layer. It is suitable for state-action value Q estimation, or for joint estimation of the advantage function A and state value V using two neurons, and is applicable to the Dueling architecture.

[0045] The physical semantics of expected value assessment is: assuming that the subsequent policy behavior pattern will continue to follow historically stable patterns, starting from the current state-operation pair, the expected value of the accumulated rewards that can be obtained in the future under the influence of the discount factor. This estimate does not rely on the current real-time local value network output of each agent in step S11, but is derived based on globally consistent parameter inference of the drift evaluation network, thus possessing comparability across time and space.

[0046] For the recent experience dataset and the benchmark experience dataset, the expected value assessment branch calculates the expected value assessment value for each sample, forming the recent value assessment set and the benchmark value assessment set. The statistical properties of the two value assessment sets will be used to calculate the mean deviation in step S23.

[0047] S225. The parameters of the drift evaluation network are not statically fixed, but are periodically updated and trained by the central coordinator during system operation intervals. The training data source is historical experience data samples collected periodically from all agents, and the training objective is to accurately reconstruct the state transition patterns and reward generation mechanisms in the historical baseline experience pool. The specific loss function can be designed as a weighted sum of latent feature reconstruction loss (such as the ELBO loss of a variational autoencoder) and value estimation temporal difference loss (such as Bellman error), and end-to-end training is performed using stochastic gradient descent or adaptive optimization algorithms.

[0048] After training, the central coordinator distributes the updated network parameters to each agent after differential compression or quantization encoding. The agents load and overwrite the old versions of the parameters locally for the next round of drift detection inference. The parameter synchronization frequency is lower than the federated learning frequency in step S3, typically set to once after several rounds of federated aggregation to balance detection accuracy and communication overhead. In extreme network isolation scenarios, agents can temporarily store multiple versions of drift evaluation network parameters and select the appropriate version based on timestamps to perform detection, ensuring the continuity of concept drift monitoring.

[0049] S23. The fusion and generation of quantitative indicators for concept drift, specifically: S231. Based on the recent latent feature set and the baseline latent feature set output in step S22, the agent performs a statistical calculation of the degree of difference in the latent feature distribution. This calculation aims to quantify the deep changes in the local environment dynamics in the state transition mechanism and reward generation structure, going beyond a simple comparison of surface statistics.

[0050] Specifically, the maximum mean difference (MMD) is used as the core metric. MMD calculates the mean distance between two distributions in this high-dimensional feature space by mapping latent features to a regenerating kernel Hilbert space. The kernel function is either a radial basis function kernel or a multinomial kernel, and the bandwidth parameter is adaptively determined through median heuristics or cross-validation. The MMD statistic is calculated as the expected difference between pairwise kernel function values ​​between two groups of samples, corrected by unbiased estimation. A larger value indicates a lower overlap between recent experience and historical benchmark distributions in the latent feature space, and a more significant degree of concept drift.

[0051] S232. Based on the recent value assessment set and the benchmark value assessment set output in step S22, the agent performs a mean deviation calculation of the expected value assessment. This calculation aims to quantify the systematic time-varying nature of the environmental reward structure or state value definition, reflecting the deviation between the expected return that the current strategy can obtain in the latest environment and that in historical stable environments.

[0052] The mean shift is calculated by first obtaining the sample means of the recent value assessment set and the benchmark value assessment set, denoted as the recent value mean and the benchmark value mean, respectively. To eliminate the influence of dimensions and ensure consistency in cross-agent comparisons, standardization is performed: the combined standard deviation of the two value assessment sets is calculated, or only the historical standard deviation of the benchmark value assessment set is used as the denominator. The mean difference is then divided by its standard deviation to obtain the standardized mean shift. This shift is a signed scalar; a positive value indicates that the recent environment tends to produce higher value estimates, while a negative value indicates that the recent environment tends to produce lower value estimates. The larger the absolute value, the more significant the shift.

[0053] S233. The agent fuses the degree of difference in latent feature distribution with the degree of deviation from the mean of expected value assessment to generate a single quantitative index of concept drift. The design of the fusion strategy needs to balance the complementary characteristics of the two types of information: the degree of distribution difference captures the structural changes in environmental dynamics, but is not sensitive to the value scale; the degree of mean deviation directly reflects the systematic bias of value estimation, but may be confused by the evolution of the strategy itself.

[0054] The fusion algorithm employs an adaptive weighting mechanism. Weight coefficients are pre-determined through meta-learning or grid search based on the validation set, or they can be dynamically adjusted: when the distribution difference exceeds a first threshold, its weight in the fusion index is increased to prioritize responding to structural drift; when the mean shift exceeds a second threshold but the distribution difference is small, the mean shift weight is increased to prioritize responding to scalar drift in the reward function. After weight normalization, a linear weighted sum is performed to output the preliminary fusion value.

[0055] Furthermore, a significance coding based on hypothesis testing is introduced. A two-sample kernel test is performed on the degree of distributional difference, and a two-sample t-test or Mann-Whitney U test is performed on the degree of mean shift. The p-values ​​obtained from the tests are transformed into significance scores through a negative logarithmic transformation, and multiplied by the weighted fusion value or used directly as the gating condition. A non-zero concept drift quantification index is output only when at least one test reaches statistical significance (e.g., p < 0.05); otherwise, a zero value or a minimum value is output to avoid false alarms caused by noise.

[0056] The final generated concept drift quantification index is a non-negative scalar, with its numerical range mapped to a fixed interval through normalization. This index encapsulates a comprehensive assessment of the environmental non-stationarity experienced by the local agent within the current detection period. Its value directly determines the probability of the agent being classified into the high-drift or low-drift group, and its feature coordinate position in the clustering analysis in step S24. The index value, along with its constituent components—the distribution difference sub-index, the mean shift sub-index, and their respective significance levels—is packaged into a detection report and uploaded to the central coordinator for subsequent collaborative adaptation group construction.

[0057] S24. Construction of co-adaptive groups based on cluster analysis; specifically including: S241. The central coordinator initiates the collection process of concept drift quantification indicators for all N geographically dispersed agents through a pre-established secure communication channel. The collection process employs an asynchronous aggregation protocol, allowing each agent to complete the calculation in step S23 locally and then upload the indicator data at different timestamps. The central coordinator maintains a dynamic receiving buffer and sets a maximum waiting time limit to ensure the overall progress of the detection cycle. For agents that do not respond within the time limit, the central coordinator can use exponential smoothing predictions of their historical indicator values ​​for imputation, or mark them as missing values ​​and handle them specially in subsequent clustering.

[0058] The uploaded indicator data includes the main indicator value and its metadata: the indicator generation timestamp, the distribution difference sub-indicators and mean offset sub-indicators that constitute the indicator, their respective statistical significance levels, and a digital signature or checksum used to verify the integrity of the indicator. The central coordinator performs integrity verification, eliminating abnormal data that has been corrupted during transmission or failed to verify the signature, ensuring that the indicator set entering subsequent processing is of reliable quality.

[0059] After the data collection is completed, the central coordinator constructs a global indicator matrix. The rows of the matrix correspond to the agents, and the columns correspond to the indicator attributes, such as the main indicator value, sub-indicator value, and significance marker, forming the structured input for processing in steps S242 to S243.

[0060] S242. The central coordinator performs standardization on all collected concept drift quantification indicators to eliminate incomparability bias caused by the basic heterogeneity of different agricultural production units. The selection of the standardization method is based on prior stratification knowledge of agro-ecological zones: if agents have been pre-classified according to crop type, soil climate zone, or management scale, stratified Z-score standardization is used, with the mean and standard deviation calculated independently within each stratum; if prior stratification is lacking, global standardization based on robust statistics is used, replacing the mean and standard deviation with the median and interquartile range to reduce the influence of outliers. The standardized indicator values ​​are zero-mean, unit-variance distributed, and possess direct numerical comparability across units.

[0061] Based on this, the central coordinator extracts the trend characteristics of each agent's concept drift quantification index within a recent continuous time window. This time window covers the most recent K drift detection cycles, where K is a preset hyperparameter, typically ranging from 5 to 20 cycles, forming the time series sequence of the agent's index. The calculation of the trend characteristics includes: First-order difference characteristics: Calculate the change in index values ​​between adjacent detection cycles. The mean of the sequence reflects the average drift velocity, and the standard deviation of the sequence reflects the volatility of the drift velocity. Second-order difference characteristics: Calculate the further difference of the first-order difference sequence to reflect the drift acceleration or deceleration and identify the turning point of environmental change from slow to rapid or from rapid to slow. Slope of sliding linear regression: A least-squares line fitted on a time series, the sign and magnitude of the slope reflect the long-term direction and intensity of the drift; Volatility index: Calculates the coefficient of variation or realized volatility of a time series to quantify the stability and predictability of the drift process; Trend reversal detection: By using change point detection algorithms, such as PELT or BinSeg, structural breakpoints in the indicator sequence are identified, and the abrupt changes in the drift pattern are marked.

[0062] The aforementioned trend features are concatenated with the standardized current indicator values ​​to form a feature vector for each agent, with a dimension of 1 plus the trend feature dimension. This feature vector fully characterizes the agent's current environmental state and its evolutionary dynamics.

[0063] S243. The central coordinator, based on the constructed feature vector set, uses a clustering algorithm to group all agents. The choice of clustering algorithm depends on the system's requirements for the group structure. If the goal is to form a well-balanced and clearly defined collaborative adaptation group, K-means or its variants can be used, such as K-means++ initialization or Mini-Batch K-means acceleration. The optimal number of clusters K can be adaptively determined by measuring the similarity between feature vectors based on Euclidean distance and using the elbow rule or silhouette coefficient method. If the goal is to identify dense clusters of arbitrary shapes and automatically mark noisy outliers, the DBSCAN algorithm is selected. The cluster boundary is defined based on the density reachability in the feature space, and the core parameters, neighborhood radius and minimum number of samples, are determined heuristically through the K-distance graph. If the goal is to construct a hierarchical nested group structure to support multi-granularity federated collaboration, such as first coarsely dividing by the degree of drift and then subdividing by the drift pattern, agglomerative hierarchical clustering or split hierarchical clustering algorithms can be selected to generate a tree-like dendrogram structure. Flat partitions of different levels can be extracted by cutting height thresholds.

[0064] The core similarity criterion for clustering is a dual constraint: feature vectors are close in terms of Euclidean or Mahalanobis distance, indicating that the agents' current concept drift quantification values ​​are similar; simultaneously, trend feature vectors are similar in terms of cosine similarity or dynamic time-warped distance, indicating that the agents experience consistent environmental evolution trajectories. This dual constraint ensures that agents in the same group not only face comparable levels of drift pressure but are also in comparable dynamic processes, thus achieving synergistic effects in policy updates during subsequent federated learning.

[0065] After clustering is completed, agents within each cluster are assigned temporary group labels, and outliers are marked as independent individuals or assigned to the nearest cluster, depending on the system's configuration strategy for group coverage completeness.

[0066] S244. The central coordinator formally defines and constructs each cluster generated by the clustering algorithm into an independent co-adaptive group. The group construction process includes the following operations: Unique Identifier Generation: A globally unique group identifier is assigned to each group. The identifier adopts a hierarchical encoding structure, including a timestamp field, a drift feature summary field, and a random salt value field, ensuring uniqueness and traceability.

[0067] Membership is solidified: Each agent within the cluster is registered with a unique identifier as a formal member of the group, creating a member list. This list is maintained in a distributed metadata store of the central coordinator, supporting high-concurrency reads and atomic updates.

[0068] Group-level metadata initialization: The central coordinator initializes group-level metadata records for each co-adaptive group, including: a list of group members and a summary of their geographical distribution, such as the geographic center coordinates and the maximum member spacing; intra-group statistical characteristics, including the average concept drift quantification index, index variance, and dominant trend pattern; the election results of the group representative agent, selecting the agent whose index is closest to Centroid within the group as a temporary coordinating node for initial gradient collection for intra-group aggregation; a group lifecycle state marker, initialized to the active state; and a group federated learning progress counter, initialized to zero.

[0069] Group Affiliation Notification: The central coordinator broadcasts a group affiliation notification message to all group member agents. The message includes the Group ID, a summary of the group member list, and a schedule for the expected federated continuous learning rounds. Upon receiving the message, agents update their local state, pause independent model updates, and await the triggering of the intra-group collaborative process.

[0070] Group structure persistence and version management: The central coordinator will persistently store the group structure snapshot formed in this detection cycle, record the group's creation time, dissolution conditions and historical evolution trajectory, and support the retrospective analysis of group-level decision-making patterns in the long-term ecological value reassessment in subsequent step S4.

[0071] At this point, step S24 is complete, and the system enters the progressive update phase of the strategy based on intra-group federated continuous learning, as described in step S3. Each collaborative adaptation group acts as an independent learning alliance, performing parameter sharing and knowledge fusion within the group, while maintaining policy separation between different groups, thus achieving a balance between differentiated adaptation and selective collaboration.

[0072] S3. Within the collaborative adaptation group, a consensus gradient is generated through federated learning to update the policy network of each agent; specifically including: S31. Initiation of the collaborative adaptive group-level federated learning process, specifically: For each co-adaptation group identified and constructed in step S24, the central coordinator independently initiates a round of federated learning for that group. This initiation operation features inter-group isolation, meaning that the federated learning processes of different co-adaptation groups are executed in parallel without interfering with each other, ensuring that each group can adaptively adjust its learning pace according to its number of members, drift urgency, and network conditions.

[0073] The startup process is triggered by the central coordinator broadcasting a federated learning startup command to all member agents within the group. The command includes: a unique round identifier for this federated learning session, a global learning rate setting, a local training iteration requirement, a gradient upload deadline window, and optional prior knowledge hints, such as a digest hash of the previous round's consensus gradient, used for version consistency verification. Upon receiving the command, the agent pauses its local model update operations in the autonomous decision-making loop, freezes the parameter states of the current policy network and the local value network, and prepares to enter the local training phase described in step S32.

[0074] The central coordinator maintains an independent federated learning state machine for each collaborative adaptation group, with states including: local training in progress, gradient collection in progress, consensus aggregation in progress, consensus distribution in progress, and current round completed. State transitions are event-driven, ensuring the orderly coordination of asynchronous operations among multiple agents within the group.

[0075] S32. Local training iterations and gradient calculations based on recent experience, specifically: S321. The data basis for local training is the local recurrent experience cache pool described in step S14. This pool retains the experience data from the most recent M decision cycles using a first-in-first-out (FIFO) rule. During training, the agent randomly samples small batches of experience data from this pool. The sampling strategy can be uniform random sampling or priority experience replay. The latter assigns sampling priority based on the temporal difference error magnitude of the experience data, so that the training focuses on key decision scenarios with high uncertainty in value estimation.

[0076] Training iterations are performed simultaneously for the policy network and the local value network, employing a reinforcement learning algorithm framework based on policy gradients. Specifically, a proximal policy optimization algorithm can be used: the local value network updates by minimizing the Bellman mean squared error to estimate the accuracy of state values; the policy network updates using a truncated alternative objective function, improving expected reward while limiting the magnitude of policy updates to ensure training stability. Alternatively, a soft actor-critic algorithm can be used, introducing an entropy regularization term to encourage policy exploration. The number of local training iterations is a preset hyperparameter, typically ranging from several to dozens of rounds, balancing local adaptation depth and global convergence efficiency.

[0077] S322. After each local training iteration, the agent calculates the change in the current network parameters relative to the initial parameters before the iteration. This change is the parameter update gradient for this iteration. After completing all local training iterations, the agent averages or weights the gradients from each iteration to form the final local policy gradient and local value gradient. This gradient represents the direction and magnitude of the agent's desired improvement to the current policy and value estimation based on its latest local experience data.

[0078] After the local gradient calculation is completed, the agent performs gradient compression encoding, using quantization and sparsity techniques to reduce upload bandwidth consumption, and adds gradient metadata, including the number of local training samples, the time coverage of empirical data, and gradient norm statistics, for subsequent aggregation weight calculation by the central coordinator.

[0079] S33. Group consensus gradient generation based on sparsity and difference weighting, specifically: S331. The central coordinator receives local policy gradients and local value gradients uploaded by all agents within the same cooperative adaptation group. The receiving process adopts a streaming processing architecture. Gradient data packets are parsed and verified immediately upon arrival, checking the integrity of digital signatures, the legitimacy of agent identities, and timestamps, and eliminating abnormal submissions that are too delayed or fail to be verified.

[0080] The validated gradients are temporarily stored in the gradient buffer of the central coordinator according to the agent's origin. The buffer maintains an independent storage slot for each agent, recording the policy gradient tensor and value gradient tensor uploaded by the agent. If there is a representative agent in the group elected in step S24, the gradient of that node can be assigned a priority processing channel, but its value does not participate in the privileged weighting of subsequent aggregation, and is only used for coordination process optimization.

[0081] S332. For each network parameter position corresponding to a gradient component, i.e., the weight or bias term of a neuron in a certain layer of the neural network, the central coordinator analyzes the set of gradient values ​​of all agents in the group on that component. Directional consistency is measured by calculating the cosine similarity or sign consistency ratio of the gradient directions: the gradient component values ​​of each agent are treated as vectors, the cosine similarity between each pair is calculated, and the average value is taken as the intra-group directional consistency score for that component; or the mode ratio of the gradient signs is statistically analyzed and used as the sign consistency score.

[0082] The central coordinator sets a directional consistency threshold, such as an average cosine similarity greater than 0.5 or a sign consistency ratio greater than 70%, and a gradient magnitude threshold, such as an absolute value greater than the 10th percentile of the historical update magnitude of that parameter. A gradient component is marked as significant only if it simultaneously satisfies both a directional consistency higher than the preset threshold and a gradient magnitude absolute value higher than the corresponding threshold. This dual-screening mechanism ensures that gradient components entering subsequent aggregation possess both a consensus basis within the group (i.e., most agents agree on the update direction) and actual update effectiveness (i.e., non-noise-level minor fluctuations).

[0083] Non-significant gradient components that fail the screening are marked as suppressed states, and their group consensus values ​​will be set to zero or minimum values ​​in subsequent steps to achieve gradient sparsity, reduce communication overhead, and suppress intra-group noise interference.

[0084] S333. Calculate an aggregate weight for each labeled significant gradient component. The weight design follows a two-factor positive correlation principle: it is positively correlated with the consistency of the gradient component's direction within the group; higher consistency indicates stronger consensus within the group for that update direction, and thus a larger weight. It is also positively correlated with the average amplitude within the group; a larger amplitude indicates a more urgent need for the update, and thus a larger weight.

[0085] The specific calculation can be performed using weighted harmonic average or neural network meta-learning prediction: the directional consistency score and normalized average magnitude are used as inputs and mapped to the final aggregate weights through a preset weight function; or a lightweight meta-network is trained to adaptively output the optimal weights based on feedback from historical aggregation effects.

[0086] Using the calculated aggregation weights, a weighted average is performed on the significant gradient components across all agents within the group. The weighted average formula is the sum of each agent's gradient value multiplied by its corresponding weight, divided by the total weights. The result is the group consensus gradient component at that parameter position. This component incorporates the common update requests of most agents within the group, while also reflecting the differences in contribution and reliability among different agents through differentiated weighting.

[0087] S334. The central coordinator traverses all parameter positions in the policy network and the local value network, performing the above filtering and aggregation operations for each position. For significant gradient components that pass the filtering, the calculated group consensus gradient component value is filled into the corresponding position; for insignificant gradient components that fail the filtering, their consensus value is set to zero, i.e., hard sparsity, or a very small default value, i.e., soft sparsity, retaining the weak update signal.

[0088] The consensus values ​​for all parameter positions are assembled according to the hierarchical structure and shape dimension of the original network parameters, restoring them to a complete tensor format. This ultimately forms the group consensus policy gradient and the group consensus value gradient, which together constitute the group consensus gradient set of this adaptive group.

[0089] The group consensus gradient is encoded and stored in a sparse tensor format, recording only the position index and value of non-zero elements, which significantly reduces the storage footprint of the central coordinator and the bandwidth required for subsequent distribution. The sparsity rate can typically reach 50% to 90%, depending on the homogeneity of the group environment and the quality of the training data.

[0090] S34. The distribution of the group consensus gradient is synchronized with the local model update, specifically: S341. The central coordinator distributes the generated group consensus gradient to all agents within the cooperative adaptation group. The distribution process is optimized using multicast or gossip protocols to ensure that group members receive consistent data versions within similar time windows. The gradient data packet includes a version stamp and integrity checksum; upon receipt, agents perform verification to detect transmission corruption or version mismatches.

[0091] S342. Each agent uses the received group consensus gradient to synchronously update the parameters of its local policy network and local value network. The update operation follows the gradient descent rule: the new parameter value equals the old parameter value minus the learning rate multiplied by the group consensus gradient value. The learning rate can be set globally or adaptively adjusted based on the convergence of the agent's local training.

[0092] S343. After the update, the agent's policy network and local value network parameters achieve intra-group co-evolution, retaining adaptive adjustments based on the latest local experience while incorporating knowledge transfer contributed by other group members through the consensus mechanism. The agent clears the temporary state of this round of federated learning, resumes the autonomous decision-making loop, and awaits the triggering of the next round of drift detection and federated learning.

[0093] S344. The central coordinator updates the federated learning progress counter for the collaborative adaptation group, recording statistics such as the sparsity rate and group consensus gradient norm of this round of aggregation, which are used as a reference for the long-term ecological value reassessment and meta-policy correction in step S4. At this point, step S3 is completed, and the system enters the next detection cycle or the ecological supervision layer operation in step S4.

[0094] S4. Based on the updated policy network, perform long-term ecosystem projection, calculate the long-term ecosystem value deficit, and generate a meta-policy correction signal to guide the calculation of subsequent group consensus gradients and mitigate long-term ecosystem risks; specifically including: S41. Maintenance and periodic updating of the regional ecological health status vector, specifically: S411. The system maintains a regional ecological health state vector, which serves as a long-term ecological monitoring benchmark beyond short-term agricultural production goals. This vector records slow-moving variables characterizing the resilience and sustainability of the agricultural ecosystem. The vector's dimensions are designed according to agricultural ecology principles and regional environmental characteristics, including but not limited to: The annual average change rate of soil organic matter reflects the dynamics of soil carbon pool and fertility maintenance capacity. It is calculated by aggregating annual monitoring data of soil organic matter content from soil profiles of various production units. Positive values ​​indicate carbon sequestration function, while negative values ​​indicate the risk of soil degradation. Groundwater balance ratio: defined as the ratio of annual groundwater recharge to extraction. It is estimated by combining hydrological monitoring well water level changes, irrigation water metering, and precipitation infiltration models. A ratio below 1 indicates an over-extraction deficit, which threatens long-term water resource security. Farmland biodiversity index: This index comprehensively represents indicators such as crop species diversity, non-crop vegetation coverage, soil microbial community richness, and pollinating insect population density. It is calculated by aggregating ecological sampling surveys and metagenomic sequencing data from each unit.

[0095] S412. The update frequency of this state vector is significantly lower than that of the production environment status collection in step S12, reflecting the monitoring characteristics of slow variables. The updated data comes from low-frequency sensing devices deployed in various agricultural production units and manual sampling inspections. Through local preprocessing at edge computing nodes and global aggregation by the central coordinator, a comprehensive assessment of ecological health at the regional scale is formed. The update cycle is usually set to quarterly or annually to ensure that the long-term evolution of the ecosystem is tracked without being drowned out by the noise of short-term agricultural fluctuations.

[0096] The current value and historical sequence of the regional ecological health status vector are stored in the ecological database of the central coordinator, which supports the initial state loading of the long-term ecological simulator in step S42 and the quantitative determination of the ecological deterioration trend in step S43.

[0097] S42. Policy Deduction and Dynamic Penalty Mechanism in a Long-Term Ecological Simulator: The updated policy network of all current agents is periodically fed into a long-term ecological simulator to perform future deductions spanning multiple planning cycles. This simulator, as a high-level form of digital twin system, integrates agricultural ecological process models, economic market models, and climate scenario data to evaluate the comprehensive performance of the current policy set over a long timescale; specifically: S421. The long-term ecological simulator acquires the latest environmental state snapshot of all agricultural production units at the current moment, including the production environment state vector mentioned in step S12, soil physicochemical parameters, crop physiological indicators, meteorological conditions, etc., as well as the historical agricultural operation trajectory accumulated in step S13; at the same time, it loads the regional ecological health state vector maintained in step S41 as the ecological baseline at the start of the simulation. The above data, after spatiotemporal alignment and format conversion, is mapped to the initial values ​​of the state variables inside the simulator, constituting the initial environmental state of the simulation.

[0098] S422. The simulator loads and maps the updated policy networks of all agents in the current system (as per step S3) onto the corresponding virtual production units in the simulation environment. Each virtual unit in the simulator has geographical coordinates, soil type, crop variety, and infrastructure configuration corresponding to the real unit. The policy networks are loaded with frozen parameters to ensure the stability of policy behavior patterns during the simulation and to avoid interference from adaptive learning within the simulator on the objectivity of the evaluation. All loaded policy networks together constitute the policy set to be evaluated, representing the current level of collaborative decision-making capability of the multi-agent system.

[0099] S423. The simulator starts from the initial environment state and runs multiple consecutive planning cycles. The cycle length is consistent with the decision cycle in step S12 or aggregated by year. In each simulation cycle, each virtual unit generates agricultural operation instructions based on the current simulation environment state according to the policy network it has loaded. The instruction types cover the entire decision space, including irrigation, fertilization, plant protection, and crop rotation.

[0100] The simulator calculates the system evolution after instruction execution based on a preset composite dynamics model: Soil process model: Simulates water transport, nutrient transformation, organic matter decomposition, and erosion processes; Hydrological model: Simulates groundwater recharge, crop evapotranspiration, and surface runoff; Crop growth model: simulating photosynthesis, dry matter accumulation, yield formation, and quality development; Market interaction model: Simulates fluctuations in agricultural product prices, changes in input costs, and the effects of policy subsidies.

[0101] The above-mentioned model runs in a coupled manner, outputting the environmental status update for the next cycle, the immediate economic returns, and the cumulative impact on various indicators of the regional ecological health status vector, such as changes in soil organic matter, groundwater level fluctuations, and biodiversity gains and losses.

[0102] The S424 simulator's reward calculation employs a phased dynamic mechanism, reflecting a trade-off between short-term economic goals and long-term ecological constraints: In the early stages of the simulation, the reward function remains consistent with the immediate reward in step S13, focusing on short-term economic performance indicators such as crop yield, resource utilization efficiency, and labor productivity. The calculation method is the same as the reward signal during the local training of each agent, ensuring the consistency between the simulator evaluation and the real training objective.

[0103] In the mid-to-late stages of the simulation, the simulator gradually introduces and amplifies the penalty term for the deterioration of the regional ecological health state vector. The penalty is introduced using a gradual strategy: in the mid-term, the penalty weight increases linearly or non-linearly from an initial zero value; in the late-term, the penalty weight reaches its peak. At this point, even if short-term economic returns are high, if accompanied by a continuous decline in soil organic matter, increased groundwater over-extraction, or a sharp reduction in biodiversity, the overall reward will be significantly negative. The penalty term is designed based on ecological threshold theory. When the state vector index crosses a key ecological boundary, such as when the groundwater balance ratio falls below 0.8 or the annual average change rate of soil organic matter falls below -2%, an additional accelerated penalty is triggered.

[0104] The immediate economic returns and ecological penalties across all planning periods are accumulated, discounted using a pre-defined discount factor, and summed to obtain the cumulative discounted return for long-term ecological constraints. This return value comprehensively represents the expected performance of the current strategy set while taking into account long-term ecological sustainability.

[0105] S43. Calculation and risk assessment of long-term ecological value deficit, specifically: S431. Through simulation, calculate the discounted cumulative return after long-term ecological constraints under the current strategy, and compare it with the discounted cumulative value of short-term training returns to obtain the long-term ecological value deficit.

[0106] The discounted cumulative value of short-term training rewards is obtained through backtracking calculation: based on the real-time reward sequence actually obtained by each agent in the local training in step S32, the discounted cumulative reward of each agent is calculated using the same discount factor as the long-term simulator, and then the short-term training reward estimate at the system level is obtained by averaging or weighted aggregation within the group.

[0107] The long-term ecological value deficit is defined as the difference between the discounted cumulative value of short-term training rewards and the discounted cumulative value of long-term ecological constraints. This difference reveals the long-term ecological value that current multi-agent systems neglect or overdraw in their pursuit of short-term economic optimization. If the deficit is zero or negative, it indicates that the current strategy has internalized ecological costs and possesses long-term sustainability; if the deficit is positive, it indicates that ecological externalities are not fully internalized.

[0108] S432. The central coordinator sets a long-term ecological value deficit threshold, such as 20% of the long-term simulated discounted value of short-term returns. When the calculated deficit exceeds this preset threshold, the system determines that there is a long-term ecological risk in the current co-evolutionary direction, triggering the meta-policy correction process in step S44. The determination result and the deficit value are recorded in the ecological monitoring log for subsequent pattern analysis and correction signal generation.

[0109] S44. Generation and Gradient Aggregation Injection of Meta-Policy Correction Signal: When a long-term ecological risk is determined, the system analyzes the key decision patterns leading to ecological degradation in the long-term ecological simulator and generates a meta-policy correction signal. This meta-policy correction signal is then injected into the gradient aggregation step in step S33 to adjust the gradient components related to harmful patterns. Specifically: S441. When the long-term ecological value deficit exceeds a threshold, the system analyzes the complete extrapolation process of the long-term ecological simulator, tracing the spatiotemporal trajectory of significant deterioration of key indicators in the regional ecological health state vector. Through causal inference algorithms, such as Granger causality tests, structural equation modeling, or counterfactual analysis based on attention mechanisms, agricultural operation decision sequences that frequently co-occur with and are statistically highly correlated with the deterioration trajectory are identified.

[0110] Typical forms of harmful decision-making patterns include: continuous aquifer depletion due to high-intensity groundwater irrigation over multiple cycles; accelerated soil organic matter mineralization due to monoculture combined with high-volume fertilizer input; and soil structure destruction and biodiversity habitat loss due to excessive mechanized farming. The system defines such patterns as harmful decision-making patterns, records their characteristic descriptions, including the types of decisions involved, temporal combinations, intensity thresholds, and the frequency and magnitude of their ecological impact in the simulation.

[0111] S442. Based on the identified harmful decision patterns, the system back-maps them to the parameter space or decision feature space of the agent's policy network, generating meta-policy correction signals. The back-mapping process employs a gradient-based attribution analysis method: in a long-term ecological simulator, sensitivity analysis is performed on the inputs of the virtual unit policy network that lead to harmful patterns, calculating the gradient contribution of each input dimension to the harmful output; further, through backpropagation of the policy network, the system traces back to the network parameter layer, identifying a subset of parameters that have a strong activation effect on harmful decisions.

[0112] The meta-policy correction signal is specifically represented as a correction coefficient vector with the same dimension as the policy network parameters. The vector elements are assigned the following values: for parameters highly correlated with harmful decision-making patterns, such as the weight of the response to a low groundwater level threshold in the irrigation decision output layer, an inhibitory correction coefficient is assigned, such as a decay factor of 0.1 to 0.5, or a negative inverse coefficient; for other parameter positions, a neutral coefficient is assigned, such as a preservation factor of 1.0. The generation of the correction coefficient vector can consider the structural correlation between parameters, such as using layer-wise or block-wise sparse correction, to avoid over-interfering with the overall expressive power of the policy.

[0113] S443. When executing step S33 in the next training cycle, the central coordinator injects the generated meta-policy correction signal into the generation process of the group consensus gradient. The injection point is located after each agent's local gradient upload and before sparsification and difference-weighted aggregation.

[0114] The specific injection method is as follows: The central coordinator performs element-wise multiplication (Hadamard product) of the correction coefficient vector with the local policy gradient uploaded by each agent. This operation pre-adjusts the local gradient, suppressing the parameter update direction associated with harmful decision patterns, weakening the update magnitude when the coefficient is less than 1, or reversing the update direction when the coefficient is negative. Thus, without changing the local experience data and training process of each agent, it implicitly guides the policy network away from decision regions identified as unsustainable in the long term.

[0115] For local value gradients, the same correction coefficient vector can be selectively applied, or an independent correction strategy can be adopted to ensure consistent adjustment between value estimation and policy behavior.

[0116] S444. The central coordinator, based on the adjusted local policy gradients of each agent according to the policy correction signal, executes the original sparsification and difference-weighted aggregation algorithm from steps S332 to S334. Directional consistency screening, saliency determination, and weighted averaging operations are all performed in the corrected gradient space, ensuring that the final aggregated group consensus gradient implicitly contains guidance for long-term ecological sustainability.

[0117] The generated group consensus gradient is distributed to all agents within the group through step S34, driving the synchronous update of the policy network. Due to the persistence of the correction signal, it can be continuously applied in multiple rounds of federated learning until the long-term ecological value deficit falls below the threshold. The co-evolutionary direction of the agents within the group is gradually guided to a more ecologically resilient decision space, achieving Pareto improvement in both short-term economic performance and long-term ecological health.

[0118] The central coordinator records the application rounds, correction intensity, and subsequent long-term ecological value deficit of the meta-strategy correction signal, forming a closed-loop feedback loop. This feedback loop is used to optimize the attribution analysis algorithm and correction coefficient generation strategy in step S442, thereby improving the adaptability and intervention accuracy of the ecological monitoring layer. Example 2

[0119] like Figure 2 As shown, an agricultural production multi-objective decision optimization system based on multi-agent reinforcement learning is used to implement the agricultural production multi-objective decision optimization method based on multi-agent reinforcement learning, including: An environment interaction and experience caching module, deployed in each agricultural production unit, is used to generate and execute agricultural operation instructions based on the local production environment, and to store the experience data generated from the decisions in a local circular experience cache pool; specifically including: Local state sensing unit, used to collect the state of the production environment; The strategy network unit is used to receive status and generate agricultural operation instructions; The instruction execution unit is used to execute operation instructions; The experience encapsulation and storage unit is used to encapsulate states, operations, rewards, and the next state into experience data and store them in a local circular experience cache pool managed by a first-in-first-out rule.

[0120] The concept drift collaborative detection module connects with various environment interaction and experience caching modules. It periodically samples data from each cache pool, calculates the concept drift quantification index for each unit through a shared drift evaluation network, and performs cluster analysis based on this index to dynamically form collaborative adaptation groups. Specifically, it includes: A dual-period data sampling unit is used to sample data from the local recurrent experience cache pool and the historical benchmark experience pool, respectively. The drift evaluation network unit, shared by all agents, is used to extract features of state-action pairs and evaluate their value. The indicator calculation unit is used to compare and analyze the characteristic distribution and value assessment of two sets of data, and generate a quantitative indicator of concept drift. The central clustering coordination unit is used to collect all indicators and form co-adaptive groups through cluster analysis.

[0121] The federated policy collaborative update module, connected to the concept drift collaborative detection module, is used to initiate a federated learning process within each collaborative adaptation group. It aggregates the model gradients of each unit within the group, generates a group consensus gradient through sparsification and difference-weighted algorithms, and synchronously updates the decision models of all agents within the group accordingly. Specifically, it includes: Local training units, deployed locally on each agent, are used to iteratively train the local model using recent empirical data and to calculate gradients. The gradient aggregation center unit is used to receive gradients uploaded by all agents in the cooperative adaptation group, run sparsification and difference weighted aggregation algorithms, filter out gradient components with consistent and significant directions, and generate group consensus gradients by weighting. The model synchronization update unit is used to distribute the group consensus gradient to all agents in the group and complete the parameter update of their local models.

[0122] The long-term ecological assessment and meta-correction module, connected to the federated strategy collaborative update module, is used to maintain the regional ecological health state vector. It uses a long-term ecological simulator to extrapolate the updated decision-making model, calculates the long-term ecological value deficit, and generates a meta-strategy correction signal when ecological risks are identified. This signal is fed back to the federated strategy collaborative update module to guide the direction of strategy updates. Specifically, it includes: The ecological status maintenance unit is used to aggregate low-frequency sensor data and periodically update the regional ecological health status vector. The long-term ecological simulator unit is used to load the current decision-making model and ecological state, perform multi-period simulations, and introduce ecological penalties in the later stages of the simulation to calculate the long-term ecological constraint discounted cumulative return. The risk quantification unit is used to compare long-term and short-term returns, calculate the long-term ecological value deficit, and determine the risk. The meta-signal generation and injection unit is used to analyze the deduction process, identify harmful decision-making patterns, and generate meta-policy correction signals when a risk is determined to exist. These signals are then fed back to the gradient aggregation center unit of the federated policy collaborative update module to adjust the generation of consensus gradients.

[0123] As can be seen from the above description, the embodiments of the present invention achieve the following technical effects: This invention achieves efficient and precise dynamic collaborative optimization of large-scale heterogeneous production units, effectively overcoming the combined challenges of scale heterogeneity and decentralized decision-making collaboration. Through a design that dynamically forms collaborative groups, the system automatically clusters agents with similar change patterns into collaborative adaptation groups based on real-time concept drift quantification indicators. This not only respects individual differences among units but also constructs precise collaborative units. Building upon this, gradient aggregation and consensus generation, through a unique sparsity and difference-weighted aggregation algorithm, extract group consensus gradients representing the common evolutionary direction within each group, driving agents within the group to update synchronously in a stable and consistent manner. This dynamic grouping + intra-group consensus evolution mechanism achieves a leap from a one-size-fits-all approach or no collaboration to adaptive and precise collaboration, significantly improving the allocation efficiency and overall effectiveness of global resources while ensuring personalized decision-making.

[0124] This invention overcomes the limitations of traditional multi-objective optimization in its insensitivity to long-term, implicit ecological risks, achieving endogenous assurance and forward-looking shaping of long-term decision-making sustainability. It introduces long-term policy extrapolation, using a long-term ecological simulator to conduct cross-cycle stress tests on the current policy and dynamically applying ecological penalties in the later stages of the simulation. This quantifies imperceptible long-term, implicit risks into explicit long-term ecological value deficits. More importantly, through meta-policy correction, the system can transform identified ecological risks into meta-policy correction signals and inject them back into the source of policy updates—the gradient aggregation process. This mechanism acts like an ethical compass for the system's evolutionary algorithm, implicitly and continuously guiding the co-evolutionary direction of all agents without interrupting short-term learning or modifying the original reward function. It proactively avoids harmful decision-making patterns, fundamentally ensuring the long-term unity of economic benefits and ecological sustainability.

[0125] A complete autonomous closed loop of perception, adaptation, auditing, and correction is constructed, enabling the system to possess continuous robustness and strategic adaptability in the face of environmental non-stationarity. This invention endows the system with higher-order intelligence through the closed-loop linkage of four core steps. Step S2 (detection) and step S3 (adaptation) constitute the inner loop for rapid response to environmental changes, while step S4 (auditing and correction) constitutes the outer loop for evaluating and correcting the evolutionary direction. In particular, the meta-policy correction signal generated in step S44 directly intervenes in the inner loop step S33, forming a reinforced feedback loop of decision-making behavior → long-term evaluation → correcting learning rules → influencing future decisions. This allows the system not only to passively adapt to changes but also to actively assess the long-term consequences of its collective behavior and adjust its learning rules accordingly. This maintains the long-term effectiveness of decision-making and the health of the evolutionary direction in complex, non-stationary agricultural environments, significantly improving the system's overall robustness and strategic-level adaptability.

[0126] The embodiments and / or implementation methods described above are merely preferred embodiments and / or implementation methods for implementing the technology of the present invention, and are not intended to limit the implementation methods of the technology of the present invention in any way. Any person skilled in the art may make some modifications or alterations to other equivalent embodiments without departing from the scope of the technical means disclosed in the present invention, but these should still be regarded as the technology or embodiments that are substantially the same as the present invention.

[0127] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are only preferred embodiments of this application. It should be noted that due to the limitations of written expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. A multi-objective decision optimization method for agricultural production based on multi-agent reinforcement learning, characterized in that, include: Each local agent executes a decision and stores the experience in a local recurrent experience cache pool; By comparing recent and historical data from each of the local recurrent experience cache pools, a concept drift quantification index is generated, and co-adaptive groups are formed based on the index. Within the collaborative adaptation group, a group consensus gradient is generated through federated learning to update the policy network of each agent; Based on the updated policy network, a long-term ecological extrapolation is performed to calculate the long-term ecological value deficit and generate a meta-policy correction signal to guide the calculation of subsequent group consensus gradients.

2. The multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production according to claim 1, characterized in that, Each local agent executes decisions and stores the experience in a local recurrent experience cache pool, specifically including: In the multi-agent system, a local decision-making agent is independently deployed for each geographically dispersed agricultural production unit; each local decision-making agent includes a policy network for generating operational instructions and a local value network for evaluating state value. In each decision cycle, each agent collects the current production environment status of its corresponding unit and inputs the status into its own policy network. The policy network then outputs a specific agricultural operation instruction. Each intelligent agent executes the agricultural operation instructions it generates. After receiving the next state and immediate reward from the environmental feedback, it encapsulates the current state, the operation executed, the reward obtained, and the next state into a piece of experience data. The encapsulated experience data is stored in the local loop experience cache pool corresponding to the agent; the local loop experience cache pool manages the data according to the first-in-first-out rule, and only retains the experience data within the most recent decision cycles.

3. The multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production according to claim 2, characterized in that, By comparing recent and historical data from each of the aforementioned local recurrent experience cache pools, a concept drift quantification index is generated, and these indexes are clustered to form co-adaptation groups, specifically including: Each agent randomly samples a certain amount of the latest experience data from its local recurrent experience cache pool to form a recent experience dataset; at the same time, it randomly samples an equal amount of historical experience data from a long-term stored historical benchmark experience pool to form a benchmark experience dataset. The state-operation pairs in the recent experience dataset and the benchmark experience dataset are respectively input into a drift evaluation network shared by all agents. The drift evaluation network outputs the latent feature representation of the state-operation pairs and their corresponding expected value evaluation. Calculate the statistical difference between the latent feature distributions of the recent experience dataset and the benchmark experience dataset, and calculate the mean deviation of their expected value assessments; fuse the difference and mean deviation to generate a concept drift quantification index for the agent. The central coordinator collects the concept drift quantification indicators of all agents and performs cluster analysis based on the value and trend of the indicators. Agents with similar indicators and similar trends are grouped into the same set, and each set forms a cooperative adaptation group.

4. The method and system for multi-objective decision optimization in agricultural production based on multi-agent reinforcement learning according to claim 3, characterized in that, The central coordinator collects the concept drift quantification indicators from all agents and performs cluster analysis based on the indicator values ​​and trends. Agents with similar indicators and trends are grouped into the same set, and each set forms a cooperative adaptation group, specifically including: The central coordinator collects concept drift quantification metrics uploaded by all agents. The central coordinator standardizes all collected indicators and extracts the trend characteristics of each indicator within a recent continuous time window. Based on the standardized index values ​​and their changing trend characteristics, the central coordinator uses a clustering algorithm to group all agents; agents whose index values ​​are close in distance in the feature space and whose changing trend characteristics are similar are grouped into the same cluster; The central coordinator formally defines and constructs each cluster generated by clustering into an independent co-adaptation group, and assigns a unique identifier to each group to coordinate the subsequent intra-group federated learning process.

5. The multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production according to claim 4, characterized in that, Within the collaborative adaptation group, a group consensus gradient is generated through federated learning to update the policy network of each agent, specifically including: For each co-adaptation group, the central coordinator initiates a round of federated learning for each group; Each agent in the group uses recent data in its local recurrent experience cache pool as the training set, performs multiple local training iterations on its own policy network and local value network, and calculates the parameter update gradient of its respective network. Each agent uploads the calculated parameter update gradient to the central coordinator. The central coordinator selects gradient components that are consistent in direction and change significantly among most agents in the group, and aggregates them to generate a unified group consensus gradient. The central coordinator distributes the group consensus gradient to all agents within the cooperative adaptation group. Each agent uses the received group consensus gradient to synchronously update the parameters of its local policy network and local value network.

6. The multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production according to claim 5, characterized in that, The central coordinator filters out gradient components that are consistent in direction and show significant changes among most agents in the group, and aggregates them to generate a unified group consensus gradient, specifically including: The central coordinator receives local policy gradients and local value gradients uploaded by all agents within the same cooperative adaptation group, and temporarily stores the gradients according to the agent's source. For each network parameter position corresponding to the gradient component, the coordinator analyzes the gradient values ​​of all agents in the group on the component; calculates the directional consistency measure of the gradient component in the group, and filters out gradient components whose directional consistency is higher than a preset threshold and whose absolute gradient magnitude is also higher than the corresponding threshold, and marks them as significant gradient components. For each labeled significant gradient component, an aggregate weight is calculated. The weight is positively correlated with the direction of the gradient component within the group and with its average magnitude. Using the calculated weight, the values ​​of the significant gradient components in each agent within the group are weighted and averaged. The result is the group consensus gradient component at the parameter position. Iterate through all network parameter positions, calculate consensus values ​​for the selected significant gradient components, and set the consensus values ​​of non-significant gradient components that fail the screening to zero or a very small default value; assemble the consensus values ​​of all positions according to the original network parameter structure to finally form the complete group consensus strategy gradient and group consensus value gradient.

7. The multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production according to claim 6, characterized in that, Based on the updated policy network, long-term ecological projection is performed, the long-term ecological value deficit is calculated, and a meta-policy correction signal is generated to guide the calculation of subsequent group consensus gradients. Specifically, this includes: The system maintains a regional ecological health status vector to record slow-moving variables, including changes in soil organic matter, groundwater balance, and biodiversity; the vector is updated periodically by aggregating low-frequency monitoring data from various production units. The updated policy network of all current agents is periodically fed into a long-term ecological simulator. The long-term ecological simulator starts from the current environmental state and simulates multiple future planning cycles, and gradually increases the penalty for the deterioration of the regional ecological health state vector in the later stage of the simulation. Through simulation, the discounted cumulative return under the current strategy and long-term ecological constraints is calculated and compared with the discounted cumulative value of the short-term training return to obtain the long-term ecological value deficit; if the deficit exceeds the preset threshold, it is determined that there is a long-term ecological risk. When a risk is identified, the system analyzes the key decision-making patterns that lead to ecological degradation in the simulation and generates a meta-policy correction signal. The meta-policy correction is then injected into the gradient aggregation stage to adjust the gradient components related to harmful patterns.

8. The multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production according to claim 7, characterized in that, The long-term ecological simulator starts from the current environmental state and simulates multiple future planning cycles. In the later stages of the simulation, the penalty for the deterioration of the regional ecological health state vector is gradually increased, specifically including: The long-term ecological simulator obtains the latest environmental state snapshot of all agricultural production units at the current moment, and loads the regional ecological health state vector as the initial environmental state for inference. The simulator loads and maps the updated policy network of all agents in the current system onto the corresponding virtual production units in the simulation environment, forming a set of policies to be evaluated. Starting from the initial environmental state, the simulator runs multiple consecutive planning cycles. In each cycle, each virtual unit generates operation instructions according to the strategy network it loads. Based on the preset soil, hydrology, crop growth and market interaction dynamics model, the simulator calculates the environmental state changes, immediate economic returns and cumulative impact on each indicator of the regional ecological health state vector after the instructions are executed. In the early stages of the simulation, the simulator calculates rewards in the same way as in short-term training, focusing on immediate goals such as output and resource efficiency. In the middle and later stages of the simulation, the simulator gradually introduces and increases the penalty term for any deterioration in any dimension of the regional ecological health state vector. The cumulative rewards and penalties after discounting over all cycles are finally obtained to obtain the cumulative return of long-term ecological constraints.

9. The multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production according to claim 8, characterized in that, When a risk is determined to exist, the system analyzes the key decision-making patterns leading to ecological degradation in the simulation and generates a meta-policy correction signal. This meta-policy correction signal is then injected into the gradient aggregation stage to adjust the gradient components related to harmful patterns. Specifically, this includes: When the long-term ecological value deficit exceeds the threshold, the system analyzes the simulation process of the long-term ecological simulator, identifies one or more high-frequency and highly correlated agricultural operation decision sequences that cause a significant deterioration of at least one key indicator in the regional ecological health state vector, and defines them as harmful decision patterns. The system back-maps the identified harmful decision patterns to the parameter space or decision feature space of the agent policy network to generate a meta-policy correction signal. The meta-policy correction signal is specifically represented as a correction coefficient vector with the same dimension as the network parameters, wherein the parameter positions that are highly correlated with the harmful decision patterns are given inhibitory correction coefficients, and the remaining positions are given neutral coefficients. During the next training cycle, the central coordinator injects the meta-policy correction signal into the generation process of the group consensus gradient; Based on the adjusted local policy gradients of each agent, the coordinator executes the original sparsification and differential weighted aggregation algorithm to generate and distribute a group consensus gradient that implicitly contains guidance on long-term ecological sustainability, so as to guide the synchronous update of the policy network of all agents in the group.

10. A multi-objective decision-making optimization system for agricultural production based on multi-agent reinforcement learning, characterized in that, The method for implementing the multi-agent reinforcement learning-based multi-objective decision optimization method for agricultural production as described in any one of claims 1-9 includes: The environmental interaction and experience caching module is deployed in each agricultural production unit. It is used to generate and execute agricultural operation instructions based on the local production environment status, and store the experience data generated by the decision in a local circulating experience caching pool. The concept drift collaborative detection module is connected to each of the aforementioned environment interaction and experience caching modules. It is used to periodically sample data from each cache pool, calculate the concept drift quantification index of each unit through a shared drift evaluation network, and perform cluster analysis based on this to dynamically form collaborative adaptation groups. The federated policy collaborative update module, connected to the concept drift collaborative detection module, is used to initiate a federated learning process within each collaborative adaptation group, aggregate the model gradients of each unit within the group, generate a group consensus gradient through sparsification and differential weighting algorithms, and synchronously update the decision models of all agents within the group accordingly. The long-term ecological assessment and meta-correction module, connected to the federated strategy collaborative update module, is used to maintain the regional ecological health state vector, use a long-term ecological simulator to extrapolate the updated decision model, calculate the long-term ecological value deficit, and generate a meta-strategy correction signal when ecological risks are identified, which is fed back to the federated strategy collaborative update module to guide the direction of strategy updates.

Images