Ecological environment impact assessment method based on multi-agent reinforcement learning

Abstract

The application discloses an ecological environment impact assessment method based on multi-agent reinforcement learning, and belongs to the technical field of ecological environment impact assessment. The method comprises the following steps: acquiring multi-source heterogeneous ecological monitoring data in a target region, and fusing the data into an ecological state tensor under a unified space-time reference; constructing a Markov game model for ecological environment impact assessment based on the ecological state tensor, and setting a joint reward function; assigning an agent to each ecological impact factor in the model, outputting a decentralized execution strategy of each agent through a centralized training process of all agents in a joint action space; determining an evaluation action sequence of each ecological impact factor in a continuous evaluation period according to the decentralized execution strategy; and calculating an ecological environment impact comprehensive index of the target region according to the evaluation action sequence. The method can dynamically depict the interactive game process among multiple ecological impact factors, adaptively generate a continuous evaluation sequence, and realize quantitative evaluation of cumulative ecological impact.

Description

Technical Field

[0001] This invention relates to the field of ecological and environmental impact assessment technology, specifically to an ecological and environmental impact assessment method based on multi-agent reinforcement learning. Background Technology

[0002] Ecological and environmental impact assessment involves the quantitative analysis of the combined impacts of various natural factors and anthropogenic disturbances. Current technologies for the collaborative assessment of multiple ecological impact factors in a target area typically employ methods such as multi-index weighted comprehensive index methods, fuzzy comprehensive evaluation models, or system dynamics simulations.

[0003] These methods rely on pre-defined static weights and linear superposition rules, treating the impact of each ecological factor as an independent and additive process. In implementation, these techniques typically perform one-time calculations based on single observations or discrete-time cross-sectional data, generating a single static assessment result. However, complex, non-independent interactions and dynamic temporal games exist among the various ecological impact factors. Regulation or change in one factor can trigger responses in others, causing the ecological environment to evolve along unpredictable trajectories. Existing static assessment paradigms cannot characterize this dynamic game pattern of simultaneous multi-factor decision-making and mutual influence, and the output assessment results fail to reflect the cumulative changes in the ecological environment over continuous time series. Furthermore, traditional assessment frameworks lack adaptive extrapolation capabilities, struggling to automatically generate assessment action sequences that fit the actual dynamics when faced with high-dimensional spatiotemporal data, leading to discrepancies between assessment results and actual ecological responses. Additionally, regarding the utilization rate of ecological monitoring data, most existing methods process multi-source data separately before synthesizing the results, failing to construct a tensor representation of multi-variable collaborative assessments directly using heterogeneous multi-source data under a unified spatiotemporal benchmark. This further restricts the precise quantification of the interactive effects of multiple factors.

[0004] To address the problem that the dynamic evolution of the ecological environment is difficult to quantify due to the interaction and game among multiple ecological impact factors, and the shortcomings of traditional methods that cannot adaptively generate continuous assessment action sequences resulting in static and discrete assessment results, a method is needed that can dynamically deduce the game behavior of each ecological impact factor in a continuous time dimension and calculate the comprehensive impact index accordingly. Summary of the Invention

[0005] The purpose of this invention is to provide an ecological and environmental impact assessment method based on multi-agent reinforcement learning, in order to solve the problems that existing assessment methods cannot characterize the dynamic game process of multiple ecological impact factors, cannot adaptively generate continuous assessment action sequences, and cannot cumulatively quantify the comprehensive ecological and environmental impact.

[0006] To achieve the above objectives, this invention provides the following technical solution: This invention offers a method for assessing ecological and environmental impacts based on multi-agent reinforcement learning. This method integrates multi-source heterogeneous ecological monitoring data into an ecological state tensor under a unified spatiotemporal benchmark. Based on this, a Markov game model for ecological and environmental impact assessment is constructed, and a joint reward function is defined. Then, an agent is assigned to each ecological impact factor. Decentralized execution strategies for each agent are obtained through centralized training. The assessment action sequence for each ecological impact factor within a continuous assessment period is determined based on the decentralized execution strategy, and the comprehensive ecological and environmental impact index of the target area is calculated based on the assessment action sequence. This method can solve the problems of traditional assessment methods being unable to effectively handle multi-source heterogeneous data and unable to characterize the dynamic interactive game relationships between ecological impact factors. It achieves quantitative assessment and dynamic extrapolation of ecological and environmental impacts, significantly improving the objectivity and foresight of the assessment results.

[0007] As a preferred technical solution of this invention, when acquiring multi-source heterogeneous ecological monitoring data within the target area and fusing it into an ecological state tensor under a unified spatiotemporal reference, ground ecological sampling data, remote sensing inversion data, and real-time dynamic sensing data are collected from ground monitoring stations, remote sensing satellites, and unmanned inspection terminals deployed within the target area. The three types of data are then time-stamp aligned and spatially registered to obtain spatiotemporally synchronized ecological data. This data is then dimensionally encoded according to ecological element categories to generate an ecological state tensor covering all ecological elements under a unified spatiotemporal reference. By fusing multi-source data and unifying the spatiotemporal reference, spatiotemporal biases caused by different observation methods are effectively eliminated, providing a highly consistent and accurate data foundation for subsequent game theory modeling.

[0008] As another preferred technical solution of the present invention, when constructing a Markov game model based on the ecological state tensor, each time-space point in the ecological state tensor is taken as a state node in the state space, and the ecological evolution probability between adjacent state nodes is defined as the state transition probability. The action space of each agent is constructed according to the optional control operations of each ecological influencing factor, and the Cartesian product of the action spaces of all agents is taken as the joint action space. For each joint action in the joint action space, a joint reward function is set. This joint reward function includes a positive contribution sub-function and a negative perturbation sub-function. The positive contribution sub-function calculates its output based on the predicted ecological restoration rate and biodiversity increment after executing the joint action, and the negative perturbation sub-function calculates its output based on the predicted water pollution diffusion distance and soil degradation area after executing the joint action. The weighted difference between the output values ​​of the positive contribution sub-function and the negative perturbation sub-function is calculated, and the result is taken as the reward value corresponding to the joint action. This setting allows the joint reward function to accurately reflect the overall positive contribution and negative perturbation of each joint action to the ecological environment, guiding multiple agents to co-evolve towards the optimal global ecological benefits during training.

[0009] As a further preferred embodiment of the present invention, after assigning an agent to each ecological impact factor, a centralized training and decentralized execution architecture is adopted. A policy network and an evaluation network are initialized for each agent. The input to the evaluation network includes joint observation information and joint action information from all agents. During centralized training, the local observation information of each agent is input into its respective policy network to obtain a local action probability distribution. All local action probability distributions are combined to obtain a joint action probability distribution. After sampling joint action samples, the joint action samples and the current joint state are input into each agent's evaluation network to obtain an action value estimate. The dominance function is used to subtract the state value baseline from the action value estimate to obtain the dominance estimate. This estimate is multiplied by the corresponding policy log probability to obtain the policy gradient estimate. After accumulating and averaging, the policy network parameters are updated using gradient ascent. Simultaneously, joint experience samples are randomly sampled from the joint experience pool. The time difference error of the evaluation network is calculated using the reward signal and the next state information, and the evaluation network parameters are updated. This update is repeated until the policy network converges. The network parameters are then frozen, and the policy network is switched to inference mode, configured to only receive local observation information from the corresponding agent and output a single action decision, which is then deployed as a decentralized execution policy. Centralized training utilizes global information for policy learning, ensuring policy quality; decentralized execution relies only on local observations, significantly reducing communication and computational overhead during policy deployment and runtime, enabling independent decision-making by various ecological influencing factors to respond quickly to environmental changes.

[0010] As another preferred technical solution of the present invention, when determining the evaluation action sequence of each ecological impact factor within a continuous evaluation period, the initial ecological state tensor of the current evaluation period is input into the decentralized execution strategy of each agent. Each agent independently outputs the first evaluation action of its respective ecological impact factor within the current period. The first evaluation actions are jointly applied to the initial ecological state tensor to obtain a first intermediate ecological state tensor. The first intermediate ecological state tensor is then input into each decentralized execution strategy to obtain the second evaluation action. This iterative execution of state transition and action generation process continues until a preset step limit is reached. The evaluation actions generated by each ecological impact factor at all steps are arranged in chronological order to obtain the evaluation action sequence. This iterative deduction method enables the prediction of the dynamic evolution of the ecological environment at continuous time nodes, can simulate the continuous effects of different impact factors, and provides managers with a complete dynamic impact chain.

[0011] As another preferred technical solution of the present invention, when calculating the comprehensive ecological environment impact index, each step of the assessment action sequence of each ecological impact factor is mapped to a corresponding ecological disturbance vector, and the ecological disturbance vectors of all ecological impact factors at the same time are superimposed to form a total disturbance vector; the initial ecological state tensor of the target area in the current assessment period is obtained, and the initial ecological state tensor is superimposed point by point with the total disturbance vector of each step to obtain the updated ecological state tensor of each step. Pre-set key ecological indicator values ​​are extracted from each updated ecological state tensor, and each key ecological indicator value is compared with the corresponding reference baseline value to obtain a single impact score. The single impact scores are then weighted and accumulated according to the number of assessment steps, and the dimensions of each ecological impact factor are aggregated to obtain the comprehensive ecological environment impact index of the target area. The resulting comprehensive index fully integrates the ecological state change information under the interaction of multiple factors, and can accurately quantify the comprehensive impact on the ecological environment from multiple dimensions, resulting in a more comprehensive and reliable evaluation result.

[0012] This invention, through the combination of the above-mentioned technical solutions, achieves efficient fusion of multi-source heterogeneous ecological monitoring data and dynamic game modeling of ecological impact factors. It utilizes multi-agent reinforcement learning to conduct centralized training and decentralized execution of the assessment process, overcoming the limitations of static assessment, single factor analysis and data fragmentation in existing technologies. It can truly reflect the linkage and synergistic effects of multiple factors in the spatiotemporal dimension, providing an intelligent, dynamic and high-precision decision support tool for ecological and environmental impact assessment.

[0013] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0014] By fusing multi-source heterogeneous ecological monitoring data into an ecological state tensor under a unified spatiotemporal benchmark, and constructing a Markov game model on this basis, an independent agent is assigned to each ecological impact factor. During centralized training, all agents share joint observation and joint action information to update the evaluation network, enabling each agent to perceive the global feedback of other factors' behavior and the overall environmental state, capturing the complex game relationships and chain responses among multiple factors. The decentralized execution strategy output after training convergence allows each agent to independently output action decisions for the corresponding ecological impact factor during the deployment phase, relying only on its own local observation information. This distributed, independent execution method eliminates the dependence on centralized coordination. Agents for different ecological impact factors can make differentiated responses based on the specific ecological state at their respective spatiotemporal locations, realistically reproducing the dynamic game characteristics of multiple factors simultaneously making autonomous decisions and mutually restraining each other in actual ecosystems. This scheme differs from conventional fixed-weighted superposition assessments, adaptively adjusting the impact assessment behavior of each factor as the ecological state evolves, making the assessment process highly consistent with the multi-agent non-cooperative game characteristics of the actual ecological environment.

[0015] This method utilizes a decentralized execution strategy to progressively generate evaluation action sequences for each ecological impact factor within a continuous evaluation cycle through a recursive approach. At each step, the evaluation actions independently output by each agent are jointly applied to the current ecological state tensor, driving a state transition to an intermediate ecological state. The updated state is then used as input to the policy network for the next time step, iterating cyclically until a preset step limit is reached. This process unfolds the game-theoretic behavior of each ecological impact factor into a continuous action sequence along the time axis. Based on the ecological perturbation vector mapped from each joint action, the initial ecological state tensor is superimposed point-by-point to calculate the updated ecological state tensor for each step. Key ecological indicator values ​​are extracted from the updated ecological state tensor and compared with a reference baseline to obtain the individual impact score for each step. Weighted summation and dimensional aggregation are then performed to obtain the comprehensive ecological and environmental impact index. This sequential evaluation method can capture the ecological state orientation shift caused by the joint perturbation of multiple factors within each local time period and accumulate this shift over time, thereby characterizing the nonlinear cumulative effect and lag effect of ecological and environmental impacts. Compared with traditional methods that only output a single static evaluation value, this technique provides an expansion of the assessment dimensions from a single time segment to a continuous dynamic process, enabling the assessment results to clearly reflect the rate of evolution, fluctuation range, and cumulative impact of the ecological environment under the continuous influence of multiple factors. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0017] Figure 1 This is a flowchart of an ecological and environmental impact assessment method based on multi-agent reinforcement learning;

[0018] Figure 2 It is the average reward change curve during the training process of a multi-agent policy network;

[0019] Figure 3 It is the convergence curve of action probability in a multi-agent policy network. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] See Figure 1 This invention provides an ecological and environmental impact assessment method based on multi-agent reinforcement learning. The method acquires multi-source heterogeneous ecological monitoring data within a target area and fuses this data into an ecological state tensor under a unified spatiotemporal reference. Based on the ecological state tensor, a Markov game model for ecological and environmental impact assessment is constructed, and a joint reward function for this Markov game model is defined. In the Markov game model, an agent is assigned to each ecological impact factor, and a decentralized execution strategy for each agent is output through a centralized training process in the joint action space. The assessment action sequence for each ecological impact factor within a continuous assessment period is determined based on the decentralized execution strategy. Finally, a comprehensive ecological and environmental impact index for the target area is calculated based on the assessment action sequence.

[0022] Example 1

[0023] In practice, ground ecological sampling data, remote sensing inversion data, and real-time dynamic sensor data are collected from ground monitoring stations, remote sensing satellites, and unmanned inspection terminals deployed within the target area. Ground monitoring stations include fixed water quality monitoring stations, soil moisture stations, and meteorological observation stations. Ground ecological sampling data includes discrete sampling records of water chemical oxygen demand concentration, soil organic matter content, air temperature, and precipitation. Each record includes the collection time and latitude / longitude coordinates of the sampling point. Remote sensing satellites acquire multispectral images of the target area at revisit intervals. Remote sensing inversion data consists of raster images of the normalized vegetation index, surface temperature, and suspended solids concentration in water, retrieved from the multispectral images. Each raster image includes the imaging time and metadata of the projection coordinate system, which uses the universal transverse Mercator projection. The unmanned inspection terminal is a drone equipped with a multi-parameter water quality sensor and a five-lens tilt camera. The real-time dynamic sensing data includes dissolved oxygen concentration, turbidity and attitude angle of aerial images continuously recorded during the drone's flight. Each sensing data point includes the recording time and the sensor's installation coordinates in the drone's body coordinate system.

[0024] After obtaining the above three types of data, a timestamp alignment operation is performed. Using the imaging time of the remote sensing inversion data as the baseline time, the acquisition time of the ground ecological sampling data and the recording time of the real-time dynamic sensing data are interpolated or extrapolated to the baseline time, respectively. For a sampling point in the ground ecological sampling data, if the acquisition time is between two adjacent acquisition times... and Recorded chemical oxygen demand concentration in water bodies and When the reference time satisfy At that time, linear interpolation was used to calculate the concentration value at the reference time. The calculation formula is:

[0025]

[0026] in, The chemical oxygen demand (COD) concentration in the water body at the reference time obtained by interpolation is expressed in milligrams per liter. For the time of data collection Recorded chemical oxygen demand (COD) concentration in water bodies; For the time of data collection Recorded chemical oxygen demand (COD) concentration in water bodies; and They are adjacent to the reference time. The two data collection times are in Coordinated Universal Time (UTC) seconds. The reference time is the imaging time of the remote sensing inversion data. The earliest collection time before ground ecological sampling data When using nearest neighbor extrapolation, direct assignment to When the reference time The latest collection time after ground ecological sampling data At that time, direct assignment to For the recording times of dissolved oxygen concentration and turbidity in real-time dynamic sensing data, the above-mentioned linear interpolation and extrapolation methods are also used to align them to the same reference time. The recording frequency of real-time dynamic sensing data is 10 Hz. When the reference time falls between two recording times, linear interpolation is used. When the reference time exceeds the beginning or end of the recording period, nearest neighbor extrapolation is used.

[0027] After timestamp alignment, spatial coordinate registration is performed. The latitude and longitude coordinates of the sampling points corresponding to the ground ecological sampling data are transformed into the universal transverse Mercator projection coordinate system used by the remote sensing inversion data using a transformation function from the geographic coordinate system to the projected coordinate system. During the transformation, the central meridian parameter of the target area and the WGS84 ellipsoid parameter are used to obtain the east and north coordinates of the sampling points in the projected coordinate system. The sensor installation coordinates corresponding to the real-time dynamic sensing data are transformed from the UAV body coordinate system to the local northeast coordinate system. Then, based on the GPS position and attitude angle recorded during the UAV flight, the coordinates in the local northeast coordinate system are further transformed into the universal transverse Mercator projection coordinate system. After coordinate transformation, both the ground ecological sampling data and the real-time dynamic sensing data obtain projected coordinates consistent with the pixel plane positions of the remote sensing inversion data.

[0028] Aligned and registered ground ecological sampling data, remote sensing inversion data, and real-time dynamic sensing data are stored together at the same spatiotemporal point. The spatial resolution of the spatiotemporal point is consistent with the pixel resolution of the remote sensing inversion data, and the temporal resolution is based on the reference time of each remote sensing image. The reference time values ​​for water chemical oxygen demand concentration and soil organic matter content obtained from the ground ecological sampling data are assigned to the grid points where the projected coordinates fall; the reference time values ​​for dissolved oxygen concentration and turbidity obtained from the real-time dynamic sensing data are assigned to the corresponding grid points; and the normalized vegetation index, surface temperature, and suspended solids concentration of each pixel in the remote sensing inversion data are used as the attribute values ​​for that grid point. If multiple data sources exist for the same grid point, they are recorded hierarchically according to the data source; if data from a certain source is missing from a grid point, the attribute field corresponding to that source is left blank. This generates spatiotemporally synchronized ecological data for each spatiotemporal point, including ground ecological sampling data, remote sensing inversion data, and real-time dynamic sensing data.

[0029] Spatiotemporally synchronized ecological data is dimensionally encoded according to ecological element categories. These categories include vegetation state, water state, soil state, atmospheric state, and topographic features. Vegetation state dimensions include the Normalized Difference Vegetation Index (NDVI) and Leaf Area Index (LAI); water state dimensions include chemical oxygen demand (COD), dissolved oxygen (DO), turbidity, and suspended solids (SSO); soil state dimensions include soil organic matter content and soil moisture; atmospheric state dimensions include air temperature, precipitation, and surface temperature; and topographic features include slope and aspect derived from the digital elevation model. For each spatiotemporal point, the values ​​of all ecological element categories are arranged in a fixed order, forming a feature vector with a length equal to the total number of ecological element categories. The feature vectors of all spatiotemporal points are then organized into a three-dimensional structure using spatial row indices, spatial column indices, and temporal indices to generate an ecological state tensor under a unified spatiotemporal reference covering all ecological elements. The three dimensions of the ecological state tensor are the number of grid points in the spatial row direction, the number of grid points in the spatial column direction, and the number of time steps, respectively. Each element is a vector containing the values ​​of all ecological element categories, and the physical units of each component in the vector are consistent with the units of the original collected data.

[0030] Example 2

[0031] In practical implementation, each spatiotemporal point in the ecological state tensor is treated as a state node in the state space of the Markov game model. The spatial dimension of the ecological state tensor includes... lines and Columns, including the time dimension If there are n time steps, then there are a total of n in the state space. There are 10 state nodes. For any given time-space node, its state vector consists of the values ​​of all ecological element categories at that node, and the dimension of the state vector is equal to the total number of ecological element categories.

[0032] The ecological evolution probability between adjacent state nodes is defined as the state transition probability. The definition of adjacent state nodes includes two cases: spatial adjacency and temporal adjacency. Spatial adjacency means that within the same time step, the spatial indices of two spatio-space points differ by one unit in the row direction and have the same column index, or differ by one unit in the column direction and have the same row index. Temporal adjacency means that at the same spatial grid point location, the temporal indices of two spatio-space points differ by one unit. For a pair of spatially adjacent state nodes... and State Node Indicates that the spatial index is the first line, number The column and time index are the first Step time space point, state node Indicates that the spatial index is the first line, number The column and time index are the first The time-space points of a step, the state transition probability between two state nodes. The calculation formula is based on the numerical gradients of the normalized vegetation index and the chemical oxygen demand (COD) concentration in the water at the two state nodes.

[0033]

[0034] in, This represents the state transition probability between spatially adjacent state nodes, with a value ranging from 0 to 1. State Node The normalized vegetation index is dimensionless and can be read directly from the ecological state tensor. State Node Normalized Difference Vegetation Index; State Node The concentration of chemical oxygen demand in water bodies, in milligrams per liter, is directly read from the ecological state tensor. State Node The concentration of chemical oxygen demand in water bodies; The weighting coefficient for the differences in the Normalized Difference Vegetation Index (NDVI) is determined based on the spatial heterogeneity index of vegetation cover type in the target area. The spatial heterogeneity index is calculated as the ratio of the sill value to the nugget value at a lag distance of 30 meters, representing the semivariogram of the NDVI within the target area. A ratio greater than 2.0 indicates a significant difference. The value is 0.8, and the ratio is between 1.0 and 2.0. The value is 0.5; when the ratio is less than 1.0... The value is 0.2; The weighting coefficient for the difference in chemical oxygen demand (COD) concentration in water bodies is determined based on the river network density of the target area. River network density is the total length of rivers per square kilometer. When the river network density is greater than 1.5 kilometers per square kilometer... The value is taken as 0.7 when the river network density is between 0.5 and 1.5 kilometers per square kilometer. The value is taken as 0.4, when the river network density is less than 0.5 kilometers per square kilometer. The value is 0.1; Let be the numerical stability constant, and let its value be . For state nodes that are adjacent in time... and State transition probability between two state nodes Using the same calculation method as the spatial adjacency state transition probability, the row index difference term is replaced with the time index difference term, that is, it is calculated on the dimension of normalized vegetation index. Calculated in terms of chemical oxygen demand concentration in water bodies and using the same weighting coefficients and and the same numerical stability constant Substitute the values ​​into the corresponding positions in the above formula.

[0035] The action space of each agent is constructed based on the optional regulatory operations of each ecological impact factor. Ecological impact factors include vegetation regulation factors, water regulation factors, soil regulation factors, and atmospheric regulation factors. Optional regulatory operations for vegetation regulation factors include afforestation, vegetation tending, logging restrictions, and enclosure protection; each optional regulatory operation is mapped to a discrete action index in the action space. Optional regulatory operations for water regulation factors include closing sewage outlets into rivers, upgrading sewage treatment plants, ecological water diversion, and sediment dredging. Optional regulatory operations for soil regulation factors include returning farmland to forest, grazing bans, organic fertilizer application, and soil improvement. Optional regulatory operations for atmospheric regulation factors include construction dust control, industrial source production restrictions, vehicle traffic restrictions, and road watering for dust suppression. The action space of each agent consists of discrete action indices of all optional regulatory operations for its corresponding ecological impact factor.

[0036] The Cartesian product of the action spaces of all agents is taken as the joint action space. The total number of joint actions contained in the joint action space is the product of the action spaces of the four ecological influence factors. A joint action is a quadruple consisting of selecting one action from the action space of the vegetation regulation factor, one action from the action space of the water regulation factor, one action from the action space of the soil regulation factor, and one action from the action space of the atmospheric regulation factor.

[0037] A positive contribution subfunction is defined for each joint action in the joint action space. The inputs to the positive contribution subfunction are the predicted ecological restoration rate and biodiversity increment after the joint action is executed. The prediction method for the ecological restoration rate is as follows: after applying the joint action to the current ecological state tensor, a pre-constructed ecosystem process model is used for simulation. The ecosystem process model is a coupled model including a vegetation growth module, a hydrological cycle module, and a soil carbon and nitrogen transformation module. The simulation step size is one month, and the simulation duration is 12 simulation steps. The change in the normalized vegetation index (NWRI) at the end of the simulation relative to the beginning of the simulation is taken as the ecological restoration rate, and the unit of the ecological restoration rate is the NWRI increment per hectare per year. The prediction method for the biodiversity increment is as follows: the area where the joint action is applied is spatially overlaid with a species distribution database. The species distribution database stores the spatial distribution raster of the population size of each species within the target area. Based on the action types of vegetation regulation factors and soil regulation factors in the joint action, the habitat suitability model is used to estimate the change in the population size of each species. The sum of the population size changes of each species is taken as the biodiversity increment, and the unit of the biodiversity increment is the change in the number of individual species.

[0038] A negative perturbation subfunction is defined for each joint action. The inputs to the negative perturbation subfunction are the predicted water pollution diffusion distance and soil degradation area after the joint action is executed. The prediction method for water pollution diffusion distance is as follows: the action type of the water body regulation factor in the joint action is input into a two-dimensional water quality diffusion model. The grid resolution of the two-dimensional water quality diffusion model is consistent with the spatial resolution of the ecological state tensor. The simulation time is 24 hours. The distance from the pollution source location to the background value is obtained, and this distance is taken as the water pollution diffusion distance, with the unit being meters. The prediction method for soil degradation area is as follows: the action types of the soil regulation factor and the vegetation regulation factor in the joint action are input into a soil erosion model. The soil erosion model is an improved general soil loss equation. The spatial area within the target area where the soil erosion modulus exceeds the allowable soil loss after the implementation of the joint action is calculated. This spatial area is taken as the soil degradation area, with the unit being hectares.

[0039] The output values ​​of the positive contribution sub-function and the negative disturbance sub-function are weighted and differ. The result of this weighted difference calculation is used as the function value of the joint reward function corresponding to the joint action. During the weighted difference calculation, the weight coefficient of the positive contribution sub-function is 0.6, and the weight coefficient of the negative disturbance sub-function is 0.4. The weight coefficients are determined based on the proportional relationship between the assessment weights of ecological protection indicators and pollution prevention indicators in the target area's ecological environment management plan.

[0040] Example 3

[0041] In specific implementation, a first strategy network and a first evaluation network are initialized for the first agent corresponding to vegetation regulation factors; a second strategy network and a second evaluation network are initialized for the second agent corresponding to water regulation factors; a third strategy network and a third evaluation network are initialized for the third agent corresponding to soil regulation factors; and a fourth strategy network and a fourth evaluation network are initialized for the fourth agent corresponding to atmospheric regulation factors. Each agent's strategy network adopts a three-layer fully connected structure: the number of nodes in the input layer is equal to the dimension of the agent's own local observation vector. The local observation vector extracts the ecological element category values ​​related to the ecological impact factor corresponding to the agent from the ecological state tensor. The local observation vector of the vegetation regulation factor includes the normalized vegetation index and leaf area index of all spatiotemporal points within the target area, with a dimension equal to the total number of spatial grid points multiplied by 2. The local observation vector of the water regulation factor includes the concentration of chemical oxygen demand, dissolved oxygen, turbidity, and suspended solids in the water. The local observation vector of the soil regulation factor includes soil organic matter content and soil moisture. The local observation vector of the atmospheric regulation factor includes air temperature, precipitation, and surface temperature. The first hidden layer contains 128 neurons, using a modified linear unit (MRU) as the activation function; the second hidden layer contains 64 neurons, also using a MRU as the activation function. The output layer contains the same number of neurons as the number of actions in the agent's action space. The vegetation regulation factor has four actions, corresponding to afforestation, vegetation tending, logging restrictions, and enclosure protection, respectively. The output layer uses a softmax activation function to convert the output values ​​into action probability distributions. Each evaluation network employs a five-layer fully connected structure: the input layer receives a concatenated vector of the joint observation vector and joint action vector from all agents. The joint observation vector is a long vector formed by concatenating the first and last local observation vectors of the four agents. The joint action vector is a concatenated vector of the one-hot encoded vectors of the current actions of the four agents. The number of nodes in the input layer is the sum of the dimensions of the joint observation vector and the joint action vector. The first hidden layer contains 256 neurons, the second hidden layer contains 128 neurons, and the third hidden layer contains 64 neurons. All three hidden layers use a MRU as the activation function. The output of the third hidden layer goes into an action value output branch and a state value output branch. The action value output branch is a fully connected layer with one neuron and no activation function, outputting the action value estimate for the agent. The state value output branch is also a fully connected layer with one neuron and no activation function, outputting the baseline state value for the agent. The four evaluation networks have identical network structures, and their parameters are initialized using independent random seeds.

[0042] During centralized training, ecological element values ​​for all time-space points at the current moment are obtained from the ecological state tensor. Local observation information for the first, second, third, and fourth agents is extracted according to the rules governing the composition of their local observation vectors. The first local observation information is input into the first policy network, which outputs a first action probability distribution, a four-dimensional vector where each component corresponds to the selection probability of a vegetation regulation action. The second local observation information is input into the second policy network, outputting a second action probability distribution; the third local observation information is input into the third policy network, outputting a third action probability distribution; and the fourth local observation information is input into the fourth policy network, outputting a fourth action probability distribution. These four action probability distributions are then combined into a joint action probability distribution, the dimension of which is the product of the sizes of the four action spaces. The first action, the first The second action, the first The third action, the first The probability of a combined action consisting of the fourth action is equal to the probability distribution of the first action. The component, the probability distribution of the second action The component, the probability distribution of the third action The component and the probability distribution of the fourth action. The product of the components. Sampling is performed according to the joint action probability distribution to obtain a set of joint action samples consisting of four specific action indices. The four action indices in the joint action sample are converted into corresponding one-hot encoded vectors, and concatenated with the joint observation vector at the current time. These vectors are then input into the first evaluation network, the second evaluation network, the third evaluation network, and the fourth evaluation network, respectively. The action value output branch of each evaluation network outputs the action value estimate of the corresponding agent, and the state value output branch outputs the state value baseline of the corresponding agent.

[0043] Based on the action value estimate and the state value baseline, the advantage estimate for each agent is calculated using an advantage function. The first action value estimate output by the first evaluation network is subtracted from the first state value baseline to obtain the first advantage estimate for the first agent; similarly, the second, third, and fourth advantage estimates are obtained. For each agent, the advantage estimate is multiplied by the agent's policy log probability to obtain the agent's policy gradient estimate. The policy log probability is the natural logarithm of the probability value corresponding to the sampled action in the action probability distribution output by the policy network. The number of random samples from the joint experience pool is... The batch of joint experience samples contains a joint observation at a historical moment, a joint action performed under that observation, a joint reward obtained after performing the joint action, and a joint observation at the next moment. For each sample in the batch of joint experience samples, the advantage estimate and policy log probability of each agent are calculated according to the above steps, and the average is calculated over all samples to obtain the average policy gradient for each agent. Taking the first agent as an example, the formula used for updating the policy network parameters is:

[0044]

[0045] in, This represents the policy gradient of the first agent corresponding to the vegetation regulation factor; The number of batch joint experience samples is randomly drawn from the joint experience pool. The value is 64, which is a trade-off between the GPU memory capacity limit and gradient estimation variance during training. This value is used when the GPU memory capacity is greater than or equal to 8 gigabytes. The value is 128, which is used when the video memory capacity is between 4 gigabytes and 8 gigabytes. The value is 64, which is used when the video memory capacity is less than 4 gigabytes. The value is 32; This is the parameter set for the first policy network; In the given number Local observations of the first agent in a sample Under these conditions, the first strategy network output is sampled action. The probability of; Represents the natural logarithm operation; For the first intelligent agent in the first The advantage estimate on each sample is the action value estimate output by the first evaluation network. Subtract the baseline state value output of the first evaluation network get, For the first Joint observations in a sample For the first The joint actions in each sample. The first policy network parameters are adjusted according to the direction of the average policy gradient. Gradient ascent updates are performed, with the update step size using the adaptive learning rate of the Adam optimizer. The initial learning rate is set to... For the second, third, and fourth agents, the same gradient calculation formula and update method as for the first agent are used to update their respective policy network parameters.

[0046] The temporal difference error of the evaluation network for each agent is calculated using the reward signal from the batch of joint experience samples and the joint observation information at the next time step. Taking the first evaluation network as an example, for the first... The nth sample, the nth Joint return value in each sample With discount factor Multiply by the first evaluation network in the next time step for joint observation The baseline of the output state value The sum is used as the time difference objective, where the discount factor The value is 0.95. The value is determined based on the monthly ecological evolution decay rate corresponding to the time step, derived from autocorrelation analysis of the ecological state time series of the target area over the past ten years. The time difference error is the action value estimate output by the first evaluation network. The difference from the time difference target. A mean squared error loss function is constructed based on the square of the time difference error. After calculating the average loss using batch samples, all parameters of the first evaluation network are updated via gradient descent. The same calculation and update are performed on the second, third, and fourth evaluation networks. The policy network parameter update and evaluation network parameter update are repeated. After each update, the change in policy network parameters for each agent is calculated as the sum of the Frobenius norms of the weight matrices of each layer of the policy network. When the change in policy network parameters for all four agents is below the preset convergence threshold of 0.001, the policy networks of all agents are considered to have converged. The convergence threshold of 0.001 is set based on the negligible critical value of the impact of parameter updates on the action probability distribution, determined through sensitivity analysis. The impact of continuing training below the convergence threshold on the difference in the evaluated action sequence does not exceed 1%.

[0047] After all agents' policy networks converge, freeze all network parameters of the first, second, third, and fourth policy networks, including the weight matrices and bias vectors of each layer. These frozen parameters will not change in subsequent use. Switch the first policy network from training mode to inference mode. In inference mode, disable dropout and batch normalization layer statistics updates; all batch normalization layers use the global mean and variance accumulated during training. In inference mode, configure the input ports of the first policy network to receive only local observation information corresponding to vegetation regulation factors. This local observation information is identical in composition to the local observation vectors used by the first agent during training. Configure the output ports of the first policy network to output only single action decisions for vegetation regulation factors. The single action decision is determined by selecting the action index with the highest probability value in the action probability distribution. Perform the same mode switching and port configuration operations on the second, third, and fourth policy networks. The first strategy network, after configuration, is deployed as a decentralized execution strategy for vegetation regulation factors; the second strategy network is deployed as a decentralized execution strategy for water regulation factors; the third strategy network is deployed as a decentralized execution strategy for soil regulation factors; and the fourth strategy network is deployed as a decentralized execution strategy for atmospheric regulation factors. Each decentralized execution strategy receives only local observation information of the corresponding ecological impact factor specified during deployment and independently generates an assessment action for that ecological impact factor. The four decentralized execution strategies do not exchange observation information or action decisions.

[0048] See Figure 2 In the graph, the horizontal axis represents the number of training rounds, increasing from 0 to 500, and the vertical axis represents the average reward value, ranging from approximately 0.5 to 4.0. The legend identifies four curves, corresponding to vegetation regulation factors (first agent, solid blue line), water regulation factors (second agent, dashed orange line), soil regulation factors (third agent, dashed green line), and atmospheric regulation factors (fourth agent, dotted red line).

[0049] As the number of training rounds increased, the average returns of the four agents all showed a significant upward trend, reflecting the gradual optimization of the policy network parameters of each agent through centralized training. In the early stages of training (from 0 to approximately 150 rounds), the average returns of all agents fluctuated significantly and remained below 1.0, indicating that the strategies were still in the exploratory stage and the effects were not yet significantly improved. Subsequently, between 150 and 300 rounds, the average returns increased significantly, with the vegetation regulation factor showing the largest increase, reaching a maximum of approximately 3.8, indicating that the vegetation regulation factor policy network converged quickly and achieved better results. The average returns of the water, soil, and atmospheric regulation factors also increased synchronously, reaching peak values ​​of approximately 3.1, 2.9, and 2.6, respectively, demonstrating that the agents of the four ecological impact factors effectively learned action strategies to improve the joint returns.

[0050] In the later training phase, from 300 to 500 rounds, the four curves tended to stabilize, and the fluctuation range of the average reward decreased, indicating that the policy networks of each agent had approached convergence. The vegetation regulation factor remained at the highest level, followed by the water regulation factor, and the soil regulation factor and the atmospheric regulation factor ranked third and fourth, respectively, reflecting the differences in policy performance of different ecological influence factors in the joint action space within this multi-agent reinforcement learning framework.

[0051] Example 4

[0052] In practice, the initial ecological state tensor for the current assessment period is obtained. This initial ecological state tensor is the ecological state tensor of the target region at the start of the current assessment period, and its data structure and dimensional definitions are consistent with those used in the training phase. The initial ecological state tensor is then input into the first decentralized execution strategy corresponding to the vegetation regulation factor, the second decentralized execution strategy corresponding to the water body regulation factor, the third decentralized execution strategy corresponding to the soil regulation factor, and the fourth decentralized execution strategy corresponding to the atmospheric regulation factor, respectively. The first decentralized execution strategy is a first strategy network that freezes all network parameters and switches to inference mode. Its input port is configured to receive only local observation information corresponding to vegetation regulation factors. This local observation information extracts the normalized vegetation index and leaf area index of all time points from the initial ecological state tensor and arranges them into a fixed-dimensional vector according to a preset order. After receiving the above local observation vector, the first decentralized execution strategy completes forward propagation calculation within its strategy network. The calculation path sequentially passes through matrix multiplication and modified linear unit activation in the first hidden layer, matrix multiplication and modified linear unit activation in the second hidden layer, and matrix multiplication and softmax normalization in the output layer. The output layer outputs a four-dimensional action probability distribution vector. The four components in the vector correspond to the selection probability of afforestation, vegetation tending, logging restriction, and enclosure protection actions, respectively. The action corresponding to the component with the highest probability value in the action probability distribution vector is selected as the first evaluation action of the vegetation regulation factor in the current evaluation period. The second decentralized execution strategy is configured to receive only local observation information corresponding to water body regulation factors. It extracts the chemical oxygen demand (COD), dissolved oxygen (DO), turbidity, and suspended solids (SSO) concentrations of all time points from the initial ecological state tensor. After forward propagation within the second strategy network, it outputs an action probability distribution. The action corresponding to the highest probability is selected as the first-step evaluation action for the water body regulation factor. Action options for the water body regulation factor include closing sewage outlets into rivers, upgrading sewage treatment plants, ecological water diversion, and sediment dredging. The third decentralized execution strategy takes soil organic matter content and soil moisture from all time points in the initial ecological state tensor as input and outputs the first-step evaluation action for the soil regulation factor. Action options for the soil regulation factor include returning farmland to forest, grazing bans, organic fertilizer application, and topsoil improvement. The fourth decentralized execution strategy takes air temperature, precipitation, and surface temperature from all time points in the initial ecological state tensor and outputs the first-step evaluation action for the atmospheric regulation factor. Action options for the atmospheric regulation factor include construction dust control, industrial source production restrictions, vehicle traffic restrictions, and road watering for dust suppression. The four decentralized execution strategies independently complete their own forward propagation computations without exchanging local observation information or output action decisions.

[0053] The initial assessment actions of vegetation regulation factors, water regulation factors, soil regulation factors, and atmospheric regulation factors are jointly applied to the initial ecological state tensor. The specific method of joint application is as follows: For each time point, based on the regulation operation type corresponding to the initial assessment actions of the four ecological impact factors, a pre-established regulation operation effect mapping table is queried. This table is a multi-dimensional lookup table, with the first dimension indexing the action type of vegetation regulation factors, the second dimension indexing the action type of water regulation factors, the third dimension indexing the action type of soil regulation factors, and the fourth dimension indexing the action type of atmospheric regulation factors. Each cell in the lookup table stores a state correction vector corresponding to the joint action. The dimension of the state correction vector is the same as the total number of ecological element categories at a single time point in the ecological state tensor. Each component in the state correction vector represents the change in the value of the corresponding ecological element category after applying the joint action for one time step.

[0054] For the components of the vegetation state element dimension, the change in the state correction vector is determined by the increase or decrease effect of afforestation, vegetation tending, logging restrictions, or enclosure protection on the normalized vegetation index and leaf area index; for the components of the water body state element dimension, the change is determined by the increase or decrease effect of closing sewage outlets into rivers, upgrading sewage treatment plants, ecological water diversion, or bottom sediment dredging on the concentrations of chemical oxygen demand, dissolved oxygen, turbidity, and suspended solids in the water body; for the components of the soil state element dimension, the change is determined by the increase or decrease effect of returning farmland to forest, grazing bans, organic fertilizer application, or topsoil improvement on the soil organic matter content and soil moisture; for the components of the atmospheric state element dimension, the change is determined by the increase or decrease effect of construction dust control, industrial source production restrictions, motor vehicle restrictions, or road watering on air temperature, precipitation, and surface temperature. The values ​​of each state correction vector in the control operation effect mapping table are obtained through paired statistical analysis of ecological monitoring data before and after the implementation of historical control measures in the target area. Each pair of samples includes the ecological state one month before the implementation of the measures and the ecological state one month after the implementation of the measures. The state correction vector is the average of the ecological state differences among all paired samples. The state vector of each time-space point in the initial ecological state tensor is added component by component to the state correction vector of the corresponding unit in the control operation effect mapping table to obtain the first intermediate ecological state tensor. The spatial dimension, temporal dimension, and ecological element dimension of the first intermediate ecological state tensor are exactly the same as those of the initial ecological state tensor. Each time-space point carries the ecological element values ​​within one time step after the application of the four initial evaluation actions.

[0055] The first intermediate ecological state tensor is input again into each decentralized execution strategy. Local observation information corresponding to vegetation regulation factors is extracted again from the first intermediate ecological state tensor and input into the first decentralized execution strategy, which outputs the second-step evaluation action for vegetation regulation factors. Local observation information corresponding to water regulation factors is extracted again from the first intermediate ecological state tensor and input into the second decentralized execution strategy, outputting the second-step evaluation action for water regulation factors. Local observation information corresponding to soil regulation factors is extracted again from the first intermediate ecological state tensor and input into the third decentralized execution strategy, outputting the second-step evaluation action for soil regulation factors. Local observation information corresponding to atmospheric regulation factors is extracted again from the first intermediate ecological state tensor and input into the fourth decentralized execution strategy, outputting the second-step evaluation action for atmospheric regulation factors. The generation method for the four second-step evaluation actions is the same as that for the first-step evaluation actions, all being independent forward propagation calculation processes for each decentralized execution strategy.

[0056] The four second-step evaluation actions are jointly applied to the first intermediate ecological state tensor in the same way as the first-step evaluation actions: the regulation operation effect mapping table is queried according to the type index of the four second-step evaluation actions to obtain the corresponding state correction vector, and the state vector of each time space point in the first intermediate ecological state tensor is added to the state correction vector component by component to obtain the second intermediate ecological state tensor.

[0057] The process of extracting local observation information from the current intermediate ecological state tensor and inputting it into each decentralized execution strategy to obtain the next evaluation action, and then jointly applying the next evaluation action to the current intermediate ecological state tensor to generate the next intermediate ecological state tensor, involves state transitions and action generation. In each repetition, the current intermediate ecological state tensor refers to the intermediate ecological state tensor generated in the previous iteration, and is the first intermediate ecological state tensor in the first iteration. The number of repetitions is determined by the preset step limit of the current evaluation period, which is determined based on the time span of the evaluation period and the length of a single time step. In some implementations, the time span of the evaluation period is one calendar year, the length of a single time step is one calendar month, and the preset step limit is set to 12. In the k-th repetition, the (k-1)-th intermediate ecological state tensor is input into each decentralized execution strategy to obtain the k-th evaluation action of the four ecological impact factors. The k-th evaluation action is then jointly applied to the (k-1)-th intermediate ecological state tensor to obtain the k-th intermediate ecological state tensor, with k counting from 1. When k equals the preset step limit, no new intermediate ecological state tensor is generated, and the repetition process terminates.

[0058] After reaching the preset step limit for the current evaluation period, all evaluation actions generated by the vegetation regulation factor at step index 1 to the preset step limit are arranged in ascending chronological order according to step index, resulting in the evaluation action sequence of the vegetation regulation factor within the continuous evaluation period. Taking a preset step limit of 12 as an example, the evaluation action sequence of the vegetation regulation factor is a sequence of length 12. The first element in the sequence is the first evaluation action of the vegetation regulation factor, the second element is the second evaluation action of the vegetation regulation factor, and so on, with the twelfth element being the twelfth evaluation action of the vegetation regulation factor. Similarly, the evaluation actions generated by the water body regulation factor at all steps are arranged in ascending chronological order according to step index, resulting in the evaluation action sequence of the water body regulation factor within the continuous evaluation period; the evaluation actions generated by the soil regulation factor at all steps are arranged in chronological order, resulting in the evaluation action sequence of the soil regulation factor within the continuous evaluation period; and the evaluation actions generated by the atmospheric regulation factor at all steps are arranged in chronological order, resulting in the evaluation action sequence of the atmospheric regulation factor within the continuous evaluation period. The length of each of the four evaluation action sequences is equal to the preset upper limit of the number of steps. The four actions at the corresponding step index positions in the sequence together constitute the joint action at that step number. Each element in the evaluation action sequence is a deterministic action index, and the value of the action index is an integer value within the action space size of the corresponding ecological impact factor.

[0059] In some implementations, the preset step limit is determined as follows: The assessment time granularity required for the ecological and environmental impact assessment task of the target area is obtained. The assessment time granularity is specified by the regional ecological and environmental status assessment technical specifications issued by the ecological and environmental management department. If the technical specifications require the assessment results to be output at a monthly frequency, then the assessment time granularity is monthly. The monthly scale variation cycle of meteorological and hydrological data of the target area is queried, and the monthly scale variation cycle is 12 months. The assessment time granularity and the monthly scale variation cycle are substituted into the step limit function. The step limit function is defined as the result of dividing the total assessment period by the assessment time granularity and rounding up. If the total assessment period is one year (12 months) and the assessment time granularity is one month, then the step limit is 12.

[0060] Optionally, when using the output of the decentralized execution strategy as an evaluation action each time, if there are multiple maximum values ​​with equal probability values ​​in the action probability distribution, then the action with the smallest action index value among the actions corresponding to these maximum values ​​is selected as the evaluation action.

[0061] See Figure 3 In the graph, the horizontal axis represents the step index during the training process, ranging from 0 to 200, and the vertical axis represents the action probability, ranging from 0 to 1. The legend identifies four action options for vegetation regulation factors: afforestation, vegetation tending, logging restriction, and enclosure protection, each corresponding to a different colored and styled curve.

[0062] The trends in the curves show that the probability of the afforestation action rapidly decreased from approximately 0.35 to around 0.25 in the early stages of training, then stabilized around 0.25 after approximately 20 steps, indicating a relatively stable action selection probability. The probabilities of vegetation tending and logging restriction actions remained between approximately 0.24 and 0.26 throughout the training process, with minimal fluctuations, suggesting that these two regulatory operations have relatively uniform selection probabilities in the policy network. The probability of the enclosure and protection action was the lowest initially, at approximately 0.15, then rapidly increased to approximately 0.25, and remained stable in the later stages of training, tending towards a level consistent with the probabilities of other actions.

[0063] Example 5

[0064] In practice, each step of the assessment action sequence for vegetation regulation factors is mapped to a corresponding vegetation ecological disturbance vector; each step of the assessment action sequence for water body regulation factors is mapped to a corresponding water body ecological disturbance vector; each step of the assessment action sequence for soil regulation factors is mapped to a corresponding soil ecological disturbance vector; and each step of the assessment action sequence for atmospheric regulation factors is mapped to a corresponding atmospheric ecological disturbance vector.

[0065] The mapping method for vegetation ecological disturbance vectors is as follows: The first step in the evaluation action sequence of vegetation regulatory factors is obtained. The action type index is used to find the corresponding vegetation ecological disturbance vector in the correspondence table between vegetation regulation operations and disturbance vectors. The correspondence table between vegetation regulation operations and disturbance vectors is a pre-constructed linear mapping table. The action options of vegetation regulation factors include afforestation, vegetation tending, logging restrictions, and enclosure protection. For each action option, the table stores a vegetation ecological disturbance vector with a dimension equal to the total number of ecological element categories. The values ​​of the vegetation ecological disturbance vector in the normalized vegetation index component and the leaf area index component are obtained by regression analysis of the differences in ecological status between areas where historical vegetation regulation measures were implemented and areas where they were not implemented. The values ​​of the vegetation ecological disturbance vector in each component of the water body state element dimension, each component of the soil state element dimension, and each component of the atmospheric state element dimension are all zero.

[0066] The mapping method for the water body ecological disturbance vector is as follows: The first step in the evaluation action sequence of water body regulatory factors is obtained. The action type index is used to find the corresponding water body ecological disturbance vector in the correspondence table between water body regulation operations and disturbance vectors. The action options of water body regulation factors include closing sewage outlets into rivers, upgrading sewage treatment plants, ecological water diversion, and bottom sediment dredging. The table stores a water body ecological disturbance vector with a dimension equal to the total number of ecological element categories for each action option. The values ​​of the water body ecological disturbance vector in the chemical oxygen demand concentration component, dissolved oxygen concentration component, turbidity component, and suspended solids concentration component are obtained by simulating the changes in water quality parameters after various regulation operations using a water quality model. The values ​​of the water body ecological disturbance vector in each component of the vegetation state element dimension, each component of the soil state element dimension, and each component of the atmospheric state element dimension are all zero.

[0067] The mapping method for soil ecological disturbance vectors is as follows: The first step in the evaluation action sequence of soil regulatory factors is obtained. The action type index is used to find the corresponding soil ecological disturbance vector in the correspondence table between soil regulation operations and disturbance vectors. The action options for soil regulation factors include returning farmland to forest, grazing ban, organic fertilizer application, and soil improvement. The table stores a soil ecological disturbance vector with a dimension equal to the total number of ecological element categories for each action option. The values ​​of the soil ecological disturbance vector in the soil organic matter content component and the soil moisture component are obtained through statistical analysis of measured data from long-term positioning experimental stations. The values ​​of the soil ecological disturbance vector in each component of the vegetation state element dimension, each component of the water body state element dimension, and each component of the atmospheric state element dimension are all zero.

[0068] The mapping method for atmospheric ecological disturbance vectors is as follows: The first step in the evaluation action sequence of atmospheric regulatory factors is obtained. The action type index is used to find the corresponding atmospheric ecological disturbance vector in the correspondence table between atmospheric control operations and disturbance vectors. The action options for atmospheric control factors include construction dust control, industrial source production restriction, motor vehicle restriction, and road watering to reduce dust. The table stores an atmospheric ecological disturbance vector with a dimension equal to the total number of ecological element categories for each action option. The values ​​of the atmospheric ecological disturbance vector in the temperature component, precipitation component, and surface temperature component are obtained by jointly estimating the impact of various control measures on local meteorological elements through atmospheric diffusion models and meteorological reanalysis data. The values ​​of the atmospheric ecological disturbance vector in each component of the vegetation state element dimension, each component of the water body state element dimension, and each component of the soil state element dimension are all zero.

[0069] By adding the vegetation ecological disturbance vector, water ecological disturbance vector, soil ecological disturbance vector, and atmospheric ecological disturbance vector at the same time (i.e., the same step index), component by component, we obtain the first... The total disturbance vector is the step-by-step disturbance vector. The dimension of the total disturbance vector is the same as that of individual ecological disturbance vectors, equal to the total number of ecological element categories. When adding components one by one, the non-zero components of the vegetation ecological disturbance vector and the water ecological disturbance vector are located in different dimensions, as are the non-zero components of the soil ecological disturbance vector and the atmospheric ecological disturbance vector. The non-zero components between the disturbance vectors do not overlap. The total perturbation vector for each step is generated by taking values ​​from 1 up to the preset step limit.

[0070] Obtain the initial ecological state tensor of the target area in the current assessment period. The initial ecological state tensor is the ecological state tensor of the target area at the start of the current assessment period, and is the same data object as the initial ecological state tensor used in Example 4. The state vector of each time-space point in the initial ecological state tensor is superimposed component-by-component with the total perturbation vector from the first step. That is, for any spatial index of the first... line, number The time-space point of the column will be the first position in the state vector of that grid point in the initial ecological state tensor. The values ​​of each ecological element component and the first step of the total disturbance vector are related to the values ​​of the first ecological element component. The numerical values ​​of the n components are algebraically added together to obtain the nth component in the updated state vector of that grid point after the first step. The values ​​of each ecological element component, among which Starting from 1, the total number of ecological element categories is taken. The above component-by-component superposition operation is performed on all time-space points in the initial ecological state tensor to obtain the first step's updated ecological state tensor. The first step's updated ecological state tensor is then component-by-component superimposed with the second step's total perturbation vector to obtain the second step's updated ecological state tensor. The updated ecological state tensor obtained in the previous step is then component-by-component superimposed with the next step's total perturbation vector until all total perturbation vectors for the preset maximum number of steps are superimposed, resulting in the updated ecological state tensor for each step. A total of one updated ecological state tensor sequence equal to the preset maximum number of steps is generated.

[0071] Pre-defined key ecological indicators are extracted from each updated ecological state tensor. These key ecological indicators include the regional average normalized vegetation index (NMR), the percentage of days with good or excellent water quality (COD concentration in water), the average soil organic matter content, and the percentage of days with good or excellent air quality. The regional average NMR is extracted as follows: for the first... To update the ecological state tensor step by step, traverse all spatial grid points, sum the normalized vegetation index components under the vegetation state element dimension in each grid point, and divide by the total number of spatial grid points to obtain the first step. The regional average normalized vegetation index was used in the first step. The extraction method for the excellent rate of water body chemical oxygen demand concentration was as follows: For the first step... The ecological state tensor is updated step by step, traversing all spatial grid points, and counting the number of grid points whose chemical oxygen demand (COD) concentration component values ​​under the water state element dimension are lower than the COD concentration limit for Class III water bodies in the "Surface Water Environmental Quality Standard". This number is then divided by the total number of grid points to obtain the first step. The water body chemical oxygen demand concentration excellent rate in step one. The extraction method for the average soil organic matter content is as follows: For the first step... To update the ecological state tensor, iterate through all spatial grid points, sum the soil organic matter content components under the soil state element dimension in each grid point, and divide by the total number of spatial grid points to obtain the first step. The average soil organic matter content of the first step. The extraction method for the percentage of days with good or excellent air quality is as follows: For the first step... The ecological state tensor is updated step by step, traversing all spatial grid points, and counting the number of grid points in the atmospheric state element dimension whose temperature component, precipitation component, and surface temperature component are all within a preset suitable range. This number is then divided by the total number of grid points to obtain the first step. The percentage of days with good or excellent air quality. The values ​​of the four key ecological indicators are represented as vectors, the first... The key ecological indicator value vector of step 1 contains the first step in sequence. The regional average normalized vegetation index of the first step, the first The first step of the excellent rate of water body chemical oxygen demand concentration, the second step The average soil organic matter content of the first step and the second step The percentage of days with good or excellent air quality.

[0072] Each key ecological indicator value is compared with its corresponding reference baseline value to obtain the individual impact score for each step. The reference baseline values ​​include the regional average normalized vegetation index baseline value, the water body chemical oxygen demand concentration good / excellent rate baseline value, the average soil organic matter content baseline value, and the air quality good / excellent days ratio baseline value. The four reference baseline values ​​are obtained by extracting the corresponding indicator values ​​at the same time points as the current assessment period from the ecological state tensor at the start of the same assessment period over the past five years for the target area, and calculating the five-year average as the reference baseline value. For the first step... The regional average normalized vegetation index (NMR) is used to calculate the difference between the regional average NMR and the baseline value. This difference is then divided by the baseline value and multiplied by 100 to obtain the result for the first step. The normalized difference in vegetation index (NDDI) affects the score; a positive difference results in a positive score, indicating a positive impact, while a negative difference results in a negative score, indicating a negative impact. For the first... The rate of excellent water quality for the first step of chemical oxygen demand (COD) concentration is calculated, and the difference between this rate and the baseline value is used as the first step. The percentage of excellent water quality in each step affects the score; the difference is expressed as a percentage point. For the first... The average soil organic matter content at step 1 is calculated, and the difference between the average soil organic matter content and the baseline average soil organic matter content is calculated. This difference is then divided by the baseline average soil organic matter content and multiplied by 100 to obtain the result for step 2. The soil organic matter content affects the score. For the first step... The percentage of days with good or excellent air quality in the first step is calculated, and the difference between this percentage and the baseline value is used directly as the percentage of days with good or excellent air quality in the second step. The score is affected by air quality. The first step... The normalized vegetation index of the first step affects the score, the second step The percentage of excellent water quality in the first step affects the score. The impact of soil organic matter on the score and the first step The combination of air quality impact scores for each step is the first step. Each step has a single impact on the score vector.

[0073] The scores for each individual impact are weighted and accumulated according to the number of assessment steps. Then, the scores for each ecological impact factor dimension are aggregated to obtain the comprehensive ecological and environmental impact index of the target area. The following comprehensive calculation formula is used for weighted accumulation:

[0074]

[0075] in, The comprehensive ecological and environmental impact index of the target area is dimensionless. The preset maximum number of steps is set to 12. The step index is from 1 to 1. ; For the first Step decay weights Calculated by the exponential decay function, ,in The time decay constant, The value is 0.1. The value of 0.1 is based on the empirical value of the half-life of the cumulative ecological impact of the target area, meaning that the closer the assessment action is to the current moment, the greater its weight in terms of the timeliness of the impact. Represented by natural constant An exponential function with base 0. This serves as a dummy variable to index the number of steps in the accumulation operation. This represents the total number of ecological impact factor dimensions, with a value of 4. Indexed by ecological impact factor dimensions. Corresponding to the dimensions of vegetation regulatory factors, Corresponding to the dimensions of water body regulation factors, Corresponding to soil regulatory factors, Corresponding to the dimensions of atmospheric regulation factors; For the first The aggregated weights of the ecological impact factors are set based on the normalized values ​​of the sub-weights of each indicator in the "Technical Specifications for Ecological Environment Status Assessment," including the aggregated weights of the vegetation regulation factor dimension. The value is 0.35, representing the aggregate weight of the water body regulation factor dimension. The value is 0.30, representing the aggregate weight of the soil regulatory factor dimension. The value is 0.20, representing the aggregate weight of the atmospheric regulation factor dimension. The value is 0.15; For the first Step in the first The individual impact scores corresponding to each ecological impact factor dimension. For the first The normalized difference in vegetation index affects the score. For the first The percentage of excellent water quality in the test affects the score. For the first The amount of soil organic matter in the soil affects the score. For the first Air quality affects the score. Weighting coefficients. satisfy Aggregate weight satisfy According to the above formula, the individual impact scores of the four ecological impact factors in each step are first weighted and summed to obtain the comprehensive score for each step. Then, the comprehensive scores of each step are weighted and accumulated according to the time decay weight to finally obtain the comprehensive ecological and environmental impact index of the target area. The higher the value of the comprehensive ecological and environmental impact index, the more significant the overall positive contribution of various control measures to the ecological environment of the target area within the current assessment period; the lower the value or the negative value, the more dominant the overall negative disturbance.

[0076] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An ecological and environmental impact assessment method based on multi-agent reinforcement learning, characterized in that, The method includes: Acquire multi-source heterogeneous ecological monitoring data within the target area, and fuse the multi-source heterogeneous ecological monitoring data into an ecological state tensor under a unified spatiotemporal reference. A Markov game model for ecological and environmental impact assessment is constructed based on the ecological state tensor, and a joint reward function of the Markov game model is defined. In the Markov game model, an agent is assigned to each ecological impact factor, and the decentralized execution strategy of each agent is output through a centralized training process of all agents in the joint action space. The decentralized execution strategy is used to determine the sequence of assessment actions for each ecological impact factor within a continuous assessment period. The comprehensive ecological and environmental impact index of the target area is calculated based on the assessment action sequence.

2. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 1, characterized in that, Acquiring multi-source heterogeneous ecological monitoring data within the target area and fusing the multi-source heterogeneous ecological monitoring data into an ecological state tensor under a unified spatiotemporal reference specifically includes: Ground ecological sampling data, remote sensing inversion data, and real-time dynamic sensing data are collected from ground monitoring stations, remote sensing satellites, and unmanned inspection terminals deployed in the target area, respectively. The ground ecological sampling data, the remote sensing inversion data, and the real-time dynamic sensing data are time-stamped and spatially registered to obtain spatiotemporally synchronized ecological data. The spatiotemporally synchronized ecological data is dimensionally encoded according to the categories of ecological elements to generate an ecological state tensor under a unified spatiotemporal benchmark covering all ecological elements.

3. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 1, characterized in that, Based on the ecological state tensor, a Markov game model for ecological and environmental impact assessment is constructed, and the joint reward function of the Markov game model is specifically defined as follows: Each timespace point in the ecological state tensor is taken as a state node in the state space of the Markov game model, and the ecological evolution probability between adjacent state nodes is defined as the state transition probability. The action space of each agent is constructed based on the optional control operations of each ecological influencing factor, and the Cartesian product of the action spaces of all agents is used as the joint action space. For each joint action in the joint action space, a joint reward function is defined to measure the positive contribution and negative disturbance of the joint action to the overall ecological environment.

4. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 1, characterized in that, In the Markov game model, an agent is assigned to each ecological impact factor. The decentralized execution strategy for each agent is output through a centralized training process across all agents in the joint action space. Specifically, this includes: A policy network and an evaluation network are initialized for each agent corresponding to an ecological impact factor, wherein the input of the evaluation network includes joint observation information and joint action information of all agents; During centralized training, the local observation information of each agent is input into the corresponding policy network to obtain the local action probability distribution of each agent, and the local action probability distributions of all agents are combined into a joint action probability distribution. Based on the joint action probability distribution, joint action samples are obtained by sampling. The joint action samples and the current joint state are input into the evaluation network of each agent to obtain the action value estimate of each agent. The policy gradient of each agent is calculated based on the action value estimation. The corresponding policy network parameters are updated according to the policy gradient. The evaluation network parameters of each agent are updated using the joint experience data of all agents until the policy networks of all agents converge. Finally, the decentralized execution policy of each agent is output.

5. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 1, characterized in that, The specific assessment action sequence for each ecological impact factor within a continuous assessment period, determined according to the decentralized execution strategy, includes: The initial ecological state tensor of the current assessment period is input into the decentralized execution strategy of each agent, and each agent independently outputs the first assessment action of its own ecological impact factor within the current assessment period. The first evaluation actions are jointly applied to the initial ecological state tensor to obtain the first intermediate ecological state tensor. The first intermediate ecological state tensor is then input into the decentralized execution strategy of each agent to obtain the second evaluation actions of each ecological impact factor. Repeat the state transition and action generation process in the previous step until the preset step limit for the current assessment cycle is reached. Arrange the assessment actions generated by each ecological impact factor in all steps in chronological order to obtain the assessment action sequence of each ecological impact factor in the continuous assessment cycle.

6. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 1, characterized in that, The calculation of the comprehensive ecological and environmental impact index of the target area based on the aforementioned assessment action sequence specifically includes: Each step of the assessment action sequence for each ecological impact factor is mapped to a corresponding ecological disturbance vector, and the ecological disturbance vectors of all ecological impact factors at the same time are superimposed to form the total disturbance vector. Obtain the initial ecological state tensor of the target area in the current evaluation period, and then superimpose the initial ecological state tensor with the total perturbation vector of each step point by point to obtain the updated ecological state tensor of each step. Preset key ecological indicator values ​​are extracted from each updated ecological state tensor, and each key ecological indicator value is compared with the corresponding reference baseline value to obtain the single impact score for each step. The individual impact scores are weighted and accumulated according to the number of evaluation steps, and then the ecological impact factors are aggregated to obtain the comprehensive ecological and environmental impact index of the target area.

7. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 2, characterized in that, The process of aligning the ground ecological sampling data, the remote sensing inversion data, and the real-time dynamic sensing data with timestamps and registering their spatial coordinates to obtain spatiotemporally synchronized ecological data specifically includes: Using the imaging time of the remote sensing inversion data as the reference time, the collection time of the ground ecological sampling data and the recording time of the real-time dynamic sensing data are interpolated or extrapolated to the reference time to complete the timestamp alignment. The coordinates of the sampling points corresponding to the ground ecological sampling data and the sensor installation coordinates corresponding to the real-time dynamic sensing data are respectively converted to the projection coordinate system used by the remote sensing inversion data to complete the spatial coordinate registration. The aligned and registered ground ecological sampling data, remote sensing inversion data, and real-time dynamic sensing data are linked and stored according to the same time and space point to generate spatiotemporally synchronized ecological data.

8. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 3, characterized in that, For each joint action in the aforementioned joint action space, a joint reward function is defined to measure the positive contribution and negative disturbance of the joint action to the overall ecological environment. Specifically, this includes: A positive contribution sub-function is defined for each joint action in the joint action space. The input of the positive contribution sub-function is the predicted ecological restoration rate and biodiversity increment after the joint action is executed. A negative perturbation sub-function is set for each joint action, and the input of the negative perturbation sub-function is the predicted water pollution diffusion distance and soil degradation area after the joint action is executed; The output value of the positive contribution sub-function and the output value of the negative perturbation sub-function are weighted and the difference is calculated. The result of the weighted difference calculation is used as the function value of the joint return function corresponding to the joint action.

9. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 4, characterized in that, The process of calculating the policy gradient for each agent based on the action value estimation, updating the corresponding policy network parameters based on the policy gradient, and updating the evaluation network parameters of each agent using the joint empirical data of all agents until the policy networks of all agents converge specifically includes: The advantage function is used to subtract the state value baseline from the action value estimate of each agent to obtain the advantage estimate of each agent. The advantage estimate is then multiplied by the policy log probability of the corresponding agent to obtain the policy gradient estimate of the agent. The average policy gradient is obtained by summing the policy gradient estimates of each agent and dividing by the number of batch samples. The policy network parameters of each agent are then updated by gradient ascent according to the direction of the average policy gradient. Randomly sample a batch of joint experience samples from the joint experience pool, use the reward signal and next state information in the batch of joint experience samples to calculate the time difference error of the evaluation network for each agent, and update the corresponding evaluation network parameters according to the time difference error; Repeatedly perform policy network parameter updates and evaluation network parameter updates until the change in policy network parameters for each agent is lower than the preset convergence threshold.

10. The method for assessing ecological and environmental impacts based on multi-agent reinforcement learning according to claim 4, characterized in that, After the policy network of all agents converges, the decentralized execution policy output for each agent specifically includes: After all agents' policy networks converge, freeze all network parameters of each agent's policy network and switch each policy network from training mode to inference mode. In inference mode, the input port of the policy network of each agent is configured to receive only the local observation information of the corresponding agent, and the output port of each policy network is configured to output only the single action decision of the corresponding ecological impact factor. The policy network of each agent after configuration is deployed as the decentralized execution policy of that agent, so that each agent can generate and execute actions independently by relying only on its own local observation information at runtime.

Images