Water microbial risk analysis method based on reinforcement learning

Abstract

The present invention discloses a water microbial risk analysis method based on reinforcement learning, comprising the following steps: S1. generating a standardized observation dataset; S2. constructing a dynamic water body graph based on the standardized observation dataset; S3. performing dynamic graph representation learning on the dynamic water body graph structure, generating a spatiotemporal risk heat matrix based on the node vector representation matrix; S4. constructing a Bayesian reward function prior distribution, initializing the reward function, and establishing a reward function parameter set; S5. using a maximum a posteriori inverse reinforcement learning model to incrementally learn the reward function parameter set and output a first propagation potential field; S6. obtaining an optimal diffusion path sequence; and S7. performing path reconstruction error analysis on the optimal diffusion path sequence and the standardized observation dataset to obtain a dynamic feedback index. In actual measurements, the present invention significantly shortened the response delay to pollution diffusion caused by emergencies, significantly improving the system's emergency modeling capabilities in sudden pollution scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of water body detection, and in particular to a water body microbial risk analysis method based on reinforcement learning. Background Art

[0002] Currently, existing water pathogen risk assessment technologies suffer from a single detection method, simple assessment models, poor spatial adaptability, and a lack of dynamic updates. Traditional risk identification methods rely heavily on on-site, fixed-point monitoring, primarily targeting a single bacterial indicator like Escherichia coli. This lacks comprehensive coverage for multiple pathogens, including viruses and parasites. Furthermore, existing detection technologies are limited by long lead times, low sensitivity, and poor field adaptability, making it impossible to capture the spread of pollution in real time, hindering the effectiveness of emergency response.

[0003] At the modeling level, most existing risk assessment models use empirical formulas or static statistical methods, which are applicable only in specific waters and with fixed parameters, making them difficult to adapt to water systems with strong nonlinear characteristics. Furthermore, mainstream methods often fail to comprehensively consider multidimensional environmental drivers such as pollution source distribution, hydrodynamic processes, meteorological disturbances, and land use changes. Risk-driving mechanisms are not clearly described, path prediction results are limited in accuracy, and the ability to deeply characterize risk transmission mechanisms is lacking. Spatial modeling methods also often rely on manual or rule-based zoning, making it impossible to extract high-quality spatiotemporal transmission clues from dynamic hydrological evolution characteristics.

[0004] In summary, there is an urgent need to introduce new modeling strategies for spatiotemporal evolution mechanisms, integrate multi-source information, enhance the intelligence of path generation, and have adaptive updating capabilities to cope with the modeling challenges of pathogenic microorganism transmission processes under complex hydrodynamic conditions. Summary of the Invention

[0005] One purpose of the present invention is to propose a water microbial risk analysis method based on reinforcement learning. In actual measurements, the present invention greatly shortens the response delay to pollution diffusion caused by emergencies, significantly improving the system's emergency modeling capabilities in sudden pollution scenarios.

[0006] A water microbial risk analysis method based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0007] S1. Collect and preprocess the original observation dataset in the target waters to generate a standardized observation dataset;

[0008] S2. Construct dynamic water body maps based on standardized observation datasets;

[0009] S3. Perform dynamic graph representation learning on the dynamic water body graph structure to obtain a node vector representation matrix. Based on the node vector representation matrix, generate a spatiotemporal risk heat matrix.

[0010] S4. Using the standardized observation dataset, the node vector representation matrix, and the spatiotemporal risk heat matrix, construct a Bayesian reward function prior distribution, initialize the reward function, and establish a set of reward function parameters.

[0011] S5. Using a standardized observation dataset as the behavioral outcome input on a dynamic water body graph, a maximum a posteriori inverse reinforcement learning model is used to incrementally learn the reward function parameter set, outputting the first propagation potential field.

[0012] S6. Using the first propagation potential field as the basis for fitness evaluation, the artificial bee colony optimization algorithm performs a global search on the dynamic water body graph structure to generate a set of candidate diffusion paths. Based on this set of candidate diffusion paths and the first propagation potential field, follower bee local refinement and scout bee random perturbation operations are performed to obtain the optimal diffusion path sequence.

[0013] S7. Perform path reconstruction error analysis on the optimal diffusion path sequence and the standardized observation dataset to obtain dynamic feedback indicators and form a joint convergence iterative loop. When the joint convergence iterative loop meets the preset convergence threshold, the final diffusion path sequence, dynamic risk zone distribution map, and reward function sensitivity indicator are output.

[0014] Optionally, the original observation data set includes pathogen concentration, water flow velocity, water temperature, pH, water flow direction, physical connectivity attributes and boundary contact attributes.

[0015] Optionally, the S2 includes the following steps:

[0016] S21. Construct a spatial node set. Each spatial node is used to represent a monitoring section, tributary inlet, potential pollution source or hydraulic control point in the target water area. All spatial nodes together constitute a spatial node set. The spatial node set covers the entire water body research range. The nodes are numbered from 1 to ,in Indicates the total number of spatial nodes;

[0017] S22. For each spatial node, extract the pathogen concentration, water temperature, and pH value of the spatial node at each time step. These pathogen concentration, water temperature, and pH value are used as the node attributes of the spatial node at the corresponding time step. The pathogen concentration, water temperature, and pH value together constitute the node attribute set of the spatial node.

[0018] S23. Based on the water flow direction and velocity recorded in the standardized observation dataset, establish directed edge relationships between spatial nodes. If there is a physical connection between two spatial nodes and the water flow direction points to the physical connection path, establish a directed edge from the upstream node to the downstream node. The directed edge set consists of all edges that meet the conditions.

[0019] S24. For each established directed edge, extract the water flow velocity corresponding to the directed edge at each time step. Use the water flow velocity value as the edge weight of the directed edge at that time step and construct an edge weight set. The edge weight is used to represent the dynamic strength of pollutants in the water body propagating along this channel.

[0020] S25. For each established directed edge, extract the edge's water flow direction attribute, physical connectivity attribute, and boundary contact attribute from the standardized observation dataset to form an edge attribute set for the edge;

[0021] S26. Assemble the spatial nodes , directed edge set , node attribute collection , edge weight set and edge attribute set Perform unified packaging to build a dynamic water body graph structure .

[0022] Optionally, S3 includes the following steps:

[0023] S31. Based on dynamic water body graph structure , from the initial time step To the final time step Extract spatial node sets sequentially The node attribute set of each spatial node in and the directed edge set The set of edge weights for each edge in , forming a dynamic graph to represent the temporal input sequence of learning;

[0024] S32. For the time series input sequence, a node vector representation model is constructed based on the dynamic graph convolutional network. The node feature update process of the node vector representation model is defined as:

[0025] ;

[0026] in, Indicates in Spatial nodes in layer networks At time step The corresponding node feature vector, represents a nonlinear activation function, Indicates the The weight matrix of the layer network, Indicates the The bias term of the layer network, represents the neighbor node aggregation function, Representation and Node A set of adjacent spatial nodes;

[0027] S33. Using node vectors to represent the model arrive The dynamic graph temporal input sequence is forward propagated to train each spatial node At each time step The node feature vector of , forming a node vector representation matrix ,The node vector representation matrix is ​​used to express the spatiotemporal dynamic characteristics of spatial nodes;

[0028] S34. Representing matrices with node vectors As input, a spatiotemporal risk heat matrix generation model is constructed;

[0029] S35. Generate a model for spatial node sets based on the spatiotemporal risk heat matrix Traverse all spatial nodes in the matrix to obtain the risk heat values ​​of all nodes at each time step, forming a complete spatiotemporal risk heat matrix .

[0030] Optionally, the S4 includes the following steps:

[0031] S41. Construct a physically driven prior space for the Bayesian reward function. The physically driven prior space uses a standardized observation dataset as input and combines a node vector representation matrix with a spatiotemporal risk heat matrix.

[0032] S42. Define the Bayesian reward function as a spatial node With time step The conditional probability distribution of each Bayesian reward function value Obey the prior mean function and covariance function is a Gaussian process distribution with parameters;

[0033] S43. Constructing the prior mean function of the Bayesian reward function ,The prior mean function consists of three weighted parts, the first part is the relative residual term of pathogenic microorganism concentration, the second part is the water temperature proportion term, and the third part is the pH deviation term;

[0034] S44. Construct the covariance function of the Bayesian reward function. The covariance function is based on the node With node Node vectors at different times With node vector The Euclidean distance between them is used as the input feature, mapped by the exponential decay function controlled by the scaling coefficient and the feature scale parameter, and the covariance value representing the correlation of the reward value is output;

[0035] S45. The prior mean function With covariance function The complete set of prior distributions that jointly form the Bayesian reward function , the prior distribution set covers all spatial nodes and all time steps, and is used to provide the initial structural relationship of the reward function in the spatial and temporal dimensions;

[0036] S46. Prior distribution set based on Bayesian reward function , initialize the reward function parameter set , each element in the sample set Representation node At time step The initial reward estimate of , all estimates constitute the initial sample set.

[0037] Optionally, the S5 includes the following steps:

[0038] S51. Using the pathogen concentration distribution in a standardized observation dataset as the expert behavior sample input, a maximum a posteriori inverse reinforcement learning model integrating a two-layer attention mechanism and dynamic Bayesian updating is constructed based on the dynamic water body graph structure, node vector representation matrix, and Bayesian reward function parameter set.

[0039] S52. In the maximum a posteriori inverse reinforcement learning model, the spatial node set A collection of node attributes As input, a node-level attention mechanism is constructed, which is used to learn spatial nodes. At time step State weight ;

[0040] S53. State weights based on node-level attention mechanism output , construct a time-level attention mechanism, which is used to capture the time-level attention weights of the pathogen diffusion path changing with time steps ;

[0041] S54. State weights based on node-level attention mechanism output and temporal attention weights , reconstruct the target optimization function of maximum a posteriori inverse reinforcement learning :

[0042] ;

[0043] in, For nodes In the reward function The concentration of pathogenic microorganisms is shown below The likelihood probability, is the prior distribution of the Bayesian reward function;

[0044] S55. Implementing the objective optimization function of maximum a posteriori inverse reinforcement learning through dynamic Bayesian updating Iterative update, update the reward function parameter set ;

[0045] S56. In the reward function parameter set After completing the iterative update and reaching the predetermined convergence threshold, the optimized reward function set is output, and the first propagation potential field is generated based on the optimized reward function set. :

[0046] ;

[0047] in, is the optimal diffusion strategy after optimization, is the discount factor, Indicates the optimal diffusion strategy The expected value calculation operator for the future propagation process.

[0048] Optionally, the S6 includes the following steps:

[0049] S61. Calculate the path fitness value of each diffusion path. The path fitness value is calculated by multiplying the propagation potential energy values ​​of all spatial nodes on the diffusion path at the corresponding time step by their risk weight coefficients in the path and then summing them. The propagation potential energy value is obtained from the first propagation potential field.

[0050] S62. Initialize the set of bee colony individuals for the artificial bee colony optimization algorithm, set the number of leader bees, follower bees, and scout bees, and generate an initial set of paths with the same number of leaders, follower bees, and scout bees. Each path individual consists of a set of spatial nodes.

[0051] S63. For each path generated by the leader bee, perform a path update operation according to the gradient direction of the propagation potential field. The path update operation inserts new spatial nodes or adjusts the path direction along the direction of the rising propagation potential energy based on the original path;

[0052] S64. For each path optimized by the follower bee, perform a local refinement operation based on the path with the highest fitness in the current propagation potential field;

[0053] S65. Scout bees are responsible for executing a fully randomized path search mechanism. Randomized paths are constructed by randomly selecting connected spatial nodes on the dynamic water body graph structure. If the cumulative propagation potential energy of a randomized path exceeds the cumulative propagation potential energy of the worst path in the current path set by more than a minimum improvement threshold, the diffusion path will replace the worst path in the original set.

[0054] S66. Repeat the global guided search by the leader bees, the local refined search by the follower bees, and the random search mechanism by the scout bees. In each iteration, the path with the highest current fitness is recorded and used as the reference optimal solution for the path population. When the maximum number of iterations is reached or the improvement in the path fitness value is less than the preset termination threshold, the artificial bee colony optimization process is terminated.

[0055] S67. After the artificial bee colony optimization process is terminated, the final optimal diffusion path is output. The optimal diffusion path is the path with the highest cumulative value of propagation potential energy among all current path individuals. The diffusion path represents the most likely propagation path for pathogenic microorganisms to spread from potential sources to high-risk areas in the dynamic water body graph structure.

[0056] Optionally, the S7 includes the following steps:

[0057] S71. Perform path reconstruction error analysis on the optimal diffusion path sequence output by the artificial bee colony optimization algorithm and the pathogen concentration distribution in the standardized observation dataset;

[0058] S72. Construct a dynamic feedback indicator set based on the path reconstruction error analysis results. The dynamic feedback indicator set includes spatial node matching error, time series concentration fitting error, and propagation direction consistency error.

[0059] S73. Using the dynamic feedback indicator set as the evaluation basis for the joint optimization update, dynamically updating the reward function parameter set using the spatial node matching error and the temporal concentration fitting error feedback information;

[0060] S74. Re-input the reward function parameter set and the updated artificial bee colony optimization control parameters into the maximum a posteriori inverse reinforcement learning model and the artificial bee colony optimization algorithm, and continue to perform reward function training and diffusion path search, forming a joint convergence iterative loop;

[0061] S75. Output the final diffusion path sequence, dynamic risk area distribution map, and reward function sensitivity index:

[0062] Final diffusion path sequence: represents the optimal path trajectory for water pathogens to diffuse from the initial high-risk source node to the downstream. The node sequence satisfies the flow direction constraint and has the maximum cumulative potential energy in the propagation potential field.

[0063] Dynamic risk zone distribution map: The risk index value of each spatial node at each time step is calculated based on the final diffusion path sequence and propagation potential field. The risk index value is used to divide the risk level area into high-risk areas with a risk index value ≥ 0.8, medium-risk areas with a risk index value of 0.5≤<0.8, and low-risk areas with a risk index value of <0.5.

[0064] Reward function sensitivity index: Based on the partial derivative response between the final converged reward function sample set and the physical and chemical indicators in the standardized observation data set, the sensitivity score of each driving factor to the change of the reward function is output. Factors with a sensitivity score greater than 0.3 are defined as key propagation drivers;

[0065] S77. Standardize the format and layer map of the final diffusion path sequence, dynamic risk area distribution map, and reward function sensitivity index.

[0066] Optionally, the spatial node matching error measures the degree of overlap between the node set in the optimal diffusion path sequence and the observed high-concentration node set;

[0067] Time series concentration fitting error: measures the degree of difference between the propagation potential field value of each node in the diffusion path at the corresponding time step and the concentration value in the standardized observation data set;

[0068] Propagation direction consistency error: measures the degree of consistency between the directions between path nodes and the flow directions recorded in the edge attribute set in the dynamic water body graph structure.

[0069] The beneficial effects of the present invention are:

[0070] (1) This paper introduces a two-layer attention mechanism based on the traditional inverse reinforcement learning structure: the node-level attention mechanism can dynamically adjust the node weight according to the propagation potential of pathogenic microorganisms at different spatial nodes; the time-level attention mechanism dynamically adjusts the weight distribution of each time step in the learning process based on the time evolution trend of the diffusion path, so that the model can focus more effectively on key propagation nodes and time periods, thereby learning a reward function that is more in line with the water flow direction and pathogen dynamic mechanism. This not only improves the strategy convergence efficiency, but also enhances the physical interpretability of the model, providing direct support for subsequent public health interventions.

[0071] (2) The present invention improves the traditional artificial bee colony optimization strategy. It introduces a propagation potential field gradient guidance mechanism in the leader bee stage to realize global guided path construction, performs path refinement optimization through Gaussian perturbation in the follower bee stage, and introduces structural consistency constraints in the scout bee stage to improve the physical rationality of path generation. At the same time, with propagation potential energy as the core of fitness evaluation, it iterates multiple rounds of searching for the optimal path sequence in the dynamic water body graph structure, effectively avoiding the common problem of falling into local optimality, and improving the structural fidelity of the optimal path in real complex watershed scenarios.

[0072] (3) The present invention establishes a dynamic convergence mechanism for the joint update of the reward function and path search, which improves the adaptability and real-time performance of the model to changes in water conditions. By introducing a path reconstruction error feedback mechanism, the joint adaptive update of the reward function parameters and the optimization algorithm control parameters is realized. In each round of iteration, dynamic feedback indicators such as spatial node matching error and propagation direction consistency error are used to adjust the inverse reinforcement learning model and the artificial bee colony optimization strategy, forming a "behavior-structure-update" closed-loop learning process. In the face of frequent hydrodynamic disturbances and dynamic changes in hydrological parameters, the system has the ability of rapid convergence and robust optimization. In actual measurements, the response delay to pollution diffusion caused by emergencies is greatly shortened, which significantly improves the system's emergency modeling capability in sudden pollution scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0074] Figure 1 This is a flow chart of a water microbial risk analysis method based on reinforcement learning proposed in the present invention. DETAILED DESCRIPTION

[0075] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0076] refer to Figure 1 , a water microbial risk analysis method based on reinforcement learning, including the following steps:

[0077] S1. Collect the original observation datasets in the target waters, perform time synchronization, patch missing data, remove anomalies, and unify dimensions on the original observation datasets, and generate a standardized observation dataset according to the waters spatial grid mapping rules;

[0078] S2. Construct a dynamic water body graph based on the standardized observation dataset. In this dynamic water body graph, monitoring sections, tributary inlets, potential pollution sources, and hydraulic control points are defined as graph nodes. Pathogen concentrations, water temperature, and pH values ​​from the standardized observation dataset are written into node attributes. Water flow velocities from the standardized observation dataset are written into edge weights. Flow directions, connectivity, and boundary attributes between nodes are defined as graph edge attributes.

[0079] S3. Perform dynamic graph representation learning on the dynamic water body graph structure. Use a standardized observation dataset containing the complete sequence of node attributes and edge weights at all time steps for training. Obtain a node vector representation matrix, and generate a spatiotemporal risk heat matrix based on the node vector representation matrix.

[0080] S4. Using the standardized observation dataset, the node vector representation matrix, and the spatiotemporal risk heat matrix, construct a Bayesian reward function prior distribution, initialize the reward function, and establish a set of reward function parameters.

[0081] S5. Using a standardized observation dataset as the behavioral outcome input on a dynamic water body graph, a maximum a posteriori inverse reinforcement learning model is used to incrementally learn the reward function parameter set, outputting the first propagation potential field.

[0082] S6. Using the first propagation potential field as the basis for fitness evaluation, the artificial bee colony optimization algorithm is invoked and the number of leader bees, follower bees, and scout bees is set. A global search is performed on the dynamic water map structure to generate a set of candidate diffusion paths. Based on this set of candidate diffusion paths and the first propagation potential field, local refinement is performed by follower bees and random perturbation by scout bees to obtain the optimal diffusion path sequence.

[0083] S7. Path reconstruction error analysis is performed on the optimal diffusion path sequence and the standardized observation dataset to obtain a dynamic feedback index. This dynamic feedback index is used to simultaneously update the reward function parameter set and the artificial bee colony optimization algorithm control parameters, forming a joint convergence iterative loop. When the joint convergence iterative loop meets the preset convergence threshold, the final diffusion path sequence, dynamic risk zone distribution map, and reward function sensitivity index are output and visualized on the dynamic water body map structure.

[0084] In this embodiment, the original observation data set includes pathogen concentration, water flow velocity, water temperature, pH, water flow direction, physical connectivity attributes and boundary contact attributes.

[0085] The boundary contact attribute identifies whether an edge in the graph is connected to a water body's boundary region, including inlets, outflows, sewage outlets, upstream confluences, and downstream discharge gates. These boundary points are often key areas where pathogens enter or leave the target water body.

[0086] The physical connectivity attribute is used to determine whether there is a real water physical channel between two spatial nodes, that is, to determine whether the two nodes are connected by a river, canal, or water pipe hydraulic structure system in the water body.

[0087] In this embodiment, S2 includes the following steps:

[0088] S21. Construct a spatial node set. Each spatial node is used to represent a monitoring section, tributary inlet, potential pollution source or hydraulic control point in the target water area. All spatial nodes together constitute a spatial node set. The spatial node set covers the entire water body research range. The nodes are numbered from 1 to ,in Indicates the total number of spatial nodes;

[0089] S22. For each spatial node, extract the pathogenic microorganism concentration of the spatial node at each time step , water temperature and pH , the pathogen concentration, water temperature and pH are used as the node attributes of the spatial node at the corresponding time step. The pathogen concentration, water temperature and pH together constitute the node attribute set of the spatial node. The node attribute set is used to characterize the physical and chemical state of the water body at the node at the time step; the unit of pathogen concentration is CFU / m³, and the unit of water temperature is °C;

[0090] S23. Based on the water flow direction and velocity recorded in the standardized observation dataset, establish directed edge relationships between spatial nodes. If there is a physical connection between two spatial nodes and the water flow direction points to the physical connection path, establish a directed edge from the upstream node to the downstream node. The directed edge set consists of all edges that meet the conditions.

[0091] S24. For each established directed edge, extract the water flow velocity corresponding to the directed edge at each time step. Use the water flow velocity value as the edge weight of the directed edge at that time step and construct an edge weight set. The edge weight is used to represent the dynamic strength of pollutants in the water body propagating along this channel.

[0092] S25. For each established directed edge, extract the edge's flow direction attribute, physical connectivity attribute, and boundary contact attribute from the standardized observation dataset to form the edge attribute set. The flow direction attribute is used to indicate whether the directed edge conforms to the actual flow direction of the water body; the physical connectivity attribute is used to indicate whether two nodes are truly connected; and the boundary contact attribute is used to indicate whether the directed edge is connected to the water body boundary. The flow direction attribute is set to "downstream" or "upstream"; the physical connectivity attribute and boundary contact attribute are Boolean values.

[0093] S26. Assemble the spatial nodes , directed edge set , node attribute collection , edge weight set and edge attribute set Perform unified packaging to build a dynamic water body graph structure ,The dynamic water graph structure is constructed based on time ,sequence, and a complete graph instance is generated at each time ,step.

[0094] In this embodiment, S3 includes the following steps:

[0095] S31. Based on dynamic water body graph structure , from the initial time step To the final time step Extract spatial node sets sequentially The node attribute set of each spatial node in and the directed edge set The set of edge weights for each edge in , forming a dynamic graph to represent the temporal input sequence of learning;

[0096] S32. For the time series input sequence, a node vector representation model is constructed based on the dynamic graph convolutional network. The node feature update process of the node vector representation model is defined as:

[0097] ;

[0098] in, Indicates in Spatial nodes in layer networks At time step The corresponding node feature vector, represents a nonlinear activation function, Indicates the The weight matrix of the layer network, Indicates the The bias term of the layer network, represents the neighbor node aggregation function, Representation and Node A set of adjacent spatial nodes;

[0099] S33. Using node vectors to represent the model arrive The dynamic graph temporal input sequence is forward propagated to train each spatial node At each time step The node feature vector of , forming a node vector representation matrix ,The node vector representation matrix is ​​used to express the spatiotemporal dynamic characteristics of spatial nodes;

[0100] S34. Representing matrices with node vectors As input, a spatiotemporal risk heat matrix generation model is constructed:

[0101] ;

[0102] in, Represents a spatial node At time step The risk heat value ranges from 0 to 1; Represents a parameter set Multilayer perceptron with parameters , is a normalization function, ensuring that the spatial nodes The sum of the risk heat values ​​at each time step is 1;

[0103] S35. Spatial node set Traverse all spatial nodes in the matrix to obtain the risk heat values ​​of all nodes at each time step, forming a complete spatiotemporal risk heat matrix , which is used to quantitatively characterize the diffusion risk intensity of water pathogenic microorganisms at different spatial nodes and time steps.

[0104] In this embodiment, S4 includes the following steps:

[0105] S41. Construct a physically driven prior space for the Bayesian reward function. This physically driven prior space uses pathogen concentration, water temperature, and pH from a standardized observational dataset as input, and combines a node vector representation matrix with a spatiotemporal risk heat matrix.

[0106] The construction process of the physical driving prior space of the Bayesian reward function is as follows:

[0107] The prior information input is based on the physical and chemical information of water bodies recorded in standardized observation datasets, mainly including three indicators: pathogen concentration, water temperature, and pH. Pathogen concentration measures the actual presence of pathogens at a spatial node, water temperature reflects the degree of environmental support for microbial survival, and pH indicates the chemical suitability of microbial survival in the water. Pathogen concentration, water temperature, and pH are spatiotemporally aligned according to spatial nodes and time steps to form the environmental driving vector for each node at each time step.

[0108] On this basis, the node environment driving vector is fused with the node vector representation matrix. The node vector representation matrix is ​​output by the dynamic graph representation learning model and is used to characterize the structural role and temporal evolution characteristics of spatial nodes in the propagation network. The fusion process splices the node physical and chemical state information and its potential diffusion ability representation in the dynamic graph into high-dimensional feature input, thereby forming a physical semantic feature set for reward function modeling.

[0109] The physical semantic feature set is combined with the spatiotemporal risk heat matrix. The spatiotemporal risk heat matrix provides the probability weight of the current node in the propagation graph being judged as a high-risk propagation channel by the model. It is used to adjust the degree of attention to different nodes and time steps in the prior modeling process. By jointly modeling the node physical and chemical state characteristics, graph structure representation characteristics and risk heat values, a Bayesian reward function physically driven prior space covering all spatial nodes and time steps is formed.

[0110] S42. Define the Bayesian reward function as a spatial node With time step The conditional probability distribution of each Bayesian reward function value Obey the prior mean function and covariance function The Gaussian process distribution is a parameter, the prior mean function is used to characterize the reward trend, and the covariance function is used to describe the degree of correlation between spatial nodes and time steps;

[0111] S43. Constructing the prior mean function of the Bayesian reward function The prior mean function consists of three weighted parts. The first part is the relative residual term of pathogen concentration, which is calculated as the target node at the time step The second part is the water temperature ratio, which is calculated as the ratio of the pathogen concentration at the node at time step Water temperature and maximum water temperature The third part is the pH deviation term, which is calculated as the node at the time step The absolute difference between the pH value and the neutral value of 7;

[0112] S44. Constructing the covariance function of the Bayesian reward function , the covariance function is based on the node With node Node vectors at different times With node vector The Euclidean distance between nodes is used as the input feature. The Euclidean distance is used to measure the similarity of nodes in the representation space. Finally, it is mapped through an exponential decay function controlled by the scaling coefficient and the feature scale parameter, and the covariance value representing the correlation of the reward value is output;

[0113] Exponential decay function with nodes With node The core input is the Euclidean distance between node vectors at a given time step. The Euclidean distance is obtained by element-by-element subtraction of the vectors of two nodes in the node vector representation matrix, squaring them, and then summing them up. This measures the difference in the ecological environment and propagation status of the two nodes in the high-dimensional feature space.

[0114] In order to convert the distance into the covariance value, a scaling factor and a characteristic scale parameter are introduced as control factors. The scaling factor is used to control the overall amplitude of the covariance function, and the characteristic scale parameter is used to adjust the sensitivity of the function to the differences between nodes. In the actual calculation, the square of the Euclidean distance is divided by twice the square of the characteristic scale parameter, and the negative value is taken. It is then mapped with the natural exponential function to obtain the final covariance value.

[0115] S45. The prior mean function With covariance function The complete set of prior distributions that jointly form the Bayesian reward function , the prior distribution set covers all spatial nodes and all time steps, and is used to provide the initial structural relationship of the reward function in the spatial and temporal dimensions;

[0116] S46. Prior distribution set based on Bayesian reward function , initialize the reward function parameter set , each element in the sample set Representation node At time step The initial reward estimate of , all estimates constitute the initial sample set.

[0117] In this embodiment, S5 includes the following steps:

[0118] S51. Using the pathogen concentration distribution in a standardized observation dataset as the expert behavior sample input, a maximum a posteriori inverse reinforcement learning model integrating a two-layer attention mechanism and dynamic Bayesian updating is constructed based on the dynamic water body graph structure, node vector representation matrix, and Bayesian reward function parameter set.

[0119] S52. In the maximum a posteriori inverse reinforcement learning model, the spatial node set A collection of node attributes As input, a node-level attention mechanism is constructed, which is used to learn spatial nodes. At time step State weight :

[0120] ;

[0121] in, represents the learnable weight vector, Represents the vector concatenation operation, and LeakyReLU is the nonlinear activation function;

[0122] In traditional inverse reinforcement learning, each spatial node is usually regarded as equally weighted in strategy learning, and the differences in the importance of nodes in the spread of pathogenic microorganisms cannot be distinguished. Especially in scenarios with complex water structures or highly uneven distribution of pollution sources, it is easy to cause strategy learning to overfit low-risk nodes.

[0123] The node-level attention mechanism introduced in S52 calculates the weight coefficient of each node at each time step by integrating the spatiotemporal representation characteristics of the node and the current reward value. , achieving risk focus on key nodes, so that the reinforcement learning model automatically tends to information-intensive areas, communication hub areas or areas of intense change during training, thereby improving the efficiency of reward function fitting, shortening the model convergence time, and enhancing the physical interpretability of node weights.

[0124] S53. State weights based on node-level attention mechanism output , construct a time-level attention mechanism, which is used to capture the time-level attention weights of the pathogen diffusion path changing with time steps :

[0125] ;

[0126] in, For parameter collection Multilayer perceptron with parameters

[0127] Traditional time series reinforcement learning methods often treat each time step in an equal-weighted manner, ignoring the fact that the propagation behavior in pollution events is phased, delayed, and sudden, which in turn affects the generalization ability of strategy learning and the accuracy of timeliness judgment.

[0128] The time-level attention mechanism introduced in S53 further aggregates the dynamic evolution trend across nodes based on node attention and generates the state weight of each time step. , dynamically emphasizes the key time window of transmission, thereby effectively enhancing the response sensitivity of the model in the key stages of pollution acceleration, turning point and diffusion, and improving the strategy's resolution and emergency adaptability in the time dimension.

[0129] S54. State weights based on node-level attention mechanism output and temporal attention weights , reconstruct the target optimization function of maximum a posteriori inverse reinforcement learning :

[0130] ;

[0131] in, For nodes In the reward function The concentration of pathogenic microorganisms is shown below The likelihood probability, is the prior distribution of the Bayesian reward function;

[0132] S54 builds on the traditional maximum a posteriori objective function by introducing the attention weights provided by S52 and S53, achieving a joint weighted optimization of space and time, thereby reconstructing the loss function. This reconstruction not only enhances the model's ability to selectively focus on data but also guides the model during the optimization process to "focus on fitting high-weight regions and reduce invalid gradient propagation." The resulting reward function is more globally representative and physically consistent.

[0133] S55. Implementing the objective optimization function of maximum a posteriori inverse reinforcement learning through dynamic Bayesian updating Iterative update, update the reward function parameter set , the update rule is defined as:

[0134] ;

[0135] in, is the learning rate, represents the gradient of the optimization function with respect to the reward function parameters;

[0136] S56. In the reward function parameter set After completing the iterative update and reaching the predetermined convergence threshold, the optimized reward function set is output, and the first propagation potential field is generated based on the optimized reward function set. :

[0137] ;

[0138] in, is the optimal diffusion strategy after optimization, is the discount factor, Indicates the optimal diffusion strategy The expected value calculation operator for the future propagation process, the first propagation potential field Used to accurately describe the optimal transmission path and spatiotemporal risk evolution trend of water pathogenic microorganisms in the dynamic water body graph structure.

[0139] In this embodiment, S6 includes the following steps:

[0140] S61. Calculate the path fitness value for each diffusion path. The path fitness value is used to measure the propagation capability of the diffusion path in the propagation potential field. The path fitness value is calculated by multiplying the propagation potential energy values ​​of all spatial nodes on the diffusion path at the corresponding time step by their risk weight coefficients in the path and then summing them. The propagation potential energy value is obtained from the first propagation potential field. The risk weight coefficient is used to reflect the importance of the pathogen's propagation at different locations along the path. The diffusion path fitness value serves as an evaluation indicator for artificial bee colony optimization.

[0141] S62. Initialize the set of bee colony individuals for the artificial bee colony optimization algorithm, set the number of leader bees, follower bees, and scout bees, and generate an initial set of paths that match the number of leader bees, follower bees, and scout bees. Each path individual consists of a set of spatial nodes. Adjacent spatial nodes in the path must meet the edge connectivity, flow direction consistency, and boundary contact requirements of the dynamic water body graph structure. All path sets together constitute the initial search solution space for the artificial bee colony optimization.

[0142] S63. For each path generated by the leader bee, perform a path update operation according to the gradient direction of the propagation potential field. The path update operation inserts new spatial nodes or adjusts the path direction along the direction of increasing propagation potential energy on the basis of the original path, so that the path tends to the area of ​​high propagation potential energy. The step size is controlled by the global step size coefficient, and the gradient direction is determined by the potential energy difference between adjacent nodes in the propagation potential field.

[0143] S64. For each path optimized by a follower bee, a local refinement operation is performed based on the path with the highest fitness in the current propagation potential field. This local refinement operation includes inserting points, deleting points, or replacing path segments in the path, and introducing fine-grained structural changes to the original path by adding Gaussian perturbations. The mean of the Gaussian perturbation is zero, the variance is controlled by a set characteristic scale parameter, and the perturbation intensity is controlled by a local perturbation coefficient to enhance the local exploration capability of the path.

[0144] S65. Scout bees are responsible for executing a fully randomized path search mechanism. Randomized paths are constructed by randomly selecting connected spatial nodes on the dynamic water body graph structure. If the cumulative propagation potential energy of a randomized path exceeds the cumulative propagation potential energy of the worst path in the current path set by more than a minimum improvement threshold, the diffusion path will replace the worst path in the original set to avoid falling into local optimality and improve global search capabilities.

[0145] S66. Repeat the global guided search by the leader bees, the local refined search by the follower bees, and the random search mechanism by the scout bees. In each iteration, the path with the highest current fitness is recorded and used as the reference optimal solution for the path population. When the maximum number of iterations is reached or the improvement in the path fitness value is less than the preset termination threshold, the artificial bee colony optimization process is terminated.

[0146] S67. After the artificial bee colony optimization process is terminated, the final optimal diffusion path is output. The optimal diffusion path is the path with the highest cumulative value of propagation potential energy among all current path individuals. The diffusion path represents the most likely propagation path for pathogenic microorganisms to spread from potential sources to high-risk areas in the dynamic water body graph structure.

[0147] In this embodiment, S7 includes the following steps:

[0148] S71. Perform path reconstruction error analysis on the optimal diffusion path sequence output by the artificial bee colony optimization algorithm and the pathogen concentration distribution in the standardized observation dataset. Path reconstruction error analysis is used to evaluate the degree of fit between the diffusion path generated by the model and the actual observation in terms of spatial node distribution and temporal concentration evolution.

[0149] S72. Construct a dynamic feedback indicator set based on the path reconstruction error analysis results. The dynamic feedback indicator set includes spatial node matching error, time series concentration fitting error, and propagation direction consistency error.

[0150] S73. Using the dynamic feedback indicator set as the evaluation basis for the joint optimization update, the reward function parameter set is dynamically updated using the spatial node matching error and the time series concentration fitting error feedback information. The update method is to perform a posteriori adjustment on the current reward function sampling distribution, regenerate the reward function sample set, and use the propagation direction consistency error feedback information to adjust the control parameters of the artificial bee colony optimization algorithm. The control parameters include the leader bee perturbation intensity, the follower bee local search range, and the scout bee reinitialization probability.

[0151] S74. Re-input the reward function parameter set and the updated artificial bee colony optimization control parameters into the maximum a posteriori inverse reinforcement learning model and the artificial bee colony optimization algorithm, and continue to perform reward function training and diffusion path search, forming a joint convergence iterative loop;

[0152] S75. Determine whether the joint convergence iteration cycle meets the preset convergence threshold. If the change in the reconstruction error of the optimal diffusion path sequence is lower than the error threshold in several consecutive rounds, or the overall model evaluation index tends to be stable, convergence is determined, the iteration is terminated, and the final result is output;

[0153] S76. Output the final diffusion path sequence, dynamic risk area distribution map, and reward function sensitivity index:

[0154] Final diffusion path sequence: represents the optimal path trajectory for water pathogens to diffuse from the initial high-risk source node to the downstream. The node sequence satisfies the flow direction constraint and has the maximum cumulative potential energy in the propagation potential field.

[0155] Dynamic risk zone distribution map: The risk index value of each spatial node at each time step is calculated based on the final diffusion path sequence and propagation potential field. The risk index value is used to divide the risk level area into high-risk areas with a risk index value ≥ 0.8, medium-risk areas with a risk index value of 0.5≤<0.8, and low-risk areas with a risk index value of <0.5.

[0156] Reward function sensitivity index: Based on the partial derivative response between the final converged reward function sample set and the physical and chemical indicators in the standardized observation data set, the sensitivity score of each driving factor to the change of the reward function is output. Factors with a sensitivity score greater than 0.3 are defined as key propagation drivers;

[0157] S77. The final diffusion path sequence, dynamic risk zone distribution map, and reward function sensitivity index are standardized and mapped to layers, and visualized on the dynamic water body map structure. It also supports the export of illustrated risk tracking reports and model traceability analysis materials.

[0158] In this embodiment, the spatial node matching error measures the degree of overlap between the node set in the optimal diffusion path sequence and the observed high-concentration node set;

[0159] Time series concentration fitting error: measures the degree of difference between the propagation potential field value of each node in the diffusion path at the corresponding time step and the concentration value in the standardized observation data set;

[0160] Propagation direction consistency error: measures the degree of consistency between the directions between path nodes and the flow directions recorded in the edge attribute set in the dynamic water body graph structure.

[0161] Example 1:

[0162] During a certain online water quality monitoring session, the system detected continuous abnormalities in the concentration of pathogenic microorganisms at a downstream water intake point. Specifically, the concentration value recorded by the monitoring device with sensor number "WS-305" between 07:30 and 08:30 rapidly increased from a stable 112 CFU / m³ to 412 CFU / m³. This increase was accompanied by abnormal changes in water temperature, from the normal 21.3°C to 24.6°C. The abnormal data exceeded the internally set concentration change rate warning threshold (an increase of more than 200 CFU / m³ per hour). The system triggered a "primary pollution warning" signal, requiring a traceability analysis of the upstream area.

[0163] The system automatically calls the pathogen diffusion path inversion method proposed in this invention and imports nearly 12 hours of historical observation data. The total data volume is 7,680, covering 72 water section nodes. Each node contains five standardized indicators: pathogen concentration, water temperature, pH value, flow velocity and flow direction. The system automatically constructs a dynamic water body graph structure with 72 nodes, 213 edges, a time step of 24 frames, and an interval of 30 minutes per frame.

[0164] During the node representation learning stage, the vector representation values ​​of the node area from "P-22" to "P-27" showed a significant clustering trend. After normalized encoding by the graph representation learning model, the risk heat value of this area exceeded 0.92 for three consecutive frames, becoming a spatiotemporal heat focus area.

[0165] During the risk-driven modeling phase, the prior mean of the reward function significantly depends on three dimensions: the distribution of standardized concentration residuals, the magnitude of water temperature fluctuations, and the degree of pH deviation from neutrality. For example, at node "P-25" at the 08:00 timeframe, its concentration was 393 CFU / m³, corresponding to a standardized residual of 0.77, a water temperature of 24.4°C (a deviation of 2.3°C from the mean), and a pH of 8.1 (a deviation of +1.1 from neutral). This resulted in a prior reward value of 0.86 and a spatial similarity of 0.94 with its neighboring nodes in the covariance function.

[0166] Subsequently, the system executed the maximum a posteriori inverse reinforcement learning process integrated with the attention mechanism. After the 13th iteration, a high-reward path was formed from nodes "P-20" to "P-28". The corresponding maximum value of the propagation potential field appeared at node "P-24" with a value of 0.97. The path was input into the artificial bee colony optimization module. The initial bee colony contained 60 path individuals. After 12 rounds of search, "P-18 → P-20 → P-23 → P-25 → P-27 → P-29" was finally selected as the optimal diffusion path.

[0167] The system automatically performed an error reconstruction assessment on the optimal path, resulting in a fitting error of 0.074, which is better than the average value (0.13) in the system's historical records. The system also recorded the cumulative risk score for this path in the propagation potential field as 5.74 (with a threshold of 3.5). Based on this result, the system labeled the area between paths "P-18 and P-29" as a "high-risk diffusion zone" and generated an automatic risk level report, assigning it a "Level II: Moderate Pollution Warning" rating. The report also included a dynamic risk heat map, a key indicator offset map, and a diffusion direction trend map.

[0168] The report was simultaneously pushed to the management terminal via the interface. Management personnel received the alert at 08:45 and, following the platform's recommendations, initiated an investigation of the upstream river inlet. On-site staff discovered recent discharge of construction wastewater at "River Inlet Number - R12." Water sample testing revealed an E. coli concentration of 678 CFU / m³, highly consistent with the starting point concentration in the system's inversion path. Subsequently, the river inlet was temporarily sealed, and source control measures were completed at 09:20.

[0169] To verify the practicality of the proposed method, the research team compared the system's performance with that of traditional reverse numerical simulation. Using the same amount of data, the traditional method identified the path "P-17 → P-22 → P-26 → P-30" in this scenario, with a path reconstruction error of 0.162 and a fitting time of 14.8 minutes. However, the proposed method achieved a 94.2% overlap between the identified path and the actual contamination point, a fitting time of only 3.7 minutes, and an 11-minute advance warning period, effectively preventing water from highly polluted areas from entering the drinking water treatment system.

[0170] The system further evaluated the sensitivity of the reward function and conducted an attribution analysis on the water temperature, pH value and concentration influence weights of each node in the path. The results showed that in the reward value change of node "P-25", the water temperature factor weight was 0.41, the concentration residual was 0.36, and the pH value was 0.23, indicating that in this scenario, the high temperature environment has a more significant driving effect on the active spread of pathogens.

[0171] In summary, this example demonstrates how, in scenarios where pollution incidents occur suddenly, monitoring points are sparse, and water structures are complex, the method of the present invention, by integrating reinforcement learning mechanisms with swarm intelligence search methods, can efficiently construct water diffusion paths, dynamically correct propagation potentials, quickly locate sources, and assist decision makers in completing effective interventions. This fully verifies the method's engineering adaptability and strategic intelligence.

[0172] This paper introduces a two-layer attention mechanism based on the traditional inverse reinforcement learning structure: the node-level attention mechanism can dynamically adjust the node weights according to the transmission potential of pathogenic microorganisms in different spatial nodes; the time-level attention mechanism dynamically adjusts the weight distribution of each time step in the learning process based on the time evolution trend of the diffusion path, so that the model can focus more effectively on key transmission nodes and time periods, thereby learning a reward function that is more in line with the water flow direction and pathogen dynamic mechanism. It not only improves the strategy convergence efficiency, but also enhances the physical interpretability of the model, providing direct support for subsequent public health interventions.

[0173] The present invention improves the traditional artificial bee colony optimization strategy. It introduces a propagation potential field gradient guidance mechanism in the leader bee stage to realize global guided path construction, performs path refinement optimization through Gaussian perturbation in the follower bee stage, and introduces structural consistency constraints in the scout bee stage to improve the physical rationality of path generation. At the same time, with propagation potential energy as the core of fitness evaluation, it conducts multiple rounds of iterative searches for the optimal path sequence in the dynamic water body map structure, effectively avoiding the common problem of falling into local optimality, and improving the structural fidelity of the optimal path in real complex watershed scenarios.

[0174] The present invention establishes a dynamic convergence mechanism for the joint update of the reward function and path search, improves the adaptability and real-time performance of the model to changes in water conditions, and realizes the joint adaptive update of the reward function parameters and the optimization algorithm control parameters by introducing the path reconstruction error feedback mechanism. In each round of iteration, the spatial node matching error and the propagation direction consistency error dynamic feedback indicators are used to adjust the inverse reinforcement learning model and the artificial bee colony optimization strategy, forming a "behavior-structure-update" closed-loop learning process. In the face of scenarios with frequent hydrodynamic disturbances and dynamic changes in hydrological parameters, the process has the ability of rapid convergence and robust optimization. In actual measurements, the response delay to pollution diffusion caused by emergencies is greatly shortened, which significantly improves the system's emergency modeling capability in sudden pollution scenarios.

[0175] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A water microbial risk analysis method based on reinforcement learning, characterized in that: The steps include: S1. Collect and preprocess the original observation dataset in the target waters to generate a standardized observation dataset; S2. Construct dynamic water body maps based on standardized observation datasets; S3. Perform dynamic graph representation learning on the dynamic water body graph structure to obtain a node vector representation matrix. Based on the node vector representation matrix, generate a spatiotemporal risk heat matrix. S4. Using the standardized observation dataset, the node vector representation matrix, and the spatiotemporal risk heat matrix, construct a Bayesian reward function prior distribution, initialize the reward function, and establish a set of reward function parameters. S5. Using a standardized observation dataset as the behavioral outcome input on a dynamic water body graph, a maximum a posteriori inverse reinforcement learning model is used to incrementally learn the reward function parameter set, outputting the first propagation potential field. S6. Using the first propagation potential field as the basis for fitness evaluation, the artificial bee colony optimization algorithm performs a global search on the dynamic water body graph structure to generate a set of candidate diffusion paths. Based on this set of candidate diffusion paths and the first propagation potential field, follower bee local refinement and scout bee random perturbation operations are performed to obtain the optimal diffusion path sequence. S7. Perform path reconstruction error analysis on the optimal diffusion path sequence and the standardized observation dataset to obtain dynamic feedback indicators. This results in a joint convergence iterative loop. When the loop ends, the final diffusion path sequence, dynamic risk zone distribution map, and reward function sensitivity indicator are output. The S6 comprises the following steps: S61. Calculate the path fitness value of each diffusion path. The path fitness value is calculated by multiplying the propagation potential energy values ​​of all spatial nodes on the diffusion path at the corresponding time step by their risk weight coefficients in the path and then summing them. The propagation potential energy value is obtained from the first propagation potential field. S62. Initialize the set of bee colony individuals for the artificial bee colony optimization algorithm, set the number of leader bees, follower bees, and scout bees, and generate an initial set of paths with the same number of leaders, follower bees, and scout bees. Each path individual consists of a set of spatial nodes. S63. For each path generated by the leader bee, perform a path update operation according to the gradient direction of the propagation potential field. The path update operation inserts new spatial nodes or adjusts the path direction along the direction of the rising propagation potential energy based on the original path; S64. For each path optimized by the follower bee, perform a local refinement operation based on the path with the highest fitness in the current propagation potential field; S65. Scout bees are responsible for executing a fully randomized path search mechanism. Randomized paths are constructed by randomly selecting connected spatial nodes on the dynamic water body graph structure. If the cumulative propagation potential energy of a randomized path exceeds the cumulative propagation potential energy of the worst path in the current path set by more than a minimum improvement threshold, the diffusion path will replace the worst path in the original set. S66. Repeat the global guided search by the leader bees, the local refined search by the follower bees, and the random search mechanism by the scout bees. In each iteration, the path with the highest current fitness is recorded and used as the reference optimal solution for the path population. When the maximum number of iterations is reached or the improvement in the path fitness value is less than the preset termination threshold, the artificial bee colony optimization process is terminated. S67. After the artificial bee colony optimization process is terminated, the final optimal diffusion path is output. The optimal diffusion path is the path with the highest cumulative value of propagation potential energy among all current path individuals. The diffusion path represents the most likely propagation path for pathogenic microorganisms to spread from potential sources to high-risk areas in the dynamic water body graph structure.

2. The water microbial risk analysis method based on reinforcement learning according to claim 1 is characterized in that: The original observation data set includes pathogen concentration, water flow velocity, water temperature, pH, water flow direction, physical connectivity attributes and boundary contact attributes.

3. The water microbial risk analysis method based on reinforcement learning according to claim 1 is characterized in that: The S2 comprises the following steps: S21. Construct a spatial node set. Each spatial node is used to represent a monitoring section, tributary inlet, potential pollution source or hydraulic control point in the target water area. All spatial nodes together constitute a spatial node set. The spatial node set covers the entire water body research range. The nodes are numbered from 1 to ,in Indicates the total number of spatial nodes; S22. For each spatial node, extract the pathogen concentration, water temperature, and pH value of the spatial node at each time step. These pathogen concentration, water temperature, and pH value are used as the node attributes of the spatial node at the corresponding time step. The pathogen concentration, water temperature, and pH value together constitute the node attribute set of the spatial node. S23. Based on the water flow direction and velocity recorded in the standardized observation dataset, establish directed edge relationships between spatial nodes. If there is a physical connection between two spatial nodes and the water flow direction points to the physical connection path, establish a directed edge from the upstream node to the downstream node. The directed edge set consists of all edges that meet the conditions. S24. For each established directed edge, extract the water flow velocity corresponding to the directed edge at each time step. Use the water flow velocity value as the edge weight of the directed edge at that time step and construct an edge weight set. The edge weight is used to represent the dynamic strength of pollutants in the water body propagating along this channel. S25. For each established directed edge, extract the edge's water flow direction attribute, physical connectivity attribute, and boundary contact attribute from the standardized observation dataset to form an edge attribute set for the edge; S26. Assemble the spatial nodes , directed edge set , node attribute collection , edge weight set and edge attribute set Perform unified packaging to build a dynamic water body graph structure .

4. The water microbial risk analysis method based on reinforcement learning according to claim 1 is characterized in that: The S3 includes the following steps: S31. Based on dynamic water body graph structure , from the initial time step To the final time step Extract spatial node sets sequentially The node attribute set of each spatial node in and the directed edge set The set of edge weights for each edge in , forming a dynamic graph to represent the temporal input sequence of learning; S32. For the time series input sequence, a node vector representation model is constructed based on the dynamic graph convolutional network. The node feature update process of the node vector representation model is defined as: ； in, Indicates in Spatial nodes in layer networks At time step The corresponding node feature vector, represents a nonlinear activation function, Indicates the The weight matrix of the layer network, Indicates the The bias term of the layer network, represents the neighbor node aggregation function, Representation and Node A set of adjacent spatial nodes; S33. Using node vectors to represent the model arrive The dynamic graph temporal input sequence is forward propagated to train each spatial node At each time step The node feature vector of , forming a node vector representation matrix ,The node vector representation matrix is ​​used to express the spatiotemporal dynamic characteristics of spatial nodes; S34. Representing matrices with node vectors As input, a spatiotemporal risk heat matrix generation model is constructed; S35. Generate a model for spatial node sets based on the spatiotemporal risk heat matrix Traverse all spatial nodes in the matrix to obtain the risk heat values ​​of all nodes at each time step, forming a complete spatiotemporal risk heat matrix .

5. The water microbial risk analysis method based on reinforcement learning according to claim 1 is characterized in that: The S4 comprises the following steps: S41. Construct a physically driven prior space for the Bayesian reward function. The physically driven prior space uses a standardized observation dataset as input and combines a node vector representation matrix with a spatiotemporal risk heat matrix. S42. Define the Bayesian reward function as a spatial node With time step The conditional probability distribution of each Bayesian reward function value Obey the prior mean function and covariance function is a Gaussian process distribution with parameters; S43. Constructing the prior mean function of the Bayesian reward function ,The prior mean function consists of three weighted parts, the first part is the relative residual term of pathogenic microorganism concentration, the second part is the water temperature proportion term, and the third part is the pH deviation term; S44. Construct the covariance function of the Bayesian reward function. The covariance function is based on the node With node Node vectors at different times With node vector The Euclidean distance between them is used as the input feature, mapped by the exponential decay function controlled by the scaling coefficient and the feature scale parameter, and the covariance value representing the correlation of the reward value is output; S45. The prior mean function With covariance function The complete set of prior distributions that jointly form the Bayesian reward function , the prior distribution set covers all spatial nodes and all time steps, and is used to provide the initial structural relationship of the reward function in the spatial and temporal dimensions; S46. Prior distribution set based on Bayesian reward function , initialize the reward function parameter set , each element in the sample set Representation node At time step The initial reward estimate of , all estimates constitute the initial sample set.

6. The water microbial risk analysis method based on reinforcement learning according to claim 1 is characterized in that: The S5 comprises the following steps: S51. Using the pathogen concentration distribution in a standardized observation dataset as the expert behavior sample input, a maximum a posteriori inverse reinforcement learning model integrating a two-layer attention mechanism and dynamic Bayesian updating is constructed based on the dynamic water body graph structure, node vector representation matrix, and Bayesian reward function parameter set. S52. In the maximum a posteriori inverse reinforcement learning model, the spatial node set A collection of node attributes As input, a node-level attention mechanism is constructed, which is used to learn spatial nodes. At time step State weight ; S53. State weights based on node-level attention mechanism output , construct a time-level attention mechanism, which is used to capture the time-level attention weights of the pathogen diffusion path changing with time steps ; S54. State weights based on node-level attention mechanism output and temporal attention weights , reconstruct the target optimization function of maximum a posteriori inverse reinforcement learning : ； in, For nodes In the reward function The concentration of pathogenic microorganisms is shown below The likelihood probability, is the prior distribution of the Bayesian reward function; S55. Implementing the objective optimization function of maximum a posteriori inverse reinforcement learning through dynamic Bayesian updating Iterative update, update the reward function parameter set ; S56. In the reward function parameter set After completing the iterative update and reaching the predetermined convergence threshold, the optimized reward function set is output, and the first propagation potential field is generated based on the optimized reward function set. : ； in, is the optimal diffusion strategy after optimization, is the discount factor, Indicates the optimal diffusion strategy The expected value calculation operator for the future propagation process.

7. The water microbial risk analysis method based on reinforcement learning according to claim 1 is characterized in that: The S7 comprises the following steps: S71. Perform path reconstruction error analysis on the optimal diffusion path sequence output by the artificial bee colony optimization algorithm and the pathogen concentration distribution in the standardized observation dataset; S72. Construct a dynamic feedback indicator set based on the path reconstruction error analysis results. The dynamic feedback indicator set includes spatial node matching error, time series concentration fitting error, and propagation direction consistency error. S73. Using the dynamic feedback indicator set as the evaluation basis for the joint optimization update, dynamically updating the reward function parameter set using the spatial node matching error and the temporal concentration fitting error feedback information; S74. Re-input the reward function parameter set and the updated artificial bee colony optimization control parameters into the maximum a posteriori inverse reinforcement learning model and the artificial bee colony optimization algorithm, and continue to perform reward function training and diffusion path search, forming a joint convergence iterative loop; S75. Output the final diffusion path sequence, dynamic risk area distribution map, and reward function sensitivity index: Final diffusion path sequence: represents the optimal path trajectory for water pathogens to diffuse from the initial high-risk source node to the downstream. The node sequence satisfies the flow direction constraint and has the maximum cumulative potential energy in the propagation potential field. Dynamic risk zone distribution map: The risk index value of each spatial node at each time step is calculated based on the final diffusion path sequence and propagation potential field. The risk index value is used to divide the risk level area into high-risk areas with a risk index value ≥ 0.8, medium-risk areas with a risk index value of 0.5≤<0.8, and low-risk areas with a risk index value of <0.

5. Reward function sensitivity index: Based on the partial derivative response between the final converged reward function sample set and the physical and chemical indicators in the standardized observation data set, the sensitivity score of each driving factor to the change of the reward function is output. Factors with a sensitivity score greater than 0.3 are defined as key propagation drivers; S76. Standardize the format and layer map the final diffusion path sequence, dynamic risk area distribution map, and reward function sensitivity index; The spatial node matching error measures the degree of overlap between the node set in the optimal diffusion path sequence and the observed high-concentration node set; Time series concentration fitting error: measures the degree of difference between the propagation potential field value of each node in the diffusion path at the corresponding time step and the concentration value in the standardized observation data set; Propagation direction consistency error: measures the degree of consistency between the directions between path nodes and the flow directions recorded in the edge attribute set in the dynamic water body graph structure.

Images