Infectious disease prevention and control method and system, computer equipment and medium thereof

Abstract

The invention discloses an infectious disease prevention and control method and system, computer equipment and a medium thereof, and is applied to the technical field of intelligent control optimization, and the method comprises the steps: formalizing an infectious disease prevention and control problem into a Markov decision process based on a constructed meta-population warehouse model; defining a plurality of observation levels including complete observability, complete report observation and partial observability; estimating an unobserved infection state by adopting an information reconstruction method based on the current observation information under the partially observable level to obtain reconstructed state information; inputting the reconstructed state information into a target agent, and outputting a target prevention and control strategy; according to the method, the meta-population chamber model is constructed and formalized into the Markov decision process, the observation level is defined, incomplete observation is made up through information reconstruction, and a prediction method combining an ODE mechanism and GCN space weighting is adopted, so that the target agent can output a more scientific and accurate prevention and control strategy, and the efficiency and accuracy of infectious disease prevention and control are effectively improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent control optimization, and in particular to an infectious disease prevention and control method and system, a computer device and a medium thereof. BACKGROUND

[0002] In the early stage of a new infectious disease, specific vaccines and drugs have not yet been developed, and non-drug public health interventions are usually needed to break the transmission chain. Although such measures can effectively inhibit the spread of the disease, they still face many challenges in the implementation process: on the one hand, the infection rate needs to be effectively controlled, and on the other hand, the implementation cost and social impact need to be considered comprehensively. In particular, in the context of significant regional differences, how to achieve precise and efficient prevention and control becomes a key problem.

[0003] To overcome these defects, the application provides an infectious disease prevention and control method, system, computer device and medium thereof. SUMMARY

[0004] The purpose of the present application is to provide an infectious disease prevention and control method, system, computer device and medium thereof, which aims to solve the problem of incomplete observation information in actual infectious disease prevention and control on the basis of complete observation information.

[0005] To achieve the above-mentioned purpose, the application provides the following technical solutions: In a first aspect, the application provides an infectious disease prevention and control method, comprising the steps of: constructing a meta-population compartment model according to the transmission dynamics of the infectious disease; formalizing the infectious disease prevention and control problem as a Markov decision process based on the meta-population compartment model; defining multiple observation levels, including complete observability, complete reporting observability and partial observability; wherein, under the partial observability level, based on the current observation information, an information reconstruction method is used to estimate the unobserved infection state to obtain reconstructed state information; inputting the reconstructed state information into a target agent to obtain a target prevention and control strategy output by the target agent; wherein, the target agent is an agent model trained based on the Markov decision process.

[0006] Further, the step of constructing a meta-population compartment model according to the transmission dynamics of the infectious disease specifically comprises: the meta-population compartment model is a common differential equation model based on a meta-population, constructed from an SEIQR model; the meta-population compartment model divides exposed individuals into undetected exposed individuals and detected exposed individuals, divides infected individuals into undetected infected individuals, detected infected individuals and reported infected individuals, and divides quarantined individuals into quarantined exposed individuals and quarantined infected individuals.

[0007] Furthermore, the step of formalizing the infectious disease control problem into a Markov decision process based on the meta-population cell model specifically includes: The Markov decision process includes the observed state, control actions, state transition probabilities, and reward function. The observation status includes local status and global status. The local status includes infection scale indicators and newly exposed person indicators for each region. The global status is an aggregation of the local status of each region. The aforementioned prevention and control measures include detection and isolation measures in each region; The reward function is a weighted negative value of infection cost, testing cost, and isolation cost.

[0008] Furthermore, the information reconstruction method includes any one or a combination of the following: Mechanism reconstruction method based on propagation dynamics; Data-driven reconstruction methods based on spatial interpolation; A data-driven reconstruction method based on neural networks; A hybrid reconstruction method that integrates propagation dynamics mechanism and neural network weighting.

[0009] Furthermore, the hybrid reconstruction method combining the fusion propagation dynamics mechanism and the neural network weighting method includes: Using the flow matrix and population data in the metapopulation cell model, spatial dependency weights are calculated through a mechanistic model. The spatial dependency weights are embedded into a graph convolutional network for feature extraction, and the temporal dependency is obtained by combining a recurrent neural network. Output the reconstructed status information.

[0010] Secondly, this application provides an infectious disease prevention and control system, specifically including: The model building module constructs a metapopulation chamber model based on the transmission mechanics of infectious diseases; based on the metapopulation chamber model, the infectious disease prevention and control problem is formalized into a Markov decision process. The observation definition module defines multiple observation levels, including fully observable, fully reported observation, and partially observable. Under the partially observable level, based on the current observation information, an information reconstruction method is used to estimate the unobserved infection status and obtain the reconstructed status information. The information reconstruction and strategy generation module inputs the reconstructed state information into the target agent to obtain the target prevention and control strategy output by the target agent; wherein, the target agent is an agent model trained based on the Markov decision process.

[0011] Thirdly, this application provides a computer device, the computer device including a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a method for infectious disease prevention and control; the processor is used to execute the program instructions stored in the memory to implement an infectious disease prevention and control method.

[0012] Fourthly, this application provides a storage medium storing processor-executable program instructions for executing an infectious disease prevention and control method.

[0013] This application provides a method, system, computer equipment, and medium for infectious disease prevention and control, which have the following beneficial effects: To address the problem of incomplete observation, this application constructs a meta-population chamber model to simulate the dynamic spread of infectious diseases in different regions and populations. It further formalizes the infectious disease control problem into a Markov decision process, clarifying key elements such as states, actions, and rewards in the control process, providing a solid theoretical framework for precise decision-making. Addressing the incompleteness of observational information in real-world control efforts, multiple observation levels are defined, including fully observable, fully reported observation, and partially observable. Particularly at the partially observable level, an information reconstruction method is used to accurately estimate unobserved infection states, obtaining reconstructed state information that effectively compensates for missing observational data, providing more comprehensive and accurate data support for subsequent decision-making. The reconstructed state information is input into a target agent trained based on the Markov decision process, outputting a target control strategy. Compared to traditional methods, this approach improves the scientific rigor and relevance of the control strategy. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating an infectious disease prevention and control method according to Embodiment 1 of this application; Figure 2 This is a schematic diagram of the architecture of the target intelligent agent in Embodiment 1 of this application; Figure 3 This is a schematic diagram of the structure of an infectious disease prevention and control system according to Embodiment 2 of this application; Figure 4 This is a schematic diagram of the computer device structure according to Embodiment 3 of this application; Figure 5 This is a schematic diagram of the storage medium structure of Embodiment 4 of this application. Detailed Implementation

[0015] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0016] The following analysis, based on relevant technologies, examines existing solutions.

[0017] The optimization of intervention strategies for infectious diseases can be viewed as a control optimization problem from the perspective of modern control theory. Specifically, a mathematical model is constructed based on the transmission dynamics of infectious diseases, and control variables are embedded in the mathematical model to optimize the strategy by combining control effects and control costs [1-3]. Among the classic methods are optimal control and model predictive control (MPC). For example, Bolzoni et al. [4] constructed a SIR model, considering strategies such as vaccination and isolation treatment, and used the Pontryagin maximum principle to solve for the optimal control measures. Morato et al. [5] constructed a SIR model and used model prediction. It can adjust measures in real time according to the dynamics of case development, and minimize the number of deaths while minimizing economic and social costs. However, both optimal control and MPC require certain simplification of the infectious disease model, making it difficult to accurately describe the complex dynamics of infectious disease transmission in real-world scenarios.

[0018] Unlike traditional control methods, reinforcement learning (RL) does not rely on precise models and learns optimal policies through interaction with the environment, providing a new path. Thanks to the development of deep reinforcement learning (DRL) technology in recent years, some researchers have begun to apply DRL to the problem of optimizing infectious disease control strategies [6-7]. Libin et al. [8] used the PPO (Proximal Policy Optimization) algorithm to optimize the regional control of infectious disease intervention strategies. The agent was trained in a virtual environment based on the metapopulation ODE model, and the state information and decision budget were used as observation conditions, with the goal of minimizing the number of infected persons. Experimental results showed that the intervention strategy learned by DRL was better than the optimal control measures obtained by exhaustive methods.

[0019] However, current optimization of infectious disease control strategies is often based on perfect observations, lacking research on optimization of control strategies under incomplete observations. In reference [9], the SEIR model was extended to construct a metapopulation ODE model with 8 cells. In this model, although only the number of people in cells I_t and I_h can be observed, the incomplete observations of the real scenario were modeled and analyzed. However, this approach still assumes that each cell will transfer probabilities at a fixed ratio, which is inconsistent with reality. Therefore, it is necessary to conduct further research and analysis on infectious disease control strategies under incomplete observations.

[0020] [1]Kar T K, Batabyal A. Stability analysis and optimal control of anSIR epidemic model with vaccination[J]. Biosystems, 2011, 104(2-3): 127-135. [2]Sharomi O, Malik T. Optimal control in epidemiology[J]. Annals ofOperations Research, 2017, 251: 55-71. [3]Perkins T A, España G. Optimal control of the COVID-19 pandemicwith non-pharmaceutical interventions[J]. Bulletin of mathematical biology,2020, 82(9): 118. [4]Bolzoni L, Bonacini E, Soresina C, et al. Time-optimal controlstrategies in SIR epidemic models[J]. Mathematical biosciences, 2017, 292:86-96. [5]Morato M M, Bastos S B, Cajueiro D O, et al. An optimal predictivecontrol strategy for COVID-19 (SARS-CoV-2) social distancing policies inBrazil[J]. Annual reviews in control, 2020, 50: 417-431. [6]Arulkumaran K, Deisenroth M P, Brundage M, et al. Deepreinforcement learning: A brief survey[J]. IEEE Signal Processing Magazine,2017, 34(6): 26-38. [7]Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. nature, 2015, 518(7540): 529-533. [8]Libin PJK, Moonens A, Verstraeten T, et al. Deeplearning for large-scale epidemic control[C] / / Machine Learning and KnowledgeDiscovery in Databases. Applied Data Science and Demo Track: EuropeanConference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020,Proceedings, Part V. Springer International Publishing, 2021: 155-170. [9]Hao Q, Xu F, Chen L, et al. Hierarchical Multi-agent Model for Reinforced Medical Resource Allocation with Imperfect Information[J]. ACMTransactions on Intelligent Systems and Technology, 2022, 14(1): 1-27. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] Example 1 Please see Figure 1 This is a flowchart illustrating an infectious disease prevention and control method according to Embodiment 1 of this application; the steps include: S1: Construct a metapopulation chamber model based on the transmission mechanics of infectious diseases.

[0022] In this embodiment, when facing the complex and spatially heterogeneous problem of infectious disease transmission, traditional single-compartment models are insufficient to accurately depict the transmission dynamics between different areas and the changes in population under different infection states. To more accurately simulate the transmission process of infectious diseases and provide a scientific basis for the formulation of infectious disease prevention and control strategies, this application constructs a meta-population compartment model based on infectious disease transmission mechanics.

[0023] Based on the SEIQR model, which divides the population into five basic compartments: susceptible (S), exposed (E), infected (I), isolated (Q), and recovered (R), this study further subdivides the exposed and infected compartments to more precisely describe the impact of different testing statuses and reporting behaviors on the spread of infectious diseases.

[0024] Exposure compartment segmentation: Exposures (E) are segmented into undetected exposures (Eundetected, denoted as E). ) and those already exposed (Edetected, denoted as Undetected exposure refers to individuals who have been exposed to the pathogen but have not yet been detected by testing; detected exposure refers to individuals who have been confirmed to be in an exposed state through testing.

[0025] Infected individuals are further subdivided into infected individuals (I) and undetected individuals (Iundetected, denoted as I). ), those who have been detected (Idetected, denoted as ) and those who proactively report infection (Ireported, denoted as An undetected infection is an individual who has been infected with the pathogen but has not been detected; a detected infection is an individual whose infection has been confirmed through testing; and a voluntary reporting infection is an individual among the detected infections who voluntarily reports their infection status to the relevant authorities.

[0026] Definition of Quarantine Units: Quarantined individuals (Q) are divided into those exposed to the virus (Q) and those in quarantine. ) and isolated infected persons ( Isolated exposed individuals refer to individuals who are isolated and in a state of exposure; isolated infected individuals refer to individuals who are isolated and in a state of infection.

[0027] Based on the above compartmentalization, the following propagation dynamics equations are constructed to describe the change in the number of people in each compartment over time.

[0028] Susceptible Equation:

[0029] in, Let be the number of susceptible individuals in region i at time t. Let be the infection rate in region i; the equation indicates that the rate of decrease in the number of susceptible individuals is proportional to the product of the infection rate and the current number of susceptible individuals.

[0030] Equation for no exposed individuals detected: ; in, The conversion rate from exposure (E) to infection (I). The number of undetected exposed individuals who subsequently became infected provides an overall description of the dynamic changes in the number of undetected exposed individuals.

[0031] Equation of exposed individuals detected: ; This equation indicates that detected exposed individuals will only be identified by conversion rate. The infection originated from no other source.

[0032] Equation for no detected infected individuals: ; in, The recovery rate from infection to recovery This represents the number of recovered individuals who were previously undetected for infection.

[0033] Equation of detected infected individuals: ; This equation describes the dynamic process of a detected exposed person becoming a detected infected person, and the recovery of a detected infected person.

[0034] Equation for proactively reporting infected individuals: ; in, To proactively report the number of recovered infected individuals.

[0035] Equation for isolating exposed individuals: ; The conversion rate indicates the number of exposed individuals in isolation. They were converted into infected individuals and placed in isolation.

[0036] Equation for isolated infected individuals: ; This equation describes the dynamic changes in the process of an isolated exposed person transforming into an isolated infected person, and the recovery of an isolated infected person.

[0037] The Recovery Equation: ; The equation indicates that the increase in the number of recovered patients comes from the recovery of undetected infected individuals, detected infected individuals, those who voluntarily reported infected individuals, and those who were in isolation.

[0038] Infection rate The rate of transmission The calculation formula is as follows, taking into account the influence of various factors such as the flow matrix M and the number of existing infected individuals: ; Where n is the total number of regions, For each element in the flow matrix M, the proportion of population flow from region j to region i is represented. The total number of regions l; The proportion of movement of exposed person E relative to infected person I; The proportion of infected individuals is denoted as . This formula comprehensively considers the impact of population movement between different regions and the distribution of exposed individuals and infected individuals in different regions on the infection rate in region i.

[0039] To more realistically simulate the impact of infected individuals' behavior on the spread of infectious diseases, a model is developed for the voluntary reporting behavior of infected individuals. When infected individuals voluntarily report, the number of undetected infected individuals decreases, while the number of voluntarily reported infected individuals increases. This is specifically described by the following two equations:

[0040] in, The active reporting rate for region i represents the proportion of infected individuals who do not actively report their infection status.

[0041] By constructing the aforementioned metapopulation cell model, we can more comprehensively and accurately simulate the spread of infectious diseases in different regions, as well as the dynamic changes of populations under different infection states and reporting behaviors.

[0042] Optionally, the steps of constructing a meta-population compartment model based on the transmission mechanics of infectious diseases specifically include: The metapopulation cell model is an ordinary differential equation model based on the metapopulation, constructed from the SEIQR model; The meta-population chamber model categorizes exposed individuals into undetected exposed individuals and detected exposed individuals, infected individuals into undetected infected individuals, detected infected individuals, and reported infected individuals, and those in isolation into isolated exposed individuals and isolated infected individuals.

[0043] S2: Based on the aforementioned metapopulation chamber model, the infectious disease prevention and control problem is formalized into a Markov decision process.

[0044] In this embodiment, after constructing a meta-population chamber model to describe the dynamics of infectious disease transmission, further optimization of infectious disease control strategies requires identifying which control actions under different states yield the optimal control effect. Markov Decision Processes (MDPs) provide an effective framework for solving such sequential decision-making problems. By defining the observed state, action, state transition probability, and reward function, the control strategy that maximizes the cumulative reward in the long run can be found. Therefore, formalizing the infectious disease control problem into an MDP based on the constructed meta-population chamber model has significant practical implications. Specific steps include: 1. Define the state space State s is an observation of the actual state, using Let S represent the susceptible population, E the exposed population (including undetected and detected exposed individuals), I the infected population (including undetected, detected, and proactively reported infected individuals), Q the isolated population (including isolated exposed individuals and isolated infected individuals), and R the recovered population. The state space encompasses all possible combinations of population distributions, reflecting the different states of the system during the spread of infectious diseases.

[0045] 2. Define the action space Action a includes detection and isolation levels; for region i, its action is... .in, This indicates the detection level of region i. The isolation level of region i is shown.

[0046] Detection rate Isolation rate The correspondence between detection level and isolation level is as follows:

[0047] in, Let i be the total number of people in region i. For the total number of infected individuals in region i, the function... In order to increase the infection rate At a given level, the number of tests required to detect a single infected person is taken as the function. ; Let be the number of known infected individuals in region i.

[0048] 3. Determine the state transition probability Detection and isolation actions affect the system's state transitions, which are described by the following differential equations: No status transition was detected in the exposed individuals: ; in, Let be the effectiveness of the detection reagent for exposed individual E. This equation indicates that the rate of decrease in the number of undetected exposed individuals is proportional to the detection rate, the current number of undetected exposed individuals, and the effectiveness of the detection reagent for exposed individuals.

[0049] Status shift of no infected individuals: ; in, Let be the effectiveness of the detection reagent against infected individual I. This equation describes the relationship between the rate of decrease in the number of undetected infected individuals and the detection rate, the current number of undetected infected individuals, and the effectiveness of the detection reagent against infected individuals.

[0050] Status shifts have been detected in exposed individuals: ; Status change of the detected infected person: ; Status shifts have been detected in exposed individuals: ; This indicates that the previously detected exposed individuals have completed the state transition by changing from the status of a detected exposed individual to an isolated exposed individual through the isolation procedure.

[0051] Status change of the detected infected person: ; This indicates that the detected infected person has changed from being a detected infected person to an isolated infected person through isolation procedures, thus achieving a change in status.

[0052] Actively report changes in the status of infected individuals: ; Status transition of exposed individuals in isolation: ; Status shift of infected individuals in isolation: ; By solving these differential equations, we can obtain the probability of the system transitioning from one state to another under different actions, and thus determine the state transition probability.

[0053] 4. Define the reward function The immediate reward *r* measures the benefit gained from taking an action in a given state. For region *i*, its reward value is... The definition is as follows: ; in, Cost of infection Testing costs and isolation costs The weighting coefficients are used to balance the impact of different costs on the reward function.

[0054] Infection cost for region i The calculation formula is: ; in, This represents the effective number of infections in region j. This indicates the proportion of movement of exposed person E relative to infected person I. Infection cost. The calculation consists of two parts, the first set of parentheses indicates... The percentage of infected individuals in region j is represented by the first parenthesis, and the second parenthesis indicates the total number of newly exposed individuals in region j. .

[0055] Detection cost for region i and isolation costs The calculation formula is: ; That is, the detection cost and isolation cost of region i are equal to its detection level and isolation level, respectively.

[0056] 5. Constructing a Markov Decision Process By defining the state space, action space, state transition probabilities, and reward function above, the infectious disease control problem is constructed as a Markov decision process (MDP). In this MDP, the decision-maker (control department) selects an action (detection and isolation level) based on the current state at each state, transitions to the next state based on the selected action and state transition probabilities, and receives the corresponding immediate reward. The decision-maker's goal is to maximize the cumulative reward over the long term by selecting the optimal sequence of actions, thereby achieving the optimal infectious disease control strategy.

[0057] Optionally, the Markov decision process includes the observed state, prevention and control actions, state transition probabilities, and reward function; the observed state includes local state and global state, the local state includes infection scale indicators and new exposure indicators for each region, and the global state is the aggregation of local states for each region; the prevention and control actions are the detection actions and isolation actions for each region; the reward function is the weighted negative value of infection cost, detection cost, and isolation cost.

[0058] S3: Define multiple observation levels, including fully observable, fully reported observation, and partially observable; wherein, under the partially observable level, based on the current observation information, the information reconstruction method is used to estimate the unobserved infection status and obtain the reconstructed status information.

[0059] In this embodiment, in the context of infectious disease prevention and control, the completeness of observational information directly affects the effectiveness of prevention and control strategies. This application classifies observation levels into three levels: fully observable, fully reported observations, and partially observable, and uses a metapopulation chamber model to mathematically model each level: Fully observable: Assuming accurate data for all cells is available, allowing direct calculation of the existing infection scale and new exposures in region i: ; This level is theoretically optimal, but in practice it is almost impossible to achieve due to limitations in testing resources (such as reagent shortages and sampling deviations).

[0060] Complete reporting and observation: Assuming only Ede, Ide, and Ire compartments can be observed, and undetected infected individuals (Iun) are reported to the Ire compartment with 100% probability. In this case, the scale of infection and the calculation of new exposures are as follows: ; This level relies on the assumption of a fixed reporting rate, but in reality, the reporting rate is affected by factors such as willingness to be tested and allocation of medical resources, and therefore fluctuates dynamically.

[0061] Partially observable: Information on undetected exposed persons (Eun) and infected persons (Iun) is missing, and the probability of Iun transferring to Ire is unknown (reporting rate varies with time and region). In other words, the reporting rate varies over time and differs by region. In this partially observable scenario, complete observation of both E and I is impossible, and this type of observation best reflects real-world prevention and control scenarios. At this point, only: ; This level is closest to the real-world scenario, but the missing observations need to be compensated for through information reconstruction.

[0062] Based on the definitions of the three observation levels mentioned above, the information reconstruction task can be defined as: using information obtained from partially observable data... , Predicting fully known observations .

[0063] To estimate the fully observable state from partial observation data, this application proposes three types of information reconstruction methods and improves the state information through a backfilling mechanism. Specific steps include: 1. Mechanism-driven reconstruction method Based on the spatial propagation dynamics of the metapopulation ODE model, a reconstruction model is constructed using the population flow matrix and regional population vectors. For the observation vector... Undetected areas are marked as The reconstruction formula is: ; This method characterizes inter-regional dependencies through the physical constraints of the ODE model, but it relies on accurate flow matrix M and population data N and is sensitive to data noise.

[0064] 2. Data-driven reconstruction method Inverse Distance Weighted (IDW) Interpolation: The formula for filling unobserved region data using a reciprocal weighted average based on spatial distance is: ; in, Let be the distance between regions i and j, and p represent the power parameter. This method is simple but cannot capture temporal dynamics and complex spatial dependencies.

[0065] Graph Convolutional Neural Network - Gated Recurrent Unit (GCN-GRU): By combining the spatial feature extraction capabilities of GCN and the temporal series modeling capabilities of GRU, a spatiotemporal joint prediction model is constructed. The flow matrix M is used as the adjacency matrix of GCN, historical observation sequences are input, and the current reconstructed state is output. .

[0066] This method automatically learns spatiotemporal dependencies through neural networks, but it requires a large amount of training data and may overfit.

[0067] 3. Mechanism-Data Fusion Reconstruction Method (ODE-GCN-GRU) To address the limitations of the GCN-GRU spatial weighting method, this application proposes incorporating the ODE mechanism model into the spatial feature extraction process of GCN. Specifically, the improvement is as follows: Original GCN weighted: ; Where W is the weight to be learned, It is an activation function.

[0068] ODE-GCN-GRU weighted: Introduce the ODE reconstruction results as a physical constraint for spatial dependence: ; This method describes the nature of spatial propagation through a mechanistic model, and at the same time uses neural networks to fit nonlinear relationships, which significantly improves the accuracy of reconstruction.

[0069] After obtaining the reconstructed state information using the above method, the unobserved areas are backfilled: ; The backfilled complete state information is used to update the local and global states in the MDP, providing data support for subsequent PPO strategy optimization.

[0070] Optionally, the information reconstruction method includes any one or a combination of the following: Mechanism reconstruction method based on propagation dynamics; Data-driven reconstruction methods based on spatial interpolation; A data-driven reconstruction method based on neural networks; A hybrid reconstruction method that integrates propagation dynamics mechanisms and neural network weighting includes: Using the flow matrix and population data in the metapopulation cell model, spatial dependency weights are calculated through a mechanistic model; the spatial dependency weights are embedded into a graph convolutional network for feature extraction, and a recurrent neural network is used to obtain temporal dependencies; the reconstructed state information is then output.

[0071] S4: Input the reconstructed state information into the target agent to obtain the target control strategy output by the target agent; wherein, the target agent is an agent model trained based on the Markov decision process.

[0072] In this embodiment, after constructing the meta-population ODE propagation model and formalizing it as an MDP, the optimal control strategy needs to be solved using a reinforcement learning algorithm. The PPO (Proximal Policy Optimization) algorithm, with its stable training performance, support for continuous action spaces, and multi-agent cooperation capabilities, has become the core choice in this embodiment. This algorithm achieves joint optimization of policy and value through an Actor-Critic architecture, and combined with a local-global state fusion mechanism, it can adapt to complex decision-making scenarios with inter-region coupling in the meta-population model. Specific steps include: 1. Target Intelligent Agent Architecture Design Agent partitioning: Each region is considered an independent agent, possessing a local Actor network and a shared Critic network. The Actor network takes the local state of region i as input and outputs an action policy; the Critic network takes the combined local and global states as input and outputs a learning value function.

[0073] Network structure: The Actor uses a fully connected network, and the output layer is the mean and variance of two continuous action dimensions, with actions sampled through a Gaussian distribution; the Critic uses a deeper network, which integrates local and global features and outputs a scalar value estimate.

[0074] 2. Training process At the start of each training round, each agent independently samples actions according to the current Actor policy. After executing an action, the state transition is calculated using the meta-population ODE model, and the immediate reward is observed. Complete trajectory data (state, action, reward, next state) is collected and stored in the experience replay pool. Furthermore, the Actor is updated using the policy gradient method, and the Critic is updated using the TD error optimization value function.

[0075] 3. Multi-agent cooperation mechanism The Critic network input includes global state, allowing value estimation to account for inter-regional propagation coupling effects. For example, the reward for region i depends on the infection status of other regions; the global state helps the Critic accurately assess such indirect effects. Actor network parameters are independent across regions, allowing each region to develop differentiated strategies based on its local state; Critic network parameters are shared globally, promoting consistency in policy evaluation. This design balances policy diversity with evaluation uniformity.

[0076] 4. Optimization of training stability and efficiency Adding a policy entropy term to the Actor loss encourages policy exploration and prevents premature convergence to a suboptimal solution. Introducing a target network into the Critic reduces value estimation fluctuations during training and improves stability. Utilizing multi-process parallel collection of trajectory data accelerates the training process and improves data utilization.

[0077] Through optimized training using the PPO algorithm, the metapopulation MDP model can output a region-level dynamic detection-isolation strategy. When validated in a simulated environment, this strategy significantly reduces infection costs while balancing detection and isolation costs.

[0078] After completing the information reconstruction in step S3, the reconstructed state information is obtained. This reconstructed state information (obs) is then used as input to the target agent. Upon receiving this information, the target agent analyzes and evaluates the current state of the infectious disease transmission based on its internally learned strategies.

[0079] The target control strategy output by the target agent is applied to actual infectious disease control work. After a period of implementation, new observational data is collected to evaluate the control effect. Evaluation indicators include changes in infection rate, efficiency of medical resource utilization, and rationality of personnel movement. If the evaluation results show that the control effect is good, the strategy is continued or fine-tuned; if the effect is poor, the reasons are analyzed, the target agent is further trained, or the information reconstruction method is adjusted to optimize the generation of subsequent control strategies.

[0080] In one embodiment, this application uses Shenzhen, China as the study area, subdividing the study area into communities, and uses anonymous mobile phone location data from China Unicom to establish a spatiotemporal propagation network. Infectious disease simulation parameter settings: At the start of the simulation, 100 initial infected individuals were distributed across the 100 largest communities. The simulation lasted 120 days, with one new infected individual randomly introduced each day. To simulate the delay in detecting isolation in a real-world scenario, the number of people isolated on day t was delayed to days t, t+1, and t+2 in a ratio of 0.3:0.4:0.3. The parameters of the Markov decision process, including the weights of the reward function... The detection level action selection is [0,0.05,0.2,0.5,1.0] ×0.1, a total of 5 levels, and the isolation level action selection is [0,0.2,1.0] ×0.01, a total of 3 levels.

[0081] For the parameters involved in several prediction methods: In mechanistic prediction methods, the parameters can be directly calculated using formulas. The data-driven IDW method uses p=2. The data-driven GCN-GRU method uses a single-layer GCN with 32 hidden neurons and a two-layer GRU with 128 hidden neurons. The sequence length is k=7, and the output is a fully connected layer. The GRU uses the ReLU activation function, and the Adam optimizer and DropOut method are used to prevent overfitting. The final ODE-GCN-GRU network uses the same network parameters. During neural network training, the dataset is obtained through the constructed meta-population ODE model, and early stopping is used to prevent overfitting.

[0082] For the evaluation of experimental results, two aspects were considered: the reconstruction accuracy of the prediction method and the impact of information reconstruction on the control effect. Reconstruction accuracy was assessed using the root mean square error (RMSE) metric to evaluate the prediction accuracy of the overall trend of existing infected individuals and the number of existing infected individuals in each region on a given day. The target value represents the observed state under completely known conditions. The Score metric was introduced to evaluate the control effect. ; Where I represents the cumulative number of infections, and Q represents the cumulative control cost. This represents the weighted sum of the number of people tested (TestNum) and the number of people in quarantine (QuaraNum). The weight ratio in the reward function. This indicates the weighting ratio of infection and control costs, using This method allows for the comprehensive evaluation of control effectiveness by considering both the number of infections and the control costs.

[0083] The experimental results are shown in the table below:

[0084] As shown in the table above, there are significant differences in control effectiveness between complete and partially observable (incomplete) scenarios under both basic conditions. Both the number of infections and control costs are an order of magnitude higher in the partially observable scenario than in the complete reporting scenario, indicating that incomplete observation significantly reduces the effectiveness of infectious disease control strategies. However, by introducing prediction methods, relatively effective control can be achieved based on partially observable data, bringing the control effect closer to that of the complete reporting scenario. Regarding the differences between the various prediction methods, from the perspective of evaluation metrics, the ODE-GCN-GRU method has a lower RMSE, higher prediction accuracy, and a lower overall control effectiveness evaluation score, reflecting the correctness of its comprehensive mechanism and data-driven approach.

[0085] Please see Figure 2 This is a schematic diagram of the architecture of the target intelligent agent in Embodiment 1 of this application. Based on the multi-agent reinforcement learning framework used by the PPO algorithm, the agent has built-in Actor and Critic networks, and can complete action selection and value judgment based on state s and reward r. The output action a affects detection and isolation. The metapopulation ODE model is affected by the action and performs state transition, feeding back the next state and immediate reward r.

[0086] In summary, Embodiment 1 of this application constructs a metapopulation ODE infectious disease transmission model based on the SEIQR model, and transforms it into a Markov decision process that includes the definitions of observation state, action, and reward. The PPO reinforcement learning algorithm is used to learn and optimize the prevention and control strategy. The ODE model clarifies the complete, fully reported, and partially observable modes in the real prevention and control scenario, and defines incomplete observations as partially observable to determine the information reconstruction goals and challenges. The information reconstruction technology based on prediction methods solves the state estimation problem, realizes the optimization of prevention and control strategies in the scenario of incomplete observation, and analyzes the differences in the effects of different prediction methods.

[0087] Example 2 Please see Figure 3 This is a schematic diagram of the structure of an infectious disease prevention and control system according to Embodiment 2 of this application; the specific content includes: The model building module 100 constructs a meta-population chamber model based on the transmission mechanics of infectious diseases; based on the meta-population chamber model, the infectious disease prevention and control problem is formalized into a Markov decision process. The observation definition module 200 defines multiple observation levels, including fully observable, fully reported observation, and partially observable. Under the partially observable level, based on the current observation information, an information reconstruction method is used to estimate the unobserved infection status and obtain the reconstructed status information. The information reconstruction and strategy generation module 300 inputs the reconstructed state information into the target agent to obtain the target prevention and control strategy output by the target agent; wherein, the target agent is an agent model trained based on the Markov decision process.

[0088] In this embodiment, through the collaborative work of various modules of the system, accurate state estimation and effective prevention and control strategy generation are achieved. The specific details are as follows: The model building module 100 first constructs a meta-population cell model based on the transmission dynamics of the infectious disease. Considering the differences in population flow, population density, and social activity patterns across different urban areas, the city is divided into multiple sub-regions (meta-populations). Each sub-region is further divided into five cells: susceptible individuals, exposed individuals, infected individuals, isolated individuals, and recovered individuals. By collecting historical infectious disease data, demographic data, and social behavior survey data, the conversion rate parameters between cells are determined. For example, based on the incubation period distribution of similar infectious diseases in the past, the conversion rate from exposed individuals to infected individuals is determined; based on the isolation capacity of medical resources and the public's cooperation in isolation, the conversion rate from infected individuals to isolated individuals is set.

[0089] Based on the constructed metapopulation-cell model, the infectious disease control problem is formalized as a Markov Decision Process (MDP). State Space: Defined as the combination of the proportion of people in different cells within each sub-region. Action Space (A): Contains a series of control measures, such as restricting population movement (divided into mild, moderate, and severe restrictions), conducting large-scale nucleic acid testing (at different frequencies), and allocating medical resources (increasing isolation beds, medical personnel, etc.). Reward Function: Set according to the control objectives. Positive rewards are given if the control actions effectively reduce the number of infected individuals and control the spread; conversely, negative rewards are given if the actions lead to an increase in the number of infected individuals and the spread of the infectious disease. For example, a positive reward value is given for each successful reduction in the number of infected individuals; a negative reward value is given for each new high-incidence area of ​​the infectious disease. State Transition Probability (P): Based on the metapopulation-cell model and historical data, the probability of transitioning from one state to another is estimated by simulating the changes in the number of people in each cell under different control actions.

[0090] The observation definition module 200 defines three observation levels: fully observable, fully reporting observation, and partially observable. Fully observable: This assumes that the number of people in all cells within each sub-region can be obtained in real time and accurately. However, in real-world scenarios, this is almost impossible to achieve due to the difficulty and cost of data collection. Fully reporting observation: Each sub-region can fully report its cell occupancy information according to the prescribed time and standards. However, in practice, there may be instances where some regions report untimely or inaccurately. Partially observable: This is the most common situation in actual epidemic prevention and control scenarios. Some sub-regions may be unable to fully report their cell occupancy information for various reasons, especially regarding the number of infected individuals, which may be missed or misreported.

[0091] At the partially observable level, the system employs information reconstruction methods to estimate the unobserved infection status. For example, by utilizing the transmission patterns and trends in historical infectious disease data, combined with currently observed data on the number of people in some isolation wards, machine learning algorithms (such as neural networks) are used to predict the number of infected individuals in unobserved areas. Specifically, data on the transmission of similar infectious diseases in different regions in the past are collected to train a prediction model. The model is then fed with the currently observed number of susceptible individuals, exposed individuals, and other individuals in isolation wards, as well as data reported by some infected individuals. The output is an estimated number of infected individuals in unobserved areas, thus obtaining the reconstructed status information.

[0092] The information reconstruction and policy generation module 300 inputs the reconstructed state information obtained by the observation definition module 200 into the target agent. The target agent is an agent model trained based on the Markov decision process defined in the preceding model construction module 100. During training, the agent's decision-making ability is continuously optimized through reinforcement learning algorithms, enabling it to select the optimal control action under different states.

[0093] After receiving the reconstructed state information, the target agent analyzes the current trend of infectious disease transmission based on its internally learned strategies and outputs target prevention and control strategies. For example, if the reconstructed state information shows that the number of infected people in a certain area is rapidly increasing and the medical resources in that area are relatively strained, the target agent may output a prevention and control strategy of "implementing moderate restrictions on the movement of people in the area and allocating additional medical resources (such as increasing isolation beds and medical staff)".

[0094] After applying the infectious disease prevention and control system for a period of time, the spread of infectious diseases in the city was assessed. The information reconstruction module enabled a more accurate estimation of the infection status in unobserved areas, providing a more reliable basis for formulating prevention and control strategies. The target-oriented prevention and control strategies generated by the target agent can be adjusted promptly based on real-time changes in status information, improving the targeting and effectiveness of prevention and control measures. Simultaneously, a comparative analysis of different prediction methods in information reconstruction was conducted. Machine learning-based prediction methods demonstrated superior performance in reconstruction accuracy and final prevention and control effects, enabling more accurate prediction of the infection status in unobserved areas, thus supporting the development of more precise prevention and control strategies.

[0095] Example 3 Please see Figure 4 This is a schematic diagram of the computer device structure according to Embodiment 3 of this application. The computer device 50 includes a processor 51 and a memory 52 coupled to the processor 51.

[0096] The memory 52 stores program instructions for implementing the above-mentioned infectious disease prevention and control method.

[0097] The processor 51 is used to execute program instructions stored in the memory 52 to implement an infectious disease control measure.

[0098] The processor 51 can also be referred to as CPU (Central Processing Unit).

[0099] Processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0100] Example 4 Please see Figure 5This is a schematic diagram of the storage medium in Embodiment 4 of this application. The storage medium in this embodiment stores a program file 61 capable of implementing all the above methods. This program file 61 can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or devices such as computers, servers, mobile phones, and tablets.

[0101] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0102] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0103] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

[0104] Of course, the present invention may have many other embodiments. Based on this embodiment, other embodiments obtained by those skilled in the art without any creative effort are all within the scope of protection of the present invention.

Claims

1. A method for controlling infectious diseases, characterized in that, include: A meta-population cell model was constructed based on the transmission mechanics of infectious diseases; Based on the aforementioned metapopulation cell model, the problem of infectious disease prevention and control is formalized as a Markov decision process; Multiple observation levels are defined, including fully observable, fully reported observation, and partially observable; wherein, under the partially observable level, based on the current observation information, an information reconstruction method is used to estimate the unobserved infection status and obtain the reconstructed status information. The reconstructed state information is input into the target agent to obtain the target control strategy output by the target agent; wherein, the target agent is an agent model trained based on the Markov decision process.

2. The infectious disease prevention and control method according to claim 1, characterized in that, The steps of constructing a meta-population cell model based on the transmission mechanics of infectious diseases specifically include: The metapopulation cell model is an ordinary differential equation model based on the metapopulation, constructed from the SEIQR model; The meta-population chamber model classifies exposed individuals into undetected exposed individuals and detected exposed individuals, infected individuals into undetected infected individuals, detected infected individuals, and reported infected individuals, and isolated individuals into isolated exposed individuals and isolated infected individuals.

3. The infectious disease prevention and control method according to claim 1, characterized in that, The steps of formalizing the infectious disease control problem into a Markov decision process based on the meta-population cell model specifically include: The Markov decision process includes the observed state, control actions, state transition probabilities, and reward function. The observation status includes local status and global status. The local status includes infection scale indicators and newly exposed person indicators for each region. The global status is an aggregation of the local status of each region. The aforementioned prevention and control measures include detection and isolation measures in each region; The reward function is a weighted negative value of infection cost, testing cost, and isolation cost.

4. The infectious disease prevention and control method according to claim 3, characterized in that, The information reconstruction method includes any one or a combination of the following: Mechanism reconstruction method based on propagation dynamics; Data-driven reconstruction methods based on spatial interpolation; A data-driven reconstruction method based on neural networks; A hybrid reconstruction method that integrates propagation dynamics mechanism and neural network weighting.

5. The infectious disease prevention and control method according to claim 4, characterized in that, The hybrid reconstruction method combining the fusion propagation dynamics mechanism and the neural network weighting method includes: Using the flow matrix and population data in the metapopulation cell model, spatial dependency weights are calculated through a mechanistic model. The spatial dependency weights are embedded into a graph convolutional network for feature extraction, and the temporal dependency is obtained by combining a recurrent neural network. Output the reconstructed status information.

6. An infectious disease prevention and control system, characterized in that, The infectious disease prevention and control system is used to perform the infectious disease prevention and control method according to any one of claims 1 to 5, and comprises: The model building module constructs a metapopulation chamber model based on the transmission mechanics of infectious diseases; based on the metapopulation chamber model, the infectious disease prevention and control problem is formalized into a Markov decision process. The observation definition module defines multiple observation levels, including fully observable, fully reported observation, and partially observable. Under the partially observable level, based on the current observation information, an information reconstruction method is used to estimate the unobserved infection status and obtain the reconstructed status information. The information reconstruction and strategy generation module inputs the reconstructed state information into the target agent to obtain the target prevention and control strategy output by the target agent; wherein, the target agent is an agent model trained based on the Markov decision process.

7. A computer device, characterized in that, The computer device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing the infectious disease prevention and control method according to any one of claims 1-5; the processor is used to execute the program instructions stored in the memory to implement infectious disease prevention and control.

8. A storage medium, characterized in that, The device stores processor-executable program instructions for performing the infectious disease prevention and control method according to any one of claims 1-5.

Images