Early epidemic detection method under arbitrary infection graph based on Markov decision process

Abstract

The present invention discloses a method for early epidemic detection under arbitrary infection graphs based on a Markov decision process. The method implementation process includes defining the state space, action space, and state transition probabilities of the Markov decision process according to detection constraints, calculating the transition probabilities between individual infection states based on the infection graph model, defining the reward function of the Markov decision process based on the conditional probability of a negative test, and obtaining the decision result through a policy iteration algorithm. This method, based on the Markov decision process, designs a detection strategy that satisfies the detection constraints for a group of individuals whose mutual infection probabilities are given by the infection graph, and minimizes the average delay in detecting the appearance of infected individuals.

Description

Technical Field

[0001] The present invention relates to the field of information technology, in particular to a method for detecting an arbitrary number of individuals n and an arbitrary infection graph. Under certain detection constraints, the goal is to minimize the average delay in detecting the presence of infected people. Background Art

[0002] When it comes to infectious disease population detection, most work focuses on static, single-shot detection. However, in reality, the emergence of infected individuals in a population, the spread of infection, and detection are dynamic and ongoing. A small amount of work has addressed dynamic population detection. These efforts focus on symmetric transmission models and algorithms or select fixed individuals for detection at each stage. However, it would be more effective to leverage the connection structure of individuals in a population or select different individuals for detection at each stage to improve detection efficiency. Furthermore, detecting the occurrence of an epidemic in a population as early as possible is crucial to preventing its widespread spread. Existing detection methods are only applicable to situations where the infection graph is symmetric or the detection scheme cannot detect different individuals at each stage. Summary of the Invention

[0003] The present invention aims to provide an early epidemic detection method under any infection graph based on Markov decision process, for any number of individuals n, any infection graph Under the detection constraint of daily testing capacity m and all individuals being tested at least once within τ days, the goal is to determine the detection strategy to minimize the average delay in detecting the infected person. The problem is modeled as a Markov decision process using the conditional probability of a negative test result under the model, and solved using the strategy iteration method. The method of early epidemic detection under arbitrary infection graph based on Markov decision process is obtained, which solves the technical problem of early epidemic detection that can detect different individuals at each stage under arbitrary infection graph.

[0004] Technical solution: The technical solution adopted by the present invention is as follows:

[0005] A method for early epidemic detection under arbitrary infection graph based on Markov decision process includes the following steps:

[0006] Step 1: For n individuals, define the state space of the system based on the Markov decision process model according to the daily test capacity m and the detection constraint that all individuals must be tested at least once within τ days.

[0007] Step 2: Define the action space of the system is the action space of state s, namely: in is the system action at time t, defined as the set of individuals selected for detection by the system at that time, [n] = {1, 2, ... n} is a set of n individuals, the test capacity m is the number of individuals allowed to be detected at each moment, s(i) is the i-th item in the vector s, represents the union of the sets s(1),s(2),…,s(τ-1);

[0008] Step 3: Define the state transition probability of the system Where p(s'|s,a) represents the probability that the system takes action a in the current state s and enters the next state s', which is set as follows:

[0009]

[0010] s(i) is the i-th item in vector s;

[0011] Step 4: Define the system’s rewards Where r(s,a) represents the system in state Next, take action Time reward;

[0012] Step 5: Set the discount factor λ = (1-p) n , p is the probability that each uninfected individual will be infected at the next moment when n individuals are not infected. The Markov decision process model established by the strategy iteration algorithm is solved Get the optimal strategy π * , the policy π is a mapping from the state space to the action space, a = π(s).

[0013] Furthermore, step 1 sets the state space as follows:

[0014] The system state s at time t t is the record of the detection strategy at τ-1 moments before time t, that is in represents the set of individuals detected at time i, represents the set of integers, the state space The set of all existing individuals The set of states that satisfy the detection constraints, namely:

[0015]

[0016] The test capacity m is the number of individuals allowed to be tested at each moment. Representing a collection And.

[0017] Furthermore, in step 4, the system is in state Next, take action The reward r(s,a) is set as follows:

[0018] First, for n individuals and infection graph The state transition probability of the individual infection state U(t) is obtained, that is, the probability π that the individual infection state changes from u to v after a stage of infection spread uv :

[0019]

[0020] Where p is the probability that each uninfected individual will be infected at the next moment when all n individuals are uninfected, ·1 represents the norm of vector ·, and vector u∈{0,1} n and v∈{0,1} n represents the infection status of n individuals, u i represents the i-th element of vector u, v i represents the i-th element of vector v, u i =1 means individual i is infected, otherwise u i =0, infection graph It is established based on the connections between these individuals with a vertex set and random edge sets An undirected random graph, where each vertex represents a unique individual, each edge represents a connection between individuals, and each edge between node i and node j has a probability c ij Independent existence, that is, the weight c of the edge of the infection graph ij It indicates the probability that individual j will be infected by individual i at the next moment when individual i is infected;

[0021] Then calculate the system in state Next, take action The reward r(s,a) when :

[0022]

[0023] in, For i∈[2,τ-1], The calculation of is divided into the following sub-steps:

[0024] Step 4.1, let the number of iterations k = 0, for all u∈{0,1} n Calculate the function g0(u,s,a,i):

[0025]

[0026] in is the set of individual states that satisfy the negative test result, that is, s(τ-i+1) is the set of individuals detected at the i-1 moments before the current one, and is the τ-i+1th item of the vector s;

[0027] Step 4.2: Set k = k + 1; if k < i, continue executing the sequence; otherwise, jump to step 4.6;

[0028] Step 4.3: Based on the individual set s(τ-i+1+k) or a detected at the current ik-1 time, obtain the set of individual states that satisfy the negative test result.

[0029]

[0030] Step 4.4, for all u∈{0,1} n Calculate the function h in sequence k (u,s,a,i),f k (s,a,i),g k (u,s,a,i):

[0031]

[0032]

[0033]

[0034] Step 4.5: Return to step 4.2.

[0035] Step 4.6, calculation function

[0036]

[0037] Furthermore, the policy iteration algorithm in step 5 includes the following sub-steps:

[0038] Step 5.1: Initialize the state space Action Space State transition probability matrix Reward Function The discount factor λ is used to Arrange in a certain order, denoted as s (1) ,s (2) ,…,s (|S|) ;

[0039] Step 5.2: Arbitrarily give the initialization strategy π0 and initialize the number of iterations k to 0;

[0040] Step 5.3. Calculate the expected value when the strategy is π0 The update strategy is π1:

[0041]

[0042] Step 5.4, the number of iterations k is incremented by 1;

[0043] Step 5.5, calculate the strategy as π k Expected value at The update strategy is π k+1 :

[0044]

[0045] Step 5.6. Repeat steps 5.4 and 5.5 until π k =π k+1 , that is, strategy π k and strategy π k+1 consistent;

[0046] Step 5.7: Get the optimal detection strategy π * =π k ;

[0047] in, It represents the strategy when the number of iterations is k, which is a mapping from state space to action space, a=π k (s), represents the set of natural numbers, is the state transition matrix when the strategy is π, which is a The matrix, is the reward vector when the strategy is π, which is a vector, I is The identity matrix of .

[0048] Beneficial effects: The present invention proposes an early epidemic detection method under any infection graph based on the Markov decision process. This method is based on the Markov decision process. By mathematically modeling and calculating the average delay in detecting the occurrence of infected persons and reasonably designing the state space, action space, state transfer matrix, reward and discount coefficient of the Markov decision process, the problem of minimizing the average delay under certain detection constraints is converted into a solution of a common Markov decision process. Finally, for a group of individuals whose probability of mutual infection is given by the infection graph, a detection strategy that meets certain detection constraints is designed, and the average delay in detecting the occurrence of infected persons is minimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1(a) is an infection diagram containing n=3 individuals in an embodiment of the present invention

[0050] Figure 1(b) is an infection diagram in an embodiment of the present invention is the state transition probability π of the individual infection state in Figure 1(a) uv Calculation result diagram of ;

[0051] Figure 1(c) is an infection diagram in an embodiment of the present invention Figure 1(a) shows the state space of the Markov decision process when p = 0.01, the specific settings of the reward, and the optimal strategy results.

[0052] Figure 2 This is a flow chart of the method for early epidemic detection under any infection graph based on the Markov decision process of the present invention;

[0053] Figure 3 Flowchart for calculating r(s,a) of the present invention;

[0054] Figure 4 Flowchart of the strategy iteration algorithm of the present invention. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the implementation examples.

[0056] like Figure 2 As shown in the figure, the present invention proposes an early epidemic detection method under any infection graph based on Markov decision process. For n individuals in a closed area, the infection graph It is established based on the connections between these individuals with a vertex set and random edge sets An undirected random graph, where each vertex represents a unique individual, each edge represents a connection between individuals, and each edge between node i and node j has a probability c ij Independent existence, that is, the weight c of the edge of the infection graph ij represents the probability that individual j will be infected by individual i at the next moment when individual i is infected. When n individuals are uninfected, the probability that each uninfected individual will be infected at the next moment is p. The system's testing constraints require a daily testing capacity of m, and all individuals must be tested at least once within τ days.

[0057] An embodiment is given below:

[0058] This example considers n=3 individuals in a closed area, and the infection graph As shown in Figure 1(a), each edge between node i and node j has a probability c ij Independent existence, specific value: c 12 =c 21 =0.5, c 13 =c 31 =0.1, c 23 =c 32= 0. When n individuals are uninfected, the probability p of each uninfected individual being infected at the next moment is 0.01. The detection constraint of the system requires that the daily detection capacity is m = 1, and all individuals must be tested at least once within τ = 4 days. The specific plan is as follows:

[0059] (1) For n = 3 individuals, based on the daily testing capacity m = 1 and the testing constraint that all individuals must be tested at least once within τ = 4 days, the state space of the system is defined based on the Markov decision process model The elements s in are shown in the first column of Figure 1(c).

[0060] (2) Define the system's action space is the action space of state s, namely: where a t is the system action at time t, defined as the set of individuals selected for detection by the system at that time, s(i) is the i-th item in the vector s, represents the union of the sets s(1), s(2), and s(3), as shown in the second, third, and fourth columns of Figure 1(c). If r(s, a) has a value, then For example

[0061] (3) Define the state transition probability of the system Where p(s'|s,a) represents the probability that the system takes action a in the current state s and enters the next state s', which is set as follows:

[0062]

[0063] s(i) is the i-th item in vector s.

[0064] (4) Define the system’s rewards Where r(s,a) represents the system in state Next, take action The reward when the system is in state Next, take action The reward r(s,a) is set as follows: First, for n=3 individuals and infection graph Get the state transition probability π of the individual infection state U(t) uv , the specific results are shown in Figure 1(b), then calculate the system in state Next, take action The reward r(s,a) when , the specific calculation results are shown in the second, third and fourth columns of Figure 1(c). The flowchart for calculating r(s,a) is as follows Figure 3 shown.

[0065] (5) Set the discount factor λ = (1-0.01) 3 , using the strategy iteration algorithm to solve the Markov decision process model established The optimal strategy π is obtained as shown in the last column of Figure 1(c). Strategy π is a mapping from state space to action space, a=π(s). The flow chart of the policy iteration algorithm is as follows Figure 4 shown.

Claims

1. A method for early epidemic detection under arbitrary infection graph based on Markov decision process, characterized by: The steps include: Step 1: For n individuals, based on the daily testing capacity m and the testing constraint that all individuals must be tested at least once within τ days, define the state space of the system based on the Markov decision process model. Step 2: Define the action space of the system is the action space of state s, namely: in is the system action at time t, is the set of individuals selected for detection by the system at time t, [n] = {1, 2, ... n} is a set of n individuals, the test capacity m is the number of individuals allowed to be detected at each moment, s(i) is the i-th item in the vector s, represents the union of the sets s(1),s(2),…,s(τ-1); Step 3: Define the state transition probability of the system Where p(s'|s,a) represents the probability that the system takes action a in the current state s and enters the next state s', which is set as follows: s(i) is the i-th item in vector s; Step 4: Define the system’s rewards Where r(s,a) represents the system in state Next, take action Time reward; Step 5: Set the discount factor λ = (1-p) n , p is the probability that each uninfected individual will be infected at the next moment when n individuals are not infected. The Markov decision process model established by the strategy iteration algorithm is solved Get the optimal strategy π * , the strategy π is a mapping from the state space to the action space, a = π(s); Step 4 The system is in status Next, take action The reward r(s,a) is set as follows: First, for n individuals and infection graph The state transition probability of the individual infection state U(t) is obtained, that is, the probability π that the individual infection state changes from u to v after a stage of infection spread uv : Where p is the probability that each uninfected individual will be infected at the next moment when all n individuals are uninfected, ||·||1 represents the norm of vector ·, and vector u∈{0,1} n and v∈{0,1} n represents the infection status of n individuals, u i represents the i-th element of vector u, v i represents the i-th element of vector v, u i =1 means individual i is infected, otherwise u i =0, infection graph It is established based on the connections between these individuals with a vertex set and random edge sets An undirected random graph, where each vertex represents a unique individual, each edge represents a connection between individuals, and each edge between node i and node j has a probability c ij Independent existence, that is, the weight c of the edge of the infection graph ij It represents the probability that individual j will be infected by individual i at the next moment when individual i is infected; Then calculate the system in state Next, take action The reward r(s,a) when : in, For i∈[2,τ-1], The calculation of is divided into the following sub-steps: Step 4.1, let the number of iterations k = 0, for all u∈{0,1} n Calculate the function g0(u,s,a,i): in is the set of individual states that satisfy the negative test result, that is, s(τ-i+1) is the set of individuals detected at the i-1 moments before the current one, and is the τ-i+1th item of the vector s; Step 4.2: Set k = k + 1; if k < i, continue executing the sequence; otherwise, jump to step 4.6; Step 4.3: Based on the individual set s(τ-i+1+k) or a detected at the current ik-1 time, obtain the set of individual states that satisfy the negative test result. Step 4.4, for all u∈{0,1} n Calculate the function h in sequence k (u,s,a,i),f k (s,a,i),g k (u,s,a,i): Step 4.5: Return to step 4.

2. Step 4.6, calculation function 2. The method for early epidemic detection under arbitrary infection graph based on Markov decision process according to claim 1, characterized in that: Step 1: Set up the state space as follows: The system state s at time t t is the record of the detection strategy at τ-1 moments before time t, that is in represents the set of individuals detected at time i, represents a set of integers, and the state space S is the set of all existing individuals The set of states that satisfy the detection constraints, namely: The test capacity m is the number of individuals allowed to be tested at each moment, Representing a collection And.

3. The method for early epidemic detection under arbitrary infection graph based on Markov decision process according to claim 1, characterized in that: The policy iteration algorithm in step 5 includes the following sub-steps: Step 5.1: Initialize the state space Action Space State transition probability matrix Reward Function The discount factor λ is used to Arrange in a certain order, denoted as s (1) ,s (2) ,…,s (|S|) ; Step 5.2: Arbitrarily give the initialization strategy π0 and initialize the number of iterations k to 0; Step 5.

3. Calculate the expected value when the strategy is π0 The update strategy is π1: Step 5.4, the number of iterations k is incremented by 1; Step 5.5, calculate the strategy as π k Expected value at The update strategy is π k+1 : Step 5.

6. Repeat steps 5.4 and 5.5 until π k =π k+1 , that is, strategy π k and strategy π k+1 consistent; Step 5.7: Get the optimal detection strategy π * =π k ; Among them, π k It represents the strategy when the number of iterations is k, which is a mapping from the state space to the action space, where a=π k (s), represents the set of natural numbers, is the state transition matrix when the strategy is π, which is a The matrix, is the reward vector when the strategy is π, which is a vector, I is The identity matrix of .

Images