Infectious disease model parameter identification method and system based on reinforcement learning

Abstract

The invention provides an infectious disease model parameter identification method and system based on reinforcement learning, and belongs to the technical field of model parameter identification, and the method comprises the steps: obtaining infectious disease past data of a region needing fitting prediction; constructing a proper fitting model according to the transmission characteristics of infectious diseases, and determining a model parameter identification problem objective function; and parameter identification optimization is carried out according to the constructed fitting model. According to the method, the initial population is generated by adopting Circle chaotic mapping, the characteristic of the initial population can cover a wider parameter space, and compared with a traditional initialization method, the method has higher ergodicity and aperiodicity, the diversity of the initial population is effectively improved, and the risk of premature convergence of the algorithm is reduced; the invention provides a novel optimization algorithm integrating a grey wolf algorithm (GWO), a star-graffiti optimization algorithm (NOA) and reinforcement learning (RL), a hierarchical hunting strategy of the GWO and a dynamic learning mechanism of the RL are combined, the intelligence of the algorithm for autonomously adjusting a search strategy is enhanced, and therefore the search performance of the algorithm is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of model parameter identification, and in particular to a method and system for identifying infectious disease model parameters based on reinforcement learning. Background Art

[0002] Infectious disease dynamics models have become a crucial tool for understanding and responding to epidemics. Using mathematical methods and computer simulations, these models can simulate and predict the spread of infectious diseases within a population, helping us better understand epidemic trends, evaluate the effectiveness of different prevention and control strategies, and guide public health emergency response decisions.

[0003] It's worth noting that the key to truly effective infectious disease dynamics models lies in accurately defining model parameters, including infection rate, contact rate, and incubation period. Accurate identification of these parameters directly impacts the model's predictive and application performance. Parameter identification in infectious disease models is essentially a nonlinear optimization problem. Traditional methods for model parameter identification include least squares, Newton's method, and maximum likelihood. These methods employ an iterative approach to obtain a global optimal solution, based on information about the analytical properties of the optimization objective. Researchers have also improved upon these algorithms to achieve more accurate estimation of model parameters. Chi Ho-Leung et al. proposed a recursive least squares method that incorporates a relaxation property called initial excitation to estimate the parameters of the SIS infectious disease model. Hari M. Srivastava et al. used the Chebyshev spectral collocation method to numerically simulate the parameters of their proposed fractional-order epidemiological model for Ebola virus.

[0004] With the development of computer technology, researchers have been trying to develop more efficient optimization algorithms to solve complex optimization problems. Consequently, swarm intelligence optimization algorithms have emerged. Compared to traditional optimization algorithms, swarm intelligence optimization algorithms have fewer constraints on the optimization object, and most swarm intelligence optimization algorithms have fewer parameters and are easier to implement. Therefore, they exhibit stronger optimization capabilities for solving complex optimization problems. For example, there is the particle swarm optimization algorithm (PSO), which was inspired by the predation of bird flocks; the artificial bee colony algorithm, which was proposed by simulating the honey bee collection process; and the Grey Wolf Optimizer (GWO), which was proposed by imitating the hunting process of wolves. In addition to swarm intelligence algorithms that are derived from simulating biological information systems in nature, some swarm intelligence algorithms are also proposed by observing physical phenomena in nature, such as the fireworks algorithm, which was inspired by the sparks produced by fireworks and the continued explosion and fragmentation.

[0005] In addition to classic swarm intelligence optimization algorithms, researchers have also proposed a variety of improved algorithms. Bing Xie et al. applied Lévy flight to improve the PSO algorithm, resulting in the Lévy-PSO algorithm, which performs better than other PSO improvements in parameter identification for infectious disease models. Leilei Shi et al. proposed a hybrid optimization algorithm that combines the PSO algorithm with a genetic algorithm (GA), incorporating some of the genetic operations used in the GA, including selection, crossover, and mutation, into the PSO algorithm to increase population diversity.

[0006] Traditional parameter identification methods rely on model structure and computational complexity. When applied to complex infectious disease systems, they are limited in the scale of the problem they can solve, making it difficult to obtain optimal or near-optimal solutions in a short period of time, which in turn affects model prediction accuracy. While swarm intelligence optimization algorithms offer certain advantages, they are prone to falling into local optima, lacking the ability to explore optimal solutions in later iterations, and resulting in slow convergence. This results in low parameter identification accuracy and fails to meet the accuracy and efficiency requirements of infectious disease models. Summary of the Invention

[0007] The purpose of the present invention is to provide a method and system for identifying infectious disease model parameters based on reinforcement learning to solve at least one technical problem existing in the above-mentioned background technology.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a method for identifying parameters of an infectious disease model based on reinforcement learning, comprising:

[0010] Obtaining historical data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region;

[0011] Construct a suitable fitting model based on the propagation characteristics of infectious diseases and determine the objective function of the model parameter identification problem;

[0012] According to the constructed fitting model, parameter identification and optimization are performed, including: population initialization, for each individual X in the initial population X i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U iSubstitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best , which are the identified infectious disease model parameters that best fit the real data.

[0013] As a further limitation of the first aspect of the present invention, it also includes preprocessing of the data, including deleting outliers and missing values, and storing the data in .csv format to facilitate subsequent data reading and processing during the infectious disease model parameter identification process, ensuring that the data can be successfully applied to model calculations.

[0014] As a further limitation of the first aspect of the present invention, the reinforcement learning action is set to:

[0015]

[0016] in, They represent the top three search individuals that currently obtain the best fit, r3 is a random number between [0,1], λ is a random number that obeys the Lévy distribution, and the coefficient l decreases from 1 to 0.

[0017] As a further limitation of the first aspect of the present invention, the reinforcement learning reward is set as follows: after the individual position is updated, when a better solution is obtained, the reward value r=1 is set; when no better solution is obtained after the position is updated, the reward value r=0 is set.

[0018] As a further limitation of the first aspect of the present invention, the reinforcement learning state is set as: after updating the position through the three actions α1, α2, and α3, the reward value r=1 is set to state 1, and the reward value r=0 is set to state 2; after updating the position through the three actions α4, α5, and α6, the reward value r=1 is set to state 3, and the reward value r=0 is set to state 4.

[0019] As a further limitation of the first aspect of the present invention, the reinforcement learning action selection strategy is: using an ε-greedy algorithm to implement action selection, the ε-greedy algorithm selects the action with the largest current Q value with a probability of 1-ε, and randomly selects the remaining actions with a probability of ε:

[0020]

[0021] For each search individual, a position update strategy is selected and a Q table is established for each individual.

[0022] In a second aspect, the present invention provides an infectious disease model parameter identification system based on reinforcement learning, comprising:

[0023] An acquisition module is used to obtain past data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region;

[0024] A construction module is used to build a suitable fitting model based on the propagation characteristics of infectious diseases and determine the objective function of the model parameter identification problem;

[0025] The identification and optimization module is used to perform parameter identification and optimization based on the constructed fitting model, including: population initialization, for each individual X in the initial population X i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U i Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best , which are the identified infectious disease model parameters that best fit the real data.

[0026] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the reinforcement learning-based infectious disease model parameter identification method as described in the first aspect is implemented.

[0027] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the infectious disease model parameter identification method based on reinforcement learning as described in the first aspect.

[0028] In a fifth aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the infectious disease model parameter identification method based on reinforcement learning as described in the first aspect.

[0029] Explanation of terms:

[0030] (1) Infectious disease dynamics model: A model that uses mathematical methods and computer simulation to describe the spread of infectious diseases in the population, such as the SIR and SEIR models.

[0031] (2) Parameter identification: The process of determining the specific values of parameters such as infection rate and contact rate in the infectious disease model is crucial to the model's prediction accuracy.

[0032] (3) Swarm intelligence optimization algorithm: a type of heuristic algorithm inspired by natural phenomena or social behavior, used to find the optimal solution in the search space, such as the gray wolf algorithm, the star crow optimization algorithm, the particle swarm optimization algorithm, etc.

[0033] (4) Reinforcement learning (RL): A learning mechanism in which an agent learns the optimal behavior strategy based on reward feedback by interacting with the environment.

[0034] Beneficial effects of the present invention: The existing swarm intelligence optimization algorithms have the problems of rapid decline in population diversity and easy falling into local optimality. The present invention significantly enhances the algorithm's exploration ability in parameter space by integrating the gray wolf algorithm, the nutcracker optimization algorithm and the dynamic learning adjustment mechanism of reinforcement learning. At the same time, it realizes the collaborative optimization of global search and local search by setting a variety of position update strategies. In the parameter identification problem of integer-order and fractional-order SIR, SEIR, and SEIAR models, compared with GWO, PSO, CS, I-GWO, EN-GWO, RLDE and NRLPSO, it has higher identification accuracy and faster convergence speed, and is more practical in the parameter identification problem of infectious disease models than existing solutions.

[0035] Additional advantages of the present invention will be more clearly given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 This is a flow chart of the infectious disease model parameter identification method based on reinforcement learning according to an embodiment of the present invention.

[0038] Figure 2 This is a flow chart of the infectious disease model parameter identification and optimization algorithm described in an embodiment of the present invention.

[0039] Figure 3 Schematic diagram showing the comparison of convergence curves of the GWO-RL algorithm described in an embodiment of the present invention and eight different optimization algorithms in the fractional-order SEIR model parameter identification problem. DETAILED DESCRIPTION

[0040] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention.

[0041] Those skilled in the art will understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0042] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0043] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0044] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise contradictory.

[0045] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0046] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0047] In the problem of parameter identification of infectious disease models, due to the complexity of infectious disease systems and the mutual influence between parameters, traditional parameter identification methods are limited by model structure and computational complexity. Most swarm intelligence optimization algorithms have problems such as easy falling into local optimality, premature convergence and slow convergence speed. In the practical application of infectious disease model parameter identification, each has room for improvement. The present invention aims to provide a more efficient and reliable solution to the problem of parameter identification of infectious disease models. The present invention aims to improve the global search capability and convergence speed of the algorithm, more accurately identify infectious disease model parameters, improve the accuracy of infectious disease model predictions, and provide a more reliable scientific basis for public health policy formulation and epidemic prevention and control decisions.

[0048] Example 1

[0049] In this embodiment 1, a reinforcement learning-based infectious disease model parameter identification system is first provided, comprising: an acquisition module for acquiring past infectious disease data for a region where fitting prediction is required; wherein the data at least includes daily case data within a period of time within the region and the total population of the region; a construction module for constructing a suitable fitting model based on the propagation characteristics of the infectious disease and determining the objective function of the model parameter identification problem; an identification optimization module for performing parameter identification optimization based on the constructed fitting model, comprising: performing population initialization, performing a population initialization on each individual X in the initial population X, and performing a population initialization on each individual X in the initial population X. i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U i Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best, which are the identified infectious disease model parameters that best fit the real data.

[0050] In this embodiment, the above-mentioned system is used to implement a parameter identification method for an infectious disease model based on reinforcement learning, including:

[0051] Obtain past data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region.

[0052] Specifically, prepare historical data on infectious diseases in a specific location for fitting and prediction. The infectious disease model needs to determine its parameters based on actual epidemic data to accurately simulate and predict the spread of infectious diseases. The data should at least include daily case data for a period of time and the local total population. Daily case data reflects the actual spread of infectious diseases in the population and is a key indicator for measuring the development trend of infectious diseases. It directly affects the determination of parameters such as infection rate and transmission speed in the model. The local total population is the basic data for calculating parameters such as infection rate and transmission range. The data should also be preprocessed, including deleting outliers and missing values, and stored in .csv format to facilitate data reading and processing during the subsequent infectious disease model parameter identification process, ensuring that the data can be successfully applied to model calculations.

[0053] According to the propagation characteristics of infectious diseases, a suitable fitting model is constructed and the objective function of the model parameter identification problem is determined. Specifically, the objective function of the model parameter identification problem is constructed as follows:

[0054] Construct a suitable fitting model based on the characteristics of infectious diseases. The model structure is as follows:

[0055]

[0056] Here, X0 is the number of population categories in the initial state during the time period, and q and ζ are the order and parameters that need to be identified for the model. Different infectious diseases have different transmission characteristics, which determine the order and parameters of the model. For example, factors such as the spread rate of infectious diseases, the length of the incubation period, and the probability of recovery are all reflected in these parameters. An optimization objective function is then constructed for these order and parameters, as shown below:

[0057]

[0058] in,

[0059] According to the constructed fitting model, parameter identification and optimization are performed, including: population initialization, for each individual X in the initial population X iSubstitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U i Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best , which are the identified infectious disease model parameters that best fit the real data.

[0060] Specifically, follow the steps below to execute the parameter identification and optimization algorithm:

[0061] Step 1: Initialize the population, which is X=[X1,X2,…,X N ], the number of individuals in the population is N, and each individual in the population represents a set of candidate parameters. In this embodiment, population initialization based on Circle chaotic mapping: The initial population is generated using Circle chaotic mapping. Its characteristics can cover a wider parameter space. Compared with traditional initialization methods, it has stronger ergodicity and aperiodicity, effectively improving the diversity of the initial population and reducing the risk of premature convergence of the algorithm.

[0062] Step 2: Substitute each individual in the population into the fitting model for numerical simulation, and calculate the deviation value y=y1,y2,…,y from the real data by optimizing the objective function n , and record the optimal solution y best and its corresponding individual x best .

[0063] Step 3: Establish a reinforcement learning Q table for each individual in the population. The reinforcement learning design is as follows:

[0064] (1) Action settings

[0065]

[0066] in, They represent the top three search individuals that currently obtain the best fit, r3 is a random number between [0,1], λ is a random number that obeys the Lévy distribution, and the coefficient l decreases from 1 to 0. Calculated by the following formula:

[0067]

[0068] (2) Reward Setting

[0069] After the individual position is updated, the reward value r=1 is set when a better solution is obtained. If no better solution is obtained after the position is updated, the reward value r=0 is set.

[0070] (3) Status setting

[0071] After updating the position through the three actions α1, α2, and α3, the reward value r = 1 is set to state 1, and the reward value r = 0 is set to state 2;

[0072] After updating the position through the three actions α4, α5, and α6, the reward value r = 1 is set to state 3, and the reward value r = 0 is set to state 4.

[0073] (4) Action selection strategy

[0074] The ε-greedy algorithm is used to implement action selection. The ε-greedy algorithm selects the action with the largest current Q value with a probability of 1-ε, and randomly selects the remaining actions with a probability of ε. The specific method is as follows:

[0075]

[0076] Based on the above settings, for each search individual, a reinforcement learning algorithm is used to select a position update strategy, and a Q table can be established for each individual as shown in Table 1 below:

[0077] Table 1 GWO-RL algorithm search individual Q table settings

[0078]

[0079] In this example, a novel optimization algorithm is proposed that integrates the Grey Wolf Algorithm (GWO), the Nutcracker Optimization Algorithm (NOA), and reinforcement learning (RL) based on the selection of population position update strategies using reinforcement learning. This algorithm combines the hierarchical hunting strategy of GWO with the dynamic learning mechanism of RL to enhance the algorithm's intelligent ability to autonomously adjust its search strategy, thereby improving its search performance. The algorithm fusion method includes selectable strategy settings, action settings, reward settings, and state settings in reinforcement learning, as well as optimization logic for parameter identification of infectious disease models.

[0080] Step 4: Each individual in the population selects a position update strategy according to the action selection strategy and obtains a new set of parameters U = [U1, U2,…, UN].

[0081] Step 5: Convert individual U iSubstitute the fitting model again for numerical simulation, and calculate the deviation value f from the real data by optimizing the objective function i , if f i <y i , then update X i For U i , and update the Q table according to the reinforcement learning setting. If f i <y best , then use U i Replace X best .

[0082] If the termination condition is not met, return to step 3. If the termination condition is met, output X best .

[0083] 4. Output X best These are the identified infectious disease model parameters that best fit the real data. Optimization is complete.

[0084] In order to verify the practicality of the proposed algorithm in the parameter identification problem of infectious disease models, simulation experiments were completed on integer-order and fractional-order SIR, SEIR, and SEIAR models, and compared with GWO, PSO, CS, I-GWO, EN-GWO, RLDE and NRLPSO algorithms. Taking the fractional-order SEIR model as an example, the performance comparison of the GWO-RL algorithm and 8 different optimization algorithms in the parameter identification problem of the fractional-order SEIR model is shown in Table 2 below:

[0085] Table 2

[0086]

[0087] Experimental results show that the GWO-RL algorithm is more accurate in identifying the order of the SEIR model than other optimization algorithms, and its speed of convergence to the true parameter value is also faster than other algorithms after 20 iterations, which has certain application value and practical significance.

[0088] Example 2

[0089] This embodiment 2 provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the above-mentioned method for identifying parameters of an infectious disease model based on reinforcement learning is implemented. The method includes:

[0090] Obtaining historical data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region;

[0091] Construct a suitable fitting model based on the propagation characteristics of infectious diseases and determine the objective function of the model parameter identification problem;

[0092] According to the constructed fitting model, parameter identification and optimization are performed, including: population initialization, for each individual X in the initial population X i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U i Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best , which are the identified infectious disease model parameters that best fit the real data.

[0093] Example 3

[0094] This embodiment 3 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the above-mentioned reinforcement learning-based infectious disease model parameter identification method, which includes:

[0095] Obtaining historical data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region;

[0096] Construct a suitable fitting model based on the propagation characteristics of infectious diseases and determine the objective function of the model parameter identification problem;

[0097] According to the constructed fitting model, parameter identification and optimization are performed, including: population initialization, for each individual X in the initial population X i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual X best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in Ui Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best , which are the identified infectious disease model parameters that best fit the real data.

[0098] Example 4

[0099] This embodiment 4 provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the above-mentioned method for identifying infectious disease model parameters based on reinforcement learning. The method includes:

[0100] Obtaining historical data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region;

[0101] Construct a suitable fitting model based on the propagation characteristics of infectious diseases and determine the objective function of the model parameter identification problem;

[0102] According to the constructed fitting model, parameter identification and optimization are performed, including: population initialization, for each individual X in the initial population X i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U i Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best, which are the identified infectious disease model parameters that best fit the real data.

[0103] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solutions disclosed in the present invention without the need for creative work should be included in the scope of protection of the present invention.

Claims

1. A method for identifying parameters of an infectious disease model based on reinforcement learning, characterized in that: include: Obtaining historical data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region; Construct a suitable fitting model based on the propagation characteristics of infectious diseases and determine the objective function of the model parameter identification problem; According to the constructed fitting model, parameter identification and optimization are performed, including: population initialization, for each individual X in the initial population X i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U i Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best , which are the identified infectious disease model parameters that best fit the real data.

2. The infectious disease model parameter identification method based on reinforcement learning according to claim 1, characterized in that: It also includes data preprocessing, including deleting outliers and missing values, and storing the data in .csv format to facilitate subsequent data reading and processing during the identification of infectious disease model parameters, ensuring that the data can be successfully applied to model calculations.

3. The infectious disease model parameter identification method based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning action is set as: in, They represent the top three search individuals that currently obtain the best fit, r3 is a random number between [0,1], λ is a random number that obeys the Lévy distribution, and the coefficient l decreases from 1 to 0.

4. The infectious disease model parameter identification method based on reinforcement learning according to claim 3, characterized in that: The reinforcement learning reward is set as follows: after the individual position is updated, when a better solution is obtained, the reward value r=1 is set; when no better solution is obtained after the position is updated, the reward value r=0 is set.

5. The infectious disease model parameter identification method based on reinforcement learning according to claim 4, characterized in that: The reinforcement learning state is set as follows: after updating the position through the three actions α1, α2, and α3, the reward value r = 1 is set as state 1, and the reward value r = 0 is set as state 2; after updating the position through the three actions α4, α5, and α6, the reward value r = 1 is set as state 3, and the reward value r = 0 is set as state 4.

6. The infectious disease model parameter identification method based on reinforcement learning according to claim 5, characterized in that: The reinforcement learning action selection strategy is to use the ε-greedy algorithm to implement action selection. The ε-greedy algorithm selects the action with the largest current Q value with a probability of 1-ε, and randomly selects the remaining actions with a probability of ε: For each search individual, a position update strategy is selected and a Q table is established for each individual.

7. An infectious disease model parameter identification system based on reinforcement learning, characterized in that: include: An acquisition module is used to obtain past data on infectious diseases in the region where fitting prediction is required; wherein the data at least includes daily case data within a period of time in the region and the total population of the region; A construction module is used to build a suitable fitting model based on the propagation characteristics of infectious diseases and determine the objective function of the model parameter identification problem; The identification and optimization module is used to perform parameter identification and optimization based on the constructed fitting model, including: population initialization, for each individual X in the initial population X i Substitute the fitting model for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the first deviation value; and record the optimal solution Y best and its corresponding individual x best ; Establish a reinforcement learning Q table for each individual in the population; Each individual in the population selects a position update strategy according to the action selection strategy to obtain a new set of parameters U; The individual U in U i Substitute the fitting model again for numerical simulation, and calculate the deviation value from the real data by optimizing the objective function, which is the second deviation value; if the second deviation value is less than the first deviation value, update X i For U i , and update the Q table according to the reinforcement learning setting; if the second deviation value <y best , then use U i Replace X best ; Until the termination condition is reached, output X best , which are the identified infectious disease model parameters that best fit the real data.

8. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the reinforcement learning-based infectious disease model parameter identification method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the infectious disease model parameter identification method based on reinforcement learning as described in any one of claims 1 to 6.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute instructions for implementing the infectious disease model parameter identification method based on reinforcement learning as described in any one of claims 1 to 6.

Images