Individual mobile intervention methods, devices, storage media and program products

Abstract

This application discloses a method, device, storage medium, and program product for individual mobility intervention, relating to the field of reinforcement learning technology. This application acquires epidemic-related data from multiple spatiotemporal regions divided into areas of a city to be controlled. Through reinforcement learning algorithms corresponding to each agent, it calculates the epidemic-related data for the corresponding spatiotemporal region to obtain target intervention measures. Since each spatiotemporal region corresponds to a separate agent, each agent can calculate the epidemic-related data for each region independently, reducing the amount of data required for each agent to compute and lowering the computational burden. Therefore, even in situations of large-scale population dynamic changes, effective target intervention measures can be generated in real time. Based on these target intervention measures, mobility intervention is performed on individuals within the corresponding spatiotemporal region, thereby achieving timely individual mobility intervention.

Description

Technical Field

[0001] This application relates to the field of reinforcement learning technology, and in particular to methods, devices, storage media, and program products for individual mobile intervention. Background Technology

[0002] Currently, epidemic control strategies are typically tailored to each individual by leveraging their specific behaviors and contact networks.

[0003] However, this approach often faces a huge computational burden when dealing with a large number of individuals in complex cities, especially when dealing with dynamic changes in large populations. It is difficult to generate effective epidemic control strategies in real time and cannot intervene in individual movement in a timely manner.

[0004] Therefore, there is a technical problem with the poor real-time performance of individual mobility intervention in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide a method, device, storage medium, and program product for individual mobility intervention, aiming to solve the technical problem of poor real-time performance in individual mobility intervention.

[0006] To achieve the above objectives, this application proposes an individual mobility intervention method, the method comprising:

[0007] Acquire epidemic-related data from multiple spatiotemporal regions obtained by dividing the city under control, with each spatiotemporal region corresponding to an intelligent agent;

[0008] By using the reinforcement learning algorithms corresponding to each of the aforementioned intelligent agents, the epidemic-related data of the corresponding spatiotemporal regions are calculated to obtain the target intervention measures for the corresponding spatiotemporal regions;

[0009] Based on the aforementioned target intervention measures, movement interventions are carried out on individuals within the corresponding spatiotemporal region.

[0010] In one embodiment, the step of calculating the epidemic-related data of the corresponding spatiotemporal region using the reinforcement learning algorithm corresponding to each of the intelligent agents to obtain the target intervention measures for the corresponding spatiotemporal region includes:

[0011] Based on the epidemic-related data in the multiple spatiotemporal regions, global epidemic-related data are calculated;

[0012] By using the reinforcement learning algorithms corresponding to each of the aforementioned intelligent agents, the epidemic-related data of the corresponding spatiotemporal region and the global epidemic-related data are calculated to obtain the target intervention measures for the corresponding spatiotemporal region.

[0013] In one embodiment, after the step of performing movement intervention on individuals within the corresponding spatiotemporal region based on the target intervention measure, the method includes:

[0014] Obtain the number of infected individuals in each spatiotemporal region after the mobile intervention;

[0015] Based on the number of infected individuals corresponding to each spatiotemporal region, the preset intervention cost, the preset balance weight between health and socioeconomic status, and the preset tendency weight, the reward value is calculated through a preset reward function, wherein the preset tendency weight is used to characterize the tendency to take intervention measures or the tendency to control costs.

[0016] Based on the reward value, the reinforcement learning algorithm is iteratively trained until the goal of minimizing the number of infected people and the intervention cost is achieved, so as to update the agent.

[0017] In one embodiment, the preset intervention cost is the sum of the costs of the target intervention measures taken in the corresponding spatiotemporal region; the preset balance weight is determined based on the affordability of the economic system and the affordability of the medical system in the corresponding spatiotemporal region.

[0018] In one embodiment, the step of performing movement intervention on individuals within a corresponding spatiotemporal region based on the target intervention measure includes:

[0019] Based on the target intervention measures corresponding to each spatiotemporal region and the preset intervention number limit for different intervention measures, the proportion of individuals that can be intervened in each spatiotemporal region is determined;

[0020] Based on the proportion of interveneable individuals in each of the aforementioned spatiotemporal regions, target individuals in each of the aforementioned spatiotemporal regions are determined, and mobile intervention is carried out on the target individuals in the corresponding spatiotemporal regions.

[0021] In one embodiment, the step of determining the target individual in each of the said spatiotemporal regions based on the proportion of intervenable individuals in each of the said spatiotemporal regions includes:

[0022] Based on the locations visited by confirmed cases in the first time period from the epidemiological data of each of the aforementioned spatiotemporal regions, high-risk locations in the first time period are identified.

[0023] Based on the aforementioned epidemic-related data, the infection risk of the first individual who visited the high-risk location during the second time period was determined.

[0024] Based on the social attributes of the first individual, determine the social contact risk of the first individual;

[0025] Based on the infection risk, social contact risk, and the proportion of individuals who can be intervened, the target individual is determined from the first individual corresponding to each of the aforementioned spatiotemporal regions.

[0026] In one embodiment, the step of determining the target individual from the first individual corresponding to each of the spatiotemporal regions based on the infection risk, social contact risk, and the proportion of interventionable individuals includes:

[0027] The first individuals are sorted from high to low infection risk. When there are individuals with zero infection risk and / or individuals with the same infection risk among the first individuals, the individuals with zero infection risk and / or individuals with the same infection risk are sorted from high to low social contact risk to obtain the second sorted individuals. The higher the ranking, the higher the intervention priority.

[0028] Based on the proportion of individuals eligible for intervention and the intervention priority, target individuals are determined from the second group of individuals.

[0029] In addition, to achieve the above objectives, this application also proposes an individual mobility intervention device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the individual mobility intervention method as described above.

[0030] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the individual mobility intervention method described above.

[0031] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the individual mobility intervention method described above.

[0032] One or more technical solutions proposed in this application have at least the following technical effects:

[0033] This application obtains epidemic-related data from multiple spatiotemporal regions obtained by dividing the city under control; through reinforcement learning algorithms corresponding to each agent, it calculates the epidemic-related data of the corresponding spatiotemporal region to obtain the target intervention measures for the corresponding spatiotemporal region. Since each spatiotemporal region corresponds to an agent, each agent can calculate the epidemic-related data of each spatiotemporal region separately, reducing the amount of data that each agent needs to calculate and lowering the computational burden; therefore, even in the case of dynamic changes in a large population, effective target intervention measures can be generated in real time. Based on the target intervention measures, mobile intervention is carried out on individuals in the corresponding spatiotemporal region, thereby achieving timely individual mobile intervention. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 This is a flowchart illustrating an embodiment of the individual mobility intervention method of this application.

[0037] Figure 2 This is a schematic diagram of the first scenario provided in Embodiment 1 of the individual mobility intervention method of this application;

[0038] Figure 3 This is a schematic diagram of the second scenario provided in Embodiment 1 of the individual mobility intervention method of this application;

[0039] Figure 4 This is a flowchart illustrating Embodiment 2 of the individual mobility intervention method of this application;

[0040] Figure 5 This is a schematic diagram of the third scenario provided in Embodiment 2 of the individual mobility intervention method of this application;

[0041] Figure 6 This is a schematic diagram of the fourth scenario provided in Embodiment 2 of the individual mobility intervention method of this application;

[0042] Figure 7 This is a schematic diagram of the module structure of the individual mobility intervention device according to an embodiment of this application;

[0043] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the individual movement intervention method in the embodiments of this application.

[0044] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0046] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0047] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or individual mobile intervention device capable of performing the above functions. The following description uses an individual mobile intervention device as an example to illustrate this embodiment and the subsequent embodiments.

[0048] Based on this, embodiments of this application provide an individual mobility intervention method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the individual mobility intervention method of this application.

[0049] In this embodiment, the individual mobility intervention method includes steps S10 to S30:

[0050] Step S10: Obtain epidemic-related data in multiple spatiotemporal regions obtained by dividing the city to be controlled, with each spatiotemporal region corresponding to an intelligent agent;

[0051] In this embodiment, the individual mobility intervention device can belong to the individual mobility intervention equipment, and the individual mobility intervention equipment can belong to the individual mobility intervention system. The individual mobility intervention system can be a Regional Multi-Agent Reinforcement Learning (R-MARL) control system created based on ABM (Agent-Based Modeling), as described above. Figure 1 .

[0052] To address the complexity of individual interventions in large-scale cities, the regional multi-agent reinforcement learning control system divides cities requiring epidemic control (cities to be controlled) into multiple spatiotemporal regions, which can be based on residential communities or streets.

[0053] In order to conduct epidemic control in different time and space regions, epidemic-related data in each time and space region were obtained.

[0054] Specifically, epidemic-related data can include the number of infections, the implementation of intervention measures, and individual infection risk.

[0055] Step S20: Using the reinforcement learning algorithm corresponding to each of the aforementioned intelligent agents, calculate the epidemic-related data of the corresponding spatiotemporal region to obtain the target intervention measures for the corresponding spatiotemporal region;

[0056] It should be noted that the regional multi-agent reinforcement learning control system assigns an agent to each spatiotemporal region. Each agent learns how to take the most effective intervention measures in its own spatiotemporal region through reinforcement learning algorithms. That is, each agent only needs to calculate the epidemic-related data at the spatiotemporal region level and make spatiotemporal region-level decisions, without needing to customize control measures for individuals.

[0057] Therefore, by using the reinforcement learning algorithms corresponding to each of the aforementioned agents to calculate the epidemic-related data for the corresponding spatiotemporal regions, it is possible to obtain the target intervention measures for the corresponding spatiotemporal regions. This can reduce the computational load of the agents, improve computational efficiency, and thus enhance the real-time nature of the intervention.

[0058] Specifically, the framework of the reinforcement learning algorithm for each agent can be Actor-Critic, referring to... Figure 3 Among them, the Critic network V π The Actor Network is responsible for estimating the long-term reward value of agent actions, while determining the optimal action to balance epidemic control with socioeconomic costs. This is done to capture time-related patterns and effectively assess the risk of epidemic spread.

[0059] Reference Figure 3 Both the Actor and Critic networks utilize Long Short-Term Memory (LSTM) network models. LSTM models encode past sequences of states, helping the regional multi-agent reinforcement learning control system dynamically adjust its control strategy. The hidden states of the LSTM are output through two layers of Multilayer Perceptrons (MLPs), and the outputs are converted into probability values ​​and state values ​​for different actions. Therefore, by employing the aforementioned reinforcement learning algorithm, the regional multi-agent reinforcement learning control system can make more accurate decisions in complex and dynamic epidemic environments, effectively suppressing the spread of epidemics.

[0060] Specifically, the specific implementation of calculating the epidemic-related data of the corresponding spatiotemporal region using the reinforcement learning algorithm corresponding to each of the aforementioned intelligent agents to obtain the target intervention measures for the corresponding spatiotemporal region can be as follows:

[0061] Based on the epidemic-related data in the multiple spatiotemporal regions, global epidemic-related data is calculated; through the reinforcement learning algorithms corresponding to each of the aforementioned agents, the epidemic-related data in the corresponding spatiotemporal regions and the global epidemic-related data are calculated to obtain the target intervention measures for the corresponding spatiotemporal regions.

[0062] It should be noted that, referring to Figure 3To improve the accuracy of the calculation of target intervention measures, in addition to considering the epidemic-related data (local variables or regional status) in each spatiotemporal region, global epidemic-related data (global variables or global status) can also be combined to calculate the target intervention measures for the corresponding spatiotemporal region.

[0063] Specifically, the local variable can be defined as s m,t =(H m,t ,ΔI m,t C m,t D m,t ), where H m,t ΔI represents the number of individuals in different health states. m,t C represents the daily number of new infections. m,t D represents the number of various intervention measures. m,t This indicates the number of individuals whose risk of infection is greater than a preset threshold (e.g., 0.1).

[0064] Specifically, global epidemic-related data can be calculated based on epidemic-related data in the multiple spatiotemporal regions. Global variables are important variables for comprehensively evaluating the state of the multi-agent reinforcement learning control system of the entire region at time step t. Global variables can be defined as the sum of local variables in each spatiotemporal region.

[0065] Furthermore, the agent in each spatiotemporal region, based on the local variable s m,t Make decisions and choose appropriate interventions; different levels of interventions can be used... This indicates that the three intervention measures correspond to restrictions on movement within a specific time and space area: home isolation and complete isolation.

[0066] Step S30: Based on the target intervention measures, conduct movement intervention on individuals within the corresponding spatiotemporal region.

[0067] It is understandable that once the target intervention measures for each spatiotemporal region are determined, mobile interventions can be carried out on individuals within the corresponding spatiotemporal region (for example, prompting that mobile intervention is needed for the individual, or sending control information to the individual, etc.) to achieve the control of the epidemic. Using the above method, there is no need to formulate intervention measures for individuals separately, which improves the timeliness of intervention.

[0068] Specifically, the specific implementation method for conducting movement intervention on individuals within the corresponding spatiotemporal region based on the target intervention measures can be:

[0069] The number of infected individuals in each spatiotemporal region after mobile intervention is obtained; based on the number of infected individuals corresponding to each spatiotemporal region, the preset intervention cost, the preset balance weight between health and socioeconomic status, and the preset tendency weight, a reward value is calculated through a preset reward function, wherein the preset tendency weight is used to characterize the tendency to take intervention measures or the tendency to control costs; based on the reward value, the reinforcement learning algorithm is iteratively trained until the goal of minimizing the number of infected individuals and the intervention cost is achieved, so as to update the agent.

[0070] It should be noted that, in order to improve the accuracy of the intervention measures formulated by the agent, feedback can be provided to the agent after each decision is made, for reference... Figure 3 This allows the agent to continuously optimize the reinforcement learning algorithm, resulting in an agent with better performance.

[0071] Specifically, it can involve obtaining the number of infected individuals in each spatiotemporal region after the mobile intervention, and calculating the reward value based on the number of infected individuals corresponding to each spatiotemporal region, the preset intervention cost, the preset balance weight between health and socioeconomic status, and the preset tendency weight, through a preset reward function.

[0072] The preset reward function can be expressed as:

[0073]

[0074]

[0075]

[0076] Specifically, for the m-th spatiotemporal region, This represents the number of infected individuals residing in the m-th spatiotemporal region at time step t. This number includes all individuals infected within the spatiotemporal region. And individuals infected by the spread from the m-th spatiotemporal region to other spatiotemporal regions. (Where, j represents other spatiotemporal regions besides the m-th spatiotemporal region); it should be noted that this number of infected individuals is used to measure the spread of the epidemic in each spatiotemporal region, so as to provide feedback for the agent's decision-making.

[0077] Meanwhile, in the aforementioned preset reward function This represents the preset intervention cost for the m-th spatiotemporal region at time step t. The preset intervention cost can be calculated by weighting and summing the intervention measures a in each spatiotemporal region. m,t The cost is obtained; that is, the preset intervention cost is the sum of the costs of the target intervention measures taken in the corresponding spatiotemporal region.

[0078] Furthermore, the preset balance weight β in the aforementioned preset reward function is used to balance health and socioeconomic factors. This preset balance weight is determined based on the carrying capacity of the economic system and the medical system in the corresponding spatiotemporal region. The specific calculation method for the preset balance weight can be: Where, ω Q ω represents the carrying capacity of the economic system in the corresponding spatiotemporal region. I This indicates the capacity of the medical system in the corresponding time and space region.

[0079] In addition, the preset tendency weight α in the preset reward function is used to characterize the agent's tendency to take intervention measures. For example, if the value of α is greater than the preset value, it is believed that the regional multi-agent reinforcement learning control system is more inclined to take intervention measures rather than control costs, so that the trained agent can prioritize controlling the risk of infection rather than prioritizing the control costs.

[0080] Furthermore, based on the reward value, the reinforcement learning algorithm is iteratively trained until the goal of minimizing the number of infected people and the intervention cost is achieved, so as to update the agent. By adopting this continuous reinforcement learning method, it can be ensured that the agent determines the optimal intervention measures while minimizing the number of infected people and the intervention cost.

[0081] In this embodiment, a regional multi-agent reinforcement learning system simplifies the complex individual-level control problem into a regional-level control problem, significantly reducing computational complexity and enabling efficient intervention in large-scale urban scenarios. By using local variables (such as the number of infected individuals and the intensity of intervention measures), the regional multi-agent reinforcement learning system can select the optimal intervention measures for each spatiotemporal region, achieving a balance between suppressing the spread of the epidemic and reducing socioeconomic costs. By combining spatiotemporal regional control with individual intervention, the system addresses the real-time performance and computational complexity issues of individual mobility control in large-scale urban environments, thereby improving real-time performance.

[0082] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 Step S30 includes steps S01 to S02:

[0083] Step S01: Based on the target intervention measures corresponding to each spatiotemporal region and the preset intervention number limit for different intervention measures, determine the proportion of individuals that can be intervened in each spatiotemporal region;

[0084] In this embodiment, in order to improve the accuracy of intervention, individual risk assessment can be combined with spatiotemporal regional intelligent control to dynamically optimize individual movement intervention, aiming to solve the problems of computational complexity and real-time performance in epidemic prevention and control in large-scale urban environments.

[0085] Specifically, after the agent determines the target intervention measures, the regional multi-agent reinforcement learning system can determine the proportion of individuals that can be intervened in each of the spatiotemporal regions based on the target intervention measures corresponding to each spatiotemporal region and the preset intervention number limit for different intervention measures.

[0086] The proportion of individuals who can be intervened for different intervention measures can be defined as follows: and

[0087] Specifically, the proportion of individuals eligible for intervention can be determined based on pre-defined limits on the number of individuals involved in different intervention measures. For example, the calculation method could be expressed as:

[0088]

[0089] Where, κ=[κ (c) ,κ (q) ,κ (iso) ], κ represents the preset limit on the number of people involved in different intervention measures (capacity limit) to ensure that the intervention measures do not exceed the capacity of resources.

[0090] Based on the above calculation formula, the proportion of individuals that can be intervened upon is d. m,t The target intervention measures corresponding to the spatiotemporal region are a m,t The term-by-term product of the capacity constraint κ (Hadmar product) allows us to convert the intervention measures selected by agents in the spatiotemporal region into the actual proportion of agents that can be intervened.

[0091] By adopting the above approach, it can be ensured that the regional multi-agent reinforcement learning control system can dynamically adjust the coverage of intervention measures during mobile interventions, without exceeding the actual available public resource capacity.

[0092] Step S02: Based on the proportion of intervenable individuals in each of the aforementioned spatiotemporal regions, determine the target individuals in each of the aforementioned spatiotemporal regions, and perform mobile intervention on the target individuals in the corresponding spatiotemporal regions.

[0093] Furthermore, in order to accurately intervene in the movement of certain individuals within a spatiotemporal region, the target individuals within each of the aforementioned spatiotemporal regions can be identified, and movement intervention can be performed on the target individuals within the corresponding spatiotemporal regions.

[0094] Specifically, the specific implementation method for determining the target individuals in each of the said spatiotemporal regions based on the proportion of interventionable individuals in each of the said spatiotemporal regions can be:

[0095] Based on the locations visited by confirmed cases within a first time period in the epidemiological data of each of the aforementioned spatiotemporal regions, high-risk locations within the first time period are identified; based on the epidemiological data, the infection risk of the first individual who visited the high-risk location within a second time period is determined; based on the social attributes of the first individual, the social contact risk of the first individual is determined; based on the infection risk, social contact risk, and the proportion of interventionable individuals, target individuals are identified from the first individuals corresponding to each of the aforementioned spatiotemporal regions.

[0096] It should be noted that an Infectious-Risk Assessment Model (I-RAM) can be used to analyze the spatiotemporal behavior of individuals and dynamically assess the infection risk of each individual, referring to... Figure 5 .

[0097] Specifically, through an individual infection risk dynamic assessment model, the first time period (|t-τ) in the epidemiological data of each spatiotemporal region is determined. a The confirmed cases within the region visited places such as schools, offices, and communities. For each place, the regional multi-agent reinforcement learning control system determines the number of infected cases in that place within the first time period based on the epidemic-related data, thereby determining the infection risk of that place and ultimately identifying high-risk places. For example, if the number of infected cases visiting a place exceeds a preset number, the place is considered a high-risk place.

[0098]

[0099]

[0100]

[0101] in, It is an indicator function used to determine the past time window τ of a certain infection case. a Have you visited this location? This indicates the risk of infection at the location.

[0102] Furthermore, by using an individual infection risk dynamic assessment model and combining spatiotemporal behavioral data of individuals in epidemiological data, the infection risk of the first individual who visits the high-risk location during the second time period is calculated.

[0103] Specifically, the individual infection risk dynamic assessment model dynamically assesses each individual's infection risk by considering the infection risk of high-risk locations they have visited, and takes into account the time window τ. p To reflect an individual's time window τ p The likelihood of contact with infected cases; specifically, an individual's risk of infection can be expressed as:

[0104]

[0105]

[0106] Among them, W n,e (t) is an indicator function representing whether an individual visited a high-risk location at a specific time (e.g., the second time period). This indicates the individual's risk of infection.

[0107] In addition, the dynamic assessment model of individual infection risk also determines the social contact risk of the first individual based on the first individual's social attributes (e.g., the first individual's occupation and age); specifically, students have a higher frequency of daily contact than the elderly, so students have a higher social contact risk than the elderly.

[0108] Furthermore, based on the infection risk, social contact risk, and the proportion of individuals eligible for intervention, target individuals can be identified from the first individual corresponding to each of the aforementioned spatiotemporal regions, thereby achieving precise intervention and improving the effectiveness of intervention.

[0109] Specifically, the specific implementation method for determining the target individual from the first individual corresponding to each of the aforementioned spatiotemporal regions based on the infection risk, social contact risk, and the proportion of interventionable individuals can be as follows:

[0110] The first individuals are sorted from high to low infection risk. When there are individuals with zero infection risk and / or individuals with the same infection risk among the first individuals, the individuals with zero infection risk and / or individuals with the same infection risk are sorted from high to low social contact risk to obtain the second sorted individuals. The higher the ranking, the higher the intervention priority. Based on the proportion of interventionable individuals and the intervention priority, the target individuals are determined from the second individuals.

[0111] Specifically, it can be based on individual infection risk. All individuals in the first group are sorted from highest to lowest infection risk. When there are individuals in the first group with zero infection risk and / or individuals with the same infection risk, their social contact risk can be considered. Sort them from highest to lowest to get the second sorted entity.

[0112] It is understandable that individuals ranked higher in the second sorted group have a higher intervention priority and are subject to stricter isolation; while individuals ranked lower have a lower intervention priority.

[0113] The proportion of interveneable individuals determined by the control agent in the spatiotemporal region. and Based on the aforementioned intervention priorities, the regional multi-agent reinforcement learning control system identifies the target individuals among the second agent who require home isolation or complete isolation.

[0114] It is understandable that, based on the proportion of individuals who can be intervened, it is determined that 100 people can be placed under home quarantine and 50 people can be placed under complete quarantine. Since people with higher intervention priority are given priority for strict quarantine, if the second group includes 500 people, then the top 50 people in the second group can be selected as target individuals for complete quarantine, and the 100 people in the second group ranked from the top 50 to the top 150 can be selected as target individuals for complete quarantine.

[0115] In this embodiment, the dynamic individual infection risk assessment model can evaluate the infection risk of each individual in real time, allowing for timely adjustments to intervention targets and ensuring the flexibility and timeliness of epidemic prevention and control. It employs two time-backtracking windows to calculate the infection risk in high-risk locations and the individual infection risk separately, accurately identifying high-risk individuals and providing data support for subsequent spatiotemporal and individual interventions. This prioritizes interventions for high-risk individuals. By combining individual spatiotemporal behavior and social contact risk, it provides precise individual intervention measures, reducing unnecessary intervention costs and balancing epidemic control with socioeconomic sustainability. It can accurately calculate an individual's infection risk based on their spatiotemporal behavior and social contact network.

[0116] Furthermore, to verify the effectiveness of the above method, by collecting micro-family structure survey data and macro-demographic data of Shenzhen, a community-level synthetic population containing 1.7 million agents and their spatial unit attributes was generated at a sampling rate of 10%. Based on the real geography, population, social networks, and epidemic transmission data of Shenzhen, the epidemic transmission risk in different spatiotemporal regions was simulated. A discrete-time susceptible-latent-infectious-removed (SLIR) model was set up according to COVID-19 to simulate changes in individual health status. Experimental results show that this application can effectively reduce the number of infections under different epidemic scenarios. Compared with traditional epidemic control strategies (such as unified isolation policies or global reinforcement learning strategies), the multi-agent reinforcement learning system of this application effectively suppressed the spread of the epidemic through refined individual movement control, successfully reduced unnecessary isolation and movement restrictions, and enabled socio-economic activities to operate relatively normally during epidemic prevention and control. Compared with excessive isolation measures, this application balances epidemic control and economic costs, effectively controlling the epidemic while reducing intervention costs. Figure 7 .

[0117] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the individual movement intervention method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0118] This application also provides an individual mobility intervention device, please refer to... Figure 7 The individual mobility intervention device includes:

[0119] The acquisition module 10 is used to acquire epidemic-related data in multiple spatiotemporal regions obtained by dividing the city to be controlled, and each spatiotemporal region corresponds to an intelligent agent;

[0120] The calculation module 20 is used to calculate the epidemic-related data of the corresponding spatiotemporal region through the reinforcement learning algorithm corresponding to each of the intelligent agents, and obtain the target intervention measures for the corresponding spatiotemporal region.

[0121] The intervention module 30 is used to perform mobile intervention on individuals within the corresponding spatiotemporal region based on the target intervention measures.

[0122] The individual mobility intervention device provided in this application, employing the individual mobility intervention method in the above embodiments, can solve the technical problem of poor real-time performance in individual mobility intervention. Compared with the prior art, the beneficial effects of the individual mobility intervention device provided in this application are the same as those of the individual mobility intervention method provided in the above embodiments, and other technical features in the individual mobility intervention device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0123] This application provides an individual mobility intervention device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the individual mobility intervention method in Embodiment 1 above.

[0124] The following is for reference. Figure 8 The diagram illustrates a structural schematic suitable for implementing the embodiments of this application. The individual mobile intervention device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The individual mobile intervention device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0125] like Figure 8As shown, the individual mobility intervention device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the individual mobility intervention device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following regional multi-agent reinforcement learning control systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the individual mobile intervention device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show individual mobile intervention devices with various regional multi-agent reinforcement learning control systems, it should be understood that it is not required to implement or possess all of the shown regional multi-agent reinforcement learning control systems. Alternatively, more or fewer regional multi-agent reinforcement learning control systems may be implemented or possessed.

[0126] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0127] The individual mobility intervention device provided in this application, employing the individual mobility intervention method in the above embodiments, can solve the technical problem of poor real-time performance in individual mobility intervention. Compared with the prior art, the beneficial effects of the individual mobility intervention device provided in this application are the same as those of the individual mobility intervention method provided in the above embodiments, and other technical features of this individual mobility intervention device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0128] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0129] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0130] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the individual mobility intervention method in the above embodiments.

[0131] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor regional multi-agent reinforcement learning control system, regional multi-agent reinforcement learning control system or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used or combined with the regional multi-agent reinforcement learning control system, regional multi-agent reinforcement learning control system, or device by instructions. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0132] The aforementioned computer-readable storage medium may be included in an individual mobility intervention device; or it may exist independently and not incorporated into an individual mobility intervention device.

[0133] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the individual mobility intervention device, cause the individual mobility intervention device to perform the aforementioned individual mobility intervention method.

[0134] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of regional multi-agent reinforcement learning control systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based regional multi-agent reinforcement learning control system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0136] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0137] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described individual mobility intervention method, thereby solving the technical problem of poor real-time performance in individual mobility intervention. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the individual mobility intervention method provided in the above embodiments, and will not be repeated here.

[0138] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the individual mobility intervention method described above.

[0139] The computer program product provided in this application can solve the technical problem of poor real-time performance in individual mobility intervention. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the individual mobility intervention method provided in the above embodiments, and will not be repeated here.

[0140] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for individual mobility intervention, characterized in that, The method includes: Acquire epidemic-related data from multiple spatiotemporal regions obtained by dividing the city under control, with each spatiotemporal region corresponding to an intelligent agent; Based on the epidemic-related data in the multiple spatiotemporal regions, global epidemic-related data are calculated; By using the reinforcement learning algorithms corresponding to each of the aforementioned agents, the epidemic-related data of the corresponding spatiotemporal region and the global epidemic-related data are calculated to obtain the target intervention measures for the corresponding spatiotemporal region; wherein, the framework of the reinforcement learning algorithm of each agent is Actor-Critic, and both the Actor network and the Critic network include a long short-term memory network model. Based on the target intervention measures corresponding to each spatiotemporal region and the preset intervention number limit for different intervention measures, the proportion of individuals that can be intervened in each spatiotemporal region is determined; Based on the locations visited by confirmed cases in the first time period from the epidemiological data of each of the aforementioned spatiotemporal regions, high-risk locations in the first time period are identified. Based on the aforementioned epidemic-related data, the infection risk of the first individual who visited the high-risk location during the second time period was determined. Based on the social attributes of the first individual, determine the social contact risk of the first individual; Based on the infection risk, social contact risk, and the proportion of individuals eligible for intervention, target individuals are identified from the first individuals corresponding to each of the aforementioned spatiotemporal regions, and mobile intervention is performed on the target individuals within the corresponding spatiotemporal regions.

2. The method as described in claim 1, characterized in that, Following the step of intervening in the movement of the target individual within the corresponding spatiotemporal region, the following steps are included: Obtain the number of infected individuals in each spatiotemporal region after the mobile intervention; Based on the number of infected individuals corresponding to each spatiotemporal region, the preset intervention cost, the preset balance weight between health and socioeconomic status, and the preset tendency weight, the reward value is calculated through a preset reward function, wherein the preset tendency weight is used to characterize the tendency to take intervention measures or the tendency to control costs. Based on the reward value, the reinforcement learning algorithm is iteratively trained until the goal of minimizing the number of infected people and the intervention cost is achieved, so as to update the agent.

3. The method as described in claim 2, characterized in that, The preset intervention cost is the sum of the costs of the target intervention measures taken in the corresponding spatiotemporal region; the preset balance weight is determined based on the affordability of the economic system and the affordability of the medical system in the corresponding spatiotemporal region.

4. The method as described in claim 1, characterized in that, The step of determining the target individual from the first individual corresponding to each of the aforementioned spatiotemporal regions based on the infection risk, social contact risk, and the proportion of interventionable individuals includes: The first individuals are sorted from high to low infection risk. When there are individuals with zero infection risk and / or individuals with the same infection risk among the first individuals, the individuals with zero infection risk and / or individuals with the same infection risk are sorted from high to low social contact risk to obtain the second sorted individuals. The higher the ranking, the higher the intervention priority. Based on the proportion of individuals eligible for intervention and the intervention priority, target individuals are determined from the second group of individuals.

5. A personal mobility intervention device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the individual mobility intervention method as described in any one of claims 1 to 4.

6. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the individual mobility intervention method as described in any one of claims 1 to 4.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the individual mobility intervention method as described in any one of claims 1 to 4.

Images