Reinforced learning and dynamic feedback driven chronic disease resource scheduling method

Abstract

The invention discloses a chronic disease resource scheduling method driven by reinforcement learning and dynamic feedback, and relates to the technical field related to medical data management, and the method comprises the steps: collecting an electronic health archive of a chronic disease patient group; multi-class training sample data sets corresponding to the multi-class chronic disease types are obtained; training a disease fluctuation distribution prediction model for outputting a plurality of resource fluctuation probability distributions; connecting a chronic disease resource scheduling system, loading a resource demand index set, and dividing the resource demand index set into multiple types of resource demand indexes; performing dynamic feedback updating, and outputting an updated resource demand index set; and carrying out chronic disease resource scheduling according to the updated resource demand index set. The technical problems of unreasonable medical resource allocation and low scheduling efficiency caused by neglect of dynamic change of patient conditions and supply and demand fluctuation of medical resources in the prior art are solved, the accuracy and flexibility of resource allocation are realized, and the technical effects of improving chronic disease resource scheduling efficiency and patient treatment timeliness are achieved.

Description

Technical Field

[0001] The present application relates to the technical field related to medical data management, and in particular to a chronic disease resource scheduling method driven by reinforcement learning and dynamic feedback. Background Art

[0002] The management of patients with chronic diseases requires attention to the long-term changes in the patient's health status and continuous treatment, while optimizing the allocation and scheduling of medical resources. However, their conditions are volatile and vary from person to person, which involves not only the allocation of medical resources, but also changes in the patient's condition, adjustments to the treatment plan, and the allocation of medical personnel. The treatment process involves a large amount of data, such as electronic health records, disease diagnosis and treatment records, and the patient's physiological data. The needs of patients with different types of chronic diseases vary greatly, and the condition of chronic disease patients may fluctuate, resulting in changes in their demand for medical resources, which increases the complexity of resource scheduling. Traditional static resource scheduling methods mostly rely on static resource scheduling plans, which cannot effectively respond to the dynamic changes in the patient's condition and fluctuations in the supply and demand of medical resources, resulting in irrational allocation of medical resources, such as waste of resources or untimely scheduling, which affects the patient's treatment effect and the overall efficiency of the medical system, making it difficult to ensure that the patient's treatment needs are met in a timely manner, thereby reducing the quality of medical services.

[0003] At the current stage, relevant technologies have technical problems that ignore the dynamic changes in patients' conditions and fluctuations in the supply and demand of medical resources, which in turn leads to unreasonable allocation of medical resources and low scheduling efficiency. Summary of the invention

[0004] This application provides a chronic disease resource scheduling method driven by reinforcement learning and dynamic feedback, which solves the technical problems in the prior art that the dynamic changes of patients' conditions and fluctuations in the supply and demand of medical resources are ignored, leading to unreasonable allocation of medical resources and low scheduling efficiency. It realizes the accuracy and flexibility of resource allocation and achieves the technical effect of improving the efficiency of chronic disease resource scheduling and the timeliness of patient treatment.

[0005] The present application provides a reinforcement learning and dynamic feedback driven chronic disease resource scheduling method, including: collecting an electronic health record library of a chronic disease patient group; performing reinforcement learning on the data of the electronic health record library according to the chronic disease type to obtain a plurality of training sample data groups corresponding to the plurality of chronic disease types; using the plurality of training sample data groups corresponding to the plurality of chronic disease types to train a disease fluctuation distribution prediction model, which is used to output a plurality of resource fluctuation probability distributions corresponding to the plurality of chronic disease types; connecting to a chronic disease resource scheduling system, loading a resource demand indicator set corresponding to the chronic disease patient group, and dividing the resource demand indicator set into a plurality of resource demand indicator classes according to the plurality of chronic disease types; driving the plurality of resource fluctuation probability distributions to perform dynamic feedback update on the plurality of resource demand indicators, and outputting an updated resource demand indicator set; the chronic disease resource scheduling system performs chronic disease resource scheduling according to the updated resource demand indicator set.

[0006] In a possible implementation, the reinforcement learning and dynamic feedback driven chronic disease resource scheduling method also performs the following processing: defining reinforcement learning parameters; learning each of the multiple training sample data groups with the reinforcement learning parameters according to the reinforcement learning algorithm to obtain multiple reinforcement learners that meet preset effects, wherein each reinforcement learner corresponds to a chronic disease type; outputting patient health status fluctuation data and corresponding resource consumption fluctuation data for each chronic disease type that meet the preset effects according to the multiple reinforcement learners; outputting patient health status fluctuation data and corresponding resource consumption fluctuation data for multiple chronic disease types as multiple training sample data groups.

[0007] In a possible implementation, the reinforcement learning and dynamic feedback driven chronic disease resource scheduling method also performs the following processing: the reinforcement learning parameters include state space S, action space A and reward function Q; the state space S defines the health status data and resource consumption status data of each patient, and the action space A defines different intervention action strategies for resource consumption; the reward function Q is used to evaluate the effect of taking different intervention action strategies under each health status data and resource consumption status data, wherein the evaluation effect includes health status improvement indicators and resource waste indicators.

[0008] In a possible implementation, the reinforcement learning and dynamic feedback driven chronic disease resource scheduling method further performs the following processing: learning each of the multiple training sample data groups according to Q-learning with the reinforcement learning parameters, and the expression of Q-learning is as follows: ; in, Indicates in status Take action The expected reward, represents the learning rate, The larger it is, the greater the impact on the update of the Q value. Indicates that an action is performed at time step t After receiving the instant reward, represents the discount factor, which is used to weigh the importance of future rewards and current rewards. , The value range of is [0,1], Indicates in status Take action The expected reward, Indicates that in the next state The maximum expected reward after selecting all possible actions a is used to calculate When the expected reward error approaches 0, learning converges and multiple reinforcement learners are output.

[0009] In a possible implementation, the reinforcement learning and dynamic feedback driven chronic disease resource scheduling method also performs the following processing: Gaussian process regression is used to perform Gaussian modeling with the patient health status fluctuation data in each type of training sample data group as input and the corresponding resource consumption fluctuation data as predicted output to obtain multiple Gaussian process regression models; and the multiple resource fluctuation probability distributions corresponding to the multiple types of chronic diseases are obtained using the multiple Gaussian process regression models.

[0010] In a possible implementation, the reinforcement learning and dynamic feedback driven chronic disease resource scheduling method further performs the following processing: the resource fluctuation probability distribution in the multiple resource fluctuation probability distributions is a Gaussian distribution, and each resource fluctuation probability distribution expression is as follows: ; in, is the probability distribution of resource fluctuation corresponding to the kth chronic disease type, is the resource consumption fluctuation of the kth chronic disease type The mean of is the variance of the kth chronic disease type, indicating the fluctuation range of resource consumption fluctuations.

[0011] In a possible implementation, the reinforcement learning and dynamic feedback driven chronic disease resource scheduling method also performs the following processing: driving the multiple resource fluctuation probability distributions to dynamically feedback update the multiple categories of resource demand indicators, and outputting the multiple categories of resource demand indicators corresponding to the multiple categories of chronic disease types; fitting the multiple categories of resource demand indicators, and outputting a set of resource demand indicators based on the merger of resource items.

[0012] In a possible implementation, the reinforcement learning and dynamic feedback driven chronic disease resource scheduling method also performs the following processing: according to the chronic disease resource scheduling system, historical burst resource scheduling information is obtained; the historical burst resource scheduling information is divided according to the multiple types of chronic diseases, and multiple groups of burst resource scheduling information are output; burst probability identification is performed on the multiple groups of burst resource scheduling information, and multiple resource mutation probabilities are output; and the resource demand indicator set is updated for the second time according to the multiple resource mutation probabilities.

[0013] Through the reinforcement learning and dynamic feedback-driven chronic disease resource scheduling method proposed in this application, the electronic health archives of chronic disease patients are collected; multiple training sample data groups corresponding to multiple types of chronic diseases are obtained; a disease fluctuation distribution prediction model is trained to output multiple resource fluctuation probability distributions; the chronic disease resource scheduling system is connected, a resource demand indicator set is loaded, and divided into multiple resource demand indicators; dynamic feedback updates are performed to output an updated resource demand indicator set; and chronic disease resources are scheduled according to the updated resource demand indicator set. This solves the technical problem that the dynamic changes in the patient's condition and the fluctuations in the supply and demand of medical resources in the prior art are ignored, which leads to unreasonable allocation of medical resources and low scheduling efficiency, realizes the accuracy and flexibility of resource allocation, and achieves the technical effect of improving the efficiency of chronic disease resource scheduling and the timeliness of patient treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0015] Figure 1 A flowchart of a method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback provided in an embodiment of the present application; Figure 2 A schematic diagram of a process for obtaining multiple types of training sample data groups in a reinforcement learning and dynamic feedback driven chronic disease resource scheduling method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0017] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0018] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first / second" involved are merely to distinguish similar objects and do not represent a specific ordering of the objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, product, or server comprising a series of steps is not necessarily limited to those steps clearly listed, but may include other steps that are not clearly listed or inherent to these processes, methods, products, or devices, and unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0019] The present application embodiment provides a method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback, such as Figure 1 As shown, the method includes: Step S100, collecting the electronic health records of the chronic disease patient group.

[0020] Preferably, relevant health information and medical record data of patients with chronic diseases are collected and managed electronically to establish a database of health data of patients with chronic diseases, namely, an electronic health archive. The electronic health archive usually includes basic patient information, health records, disease information, physiological data, treatment data, etc. Specifically, basic patient information includes name, gender, age, contact information, address, etc. Health records include medical history information (past medical history, family medical history, allergy history, etc. of chronic disease patients) and diagnosis and treatment records (such as doctor's diagnosis results, treatment plans, medication records, surgical records, examination reports, etc.). Disease information includes chronic disease types (recording the specific types of chronic diseases suffered by patients, such as diabetes, hypertension, chronic heart disease, chronic respiratory diseases, etc., as well as the progression of these diseases). The electronic health data includes information on the course of diabetes, such as disease progression, complications, treatment plans, etc.) and disease stages (diabetic patients may be divided into different stages of the disease, such as early, middle and late stages, and patients with different conditions have different resource needs). Physiological data includes vital sign monitoring data (such as routine physiological parameters such as blood pressure, blood sugar, body temperature, heart rate, etc.) and laboratory test data (such as blood tests, urine tests, imaging tests, etc.). Treatment data records drug types, dosages, medication times, efficacy of drugs and treatment plans, patient responses, etc. Through standardized data sets, a comprehensive electronic health data file for chronic disease patients is formed, which not only helps medical service providers (such as doctors, nurses, etc.) understand the patient's health status and changes in their condition, but also provides personalized treatment and management plans for chronic disease patients through data analysis.

[0021] Step S200, performing reinforcement learning on the data in the electronic health archive according to the chronic disease types to obtain multiple types of training sample data groups corresponding to the multiple types of chronic diseases.

[0022] Preferably, a reinforcement learning algorithm is used to process and learn the data collected from the electronic health records of chronic disease patients, so as to extract corresponding training sample data groups according to different chronic disease types. Specifically, patients are classified according to the medical history information, diagnosis records, treatment plans, etc. in the electronic health records, that is, the chronic disease patient group is classified according to different disease types. For example, patients can be divided into diabetic patients, hypertensive patients, chronic respiratory disease patients, etc. These chronic disease types may have different pathological characteristics, treatment needs and resource requirements. Based on each chronic disease type, relevant data is extracted from the electronic health record library. For example, for diabetic patients, the extracted relevant data may include the patient's blood sugar level, weight, medication status, eating habits, etc.; for hypertensive patients, it may include blood pressure records, medication status, exercise numbers, etc. data, etc.; then these health data are used as training samples for reinforcement learning, that is, by learning the relationship between the health data and resource demand data of chronic disease patients, the scheduling strategy is continuously optimized to learn how to perform optimal resource scheduling according to the patient's disease type and resource demand under different health conditions. Finally, for each type of chronic disease (such as diabetes, hypertension, etc.), multiple categories of training sample data groups are constructed according to different patient health data, including patient data from different chronic disease types (for example, blood sugar levels, medication information, etc.), resource requirements (such as drugs, medical equipment, doctors' working hours, etc.), and state transitions that may occur during the treatment process (for example, improvement, deterioration, etc.), and each training sample data group will contain specific input data (such as patient health data) and output data (such as medical resource demand and allocation results).

[0023] Further, such as Figure 2 As shown, step S200 also includes step S210, defining reinforcement learning parameters; step S220, learning each type of training sample data group in the multiple types of training sample data groups with the reinforcement learning parameters according to the reinforcement learning algorithm, to obtain multiple reinforcement learners that meet the preset effect, wherein each reinforcement learner corresponds to a chronic disease type; step S230, outputting patient health status fluctuation data and corresponding resource consumption fluctuation data of each chronic disease type under the preset effect according to the multiple reinforcement learners; step S240, outputting the patient health status fluctuation data and corresponding resource consumption fluctuation data of multiple chronic disease types as multiple types of training sample data groups.

[0024] Step S210 further includes step S211, wherein the reinforcement learning parameters include state space S, action space A and reward function Q; step S212, wherein the state space S defines the health status data and resource consumption status data of each patient, and the action space A defines different intervention action strategies for resource consumption; step S213, wherein the reward function Q is used to evaluate the effect of taking different intervention action strategies under each health status data and resource consumption status data, wherein the evaluation effect includes health status improvement indicators and resource waste indicators.

[0025] Preferably, reinforcement learning parameters are defined, including state space S, action space A, and reward function Q. Specifically, state space S defines the health status data and resource consumption status data of each patient. For patients with chronic diseases, health status data may include physiological indicators such as blood sugar, blood pressure, weight, and symptom changes, while resource consumption status data includes the patient's medical resource usage, such as drug consumption, number of examinations and treatments, and number of hospital days. Action space A defines different intervention strategies for resource consumption. For example, an intervention may be to increase the patient's medication dosage, or provide more treatments, examinations, or lifestyle interventions. Each action represents an operation in resource scheduling, and these operations are used to Optimize the patient's health status; the reward function Q is used to evaluate the effect of a certain intervention action strategy under a specific health status and resource consumption status. The evaluation effect usually includes indicators of health status improvement (such as improvement of blood sugar stability, blood pressure control and other indicators) and resource waste indicators (i.e., whether there is resource waste in the scheduling process, such as over-medication, too many examinations, etc.). Through the reward function, the reinforcement learning algorithm can calculate the score according to the status of each patient and the intervention strategy adopted, and then select a more effective resource allocation strategy. For example, if the blood sugar level of diabetic patients improves and the drug dosage is moderate, the reward will be higher; if resources are overused (such as drug overdose), negative rewards may result.

[0026] Preferably, according to the defined reinforcement learning parameters (state space S, action space A, reward function Q), a reinforcement learning algorithm is used to learn each type of training sample data group, and each type of training sample data group corresponds to a type of chronic disease (such as diabetes, hypertension, etc.). Through training, the reinforcement learning algorithm gradually optimizes the resource scheduling strategy for each type of chronic disease, aiming to achieve preset effects (such as better health management effects and higher resource utilization efficiency). Among them, the reinforcement learning algorithm is used to learn what kind of resource intervention can meet the patient's treatment effect under the distribution of health status, thereby establishing multiple reinforcement learners, each reinforcement learner represents a learning model for a type of chronic disease, and optimizes the resource scheduling strategy for each type of chronic disease by training on specific patient data.

[0027] Preferably, multiple reinforcement learners are used to determine the most appropriate resource consumption strategy according to the different health states of patients (such as different values ​​of blood sugar, blood pressure, etc.), that is, for each type of chronic disease, the patient's health state fluctuation data and the corresponding resource consumption fluctuation data that meet the preset effect are output, wherein the preset effect refers to the desired goal, such as improved health status, optimal resource consumption, or the best balance between the improvement of health status and the efficiency of resource consumption, and then the health status changes of each chronic disease type patient after the implementation of different resource scheduling schemes are output, such as changes in blood sugar levels of diabetic patients, blood pressure fluctuations of hypertensive patients, etc.; at the same time, the resource consumption fluctuation data corresponding to each chronic disease type and health status fluctuation are output, indicating the medical resources consumed by patients with this disease under different health states, such as the amount of medicine, number of hospitalization days, number of medical examinations, etc.; these fluctuation data are used as model outputs to reflect the dynamic relationship between the patient's health status and resource consumption, and are organized into multiple types of training sample data groups, each type of data group represents the health changes and resource consumption of patients with a chronic disease type.

[0028] Furthermore, step S220 further includes learning each of the multiple training sample data groups according to Q-learning with the reinforcement learning parameters, and the expression of Q-learning is as follows: ; in, Indicates in status Take action The expected reward, represents the learning rate, The larger it is, the greater the impact on the update of the Q value. Indicates that an action is performed at time step t After receiving the instant reward, represents the discount factor, which is used to weigh the importance of future rewards and current rewards. , The value range of is [0,1], Indicates in status Take action The expected reward, Indicates that in the next state The maximum expected reward after selecting all possible actions a is used to calculate When the expected reward error approaches 0, learning converges and multiple reinforcement learners are output.

[0029] Preferably, Q-learning is an algorithm in reinforcement learning, the goal is to maximize long-term rewards by learning action strategies. The expression of Q-learning describes how to update the expected reward (Q value) of taking an action in a certain state. Specifically, at each time step t, the algorithm updates the current state and current action Calculate Q value and receive instant reward After that, the algorithm will update the current Q value according to the maximum Q value of the next state. The update process is done by the learning rate and discount factor To balance the importance of current rewards and future rewards, through continuous learning, Q-learning gradually optimizes the Q value, and finally reaches the best action strategy to maximize the long-term reward; then the training sample data of each chronic disease type is regarded as a state, and the different resource scheduling (actions) of each chronic disease patient are learned. The goal is to optimize the resource allocation strategy so that in the long run, the health status is maximized while the resource consumption is kept optimal.

[0030] Step S300, using the multiple types of training sample data groups corresponding to the multiple types of chronic diseases, training a disease fluctuation distribution prediction model to output multiple resource fluctuation probability distributions corresponding to the multiple types of chronic diseases.

[0031] Preferably, by using multiple training sample data groups (patient data) of different chronic disease types, a disease fluctuation distribution prediction model is established to predict the resource demand fluctuations of different diseases in the future, that is, historical health data is used to infer the fluctuations in the demand for medical resources for each chronic disease type under different states, forming multiple resource fluctuation probability distributions, wherein each type of training sample data group corresponds to a type of chronic disease. Specifically, the patient data in the training sample data group is preprocessed (cleaning, removing missing values, outliers, and normalizing), and then health status characteristics, patient treatment history, and corresponding time series data are constructed. For example, resource consumption (such as drugs, hospitalization, and examinations) is used as the target variable for modeling; then a prediction model is constructed based on the regression model to convert the multiple training samples of chronic disease types into The sample data group is input into the prediction model and trained through supervised learning to learn how to predict resource consumption (as output) from the input features (health status), that is, to predict future resource consumption fluctuations based on the health status of chronic disease patients; the disease fluctuation distribution prediction model obtained through training is used to predict the changing trend of future resource demand for patients with different chronic diseases (different health statuses), that is, to output the resource fluctuation probability distribution, which may include the estimated value of resource consumption for each chronic disease type (such as drug dosage, number of hospitalization days, etc.), and the category prediction of resource demand (such as low, medium, and high demand). Then, a dynamic resource scheduling plan can be formulated to adjust the resource allocation according to the predicted demand fluctuations, so as to more accurately predict and plan resource consumption and improve resource utilization efficiency.

[0032] Furthermore, step S300 also includes step S310, performing Gaussian modeling through Gaussian process regression with the patient health status fluctuation data in each type of training sample data group as input and the corresponding resource consumption fluctuation data as predicted output to obtain multiple Gaussian process regression models; step S320, using the multiple Gaussian process regression models to obtain multiple resource fluctuation probability distributions corresponding to the multiple types of chronic diseases.

[0033] Preferably, Gaussian process regression is a regression model based on Bayesian theory, which predicts new data points by learning the covariance structure of training data, and provides uncertainty estimates for the prediction results through Bayesian reasoning. Specifically, Gaussian process regression (GPR) takes the health status fluctuation data of patients in each type of training sample data group (such as blood sugar fluctuations, weight changes of diabetic patients, or blood pressure fluctuations of hypertensive patients, etc.) as input, and the corresponding resource consumption fluctuation data (such as drug consumption, hospitalization days, number of medical examinations, etc. of patients in different health states) as the prediction output for modeling. Gaussian process regression uses the training data set of each chronic disease type (health status fluctuation data and corresponding resource consumption fluctuation data), and establishes a function model through a covariance function (or kernel function) to describe the relationship between health status and resource consumption. The Gaussian process regression model can provide the predicted value of resource consumption and the confidence interval of each prediction (i.e., the uncertainty of the prediction) when the health status of each patient changes. For various types of chronic diseases (such as diabetes, hypertension, chronic heart disease, etc.), multiple Gaussian process regression models are trained; then multiple Gaussian process regression models are used to analyze and predict various types of chronic diseases, and the predicted values ​​of resource consumption (such as drug dosage, number of hospitalization days) and the confidence intervals of these predicted values ​​are output, that is, multiple resource fluctuation probability distributions are obtained. Specifically, the probability distribution of resource demand (such as drugs, hospitalization, examination, etc.) corresponding to the health status fluctuations of each chronic disease type is included, as well as the predicted value of resource consumption corresponding to the health status fluctuations of the model output and the uncertainty of the prediction (such as confidence interval), which helps medical institutions prepare in advance and dynamically adjust medical resources, thereby improving resource utilization efficiency.

[0034] Furthermore, step S300 also includes that the resource fluctuation probability distribution in the multiple resource fluctuation probability distributions is a Gaussian distribution, and each resource fluctuation probability distribution expression is as follows: ; in, is the probability distribution of resource fluctuation corresponding to the kth chronic disease type, is the resource consumption fluctuation of the kth chronic disease type The mean of is the variance of the kth chronic disease type, indicating the fluctuation range of resource consumption fluctuations.

[0035] Preferably, the probability distribution of resource fluctuation is Gaussian distribution (normal distribution), that is, resource scheduling adaptability under long-term periods. It is the average value of resource consumption predicted based on health status, reflecting the average resource consumption of this type of patients under a certain health status (such as blood sugar level, blood pressure, etc.), for example, the average dosage of drugs, the number of days in hospital, etc. It is the range or uncertainty of resource consumption fluctuations. The larger the variance, the greater the volatility of resource consumption, that is, the more significant the fluctuation of predicted resource demand under different circumstances. The smaller the variance, the more stable the predicted resource consumption and the lower the uncertainty. Gaussian distribution can not only predict the average value of resource consumption (such as drug usage), but also quantify the uncertainty of the prediction results (such as the volatility of drug demand). According to the probability distribution of resource fluctuations for each chronic disease type, more accurate resource scheduling and planning can be carried out to prepare the required medical resources in advance.

[0036] Step S400, connecting to the chronic disease resource scheduling system, loading a resource demand indicator set corresponding to the chronic disease patient group, and dividing the resource demand indicator set into multiple categories of resource demand indicators according to the multiple types of chronic diseases.

[0037] Preferably, the chronic disease resource scheduling system is connected, and the patient's health data, historical treatment information and resource demand data are imported, wherein the chronic disease resource scheduling system is an intelligent system for managing and optimizing the allocation of medical resources for patients with chronic diseases. It dynamically schedules and optimizes resources based on the patient's health status, course of disease, and demand and supply of medical resources, thereby ensuring that patients receive timely and effective medical resource support and improving the utilization efficiency of medical resources; the resource demand indicator set refers to the set of all resource demand data related to the chronic disease patient group, usually including different resource types, such as the demand for drugs, treatments, examinations, hospital beds, etc. Specifically, according to multiple types of chronic diseases, The resource demand indicator set is divided into multiple categories of resource demand indicators, that is, the resource demand data is classified and processed according to the type of chronic disease of the patient (such as diabetes, hypertension, chronic respiratory diseases, etc.). For example, for diabetic patients, resource demand indicators may include drug demand, blood sugar monitoring frequency, complication examination, etc. For hypertensive patients, resource demand indicators may include drug allocation, blood pressure monitoring, hospital beds, etc.; classifying resource demand indicators according to chronic disease types can more accurately dispatch resources for different types of chronic disease patients, avoid waste and over-allocation of resources, and ensure that patients of each type of chronic disease can get the required medical resources in a timely manner, so as to improve the efficiency of medical resource scheduling.

[0038] Step S500: driving the plurality of resource fluctuation probability distributions to dynamically feedback and update the plurality of resource demand indicators, and outputting an updated resource demand indicator set.

[0039] Preferably, in the chronic disease resource scheduling system, the predicted resource fluctuation probability distribution is used to adjust different types of resource demand indicators in real time, and the overall resource demand plan is updated according to the new information. Specifically, the resource fluctuation probability distribution describes the range and uncertainty of the resource demand (such as drugs, treatment, hospitalization, etc.) of chronic disease patients in a healthy state. Then, according to the predicted resource fluctuation probability distribution (such as the mean and variance of drug demand), the resource demand indicator is updated. If the variance of resource demand is large, indicating that there is a large uncertainty, the reserved amount of resources is increased to cope with possible demand fluctuations, that is, based on the new health data and fluctuation prediction, the resource demand indicator correction result is obtained (that is, the updated resource demand indicator set is output), which may include recalculating the patient's resource demand (such as drugs, medical examinations, hospitalization, etc.); adjusting the original resource allocation plan to ensure that the allocation of resources is more reasonable and meets the needs; outputting an updated resource demand plan, such as the resource demand, allocation plan and possible fluctuation range of each type of chronic disease patient. The dynamic feedback update mechanism can improve the utilization efficiency of medical resources and ensure that patients receive timely and appropriate treatment in different health states.

[0040] Furthermore, step S500 also includes step S510, driving the multiple resource fluctuation probability distributions to dynamically feedback update the multiple types of resource demand indicators, and outputting the multiple types of resource demand indicators corresponding to the multiple types of chronic diseases; step S520, fitting the multiple types of resource demand indicators, and outputting a set of resource demand indicators based on the merger of resource items.

[0041] Preferably, the resource demand index is adjusted and fed back based on the actual data of real-time monitored changes in patient health status and resource consumption, combined with the probability distribution of resource fluctuations. For example, the blood sugar of diabetic patients fluctuates greatly, and the demand for medicines will increase accordingly. Adjustments are made based on this fluctuation to update the forecast of drug demand, which may include predicting changes in resource demand (drugs, beds, equipment, etc.) and updating the quantity and allocation plan of resource demand to ensure that resources can effectively meet the needs of patients. Different types of chronic diseases (such as diabetes, hypertension, heart disease, etc.) will have different resource demands, and then multiple types of resource demand indicators corresponding to multiple types of chronic diseases are output, such as drug demand (such as insulin and hypoglycemic drugs for diabetic patients), examination demand (such as blood sugar monitoring Measurement, blood pressure check, etc.), hospitalization needs (such as hospital beds, medical services, etc.), to ensure that resources can be appropriately allocated according to the needs of different patient groups; then fit multiple types of resource demand indicators, adjust and optimize the allocation of resources, including merging resource items, classifying and integrating similar resource demand items. For example, patients with diabetes and hypertension may need regular blood pressure monitoring. By merging similar resource demand items, the efficiency of resource utilization can be optimized, and finally a set of resource demand indicators is output, including the integrated resource requirements of all chronic disease types of patients, and the merging process is used to improve the efficiency of resource scheduling, so that the chronic disease resource scheduling system can efficiently plan and allocate medical resources, reduce waste, and improve resource utilization efficiency.

[0042] Furthermore, step S500 also includes step S530, obtaining historical burst resource scheduling information according to the chronic disease resource scheduling system; step S540, dividing the historical burst resource scheduling information according to the multiple types of chronic diseases, and outputting multiple groups of burst resource scheduling information; step S550, performing burst probability identification on the multiple groups of burst resource scheduling information, and outputting multiple resource mutation probabilities; step S560, performing secondary updating on the resource demand indicator set according to the multiple resource mutation probabilities.

[0043] Preferably, historical emergency resource scheduling information is obtained from the chronic disease resource scheduling system, that is, unexpected or sudden resource scheduling events that occur in chronic disease management, which may include sudden deterioration of the patient's health status, resulting in a surge in demand for medical resources (such as worsening of the condition requiring more drugs, equipment, hospitalization, etc.); public health emergencies leading to increased demand for medical resources; seasonal fluctuations (for example, certain diseases have more attacks in winter, resulting in a surge in resource demand); then the historical emergency resource scheduling information is classified according to different chronic disease types (such as diabetes, hypertension, chronic respiratory diseases, etc.), and by dividing emergencies according to chronic disease types, resource adjustments and planning are carried out according to the specific needs of each disease type. For example, diabetic patients may need additional drugs and hospitalization resources in certain special circumstances (such as hyperglycemia crisis), and hypertensive patients may need more examinations and drug preparations when their health status fluctuates greatly, and multiple sets of emergency resource scheduling information are output, wherein each chronic disease type has a unique set of emergency resource demand information.

[0044] Preferably, based on the analysis results of historical data, burst probability identification is performed on multiple groups of burst resource scheduling information, that is, the probability of a surge in resource demand under specific conditions for each type of chronic disease is identified, for example, the probability of diabetic patients developing acute complications and requiring additional resources under specific health conditions (such as excessive blood sugar fluctuations); the probability of hypertensive patients suffering a heart attack and requiring hospitalization under drastic fluctuations; burst probability can help predict which events may lead to sudden changes in resource demand. Each type of chronic disease corresponds to a resource mutation probability, which indicates the possibility of a sharp increase in resource demand due to changes in the patient's health status under specific conditions. The resource mutation probability reflects the The ability to warn of potential emergencies in the future helps hospitals better plan resources. Finally, the resource demand indicator set is updated for a second round using the identified multiple resource mutation probabilities. For example, if the probability of resource mutation in hypertensive patients is higher (such as requiring more examinations and hospitalizations in specific seasons or due to health fluctuations), this information will be fed back to resource demand, thereby increasing the resource preparation for inpatient beds, inspection equipment, etc., and thus preparing resources in advance, such as increasing the inventory of certain drugs, arranging more inspection equipment in advance, ensuring that there are enough inpatient beds, etc., so that resource scheduling can be carried out more accurately to ensure that medical resources can be effectively utilized and meet the needs of patients.

[0045] Step S600: The chronic disease resource scheduling system schedules chronic disease resources according to the updated resource demand indicator set.

[0046] Preferably, the chronic disease resource scheduling system allocates and schedules medical resources according to the latest updated resource demand indicator set to ensure that patients of each chronic disease type can obtain appropriate and timely medical resources, ensure the optimal allocation of resources, and thus improve the efficiency and effectiveness of medical services. The updated resource demand indicator set includes not only the basic medical resource needs of chronic disease patients (such as drugs, treatment equipment, examinations, etc.), but also takes into account potential resource fluctuations, emergencies and uncertainties, which may include the expected demand for resources such as drugs, treatments, examinations, etc., the specific resource allocation needs corresponding to various chronic disease types (such as diabetes, hypertension, etc.), and specific conditions (such as seasonal fluctuations, The chronic disease resource scheduling system can adjust the resource demand fluctuations that may occur due to sudden changes in the health status of patients, changes in health status, etc. Specifically, the chronic disease resource scheduling system performs differentiated scheduling based on the updated resource demand indicators of each chronic disease type. For example, patients with diabetes may need more medications and blood sugar tests, while patients with hypertension may need more blood pressure monitoring and drug dispensing. The system can dynamically adjust the allocation of resources based on factors such as changes in health status and the probability of sudden events. For example, more medical resources may be prepared in advance based on the probability of sudden events in patients with a certain type of chronic disease to ensure timely response in the event of an emergency. The system also monitors resource usage in real time and provides feedback. If the consumption of certain resources exceeds expectations or is not allocated as needed, it will automatically adjust the resources. Through precise resource allocation, it ensures that patients of each chronic disease type can obtain medical resources on demand, maximize resource utilization efficiency, reduce resource waste, and improve treatment effects and medical service quality.

[0047] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback, characterized in that: The method comprises: Collect electronic health records of chronic disease patient groups; Performing reinforcement learning on the data in the electronic health archive according to the chronic disease types to obtain multiple types of training sample data groups corresponding to multiple types of chronic diseases; Using the multiple types of training sample data groups corresponding to the multiple types of chronic diseases, training a disease type fluctuation distribution prediction model to output multiple resource fluctuation probability distributions corresponding to the multiple types of chronic diseases; Connecting to a chronic disease resource scheduling system, loading a resource demand indicator set corresponding to the chronic disease patient group, and dividing the resource demand indicator set into multiple categories of resource demand indicators according to the multiple types of chronic diseases; Driving the multiple resource fluctuation probability distributions to dynamically feedback and update the multiple types of resource demand indicators, and outputting an updated resource demand indicator set; The chronic disease resource scheduling system performs chronic disease resource scheduling according to the updated resource demand indicator set.

2. The method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback as claimed in claim 1, characterized in that: Reinforcement learning is performed on the data of the electronic health archive according to the chronic disease types to obtain multiple types of training sample data groups corresponding to the multiple types of chronic diseases, the method comprising: Define reinforcement learning parameters; According to the reinforcement learning algorithm, each of the multiple types of training sample data groups is learned with the reinforcement learning parameters to obtain multiple reinforcement learners that meet the preset effect, wherein each reinforcement learner corresponds to a chronic disease type; According to the multiple reinforcement learners, output the health status fluctuation data of patients of each chronic disease type and the corresponding resource consumption fluctuation data that meet the preset effect; The health status fluctuation data of patients with multiple chronic diseases and the corresponding resource consumption fluctuation data are output as a multi-class training sample data group.

3. A method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback as claimed in claim 2, characterized in that: The reinforcement learning parameters include state space S, action space A and reward function Q; The state space S defines the health status data and resource consumption status data of each patient, and the action space A defines different intervention action strategies for resource consumption; The reward function Q is used to evaluate the effects of taking different intervention action strategies under each health status data and resource consumption status data, wherein the evaluation effect includes the improvement index of the health status and the resource waste index.

4. The method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback as claimed in claim 3, characterized in that: According to Q-learning, each of the multiple training sample data sets is learned with the reinforcement learning parameters. The expression of Q-learning is as follows: ； in, Indicates in status Take action The expected reward, represents the learning rate, The larger it is, the greater the impact on the update of the Q value. Indicates that an action is performed at time step t After receiving the instant reward, represents the discount factor, which is used to weigh the importance of future rewards and current rewards. , The value range of is [0,1], Indicates in status Take action The expected reward, Indicates that in the next state The maximum expected reward after selecting all possible actions a is used to calculate When the expected reward error approaches 0, learning converges and multiple reinforcement learners are output.

5. The method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback as claimed in claim 1, characterized in that: Training a disease type fluctuation distribution prediction model to output multiple resource fluctuation probability distributions corresponding to the multiple types of chronic diseases, the method comprising: Gaussian process regression is used to perform Gaussian modeling using the patient health status fluctuation data in each type of training sample data group as input and the corresponding resource consumption fluctuation data as prediction output, thereby obtaining multiple Gaussian process regression models; The multiple Gaussian process regression models are used to obtain multiple resource fluctuation probability distributions corresponding to the multiple types of chronic diseases.

6. The method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback as claimed in claim 1, characterized in that: The resource fluctuation probability distribution in the multiple resource fluctuation probability distributions is a Gaussian distribution, and each resource fluctuation probability distribution expression is as follows: ； in, is the probability distribution of resource fluctuation corresponding to the kth chronic disease type, is the resource consumption fluctuation of the kth chronic disease type The mean of is the variance of the kth chronic disease type, indicating the fluctuation range of resource consumption fluctuations.

7. The method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback as claimed in claim 1, characterized in that: Driving the multiple resource fluctuation probability distributions to dynamically feedback and update the multiple resource demand indicators, and outputting an updated resource demand indicator set, the method includes: Driving the multiple resource fluctuation probability distributions to dynamically feedback and update the multiple resource demand indicators, and outputting the multiple resource demand indicators corresponding to the multiple chronic disease types; The multiple types of resource demand indicators are fitted, and a resource demand indicator set based on the merger of resource items is output.

8. The method for scheduling chronic disease resources driven by reinforcement learning and dynamic feedback as claimed in claim 7, characterized in that: After outputting the updated resource demand indicator set, the method further includes: According to the chronic disease resource scheduling system, historical burst resource scheduling information is obtained; Dividing the historical burst resource scheduling information according to the multiple types of chronic diseases, and outputting multiple groups of burst resource scheduling information; Performing burst probability identification on the multiple groups of burst resource scheduling information, and outputting multiple resource mutation probabilities; The resource demand indicator set is updated secondarily according to the multiple resource mutation probabilities.

Images