Vitreous cavity medicine injection center scheduling method and system based on hybrid reinforcement learning and storage medium

Abstract

The invention discloses a vitreous cavity medicine injection center scheduling method and system based on hybrid reinforcement learning and a storage medium. The method mainly solves the problems of slow convergence, low exploration efficiency and the like when a traditional scheduling method is used for processing a high-dimensional state space, sparse reward signals and multi-target conflicts. The method has the beneficial effects that the scheduling efficiency is remarkably improved: in a simulation environment, the average waiting time of patients can be shortened by more than 20%, and the treatment room turnover rate is improved by more than 15%; the system is high in adaptability: through a hybrid reinforcement learning architecture, patient flow change and resource state fluctuation can be dynamically responded, and long-term steady-state optimization is realized; multi-objective collaborative optimization: efficiency, fairness and medical safety are effectively balanced through multi-objective reward function design and constraint embedding; and the system provides a decision explanation mechanism, so that the sense of trust and the use intention of medical staff to the intelligent scheduling scheme are enhanced.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of smart healthcare and operations research optimization, specifically to a method, system, and storage medium for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning. Background Technology

[0002] Intravitreal drug delivery is a mainstream day surgery procedure for treating retinal diseases such as wet age-related macular degeneration and diabetic macular edema. With the continuous increase in the number of patients, drug delivery units in large ophthalmology centers are increasingly characterized by "high frequency, short cycles, and high concurrency." In such medical scenarios, efficient, equitable, and safe resource allocation is crucial for shortening patient waiting times, improving the utilization efficiency of medical resources, and ensuring treatment quality and patient satisfaction.

[0003] Currently, the scheduling and management of drug dispensing centers mainly rely on two types of methods. The first type is static rule-based scheduling based on human experience, such as "first-come, first-served" (FCFS) or fixed doctor-clinic scheduling. Although these methods are simple and easy to implement, they heavily depend on the scheduler's subjective experience and are difficult to cope with dynamic uncertainties such as random patient arrival times, fluctuating treatment durations, doctors taking temporary leave, or emergency room interventions. This can easily lead to uneven resource utilization (some resources are idle while others are overcrowded) and increased average patient waiting times.

[0004] The second type of method applies traditional operations research optimization models, such as integer programming, mixed integer linear programming, or metaheuristic algorithms (such as genetic algorithms). However, most of these are single algorithms (such as Q-learning and DQN), which have problems such as slow convergence and low exploration efficiency when dealing with high-dimensional state spaces, sparse reward signals, and multi-objective conflicts. Furthermore, they lack specific optimizations for high-frequency, short-term operations such as ophthalmic drug injection. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides a method, system and storage medium for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning. It mainly solves the problems of slow convergence and low exploration efficiency of traditional scheduling methods when dealing with high-dimensional state spaces, sparse reward signals and multi-objective conflicts.

[0006] The technical solution of the present invention is as follows: A method for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning includes the following steps: Step 1: Data Preprocessing: Collect patient information, resource information, drug information, and treatment event time sequence information from the drug injection center, process the data, and extract key scheduling features; Step 2: Problem Modeling: Model the dynamic resource scheduling problem of the drug injection center as a partially observable Markov decision process, and define the state space, action space, and multi-objective reward function; The state space S is defined as a tuple (R, Q, D, T), where R represents the treatment room state list, Q represents the patient queue list, D represents the doctor state list, and T represents the system time. The action space includes assign actions to allocate treatment rooms to patients and wait actions that are not assigned temporarily, and the action mask ensures that the assign actions meet medical hard constraints. The reward function R is designed in a weighted sum form: R = w1*R_wait + w2*R_idle + w3*R_fair, Where R_wait is the waiting penalty, R_idle is the idle penalty, R_fair is the fairness reward, and w1, w2, and w3 are weight coefficients; Step 3, Hybrid Policy Training: The model is trained using an Actor-Critic-based reinforcement learning framework, which integrates imitation learning and meta-learning strategies. The imitation learning includes reconstructing state-action pairs based on historical decision data and screening high-quality samples for behavior cloning through counterfactual simulation and statistical testing to obtain the initial policy. Step 4, Intelligent Scheduling Steps: Deploy the trained intelligent scheduling strategy model as a decision engine, receive real-time status data, output scheduling instructions containing assign or wait, and perform resource allocation or output risk warnings based on the decision instructions.

[0007] The imitation learning process in the hybrid strategy training step includes: Reconstructing historical state-action pairs (St, At) at the moment of decision-making based on accurate timestamps from historical information systems; For each historical sample (St, At), execute the real action At and all other legal candidate action sets {Aother} in the simulation environment, and perform multiple independent counterfactual simulations for each action to construct the payoff distribution for each action; Through statistical testing, samples whose average revenue of real action At is statistically significantly better than the average revenue of all other candidate actions are selected as high-quality expert samples. The initial policy is obtained by cloning the behavior of the policy network using the high-quality expert samples.

[0008] In step one, the extraction of key scheduling features includes: extracting the patient's expected treatment time, the patient's average late arrival time, the doctor's treatment efficiency, and the patient arrival rate at different times.

[0009] In step two, each element in the state space S is further defined as follows: Each element in R includes the occupancy status of the treatment room, and if busy, it further includes the estimated remaining treatment time based on historical averages and equipment availability. Each element in Q includes a unique identifier for the anonymous patient, the priority of the condition, the waiting time, and the expected treatment duration; Each element in D includes the doctor's availability status, and if busy, it further includes the associated treatment room, the degree of match between the doctor's specialty and the current patient's condition. T represents the current time period.

[0010] The meta-learning process in the hybrid strategy training step adopts a model-independent meta-learning framework, specifically including: Utilize data from different historical periods or workdays to construct multiple meta-tasks; During the meta-training phase, the trained model obtains a set of initialization parameters that can quickly adapt to new tasks; During the online application phase, the model is updated with a small amount of newly arrived data from the current date to quickly adapt to the specific working mode of the day. Based on the initial policy and the ability to adapt quickly, online reinforcement learning algorithms such as proximal policy optimization are used for further training. The policy parameters are fine-tuned by interacting with the environment in real time to optimize the long-term cumulative reward.

[0011] The hybrid strategy training step employs the Actor-Critic reinforcement learning framework, and introduces a multi-head self-attention mechanism in the Actor network and Critic network to dynamically calculate the attention weights of each element in the patient queue and resource state.

[0012] Step four, the risk warning step, includes at least the following: Delay risk warning: Triggered when a patient's expected waiting time exceeds a preset threshold; Resource conflict warning: Triggered when the proposed allocation action causes resource occupation conflicts or overload. Medical safety warning: Triggered when the doctor-patient professional matching degree of the proposed action is lower than the safety threshold or when the patient priority rules are violated.

[0013] A scheduling system for intravitreal drug delivery centers based on hybrid reinforcement learning, including... The data processing module is used to execute step one, completing the processing and feature extraction of multi-source data; The environmental modeling module is used to perform step two, constructing and maintaining the partially observable Markov decision process model. The model training module, used to perform step three, includes an imitation learning unit, a meta-learning unit, and an online reinforcement learning unit, used to train the intelligent scheduling strategy model. The decision and execution module is used to execute step four, including a decision engine encapsulating the intelligent scheduling strategy model, used to generate scheduling instructions based on real-time status; The human-computer interaction module includes a monitoring dashboard for visually displaying core scheduling indicators, an interactive interface for receiving and displaying the scheduling instructions and risk warnings, and a simulation and deduction unit for strategy verification.

[0014] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning.

[0015] The beneficial effects of this invention are: Significantly improved scheduling efficiency: In a simulation environment, this method can reduce the average waiting time for patients by more than 20% and increase the turnover rate of treatment rooms by more than 15%. The system is highly adaptive: through a hybrid reinforcement learning architecture, it can dynamically respond to changes in patient flow and fluctuations in resource status, achieving long-term steady-state optimality; Multi-objective collaborative optimization: By designing and embedding multi-objective reward functions, efficiency, fairness, and medical safety can be effectively balanced; High clinical acceptance: The system provides a decision interpretation mechanism, which enhances medical staff's trust in and willingness to use the intelligent scheduling solution. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an embodiment of the present invention.

[0017] Figure 2 This is a diagram of the hybrid reinforcement learning algorithm architecture of the present invention. Detailed Implementation

[0018] The present invention will be further described below. A method for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning includes the following steps: Step 1: Data Preprocessing: Collect patient information, resource information, drug information, and treatment event time sequence information from the drug injection center, process the data, and extract key scheduling features; Step 2: Problem Modeling: Model the dynamic resource scheduling problem of the drug injection center as a partially observable Markov decision process, and define the state space, action space, and multi-objective reward function; The state space S is defined as a tuple (R, Q, D, T), where R represents the treatment room state list, Q represents the patient queue list, D represents the doctor state list, and T represents the system time. The action space includes assign actions to allocate treatment rooms to patients and wait actions that are not assigned temporarily, and the action mask ensures that the assign actions meet medical hard constraints. The reward function R is designed in a weighted sum form: R = w1*R_wait + w2*R_idle + w3*R_fair, Where R_wait is the waiting penalty, R_idle is the idle penalty, R_fair is the fairness reward, and w1, w2, and w3 are weight coefficients; Step 3, Hybrid Policy Training: The model is trained using an Actor-Critic-based reinforcement learning framework, which integrates imitation learning and meta-learning strategies. The imitation learning includes reconstructing state-action pairs based on historical decision data and screening high-quality samples for behavior cloning through counterfactual simulation and statistical testing to obtain the initial policy. Step 4, Intelligent Scheduling Steps: Deploy the trained intelligent scheduling strategy model as a decision engine, receive real-time status data, output scheduling instructions containing assign or wait, and perform resource allocation or output risk warnings based on the decision instructions.

[0019] The imitation learning process in the hybrid strategy training step includes: Reconstructing historical state-action pairs (St, At) at the moment of decision-making based on accurate timestamps from historical information systems; For each historical sample (St, At), execute the real action At and all other legal candidate action sets {Aother} in the simulation environment, and perform multiple independent counterfactual simulations for each action to construct the payoff distribution for each action; Through statistical testing, samples whose average revenue of real action At is statistically significantly better than the average revenue of all other candidate actions are selected as high-quality expert samples. The initial policy is obtained by cloning the behavior of the policy network using the high-quality expert samples.

[0020] In step one, the extraction of key scheduling features includes: extracting the patient's expected treatment time, the patient's average late arrival time, the doctor's treatment efficiency, and the patient arrival rate at different times.

[0021] In step two, each element in the state space S is further defined as follows: Each element in R includes the occupancy status of the treatment room, and if busy, it further includes the estimated remaining treatment time based on historical averages and equipment availability. Each element in Q includes a unique identifier for the anonymous patient, the priority of the condition, the waiting time, and the expected treatment duration; Each element in D includes the doctor's availability status, and if busy, it further includes the associated treatment room, the degree of match between the doctor's specialty and the current patient's condition. T represents the current time period.

[0022] The meta-learning process in the hybrid strategy training step adopts a model-independent meta-learning framework, specifically including: Utilize data from different historical periods or workdays to construct multiple meta-tasks; During the meta-training phase, the trained model obtains a set of initialization parameters that can quickly adapt to new tasks; During the online application phase, the model is updated with a small amount of newly arrived data from the current date to quickly adapt to the specific working mode of the day. Based on the initial policy and the ability to adapt quickly, online reinforcement learning algorithms such as proximal policy optimization are used for further training. The policy parameters are fine-tuned by interacting with the environment in real time to optimize the long-term cumulative reward.

[0023] The hybrid strategy training step employs the Actor-Critic reinforcement learning framework, and introduces a multi-head self-attention mechanism in the Actor network and Critic network to dynamically calculate the attention weights of each element in the patient queue and resource state.

[0024] Step four, the risk warning step, includes at least the following: Delay risk warning: Triggered when a patient's expected waiting time exceeds a preset threshold; Resource conflict warning: Triggered when the proposed allocation action causes resource occupation conflicts or overload. Medical safety warning: Triggered when the doctor-patient professional matching degree of the proposed action is lower than the safety threshold or when the patient priority rules are violated.

[0025] A scheduling system for intravitreal drug delivery centers based on hybrid reinforcement learning, including... The data processing module is used to execute step one, completing the processing and feature extraction of multi-source data; The environmental modeling module is used to perform step two, constructing and maintaining the partially observable Markov decision process model. The model training module, used to perform step three, includes an imitation learning unit, a meta-learning unit, and an online reinforcement learning unit, used to train the intelligent scheduling strategy model. The decision and execution module is used to execute step four, including a decision engine encapsulating the intelligent scheduling strategy model, used to generate scheduling instructions based on real-time status; The human-computer interaction module includes a monitoring dashboard for visually displaying core scheduling indicators, an interactive interface for receiving and displaying the scheduling instructions and risk warnings, and a simulation and deduction unit for strategy verification.

[0026] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning.

[0027] This method specifically includes the following steps: Multimodal data preprocessing and feature engineering Data Collection: This includes collecting patient information from the medication administration center (including anonymous patient unique identifiers, age, gender, primary diagnosis ICD-10 code, diabetic retinopathy stage, left eye visual acuity, right eye example visual acuity, and physician-assessed disease priority), resource information (including resource unique identifiers, resource type, applicable specialty field, and available time period), drug information (including drug unique identifiers, drug name, and drug quantity), and treatment event timeline information (including event unique identifiers, anonymous patient unique identifiers, planned treatment timestamp, actual arrival timestamp, actual treatment start timestamp, actual treatment end timestamp, assigned treatment room unique identifier, assigned physician unique identifier, and event status). Data processing: Records with missing data or treatment durations exceeding the mean by 3 standard deviations or other obvious outliers were directly removed. Then, using the unique identifier of each patient's single treatment event as the primary key, structured tabular data scattered across the HIS, EMR, and manual anesthesia systems were correlated and merged to resolve timestamp alignment issues. Next, numerical features such as age and waiting time were standardized using Z-Score, while categorical features such as the primary diagnosis ICD-10 code and diabetic retinopathy stage were encoded using One-Hot Encoding. Finally, key features such as the patient's expected treatment duration, average patient delay time, physician treatment efficiency, and patient arrival rate at different times were extracted.

[0028] The scheduling problem is modeled as a partially observable Markov decision process (POMDP). Define the state space as a tuple S = (R, Q, D, T). R represents the treatment room status list, where each element is (occupancy status: busy / idle; if busy, it includes the estimated remaining treatment time based on historical averages and equipment availability); Q represents the patient queue list, where each element is (unique anonymous patient identifier, disease priority, waiting time, and estimated treatment time); D represents the doctor status list, where each element is (availability status; if busy, it includes the associated treatment room, specialty, and the degree of match between the doctor and the current patient's disease); and T represents the system time (current time period, such as "morning peak").

[0029] Define the action space as {assign(Patient_i, Room_j), wait}. The assign action must satisfy medical hard constraints, implemented through action masking. For example, it prohibits assigning patients to doctors unfamiliar with their conditions or to treatment rooms that are not available or have equipment malfunctions. The wait action indicates that the system will not assign patients temporarily. This action is triggered based on the reinforcement learning model's prediction of long-term cumulative rewards. When the Critic network evaluates that the state-action value (Q-value) of all currently valid assign actions is lower than the state value (V-value) corresponding to the wait state, the model will tend to choose the wait action.

[0030] The reward function is defined as a weighted sum, R = w1*R_wait + w2*R_idle + w3*R_fair. Here, R_wait (waiting penalty) = -1 × the total increase in waiting time for all patients in the new state; R_idle (idle penalty) = -10 × the number of newly available vacant treatment rooms; and R_fair (fairness reward) = -5 × the standard deviation of the maximum and minimum waiting times in the patient queue (encouraging load balancing). The weighting coefficients (w1, w2, w3) = (0.6, 0.3, 0.1) are determined using expert scoring (AHP) combined with historical data regression analysis.

[0031] Hybrid Reinforcement Learning Algorithm Design An Actor-Critic framework is adopted. The Actor network (policy network) takes state S as input and outputs the action probability distribution; the Critic network (value network) takes state S as input and outputs state value V. A meta-learning strategy is introduced to enable the model to quickly adapt to workloads on different dates and time periods based on historical patient flow patterns. Attention Mechanism: A multi-head self-attention mechanism is introduced after the input layer of the Actor and Critic networks. This mechanism dynamically assigns weights to different elements in the patient queue and resource state by calculating the query (Q), key (K), and value (V) vectors, enabling the network to focus on the information most critical to the current decision.

[0032] Meta-learning strategy: Employing the Model-Agnostic Meta-Learning (MAML) framework. During the meta-training phase, multiple tasks (such as "Monday Task" and "Friday Task") are constructed using historical data from different workdays, training the model to obtain a set of initial parameters. In online applications, the model only requires a small amount of new data (such as arrival data from the first two hours of the day) for minor gradient updates to quickly adapt to the specific pattern of the day.

[0033] Imitation learning: In the imitation learning stage, the system state St at the decision moment is reconstructed based on the precise timestamps of the HIS and EMR systems, and the actual actions At performed by the scheduler are extracted to form (St, At) pairs. Subsequently, data cleaning is carried out through counterfactual simulation based on uncertainty estimation: First, in the revenue distribution construction stage, for each sample (St, At) In the simulated environment, in addition to executing the real action At, all other legal candidate action sets {Aother} that satisfy the medical hard constraints are simultaneously enumerated. To overcome the randomness of the medical environment (such as fluctuations in treatment duration), N independent counterfactual simulations (e.g., N=10) are performed on At and each Aother, with each simulation using a different random seed, thereby constructing a payoff distribution containing N payoff values ​​for each action. Next, in the statistical significance screening stage, statistical tests are carried out based on the constructed payoff distribution. A one-sided T-test (with a confidence level set to 95%) is used to determine whether there is a significant difference between the mean payoff of the real action At and the mean payoff of each other candidate action Aother. Finally, in the sample retention criterion stage, only when the mean payoff of the real action At is statistically significantly better than the mean payoff of all other candidate actions is the sample marked as a "high-quality sample" and retained. This criterion ensures that the retained decisions not only have excellent long-term payoff performance but also have statistical robustness. Finally, the cleaned high-quality sample set is used to clone the behavior of the Actor network, thereby obtaining a high-quality initial strategy that combines expert experience and statistical robustness.

[0034] The reinforcement and fusion phase employs a proximal policy optimization algorithm for online training based on pre-training. An ε-greedy strategy (initial ε=0.2, which linearly decays to 0.05 with each training epoch) is used to balance the exploration of new strategies with the utilization of existing strategies.

[0035] Intelligent scheduling system implementation Decision Engine: This encapsulates the trained hybrid reinforcement learning model as a RESTful API microservice. Input is real-time status data in JSON format, and output is recommended scheduling actions and confidence levels.

[0036] The system's functional modules include: ① Real-time monitoring dashboard: Displaying core indicators such as patient queue (priority, disease type, waiting time), resource status (doctor's expertise / busy status, operating room availability), and scheduling progress; ② Intelligent scheduling recommendation: Outputting the optimal assign / wait action and decision-making basis (such as disease matching degree, benefit value); ③ Simulation simulation: Supporting scenario simulation and strategy verification; ④ Risk warning: Issuing warnings for 3 types of core potential risks, specifically including: a) Delay risk: When a patient's expected waiting time exceeds a preset threshold (e.g., 30 minutes) or the waiting time increases by 20% compared to the historical average for the same period, the system will prompt "Patient P_XXX is expected to be delayed by XX minutes, it is recommended to prioritize the allocation of available resources"; b) Resource conflict risk: When the proposed allocation action triggers resource overlap (e.g., two patients are simultaneously allocated to the same operating room) or when a doctor's continuous work exceeds the limit, the system will prompt "Resource conflict: OR_XXX has already been allocated to patient P_YYY, the current action cannot be executed"; c) Medical safety risk: When the match between the patient's disease and the specialist expertise of the doctor to be assigned is less than 80%, or when a high-priority patient is cut in line by a low-priority patient, a "Medical safety warning: Doctor Z and patient P_XXX's disease match is insufficient, it is recommended to adjust the assignment" message will be displayed to assist in rapid manual intervention.

[0037] To verify the effectiveness of this patented method, a simulation experiment was conducted at the ophthalmology injection center of a tertiary hospital. Random events such as emergency insertion and doctors taking temporary leave were added to the simulation experiment. The specific implementation is as follows: 1) Experimental Environment and Data The center used approximately 80,000 medical records accumulated over nearly a year as training and testing data; The simulation platform is built using Python 3.8 and the SimPy library, and the reinforcement learning algorithm is implemented using PyTorch.

[0038] 2) Parameter settings State space dimensions: 128 dimensions (including treatment room status, patient queue, physician workload, etc.); Action space: {Assigned to treatment room 1, assigned to treatment room 2, ..., waiting}; Reward function weights: waiting time penalty coefficient w1=0.6, resource idle penalty coefficient w2=0.3, fairness penalty coefficient w3=0.1; Training rounds: 10,000 rounds, each round simulating the scheduling process of one workday.

[0039] 3) Compare to baseline Baseline 1: First-Come, First-Served (FCFS) rule; Baseline 2: Static scheduling using a genetic algorithm; Baseline 3: Deep Q-Network (DQN) scheduling method.

[0040] 4) Experimental Results

[0041] Although the above embodiments have provided a detailed description of the present invention, they are only some embodiments of the present invention, and not all embodiments. People can obtain other embodiments based on these embodiments without creative effort, and these embodiments all fall within the protection scope of the present invention.

Claims

1. A method for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning, characterized in that, Includes the following steps: Step 1: Data Preprocessing: Collect patient information, resource information, drug information, and treatment event time sequence information from the drug injection center, process the data, and extract key scheduling features; Step 2: Problem Modeling: Model the dynamic resource scheduling problem of the drug injection center as a partially observable Markov decision process, and define the state space, action space, and multi-objective reward function; The state space S is defined as a tuple (R, Q, D, T), where R represents the treatment room state list, Q represents the patient queue list, D represents the doctor state list, and T represents the system time. The action space includes assign actions to allocate treatment rooms to patients and wait actions that are not assigned temporarily, and the action mask ensures that the assign actions meet medical hard constraints. The reward function R is designed in a weighted sum form: R = w1*R_wait + w2*R_idle + w3*R_fair, Where R_wait is the waiting penalty, R_idle is the idle penalty, R_fair is the fairness reward, and w1, w2, and w3 are weight coefficients; Step 3, Hybrid Policy Training: The model is trained using an Actor-Critic-based reinforcement learning framework, which integrates imitation learning and meta-learning strategies. The imitation learning includes reconstructing state-action pairs based on historical decision data and screening high-quality samples for behavior cloning through counterfactual simulation and statistical testing to obtain the initial policy. Step 4, Intelligent Scheduling Steps: Deploy the trained intelligent scheduling strategy model as a decision engine, receive real-time status data, output scheduling instructions containing assign or wait, and perform resource allocation or output risk warnings based on the decision instructions.

2. The method for scheduling intravitreal drug delivery centers based on hybrid reinforcement learning according to claim 1, characterized in that, The imitation learning process in the hybrid strategy training step includes: Reconstructing historical state-action pairs (St, At) at the moment of decision-making based on accurate timestamps from historical information systems; For each historical sample (St, At), execute the real action At and all other legal candidate action sets {Aother} in the simulation environment, and perform multiple independent counterfactual simulations for each action to construct the payoff distribution for each action; Through statistical testing, samples whose average revenue of real action At is statistically significantly better than the average revenue of all other candidate actions are selected as high-quality expert samples. The initial policy is obtained by cloning the behavior of the policy network using the high-quality expert samples.

3. The intravitreal drug delivery center scheduling method based on hybrid reinforcement learning according to claim 1, characterized in that, In step one, the extraction of key scheduling features includes: extracting the patient's expected treatment time, the patient's average late arrival time, the doctor's treatment efficiency, and the patient arrival rate at different times.

4. The intravitreal drug delivery center scheduling method based on hybrid reinforcement learning according to claim 1, characterized in that, In step two, each element in the state space S is further defined as follows: Each element in R includes the occupancy status of the treatment room, and if busy, it further includes the estimated remaining treatment time based on historical averages and equipment availability. Each element in Q includes a unique identifier for the anonymous patient, the priority of the condition, the waiting time, and the expected treatment duration; Each element in D includes the doctor's availability status, and if busy, it further includes the associated treatment room, the degree of match between the doctor's specialty and the current patient's condition. T represents the current time period.

5. The intravitreal drug delivery center scheduling method based on hybrid reinforcement learning according to claim 1, characterized in that, The meta-learning process in the hybrid strategy training step adopts a model-independent meta-learning framework, specifically including: Utilize data from different historical periods or workdays to construct multiple meta-tasks; During the meta-training phase, the trained model obtains a set of initialization parameters that can quickly adapt to new tasks; During the online application phase, the model is updated with a small amount of newly arrived data from the current date to quickly adapt to the specific working mode of the day. Based on the initial policy and the ability to adapt quickly, online reinforcement learning algorithms such as proximal policy optimization are used for further training. The policy parameters are fine-tuned by interacting with the environment in real time to optimize the long-term cumulative reward.

6. The intravitreal drug delivery center scheduling method based on hybrid reinforcement learning according to claim 1, characterized in that, The hybrid strategy training step employs the Actor-Critic reinforcement learning framework, and introduces a multi-head self-attention mechanism in the Actor network and Critic network to dynamically calculate the attention weights of each element in the patient queue and resource state.

7. The intravitreal drug delivery center scheduling method based on hybrid reinforcement learning according to claim 1, characterized in that, Step four, the risk warning step, includes at least the following: Delay risk warning: Triggered when a patient's expected waiting time exceeds a preset threshold; Resource conflict warning: Triggered when the proposed allocation action causes resource occupation conflicts or overload. Medical safety warning: Triggered when the doctor-patient professional matching degree of the proposed action is lower than the safety threshold or when the patient priority rules are violated.

8. A scheduling system for intravitreal drug delivery centers based on hybrid reinforcement learning, characterized in that, include The data processing module is used to execute step one, completing the processing and feature extraction of multi-source data; The environmental modeling module is used to perform step two, constructing and maintaining the partially observable Markov decision process model. The model training module, used to perform step three, includes an imitation learning unit, a meta-learning unit, and an online reinforcement learning unit, used to train the intelligent scheduling strategy model. The decision and execution module is used to execute step four, including a decision engine encapsulating the intelligent scheduling strategy model, used to generate scheduling instructions based on real-time status; The human-computer interaction module includes a monitoring dashboard for visually displaying core scheduling indicators, an interactive interface for receiving and displaying the scheduling instructions and risk warnings, and a simulation and deduction unit for strategy verification.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the intravitreal drug delivery center scheduling method based on hybrid reinforcement learning as described in any one of claims 1-7.

Images