A deep reinforcement learning-based pharmaceutical service resource coordination method

Abstract

The application belongs to the field of intelligent scheduling, and discloses a medicine affair service resource coordination method based on deep reinforcement learning. In view of problems such as waste of service resources and lag of demand response existing in a traditional Chinese medicine affair service resource deployment mode, a Chinese medicine affair service resource coordination problem is modeled as a dynamic Markov decision process, and an improved D3QN algorithm is used for solving, so that an optimal medicine affair service coordination strategy is obtained, medicine affair service resources are distributed, medicine affair service resource demand response is accelerated, medicine affair service cost of patients is effectively reduced, and medicine affair service quality is improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent scheduling, specifically relating to a method for coordinating pharmaceutical service resources based on deep reinforcement learning. Background Technology

[0002] Traditional Chinese medicine (TCM) pharmaceutical services encompass a large-scale collaborative resource network covering multiple aspects such as medicinal material production, pharmaceutical logistics, decoction preparation, and hospital consultations (e.g., TCM pharmaceutical services). Figure 1 As shown in the figure, its operation process has complex characteristics such as strong resource heterogeneity, high business concurrency, and significant differences in user needs. Therefore, how to achieve efficient and accurate scheduling of service resources has become the key to the quality of traditional Chinese medicine pharmaceutical services.

[0003] Currently, the collaborative scheduling model of traditional Chinese medicine pharmaceutical services (including core links such as pharmacy resource allocation and cross-regional logistics distribution) still generally relies on fixed rules set by human experience or expert experience for decision-making. Such scheduling schemes based on static preset logic are difficult to adapt to the real-time changing service environment when facing large-scale dynamic collaborative scenarios. They have prominent problems such as waste of service resources, delayed demand response, and imbalance in resource allocation, which seriously restricts the improvement of the overall efficiency of pharmaceutical services.

[0004] To overcome the shortcomings of traditional models, existing technologies have begun to introduce optimization algorithms or learning models for medical resource scheduling optimization. However, these methods still have limitations. For example, the paper "Research on Scheduling Methods Based on Pharmaceutical Service Resource Collaboration" introduces an MDP modeling scheme for the collaborative problem of traditional Chinese medicine pharmaceutical services. While this breaks through the constraints of static preset rules, the scheme only defines a basic state space, and the reward function design relies solely on general cost, time, or satisfaction indicators. This fails to guide the model to learn refined and customized collaborative strategies that adapt to individual needs. Furthermore, the actions involved in the collaborative management of traditional Chinese medicine pharmaceutical services are essentially a high-dimensional allocation matrix. The reinforcement learning method (Q-learning) used in this method is prone to Q-value overestimation when dealing with such high-dimensional, multi-constrained resource allocation matrix action spaces, leading to learned collaborative strategies that fail to meet the high reliability requirements of actual traditional Chinese medicine pharmaceutical services. Another paper, "Research on Scheduling Model Based on Collaborative Cognition of Pharmaceutical Service Resources," optimizes the model based on evolutionary algorithms. However, when a new real-time service request is received, this type of method requires a lot of time for iterative search, resulting in high decision-making response delays and failing to meet the requirements of high concurrency and real-time response for traditional Chinese medicine pharmaceutical services.

[0005] Therefore, existing technologies are currently unable to effectively solve the multi-objective trade-off problem in the coordination of pharmaceutical service resources for traditional Chinese medicine. There is an urgent need to construct a collaborative method for pharmaceutical service resources that takes into account real-time performance, reliability, and precision, in order to solve the problem of high-precision collaborative optimization of pharmaceutical service resources. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method for coordinating pharmaceutical service resources based on deep reinforcement learning. Addressing issues such as resource waste and delayed demand response in traditional Chinese medicine pharmaceutical service resource allocation models, this invention models the problem of coordinating Chinese medicine pharmaceutical service resources as a dynamic Markov Decision Process (MDP) and employs an improved D3QN algorithm for solution. This accelerates the response to pharmaceutical service resource demands, effectively reduces patient pharmaceutical service costs, and improves the quality of pharmaceutical services.

[0007] The present invention discloses a method for collaborative pharmaceutical service resources based on deep reinforcement learning, comprising the following steps: Step 1: To address the issue of resource collaboration in traditional Chinese medicine (TCM) pharmaceutical services, an MDP model for TCM pharmaceutical services is constructed. Step 2: Based on the MDP model of traditional Chinese medicine pharmaceutical services, the improved D3QN algorithm is used to optimize the two-layer Q network to form a deep reinforcement learning decision module for the MDP model of traditional Chinese medicine pharmaceutical services. Step 3: Based on the deep reinforcement learning decision-making module, analyze the real-time data on Chinese medicine pharmaceutical service resources and demand, output the optimal pharmaceutical service collaboration strategy, and allocate pharmaceutical service resources.

[0008] Furthermore, step 1 specifically includes: The Markov decision-making process for traditional Chinese medicine pharmaceutical services can be represented as: , Where M represents the collaborative problem model of traditional Chinese medicine (TCM) pharmaceutical services; S represents the state space of TCM pharmaceutical services, and the state of TCM pharmaceutical services... It reflects the current inventory level of various pharmaceutical service resources, the distribution of each traditional Chinese medicine pharmaceutical service resource node, the current patient demand information to be coordinated and scheduled, and their service quality (QoS) status. The status of traditional Chinese medicine pharmaceutical services at time t is represented as follows: , in, The remaining amount of various Chinese medicine pharmaceutical service resources; To meet the current needs for collaborative patient pharmaceutical services resources; This includes the location and status information between each resource node and the patient; A represents the action space, indicating the allocation strategy for all Chinese medicine pharmaceutical service resources; For each traditional Chinese medicine pharmaceutical service status The following collaborative actions that can be taken to improve traditional Chinese medicine pharmaceutical services resources are represented as follows: , in, Indicates the state Let I represent the number of resources allocated from the i-th pharmaceutical service resource set to the j-th patient; I represents the number of pharmaceutical service resource sets; and J represents the number of patients to be coordinated. Define constraints for action space A: 1) The allocation of each type of pharmaceutical service resource set shall not exceed its remaining quantity; 2) The amount of resources allocated to each patient shall not exceed their required amount; Let be the state transition probability function, representing the state of traditional Chinese medicine pharmaceutical services. Taking pharmaceutical service actions Then it will be transferred to the next traditional Chinese medicine pharmaceutical service status. The probability is expressed as: ; R is the reward function. In order to take synergistic action in pharmaceutical matters Then it will be transferred to the next traditional Chinese medicine pharmaceutical service status. The reward value obtained is represented as: , in, The reward for fulfilling demand is represented as: , in, The weighting coefficient for the reward to satisfy demand. For patient j's resource requirements, The service quality requirement level for patient j; The cost of using resources for traditional Chinese medicine pharmaceutical services is expressed as: , in, This is the cost penalty coefficient. The cost per unit of resource from resource set i to patient j; Let the penalty function be expressed as: , in, This is the delay penalty coefficient; A penalty is imposed for delaying the demand of the j-th patient; when a patient's waiting time exceeds their tolerance. In such cases, a delay penalty must be paid. This is a discount factor used to balance the importance of current and future rewards; Let T be the total duration of the collaborative process for traditional Chinese medicine (TCM) pharmaceutical services. What is the optimal collaborative strategy for TCM pharmaceutical services? Represented as: , in, As expected, A strategy for resource collaboration in pharmaceutical services for traditional Chinese medicine.

[0009] Furthermore, step 2 specifically involves: Step 2-1: Initialize the MDP model, reset the pharmaceutical service resource collaboration strategy, the inventory of each type of pharmaceutical service resource set, and patient needs; Step 2-2: Evaluation of the two-layer Q-network Calculate the current Q-function value and output the Q-value estimate. Select the resource coordination action that maximizes the Q value as the action to be executed at the current moment. ; Steps 2-3: Perform actions in the MDP model And update the next environment state according to the state transition probability function. ; Steps 2-4: Based on the reward function Calculate instant rewards; Steps 2-5: Combine the current state, actions, immediate rewards, and the next state to form experience. Stored in the memory experience pool D as a transitional sample; Steps 2-6: Randomly select a batch of transitional samples from the memory experience pool. The target network of a two-layer Q-network For the next state Generate the target Q value and calculate the loss function; Steps 2-7: Update using gradient descent. Network parameters, through Network parameters are obtained using a soft synchronization strategy for the target network. Parameter updates; Steps 2-8: Repeat steps 2-2 to 2-7 until the network loss converges, obtaining the network parameters that optimize the synergy of traditional Chinese medicine pharmaceutical service resources.

[0010] Furthermore, in step 2-2, a dynamic advantage reweighting mechanism is introduced to adaptively adjust the advantage weights based on resource utilization and patient demand fluctuations under the current collaborative adaptation state. The Q-function value is then expressed as: , in, Let be the value function, representing the degree of goodness or badness of the state; The dominance function represents the state. Next action Advantages compared to other actions; For state The mean of the dominance function for all actions; To evaluate the network Network parameters, The dominant weight is represented as: , in, For the Sigmoid function, The rate of change in resource utilization. For patient demand volatility, It is a regulating factor.

[0011] Furthermore, in steps 2-6, the loss function is defined as: ; in, These are the weighting coefficients.

[0012] Furthermore, steps 2-7 are as follows: Update using gradient descent. The network parameters, and the update formula, are expressed as follows: , in, For learning rate, relative parameters of the loss function The gradient is expressed as: , in, for Error: ; pass Network parameters Using a soft synchronization strategy for the target network parameter Update, represented as: , in, This is the soft synchronization coefficient.

[0013] The beneficial effects of this invention are as follows: 1) By modeling the dynamic characteristics of the pharmaceutical service resource collaboration process, an MDP state space that can characterize inventory fluctuations, delivery delays and changes in patient needs is constructed, so that the state representation can characterize the real multidimensional constraints in pharmaceutical service resource collaboration and achieve a more accurate environmental description. 2) For the action space of high-dimensional resource allocation matrix, an improved D3QN algorithm is proposed. The existing two-layer Q network is improved by combining a dynamic advantage reweighting mechanism, so that the network can adaptively adjust the Q value composition according to the rate of change of resource utilization and the rate of demand fluctuation. This improves the policy response capability when the environment changes drastically and suppresses the overestimation problem caused by the amplification of the advantage term when the environment is stable. During the training of the algorithm, a state change regularization term and an advantage weight smoothing term are added to the loss function to further mitigate the gradient instability problem caused by sudden changes in inventory and delivery delays, making the training process easier to converge. The target network drift is reduced by a soft update strategy, thereby improving the overall training performance. 3) This invention can generate more stable and efficient resource collaboration strategies in dynamic, multi-stage coupled pharmaceutical service scenarios, improve resource utilization, shorten patient waiting time, improve patient QoS, and improve the overall operational efficiency of the pharmaceutical service system. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the pharmaceutical service process for traditional Chinese medicine. Figure 2 This is a flowchart of the method described in this invention; Figure 3 This is a diagram illustrating the specific implementation steps of the method. Figure 4 This is a network structure diagram of the method described above; Figure 5 This is a framework diagram of deep reinforcement learning methods. Detailed Implementation

[0015] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0016] like Figure 2 and Figure 3 As shown, the present invention provides a method for collaborative pharmaceutical service resources based on deep reinforcement learning, comprising the following steps:

[0017] Step 1: Regarding the issue of resource coordination in traditional Chinese medicine (TCM) pharmaceutical services, the Markov decision process for TCM pharmaceutical services is represented as follows: , Where M represents the collaborative problem model of traditional Chinese medicine (TCM) pharmaceutical services; S represents the state space of TCM pharmaceutical services, indicating all possible states in the collaborative process of pharmaceutical services, and the TCM pharmaceutical service state... This includes the current inventory levels of various pharmaceutical service resource sets, the distribution of each traditional Chinese medicine pharmaceutical service resource node, the patient demand information currently awaiting coordinated scheduling, and their Quality of Service (QoS) status; specifically, the status of traditional Chinese medicine pharmaceutical services at time t is represented as: , in, This represents the remaining quantity of various Chinese medicine pharmaceutical service resources, indicating the real-time supply status of various Chinese medicine resource sets (processed herbs, decoction processing capabilities, etc.). To meet the current needs for collaborative patient pharmaceutical services resources; This includes location and status information, such as the location, transportation, and delivery time between each resource node and the patient; A is the action space, representing all possible strategies for allocating resources for traditional Chinese medicine pharmaceutical services; actions Indicates each state The following collaborative actions that can be taken to improve traditional Chinese medicine pharmaceutical services resources are represented as follows: , in, Indicates the state Let I represent the number of resources allocated from the i-th pharmaceutical service resource set to the j-th patient, where I represents the number of pharmaceutical service resource sets and J represents the number of patients to be coordinated. To adapt to the complex pharmaceutical service resource collaboration environment, additional constraints are defined for the action space A, including: (1) the allocation amount of each type of pharmaceutical service resource set shall not exceed its remaining amount; (2) the amount of resources allocated to each patient shall not exceed its demand. Let be the state transition probability function, representing the state of traditional Chinese medicine pharmaceutical services. Taking pharmaceutical service actions Then it will be transferred to the next traditional Chinese medicine pharmaceutical service status. The probability is expressed as: ; R is the reward function, used to measure the effectiveness of a resource coordination strategy for traditional Chinese medicine pharmaceutical services. In order to take synergistic action in pharmaceutical matters Then it will be transferred to the next traditional Chinese medicine pharmaceutical service status. The reward value obtained is represented as: , in, The demand fulfillment reward represents the positive benefit gained from successfully fulfilling a patient's demand; considering the impact of QoS indicators, a definition is provided. for: , in, The weighting coefficient for the reward to satisfy demand. For patient j's resource requirements, The service quality requirement level for patient j reflects their sensitivity to QoS; The cost of using TCM pharmaceutical service resources represents the logistics and storage costs incurred during the collaborative process of TCM pharmaceutical service resources. This cost is typically a negative benefit and is expressed as: , in, This is the cost penalty coefficient. The cost per unit of resource from resource set i to patient j; Let be the penalty function, used to reflect the negative impact of resource response lag. Let the penalty for the delay in demand for the j-th patient be . When patients wait longer than they can tolerate Delay penalties will be charged. ,but Represented as: , in, This is the delay penalty coefficient; This is a discount factor used to balance the importance of current and future rewards.

[0018] Let T be the total duration of the collaborative TCM pharmaceutical services cycle. The ultimate goal of the model is to obtain the TCM pharmaceutical service resource collaboration strategy that maximizes long-term rewards. Then, the optimal TCM pharmaceutical service collaboration strategy is... Represented as: , in, For the expectation, A strategy for resource collaboration in pharmaceutical services for traditional Chinese medicine.

[0019] Step 2: In the MDP model of traditional Chinese medicine pharmaceutical services, an improved D3QN algorithm is used to optimize the two-layer Q-network, forming a deep reinforcement learning decision module, including: Step 2-1: Initialize the MDP model, reset the pharmaceutical service resource collaboration strategy, the inventory of each type of pharmaceutical service resource set, and patient needs; Step 2-2, as follows Figure 4 and Figure 5 As shown, the two-layer Q-network includes an evaluation network. and target network Evaluation network Used for calculating the current Q-function value and selecting actions, target network Used for estimating the target Q value; In each network, the Q-function is divided into two parts: the first part is the value function. Only related to state The first part is related to the degree of good or bad of the state; the second part is the advantage function. It relates to state S and action A, representing state S. Next action Advantages compared to other actions.

[0020] A dynamic advantage reweighting mechanism is introduced to calculate the Q function value; at time t, the current collaborative environment status of pharmaceutical service resources is collected. Evaluation network In the current state Down to action space Collaborative Actions of Pharmaceutical Service Resources for Each Candidate Traditional Chinese Medicine Perform Q-value evaluation and output the Q-value estimate. Select the resource coordination action that maximizes the Q value as the action to be executed at the current moment. .

[0021] A dynamic advantage reweighting mechanism is introduced when calculating the Q function. The advantage weights are adaptively adjusted based on resource utilization and fluctuations in patient demand under the current collaborative adaptation status. The Q function value is then expressed as: , in, For network parameters, The dominant weight is represented as: , in, For the Sigmoid function, The rate of change in resource utilization rate For patient demand volatility, As a regulating factor; When the environment changes drastically An increase in the value of the advantage function increases its influence on Q-value estimation, thereby accelerating the response of the collaborative adaptation strategy; when the environment tends to stabilize, The value tends to the value function in order to stabilize training.

[0022] Steps 2-3: Perform actions in the MDP model And update the next environment state according to the state transition probability function. ;

[0023] Steps 2-4: Based on the reward function Calculate instant rewards;

[0024] Steps 2-5: Combine the current state, actions, immediate rewards, and the next state to form experience. Stored in the memory experience pool D as transitional samples for subsequent network evaluation. and training of the target network;

[0025] Steps 2-6: Randomly select a batch of transitional samples from the memory experience pool. Target network For the next state Generate the target Q-value to provide stable gradient feedback for training and evaluating the network; and calculate the loss function; Wherein, the target Q value at time t Represented as: , in, For instant rewards, express Network parameters, To utilize Selected in state The action that maximizes the estimated Q value; The loss function is defined as follows: , in, These are the weighting coefficients.

[0026] Steps 2-7: Update using gradient descent. The network parameters, and the update formula, are expressed as follows: , in, For learning rate, relative parameters of the loss function The gradient is expressed as: , in, for Error: .

[0027] To ensure algorithm stability, by Network parameters Using a soft synchronization strategy for the target network parameter Updated to avoid overfitting, as follows: , in, This is the soft synchronization coefficient.

[0028] Steps 2-8: Repeat steps 2-2 to 2-7 until the network loss converges, obtaining the network parameters that enable optimal coordination of traditional Chinese medicine pharmaceutical service resources.

[0029] Step 3: Deploy the trained deep reinforcement learning decision-making module in the actual pharmaceutical service system, receive real-time resource and demand data, and output the optimal pharmaceutical service collaboration strategy; allocate pharmaceutical service resources according to the collaboration strategy to achieve dynamic scheduling, accelerate the response to pharmaceutical service resource demands, effectively reduce pharmaceutical service resource redundancy, and improve the quality of pharmaceutical service resource collaboration.

[0030] To verify the effectiveness of the present invention, comparative experiments were conducted based on historical pharmaceutical service request data of different scales. The different dataset proportions refer to the percentage of historical pharmaceutical service request data used for training and testing within the complete dataset; experimental datasets were constructed by selecting 20%, 40%, 60%, 80%, and 100% of the complete dataset, respectively.

[0031] Table 1 shows the resource utilization rates of the method described in this invention and commonly used methods in the prior art. As can be seen from Table 1, the method of this invention has the highest resource utilization rate among different dataset proportions. Compared with the comparative methods, the method of this invention maintains a high overall resource utilization level, indicating that the method can effectively improve the synergistic utilization efficiency of pharmaceutical service resources.

[0032] Table 1 Resource utilization rate of different methods

[0033] Table 2 shows a comparison of the response time of the method described in this invention and commonly used methods in the prior art. As can be seen from Table 2, the method of this invention maintains a low response time level under various data scale conditions, and is superior to the comparative methods overall. This indicates that the method can achieve a faster scheduling response while ensuring the resource coordination effect.

[0034] Table 2 Response Time Comparison

[0035] Table 3 shows a comparison of the QoS of the method described in this invention and commonly used methods in the prior art. As can be seen from Table 3, the QoS of this invention is the highest, indicating that the method can effectively take into account service quality requirements in the process of pharmaceutical service resource collaboration.

[0036] Table 3 QoS Comparison

[0037] The above description is merely a preferred embodiment of the present invention and is not intended to further limit the present invention. All equivalent changes made based on the description and drawings of the present invention are within the protection scope of the present invention.

Claims

1. A method for collaborative pharmaceutical service resources based on deep reinforcement learning, characterized in that, Includes the following steps: Step 1: To address the issue of resource collaboration in traditional Chinese medicine (TCM) pharmaceutical services, an MDP model for TCM pharmaceutical services is constructed. Step 2: Based on the MDP model of traditional Chinese medicine pharmaceutical services, the improved D3QN algorithm is used to optimize the two-layer Q network to form a deep reinforcement learning decision module for the MDP model of traditional Chinese medicine pharmaceutical services. Step 3: Based on the deep reinforcement learning decision module, analyze the real-time data on Chinese medicine pharmaceutical service resources and demand, output the optimal pharmaceutical service collaboration strategy, and allocate pharmaceutical service resources. Step 1 specifically includes: The Markov decision-making process for traditional Chinese medicine pharmaceutical services can be represented as: ， Where M represents the collaborative problem model of traditional Chinese medicine (TCM) pharmaceutical services; S represents the state space of TCM pharmaceutical services, and the state of TCM pharmaceutical services... It reflects the current inventory level of various pharmaceutical service resources, the distribution of each traditional Chinese medicine pharmaceutical service resource node, the current patient demand information to be coordinated and scheduled, and their service quality (QoS) status. The status of traditional Chinese medicine pharmaceutical services at time t is represented as follows: ， in, The remaining amount of various Chinese medicine pharmaceutical service resources; To meet the current needs for collaborative patient pharmaceutical services resources; This includes the location and status information between each resource node and the patient; A represents the action space, indicating the allocation strategy for all Chinese medicine pharmaceutical service resources; For each traditional Chinese medicine pharmaceutical service status The following collaborative actions that can be taken to improve traditional Chinese medicine pharmaceutical services resources are represented as follows: ， in, Indicates the state Let I represent the number of resources allocated from the i-th pharmaceutical service resource set to the j-th patient; I represents the number of pharmaceutical service resource sets; and J represents the number of patients to be coordinated. Define constraints for action space A: 1) The allocation of each type of pharmaceutical service resource set shall not exceed its remaining quantity; 2) The amount of resources allocated to each patient shall not exceed their required amount; Let be the state transition probability function, representing the state of traditional Chinese medicine pharmaceutical services. Taking pharmaceutical service actions Then it will be transferred to the next traditional Chinese medicine pharmaceutical service status. The probability is expressed as: ； R is the reward function. In order to take synergistic action in pharmaceutical matters Then it will be transferred to the next traditional Chinese medicine pharmaceutical service status. The reward value obtained is represented as: ， in, The reward for fulfilling demand is represented as: ， in, The weighting coefficient for the reward to satisfy demand. For patient j's resource requirements, The service quality requirement level for patient j; The cost of using resources for traditional Chinese medicine pharmaceutical services is expressed as: ， in, This is the cost penalty coefficient. The cost per unit of resource from resource set i to patient j; Let the penalty function be expressed as: ， in, This is the delay penalty coefficient; A penalty is imposed for delaying the demand of the j-th patient; when a patient's waiting time exceeds their tolerance. In such cases, a delay penalty must be paid. This is a discount factor used to balance the importance of current and future rewards; Let T be the total duration of the collaborative process for traditional Chinese medicine (TCM) pharmaceutical services. What is the optimal collaborative strategy for TCM pharmaceutical services? Represented as: ， in, For the expectation, Strategies for resource collaboration in pharmaceutical services for traditional Chinese medicine; Step 2 is as follows: Step 2-1: Initialize the MDP model, reset the pharmaceutical service resource collaboration strategy, the inventory of each type of pharmaceutical service resource set, and patient needs; Step 2-2: Evaluation of the two-layer Q-network Calculate the current Q-function value and output the Q-value estimate. Select the resource coordination action that maximizes the Q value as the action to be executed at the current moment. ; Steps 2-3: Perform actions in the MDP model And update the next environment state according to the state transition probability function. ; Steps 2-4: Based on the reward function Calculate instant rewards; Steps 2-5: Combine the current state, actions, immediate rewards, and the next state to form experience. Stored in the memory experience pool D as a transitional sample; Steps 2-6: Randomly select a batch of transitional samples from the memory experience pool. The target network of a two-layer Q-network For the next state Generate the target Q value and calculate the loss function; Steps 2-7: Update using gradient descent. Network parameters, through Network parameters are obtained using a soft synchronization strategy for the target network. Parameter updates; Step 2-8: Repeat steps 2-2 to 2-7 until the network loss converges, and obtain the network parameters that optimize the coordination of traditional Chinese medicine pharmaceutical service resources. In step 2-2, a dynamic advantage reweighting mechanism is introduced to adaptively adjust the advantage weights based on resource utilization and patient demand fluctuations under the current collaborative adaptation state. The Q-function value is then expressed as: ， in, Let be the value function, representing the degree of goodness or badness of the state; The dominance function represents the state. Next action Advantages compared to other actions; For state The mean of the dominance function for all actions; To evaluate the network Network parameters, The dominant weight is represented as: ， in, For the Sigmoid function, The rate of change in resource utilization rate For patient demand volatility, It is a regulating factor.

2. The method for collaborative pharmaceutical service resources based on deep reinforcement learning according to claim 1, characterized in that, In steps 2-6, the loss function is defined as follows: ； in, These are the weighting coefficients.

3. The method for collaborative pharmaceutical service resources based on deep reinforcement learning according to claim 2, characterized in that, Steps 2-7 are as follows: Update using gradient descent. The network parameters, and the update formula, are expressed as follows: ， in, For learning rate, relative parameters of the loss function The gradient is expressed as: ， in, for Error: ； pass Network parameters Using a soft synchronization strategy for the target network parameter Update, represented as: ， in, This is the soft synchronization coefficient.

Images