Propofol dosing method based on pharmacokinetics-pharmacodynamics and reinforcement learning and application thereof

Abstract

This invention relates to a propofol administration method based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning methods and its application. It uses a PKPD model to simulate the metabolic process of propofol in the human body, extracts time-dependent and kinetic continuous information through LSTM and Neural OED, and then establishes a nonlinear relationship between the effective concentration of propofol and the clinically observed anesthesia depth index (BIS) using a Kan network. This rapidly establishes a mapping between intraoperative monitoring information and anesthesia depth, allowing for timely adjustments to the strategy based on environmental changes. It is suitable for handling non-standardized and complex problems, enabling personalized adjustments to drug dosage and administration regimens, and improving therapeutic efficacy.

Description

Technical Field

[0001] This invention relates to the field of intelligent healthcare, and more particularly to a propofol administration method based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning methods, and its application. Background Technology

[0002] Pharmacokinetics (PK) is the science that studies the absorption, distribution, metabolism, and excretion of drugs in the body. It is crucial for the rational use of drugs and the individualization of drug therapy. Pharmacokinetics reveals the mechanisms of drug action and characterizes the biotransformation processes of drugs, such as absorption, distribution, metabolism, and excretion in the body. By revealing the mechanisms of drug action, we can better understand the pharmacological effects of drugs, thereby guiding rational drug use. The efficacy and safety of drug therapy largely depend on the drug's kinetic processes in the body. Pharmacokinetics can measure the blood concentration-time curve of a drug. By analyzing the pharmacokinetic parameters, we can predict the drug dosage and dosing regimen. For example, for dose-dependent drugs, understanding their pharmacokinetic characteristics allows for the rational adjustment of drug dosage, avoiding overdose and side effects. Furthermore, pharmacokinetics can guide the development of drug combination therapies and individualized drug treatment regimens, improving therapeutic efficacy. Pharmacokinetics can also predict drug efficacy and toxicity by establishing pharmacokinetic models. By measuring drug concentrations in the body, the maximum effect and half-maximal effect concentration (EC50) can be estimated, thus predicting therapeutic efficacy. Furthermore, pharmacokinetics can reveal potential toxicity and side effects, providing a basis for evaluating clinical drug use and reducing adverse reactions. Pharmacokinetics can also reveal individual differences in drug metabolism, providing a basis for personalized drug therapy. Different individuals have varying abilities to metabolize, absorb, and excrete drugs, which can lead to different individual responses. By utilizing pharmacokinetics, we can understand the types of drug-metabolizing enzymes and assess an individual's ability to metabolize the drug, thereby allowing for personalized adjustments to dosage and administration regimens to improve therapeutic efficacy.

[0003] In clinical medicine, general anesthesia is often required during surgery, necessitating the use of sedative drugs such as propofol. Propofol and other sedatives are dose-dependent; the dosage and rate of administration directly affect the anesthetic effect. Therefore, it is crucial to control the dosage and rate of administration within a reasonable range and dynamically adjust the dosage according to the depth of anesthesia. When the depth of anesthesia is deeper than expected, the dosage should be reduced promptly; when the depth of anesthesia is shallower than expected, the dosage should be increased promptly to ensure that the depth of anesthesia remains stable within the expected range. The most important reference indicator for assessing the depth of anesthesia is the bispectral index (BIS). This index directly reflects the patient's current level of alertness and can be used to determine whether the dosage of sedatives needs to be adjusted to ensure that the depth of anesthesia remains within a reasonable range during surgery.

[0004] CN101247809A discloses a method for quantitatively administering propofol prodrug to induce mild to moderate sedation. The method calculates the required propofol prodrug dose to induce mild to moderate sedation based on the patient's lean body mass, determines a dose appropriate for the patient's weight, and then adjusts the dose based on age. For example, for patients aged 60 years or older, the dose required to produce sedation or other effects may be approximately 0.6-0.8 times the dose required to produce the corresponding effect in younger patients of the same weight.

[0005] As a subfield of machine learning, reinforcement learning (RL) aims to enhance an agent's behavioral decision-making ability by using interactive experiences and evaluation feedback from the world. Reinforcement learning algorithms can be divided into two main categories: value function-based reinforcement learning and policy-based reinforcement learning. The proximal policy optimization (PPO) algorithm used in this invention is a reinforcement learning algorithm proposed by OpenAI in 2017, considered the state-of-the-art (SOTA) method in the field of reinforcement learning and one of the most widely applicable algorithms. Policy-based reinforcement learning no longer determines the policy for choosing actions through a value function, but directly learns the policy itself. It parameterizes the policy using a set of parameters θ and optimizes θ using neural network methods. Policy-based reinforcement learning uses a parameterized probability distribution π. θ The formula (a|s) = P(a|s; θ) replaces the deterministic policy π:s→a in value function-based reinforcement learning, sampling different actions from the returned list of action probabilities. Unlike traditional supervised learning methods (which typically rely on one-off, exhaustive, and supervisory signals), reinforcement learning simultaneously addresses the sequential decision-making problem of sampling, evaluation, and delayed feedback.

[0006] In conclusion, how to adjust the dosage of the anesthetic drug propofol in a timely manner based on human physiological indicators has become one of the urgent problems to be solved in this field. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a propofol administration method based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning methods and its application. Based on the patient's basic information such as height, weight, and gender, the BIS value can be calculated from the discrete time series of the input dosing rate data.

[0008] To achieve this objective, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a propofol administration method based on deep learning methods for pharmacokinetic-pharmacodynamics and reinforcement learning, the propofol administration method comprising the following steps:

[0010] (1) Collect basic information about the patient, including height, weight, gender and age, and calculate the patient’s lean body mass based on the basic information;

[0011] (2) Based on the collected patient information and the pharmacokinetic characteristics of propofol, determine the parameters of the pharmacokinetic-pharmacodynamic model, including the volume of each compartment, clearance rate and transfer rate;

[0012] (3) The pharmacokinetic-pharmacodynamic model includes three compartments: the central compartment, the fast peripheral compartment, and the slow peripheral compartment. Using the pre-defined parameters, a set of differential equations describing the concentration changes of propofol in the three compartments is constructed.

[0013] The pharmacokinetic-pharmacodynamic model is represented by a set of differential equations, as shown in equation (1):

[0014]

[0015] In the formula The values ​​represent the drug concentration changes in the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively. x1(t), x2(t), and x3(t) represent the drug concentrations in the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively. u(t) is the drug infusion rate, and t represents time.

[0016] constant K ij (i≠j) represents the drug transfer rate from compartment i to compartment j, where the central compartment is compartment 1, the fast distribution compartment is compartment 2, and the slow distribution compartment is compartment 3, for example, k 12 k is the drug transfer rate from the central compartment to the rapid distribution compartment. 10 The dosing rate of propofol into the central compartment;

[0017] The transfer rate and the drug metabolism rate between different compartments are expressed as follows:

[0018]

[0019] In the formula, V1, V2, and V3 represent the volumes of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively, and C1, C2, and C3 represent the clearance rates of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively.

[0020] The pharmacodynamic model introduces an additional hypothetical effect compartment to represent the site of drug action, as shown in the following formula:

[0021]

[0022] In the formula, For the change in drug concentration in the effect room, x e x1(t) represents the drug concentration in the effect chamber, x1(t) represents the drug concentration in the central chamber, t represents time, and k is a constant. e The drug transfer rate from the effect chamber to the central chamber;

[0023] (4) Establishing a mapping between intraoperative monitoring information and anesthesia depth based on deep learning methods, including using drug infusion rate and patient covariate data as input variables, extracting time-dependent parameters Lx and time-continuous parameters Nx through LSTM and Neural ODE_GRU, establishing a nonlinear function based on Kan network, and obtaining the predicted anesthesia depth by inputting drug infusion rate. The patient covariates include the patient's age, gender, height, and weight.

[0024] Constructing an accurate and effective pharmacokinetic model to describe the metabolism of propofol in the human body is a crucial foundation for the precise calculation of anesthetic dosage. There are many methods for constructing pharmacokinetic models; this invention uses the three-compartment model proposed by Sheiner in 1979 to construct the pharmacokinetic model of propofol, such as... Figure 1 As shown, the three-compartment pharmacokinetic-pharmacodynamic model consists of two parts: a pharmacokinetic (PK) model and a pharmacodynamic (PD) model, which are used to calculate and predict the optimal dose of a drug.

[0025] The PK model describes the transfer and metabolism of drugs between three compartments. The fast peripheral compartment represents the site of direct drug action, such as blood, and the drug concentration in this compartment is also called plasma concentration. The slow peripheral compartment represents tissues with slower action, such as muscle and fat.

[0026] In the PK model, u(t) refers to the drug infusion rate, usually expressed as u(t) / V1; V i (i = 1, 2, 3) and C i (i = 1, 2, 3) is related to the patient's lean body mass (LBM), which in turn depends on each individual's height, weight, and sex. The pharmacokinetic-pharmacodynamic model provided by this invention can adjust parameters according to the patient's actual physiological indicators.

[0027] The PD model links plasma concentration with drug effect. Since blood, muscle tissue, and organs are not direct manifestations of drug action, an additional hypothetical effect chamber is introduced to represent the site of drug action. The relationship between plasma concentration and effect chamber concentration is calculated based on the hypothetical effect chamber. Mathematical equations can be used to model the PD model, and the effect chamber concentration can be converted into BIS for direct observation.

[0028] In the PD model, the constant k e This represents the metabolic rate of the drug between the central chamber and the effect chamber, and its value is related to the type of anesthetic drug.

[0029] The method provided by this invention, after inputting the patient's height, weight, gender, and age data, then inputs the propofol administration rate u(t), and the patient's BIS index can be calculated through the above PKPD model.

[0030] In practical applications, it has been found that when individuals change, a large amount of data is needed to optimize the parameters of the formula. This invention uses deep learning methods to quickly establish a mapping between intraoperative monitoring information and anesthesia depth with a small amount of data.

[0031] Preferably, the input variable calculation method in step (4) includes: calculating the concentration changes and drug concentrations of propofol in the three compartments, as well as the changes and drug concentrations in the effect compartment, using a pharmacokinetic-pharmacodynamic model based on the patient's covariates and the drug infusion rate; and combining the concentration changes and drug concentrations in the three compartments within 1 minute, as well as the changes and drug concentrations in the effect compartment, with the patient covariates to form the input variable X. t .

[0032] Preferably, the input variable X t The calculation formula is as follows:

[0033]

[0034] Preferably, the Lx calculation method includes: converting X... tThe data is fed into a BiLSTM model, which captures the nonlinear dependencies in the time series and learns the transition rules between different anesthesia stages to obtain the time-dependent parameter Lx at this moment.

[0035] The LSTM (Gated Recurrent Unit) model is a recurrent neural network (RNN) architecture, and its calculation formula is shown below:

[0036] i t =σ(W ii x t +b ii +W hi h t-1 +b hi Equation (4.2);

[0037] f t =σ(W if x t +b if +W hf h t-1 +b hf Equation (4.3);

[0038] g t =tanh(W ig x t +b ig +W hg h t-1 +b hg Equation (4.4);

[0039] o t =σ(W io x t +b io +W ho h t-1 +b ho Equation (4.5);

[0040] c t =f t *c t-1 +i t *g t Equation (4.6);

[0041] h t =o t *tanh(c t Equation (4.7);

[0042] Here (W) ii W hi W fi W hf W gi W hgW io W ho and (b) ii ,b hi ,b fi ,b hf ,b gi ,b hg ,b io ,b ho ) represents the weight matrix and bias matrix, σ represents the sigmoid activation function, and tanh is the tangent function. t Determines how much of the current input is saved to the cell state; f t Determines how many cell states from the previous time step are retained in the current time step; g t Calculate the instantaneous state at the current moment; t Determines how much of the cell state is output to the current LSTM output value; c t To update the cell state; h t The filtered and updated information in the cell state is sent to the hidden state in the next time step.

[0043] BiLSTM is an extension of the LSTM model described above. Two LSTMs are applied to the input data. In the first round, the LSTM is applied to the input sequence (i.e., the forward layer). In the second round, the reversed form of the input sequence is fed into the LSTM model (i.e., the backward layer). Applying LSTM twice improves the learning of long-term dependencies, thereby increasing the model's accuracy. The calculation formula is as follows:

[0044]

[0045] Here (W) f W r ) and (B bi ) are the weight matrix and the bias matrix.

[0046] Preferably, the Nx calculation method includes: converting X... t When fed into the Neural ODE_GRU model, the ODE_GRU-based model encodes past trajectories as variables, yielding the current time-continuous parameter Nx, calculated as follows:

[0047]

[0048] Here, GRUCell stands for GRU unit. It is the unit output, and its implicit variable h t The output is controlled by ODESolve; ODESolve is the numerical differential solver for forward propagation in a neural network; MLP is the output of the fully connected network through the first d time steps of the dependent variable (h). t-d ,...,ht Calculate Nx at this moment.

[0049] Preferably, the method for calculating the predicted depth of anesthesia includes: inputting Lx, Nx, and the patient's covariates into a KAN network, and obtaining the predicted depth of anesthesia by inputting the drug infusion rate. The calculation formula is as follows:

[0050] Anesthetic_parameters t =[Lx t ,Nx t ,Age,Sex,Height,Weight] formula (4.15);

[0051] KAN(x)=(Φ3°Φ2°Φ1)(x) Equation (4.16);

[0052] Φ(x)=w(b(x)+spline(x)) Formula (4.17);

[0053]

[0054] Here, Age, Sex, Height, and Weight refer to the patient's age, sex, height, and weight; Φ refers to different network layers; and spline(x) represents the spline function. i These are the coefficients optimized during training, while B... i These are B-spline basis functions defined on a mesh.

[0055] We divided the dataset into a training set (80%) and a test set (20%), based on the number of surgical cases. The training set was input into the BIS model to obtain the training parameters. The test set was further divided, with 20% of the data input into the model for parameter fine-tuning. This ensures the model more closely approximates the patient's condition during testing, avoiding the influence of patient differences.

[0056] In this invention, the LSTM (Gated Recurrent Unit) model is a recurrent neural network (RNN) architecture that struggles to learn long-term dependencies. LSTM-based models are an extension of RNNs, cleanly resolving the vanishing gradient problem. Essentially, LSTM models extend the memory of RNNs, enabling them to retain and learn long-term dependencies of inputs. This memory extension allows information to be remembered for a longer period, enabling the reading, writing, and deletion of information. LSTM memories are called "gated" units, the word "gated" inspired by the ability to make decisions about retaining or ignoring information. LSTM models capture important features from the input and retain this information over a long period. The decision to delete or retain information is based on the weights assigned to the information during training. Thus, the LSTM model learns which information is worth retaining or deleting. BiLSTM is an extension of the aforementioned LSTM model, where two LSTMs are applied to the input data. In the first round, the LSTM is applied to the input sequence (i.e., the feedforward layer). In the second round, the reversed form of the input sequence is fed into the LSTM model (i.e., the feedback layer). Applying LSTM twice can improve the learning of long-term dependencies, thereby increasing the accuracy of the model.

[0057] In this invention, compared to ODE-based models with encoder-decoder structures, the model derived from a typical recurrent model can predict online at each time step. Unlike standard RNNs, ODE-based RNNs learn the dynamics between observations; ODEs are suitable for inferring unknown physical information and can extract continuous features from data. This is achieved by extending the hidden state transitions in the RNN to continuous-time dynamics defined by the neural ODE. In our model, the transitions between hidden states and latent states are computed using GRU and Neural ODE.

[0058] In a second aspect, the present invention provides a propofol delivery device based on deep learning methods for pharmacokinetic-pharmacodynamic and reinforcement learning, the propofol delivery device being used to perform the propofol delivery method described in the first aspect.

[0059] Preferably, the device includes:

[0060] The vital signs data acquisition module is used to collect basic information about patients, including height, weight, gender and age, and to calculate the patient's lean body mass based on the basic information.

[0061] The drug administration control analysis module is used to determine the parameters of the pharmacokinetic-pharmacodynamic model, construct the pharmacokinetic-pharmacodynamic model, link the effective concentration of propofol with the clinically observed anesthesia depth index BIS, and establish a mapping between intraoperative monitoring information and anesthesia depth based on deep learning methods.

[0062] Preferably, the drug administration control and analysis module is specifically used to perform the following steps:

[0063] (2) Based on the collected patient information and the pharmacokinetic characteristics of propofol, determine the parameters of the pharmacokinetic-pharmacodynamic model, including the volume of each compartment, clearance rate and transfer rate;

[0064] (3) The pharmacokinetic-pharmacodynamic model includes three compartments: the central compartment, the fast peripheral compartment, and the slow peripheral compartment. Using the pre-defined parameters, a set of differential equations describing the concentration changes of propofol in the three compartments is constructed.

[0065] The pharmacokinetic-pharmacodynamic model is represented by a set of differential equations, as shown in equation (1):

[0066]

[0067] In the formula Let x1(t), x2(t), and x3(t) represent the drug concentration changes in the central chamber, rapid distribution chamber, and slow distribution chamber, respectively. Let u(t) be the drug infusion rate, and K be a constant. ij (i≠j) represents the drug transfer rate from compartment i to compartment j, and t represents time;

[0068] The transfer rate and the drug metabolism rate between different compartments are expressed as follows:

[0069]

[0070] In the formula, V1, V2, and V3 represent the volumes of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively, and C1, C2, and C3 represent the clearance rates of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively.

[0071] The pharmacodynamic model introduces an additional hypothetical effect compartment to represent the site of drug action, as shown in the following formula:

[0072]

[0073] In the formula, For the change in drug concentration in the effect room, x e x1(t) represents the drug concentration in the effect chamber, x1(t) represents the drug concentration in the central chamber, t represents time, and k is a constant. e The drug transfer rate from the effect chamber to the central chamber;

[0074] (4) Establishing a mapping between intraoperative monitoring information and anesthesia depth based on deep learning methods, including using drug infusion rate and patient covariate data as input variables, extracting time-dependent parameters Lx and time-continuous parameters Nx through LSTM and Neural ODE_GRU, establishing a nonlinear function based on Kan network, and obtaining the predicted anesthesia depth by inputting drug infusion rate. The patient covariates include the patient's age, gender, height, and weight.

[0075] Thirdly, the present invention provides a computer-readable storage medium storing a computer program or instructions that, when executed by a processor, implement the propofol administration method as described in the first aspect.

[0076] Fourthly, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program or instructions, which, when executed by the processor, implement the steps of the propofol administration method described in the first aspect.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] This invention provides a propofol administration method based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning approaches. A PKPD model is used to simulate the metabolic process of propofol in the human body. Time dependence and continuous kinetic information are extracted using LSTM and Neural OED. Then, a Kan network is used to establish a nonlinear relationship between the effective concentration of propofol and the clinically observed depth of anesthesia (BIS). A reinforcement learning algorithm is employed for accurate dosage calculation. The patient's current BIS, historical BIS values, and the dosing rate over a previous period are used as the state of the reinforcement learning algorithm's environment, while the dosing rate at the next moment is used as the algorithm's action. The agent constructed by the reinforcement learning algorithm can gradually optimize its action selection strategy during interaction with the environment, adjusting the strategy in a timely manner according to environmental changes. This is highly suitable for handling non-standardized complex problems. It rapidly establishes a mapping between intraoperative monitoring information and depth of anesthesia, and adjusts the strategy in a timely manner according to environmental changes. It is suitable for handling non-standardized complex problems, allowing for personalized adjustment of drug dosage and administration regimens, and improving therapeutic efficacy. Attached Figure Description

[0079] Figure 1 This is a structural diagram of the PKPD model.

[0080] Figure 2 This is a diagram illustrating the propofol administration process of this invention.

[0081] Figure 3This is a dataset partitioning diagram for this invention.

[0082] Figure 4 This is a graph showing the BIS prediction results and PPF administration results for a certain surgery during the testing process of this invention.

[0083] Figure 5 This is a diagram showing the control results of a surgical procedure during the testing process of this invention. Detailed Implementation

[0084] To further illustrate the technical means and effects of this invention, the following description, in conjunction with embodiments and accompanying drawings, provides a further explanation of the invention. It is understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it.

[0085] Example 1

[0086] This embodiment provides a propofol administration method based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning approaches, such as... Figure 1 and Figure 2 As shown, it includes the following steps:

[0087] (1) Construct a PKPD model and use this model as the environment for reinforcement learning algorithms to train agents. The PK model formula is as follows:

[0088]

[0089] In the formula Let x1(t), x2(t), and x3(t) represent the drug concentration changes in the central chamber, rapid distribution chamber, and slow distribution chamber, respectively. Let u(t) be the drug infusion rate, and K be a constant. ij (i≠j) represents the drug transfer rate from compartment i to compartment j, and t represents time;

[0090] The transfer rate and the drug metabolism rate between different compartments are expressed as follows:

[0091]

[0092] In the formula, V1, V2, and V3 represent the volumes of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively, and C1, C2, and C3 represent the clearance rates of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively.

[0093] The PD model formula is as follows:

[0094]

[0095] In the formula, For the change in drug concentration in the effect room, xe x1(t) represents the drug concentration in the effect chamber, x1(t) represents the drug concentration in the central chamber, t represents time, and k is a constant. e0 The drug transfer rate from the effect chamber to the central chamber.

[0096] (2) Collect basic information about the patient, including height, weight, gender and age, and calculate the patient’s lean body mass using a specific formula. This parameter is crucial for setting parameters for the subsequent pharmacokinetic model.

[0097] (3) Based on the collected patient information and the pharmacokinetic characteristics of propofol, determine the parameters in the three-compartment model, including the volume, clearance rate and transfer rate of each compartment. Using the set parameters, construct a set of differential equations to describe the concentration changes of propofol in the three compartments (central compartment, fast distribution compartment and slow distribution compartment) to ensure accurate simulation of drug concentration.

[0098] (4) Establishing a mapping between intraoperative monitoring information and anesthesia depth based on deep learning methods, including using drug infusion rate and patient covariate data as input variables, extracting time-dependent parameters Lx and time-continuous parameters Nx through LSTM and Neural ODE_GRU, and then inputting Lx, Nx and patient covariates into a Kan network to establish a nonlinear function. The patient covariates include the patient's age, gender, height and weight.

[0099] Step (4) specifically includes the following steps:

[0100] The concentration changes and drug concentrations of propofol in the three compartments, as well as the effect compartment, will be calculated using a pharmacokinetic-pharmacodynamic model based on patient covariates and drug infusion rate. The concentration changes and drug concentrations in the three compartments within 1 minute, as well as the effect compartment and effect compartment, will be combined with patient covariates to form the input variable X. t ;

[0101]

[0102] X t The data is fed into a BiLSTM model, which captures the nonlinear dependencies in the time series and learns the transition rules between different anesthesia stages to obtain the time dependency parameter Lx at this moment.

[0103] The LSTM (Gated Recurrent Unit) model is a recurrent neural network (RNN) architecture, and its calculation formula is shown below:

[0104] i t =σ(W ii x t +b ii +Whi h t-1 +b hi Equation (4.2);

[0105] f t =σ(W if x t +b if +W hf h t-1 +b hf Equation (4.3);

[0106] g t =tanh(W ig x t +b ig +W hg h t-1 +b hg Equation (4.4);

[0107] o t =σ(W io x t +b io +W ho h t-1 +b ho Equation (4.5);

[0108] c t =f t *c t-1 +i t *g t Equation (4.6);

[0109] h t =o t *tanh(c t Equation (4.7);

[0110] BiLSTM is an extension of the LSTM model described above. Two LSTMs are applied to the input data. In the first round, the LSTM is applied to the input sequence (i.e., the forward layer). In the second round, the reversed form of the input sequence is fed into the LSTM model (i.e., the backward layer). Applying LSTM twice improves the learning of long-term dependencies, thereby increasing the model's accuracy. The calculation formula is as follows:

[0111]

[0112] At the same time, X t When fed into the Neural ODE_GRU model, the ODE_GRU-based model encodes past trajectories as variables, yielding the current time-continuous parameter Nx, calculated as follows:

[0113]

[0114] N x (t)=MLP(h t-d ,...,h t Equation (4.14);

[0115] Lx, Nx, and the patient's covariates are input into the KAN network, and the predicted depth of anesthesia is obtained by inputting the drug infusion rate. The calculation formula is as follows:

[0116] Anesthetic_parameters t =[Lx t ,Nx t ,Age,Sex,Height,Weight] formula (4.15);

[0117] KAN(x)=(Φ3°Φ2°Φ1)(x) Equation (4.16);

[0118] Φ(x)=w(b(x)+spline(x)) Formula (4.17);

[0119]

[0120] Test Example 1

[0121] This test case applies the propofol administration method provided in Example 1. First, data was selected from patients aged 20-60 years. Intraoperative information was recorded in real-time using the BIS (Biological Information System) and propofol administration rate. Then, the BIS and propofol administration data were aligned temporally, with the latest recorded time point selected as the starting point. Next, based on the patient's age, height, weight, and gender, the PKPD model was used to calculate drug concentrations in different organs. Then, a sliding window technique was used with a 60-second window length and a 1-second step size to preprocess the drug concentrations at different sites and their per-second changes at different propofol administration rates. The BIS at the end of the window was used as the label. The dataset was randomly divided according to the number of surgeries (without shuffling the order within each surgery), with a training set to test set ratio of 8:2. The dataset partitioning is as follows. Figure 3 As shown. The model is trained using the training set, and then tested using the test set. The model is fine-tuned using the first 20% of data from each surgery, with the learning rate adjusted to 10% of its original value. After training, the model is tested on the remaining test set. The MAE (Mastering Experience) is 8.9, which is considered a relatively advanced algorithm. Visualization of a specific surgery from the test set is shown below. Figure 4As shown, the overall trend of almost perfectly fitting BIS can be observed. The reinforcement learning part uses the Proximal Policy Optimization (PPO) algorithm, and the environment interacting with the agent uses a BIS model and a three-compartment pharmacokinetic model. The observation space of the environment is BIS(t-2), BIS(t-1), BIS(t), PPF_v(t-1), and the action space of the environment is the propofol rate (PPF_v(t)). The reward function is set to... The reward here is the difference between the current BIS value (BIS=50) and the current BIS value after reinforcement learning, following a normal distribution. The training and testing processes then proceed. Figure 5 As the test results show, the target value can be maintained well.

[0122] In summary, this invention provides a propofol administration method based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning approaches. It uses a PKPD model to simulate the metabolic process of propofol in the human body, extracts time-dependent relationships and continuous kinetic information through LSTM and Neural OED, and then establishes a nonlinear relationship between the effective concentration of propofol and the clinically observed anesthesia depth index (BIS) using a Kan network. This rapidly establishes a mapping between intraoperative monitoring information and anesthesia depth, allowing for timely adjustments to the strategy based on environmental changes. It is suitable for handling complex, non-standardized problems, and allows for personalized adjustments to drug dosage and administration regimens, thereby improving therapeutic efficacy.

[0123] The applicant declares that the above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Those skilled in the art should understand that any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention fall within the protection and disclosure scope of the present invention.

Claims

1. A propofol administration method based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning methods, characterized in that, The propofol administration method includes the following steps: (1) Collect basic information about the patient, including height, weight, gender and age, and calculate the patient’s lean body mass based on the basic information; (2) Based on the collected patient information and the pharmacokinetic characteristics of propofol, determine the parameters of the pharmacokinetic-pharmacodynamic model, including the volume of each compartment, clearance rate and transfer rate; (3) The pharmacokinetic-pharmacodynamic model includes three compartments: the central compartment, the fast peripheral compartment, and the slow peripheral compartment. Using the pre-defined parameters, a set of differential equations describing the concentration changes of propofol in the three compartments is constructed. The pharmacokinetic-pharmacodynamic model is represented by a set of differential equations, as shown in equation (1): In the formula Let x1(t), x2(t), and x3(t) represent the drug concentration changes in the central chamber, rapid distribution chamber, and slow distribution chamber, respectively. Let u(t) be the drug infusion rate, and K be a constant. ij (i≠j) represents the drug transfer rate from compartment i to compartment j, and t represents time; The transfer rate and the drug metabolism rate between different compartments are expressed as follows: In the formula, V1, V2, and V3 represent the volumes of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively, and C1, C2, and C3 represent the clearance rates of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively. The pharmacodynamic model introduces an additional hypothetical effect compartment to represent the site of drug action, as shown in the following formula: In the formula, For the change in drug concentration in the effect room, x e x1(t) represents the drug concentration in the effect chamber, x1(t) represents the drug concentration in the central chamber, t represents time, and k is a constant. e The drug transfer rate from the effect chamber to the central chamber; (4) Establishing a mapping between intraoperative monitoring information and anesthesia depth based on deep learning methods, including using drug infusion rate and patient covariate data as input variables, extracting time-dependent parameters Lx and time-continuous parameters Nx through LSTM and Neural ODE_GRU, establishing a nonlinear function based on Kan network, and obtaining the predicted anesthesia depth by inputting drug infusion rate. The patient covariates include the patient's age, gender, height, and weight.

2. The propofol administration method according to claim 1, characterized in that, The input variable calculation method described in step (4) includes: The concentration changes and drug concentrations of propofol in the three compartments, as well as the effect compartment, will be calculated using a pharmacokinetic-pharmacodynamic model based on patient covariates and drug infusion rate. The concentration changes and drug concentrations in the three compartments within 1 minute, as well as the effect compartment and effect compartment, will be combined with patient covariates to form the input variable X. t .

3. The propofol administration method according to claim 2, characterized in that, The input variable X t The calculation formula is as follows:

4. The propofol administration method according to claim 1, characterized in that, The method for calculating the time-dependent parameter Lx includes: X t The data is fed into a BiLSTM model, which captures the nonlinear dependencies in the time series and learns the transition rules between different anesthesia stages to obtain the time-dependent parameter Lx at this moment.

5. The propofol administration method according to claim 1, characterized in that, The method for calculating the time continuity parameter Nx includes: X t When fed into the Neural ODE_GRU model, the ODE_GRU-based model encodes past trajectories as variables, yielding the current time-continuous parameter Nx.

6. The propofol administration method according to claim 1, characterized in that, The method for calculating the predicted depth of anesthesia includes: inputting Lx, Nx and the patient's covariates into a KAN network, and obtaining the predicted depth of anesthesia by inputting the drug infusion rate.

7. A propofol delivery device based on deep learning-based pharmacokinetic-pharmacodynamic and reinforcement learning methods, characterized in that, The propofol delivery device is used to perform the propofol delivery method according to any one of claims 1 to 6.

8. The apparatus according to claim 7, characterized in that, The device includes: The vital signs data acquisition module is used to collect basic information about patients, including height, weight, gender and age, and to calculate the patient's lean body mass based on the basic information. The drug administration control analysis module is used to determine the parameters of the pharmacokinetic-pharmacodynamic model, construct the pharmacokinetic-pharmacodynamic model, link the effective concentration of propofol with the clinically observed anesthesia depth index BIS, and establish a mapping between intraoperative monitoring information and anesthesia depth based on deep learning methods. The drug administration control analysis module is specifically used to perform the following steps: (2) Based on the collected patient information and the pharmacokinetic characteristics of propofol, determine the parameters of the pharmacokinetic-pharmacodynamic model, including the volume of each compartment, clearance rate and transfer rate; (3) The pharmacokinetic-pharmacodynamic model includes three compartments: the central compartment, the fast peripheral compartment, and the slow peripheral compartment. Using the pre-defined parameters, a set of differential equations describing the concentration changes of propofol in the three compartments is constructed. The pharmacokinetic-pharmacodynamic model is represented by a set of differential equations, as shown in equation (1): In the formula Let x1(t), x2(t), and x3(t) represent the drug concentration changes in the central chamber, rapid distribution chamber, and slow distribution chamber, respectively. Let u(t) be the drug infusion rate, and K be a constant. ij (i≠j) represents the drug transfer rate from compartment i to compartment j, and t represents time; The transfer rate and the drug metabolism rate between different compartments are expressed as follows: In the formula, V1, V2, and V3 represent the volumes of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively, and C1, C2, and C3 represent the clearance rates of the central chamber, the rapid distribution chamber, and the slow distribution chamber, respectively. The pharmacodynamic model introduces an additional hypothetical effect compartment to represent the site of drug action, as shown in the following formula: In the formula, For the change in drug concentration in the effect room, x e x1(t) represents the drug concentration in the effect chamber, x1(t) represents the drug concentration in the central chamber, t represents time, and k is a constant. e The drug transfer rate from the effect chamber to the central chamber; (4) Establishing a mapping between intraoperative monitoring information and anesthesia depth based on deep learning methods, including using drug infusion rate and patient covariate data as input variables, extracting time-dependent parameters Lx and time-continuous parameters Nx through LSTM and Neural ODE_GRU, establishing a nonlinear function based on Kan network, and obtaining the predicted anesthesia depth by inputting drug infusion rate. The patient covariates include the patient's age, gender, height, and weight.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions for causing a processor to execute the propofol administration method as described in any one of claims 1 to 6.

10. A computer device, characterized in that, The computer device includes a storage device and a processor. The storage device stores computer programs or instructions that, when executed by the processor, implement the steps of the propofol administration method according to any one of claims 1 to 6.

Images