Anesthesia depth prediction method based on representation learning

The anesthesia depth prediction method using representation learning with a causal transformer and bottleneck network improves anesthesia depth prediction accuracy by integrating medication dosage and patient-specific data, overcoming limitations of traditional PK-PD models.

CN119092127AActive Publication Date: 2024-12-06GUANGDONG UNIV OF TECH

Patent Information

Application Number
CN202411210494.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

Traditional pharmacokinetic-pharmacodynamic (PK-PD) models for anesthesia depth prediction fail to accurately account for individual differences and dynamic changes in drug distribution and mechanism, leading to inadequate anesthesia depth control.

Method used

An anesthesia depth prediction method using representation learning, incorporating a causal transformer and bottleneck network to extract and combine medication dosage time series characteristics with static patient information such as age, gender, and weight, to predict anesthesia depth index values.

Benefits of technology

Accurately predicts anesthesia depth by learning time series associations and static information correlations, enhancing prediction accuracy and addressing individual variability and dynamic changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119092127A_ABST
    Figure CN119092127A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of anesthesia depth prediction, and provides an anesthesia depth prediction method based on representation learning, and the method comprises the following steps: obtaining the dosage data of a to-be-anesthetized object and the static information including the age, gender, height and / or weight of the to-be-anesthetized object; constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model comprises a causal Transform and a bottleneck network; wherein the causal Transform is used for extracting dosage time sequence characteristics based on dosage data, and the bottleneck network is used for predicting an anesthesia depth index value in combination with the dosage time sequence characteristics and static information corresponding to the dosage time sequence characteristics; the anesthesia depth index prediction value of the to-be-anesthetized object obtained by using the anesthesia depth prediction model can accurately learn the time sequence association of the dosage and the association between the static information and the dosage time sequence characteristics, and the accuracy of the result is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of anesthesia depth prediction, and more specifically, to an anesthesia depth prediction method based on representation learning. Background Art

[0002] In modern medicine, anesthesia depth monitoring is essential to ensure the safety and comfort of patients during surgery. Too shallow anesthesia depth may cause patients to feel pain and discomfort during surgery, or even forget the surgical process; too deep anesthesia depth may cause serious complications such as respiratory depression and cardiovascular system depression. Therefore, accurate prediction and control of anesthesia depth is of great significance to anesthesiologists.

[0003] Traditional pharmacokinetic-pharmacodynamic (PK-PD) models are widely used in the prediction of anesthesia depth. The PK-PD model predicts and controls the depth of anesthesia by establishing the relationship between drug dose, plasma concentration and drug efficacy. However, these models are usually based on simple mathematical relationships, such as linear models, Emax models and Sigmoid Emax models, which cannot fully consider the distribution and mechanism of action of drugs at the effect site, as well as individual differences and dynamic changes of drugs. Although some scholars have proposed some improved models, such as mechanistic PK-PD models and integrated PK-PD models, these models still have some shortcomings. For example, the traditional PK-PD model mainly relies on empirical data and lacks a detailed explanation of the distribution and mechanism of action of drugs at the effect site. Secondly, there are significant differences in the absorption, distribution, metabolism and excretion of drugs in different patients, and it is difficult for traditional models to effectively capture these individual differences. Finally, the drug concentration and pharmacodynamic response during anesthesia change dynamically, and it is difficult for traditional models to accurately predict these dynamic changes. Summary of the invention

[0004] In order to overcome the defect of the above-mentioned prior art that the anesthesia depth cannot be accurately predicted, the present invention provides an anesthesia depth prediction method based on representation learning with high prediction accuracy.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] Obtaining medication dosage data of the subject to be anesthetized and static information including age, gender, height and / or weight of the subject to be anesthetized;

[0007] Constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract the dosage time series characteristics based on the dosage data, and the bottleneck network is used to combine the dosage time series characteristics and the static information corresponding to the dosage time series characteristics to predict the anesthesia depth index value;

[0008] The medication dosage data of the anesthetized subject and the static information of the anesthetized subject are input into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia subject.

[0009] The present invention also proposes an anesthesia depth prediction system based on representation learning for implementing the anesthesia depth prediction method based on representation learning, the system comprising:

[0010] A data acquisition module, used to acquire dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject;

[0011] A model building module, used to build an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract a dosage time series feature based on the dosage data, and the bottleneck network is used to combine the dosage time series feature and the static information corresponding to the dosage time series feature to predict the anesthesia depth index value;

[0012] The prediction module is used to input the dosage data of the anesthetized object and the static information of the anesthetized object into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia object.

[0013] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0014] The present application predicts anesthesia depth index values ​​by extracting medication dosage time series characteristics, combining the medication dosage time series characteristics and the static information corresponding to the medication dosage time series characteristics, and accurately learning the time series association of medication dosage and the association between the static information and the medication dosage time series characteristics, thereby achieving the purpose of obtaining an accurate anesthesia depth index prediction value using an anesthesia depth prediction model by only obtaining medication dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of the anesthesia depth prediction method based on representation learning proposed in Example 1;

[0016] Figure 2 This is a schematic diagram of the framework of the anesthesia depth prediction method based on representation learning proposed in Example 1;

[0017] Figure 3 This is the overall framework diagram of the anesthesia depth prediction system based on representation learning proposed in Example 3. DETAILED DESCRIPTION

[0018] The accompanying drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0019] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0020] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0021] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0022] Example 1

[0023] This embodiment proposes a method for predicting anesthesia depth based on representation learning. Figure 1 is a flow chart of the anesthesia depth prediction method based on representation learning of this embodiment;

[0024] A method for predicting anesthesia depth based on representation learning proposed in this embodiment includes the following steps:

[0025] S1: Obtaining medication dosage data of the anesthetized subject and static information including age, gender, height and / or weight of the anesthetized subject;

[0026] S2: constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract the dosage time series feature based on the dosage data, and the bottleneck network is used to combine the dosage time series feature and the static information corresponding to the dosage time series feature to predict the anesthesia depth index value;

[0027] S3: Inputting the dosage data of the anesthetized subject and the static information of the anesthetized subject into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs a predicted value of the anesthesia depth index of the anesthesia subject.

[0028] In the specific implementation process, by extracting the time series characteristics of medication dosage, combining the time series characteristics of medication dosage and the static information corresponding to the time series characteristics of medication dosage, the time series correlation of medication dosage and the correlation between static information and the time series characteristics of medication dosage are accurately learned, thereby achieving the purpose of obtaining the predicted value of anesthesia depth index using the anesthesia depth prediction model by only obtaining the medication dosage data of the anesthetized subject and the static information including the age, gender, height and / or weight of the anesthetized subject.

[0029] As an exemplary illustration, the drugs used include a combination of propofol and remifentanil.

[0030] In an optional embodiment, before using the anesthesia depth prediction model to predict the anesthesia depth index value of the anesthesia subject, a training set is collected, the training set is preprocessed, and the anesthesia depth prediction model is trained using the preprocessed training set;

[0031] The training set includes dosage data and anesthesia depth index values ​​corresponding to several historical anesthesia subjects, as well as static information including age, gender, height and / or weight of the historical anesthesia subjects.

[0032] As an example, data of 600 patients for total intravenous anesthesia were randomly selected from the open source database. The data were processed as follows: from the start of propofol or remifentanil infusion to the end of BIS measurement, data containing the following conditions were discarded: 1. When the BIS value at the start of drug infusion was lower than 80. 2. Data over 300s were lost. 3. When the first BIS was recorded, the cumulative amount of drug infused was not 0. After our data processing, the data of 332 patients remained and were randomly divided into training, validation and test data sets, with 180 cases in the training set, 76 cases in the validation set and 76 cases in the test set.

[0033] In an optional embodiment, the step of preprocessing the training set includes:

[0034] Removing invalid data from the training set, the invalid data including: data corresponding to historical anesthesia subjects whose anesthesia depth index value is lower than a preset value when the drug infusion starts, data corresponding to historical anesthesia subjects whose drug dosage data is lost for a time period exceeding a preset time, and data corresponding to historical anesthesia subjects whose drug dosage corresponding to the first anesthesia depth index value is not 0;

[0035] In the training set after removing invalid data, the anesthetic drug dosage data with missing data are filled by linear interpolation; for the anesthetic drug dosage data with erroneous data, the erroneous data are replaced by the mean of the sum of the previous moment and the next moment of the erroneous data; the anesthesia depth index value is smoothed by local weighted scatter plot smoothing.

[0036] In an optional embodiment, the step of preprocessing the training set further includes:

[0037] Sampling the anesthetic drug dosage data of several historical anesthesia subjects in the training set at preset time intervals, and recording the anesthesia depth index value corresponding to each sampling point, to form several time series corresponding to several historical anesthesia subjects, wherein the time series includes each sampling point, the cumulative dosage of each sampling point within a preset time period, and the anesthesia depth index value of each sampling point;

[0038] Each time series is divided into a number of time window sequences using a number of time window blocks. The expressions of the time window sequences include:

[0039]

[0040] In the formula, R P×N represents a real number field with dimension P×N, P represents the time window block length, S represents the step size of the non-overlapping region between two consecutive time window blocks, N represents the number of time window blocks, and L represents the length of the time window sequence; The time window block sequence corresponding to the time window block of length P of the i-th time series;

[0041] The time window sequence is shielded by a preset shielding number using a random mask method, and the unshielded time window sequence is divided into a time window subsequences in chronological order. The size of the data distribution difference between the two time window subsequences is measured, and the a time window subsequence is divided into a / 2 time window subsequences corresponding to the time periods with the largest data distribution differences between each two through a greedy algorithm. The anesthesia depth prediction model is trained using the time window subsequences.

[0042] As an exemplary illustration, in the prediction of anesthesia depth, the data of the maintenance period accounts for about 90% of the total data, while the data of the induction period and the recovery period only account for about 10%, resulting in poor prediction performance of the model for the induction period and the recovery period. Therefore, the present application addresses this data imbalance problem by pre-training the anesthesia depth data through self-masking reconstruction. Specifically, after the anesthesia depth sequence is divided into time window blocks, the time window blocks are masked at a high ratio, and then the complete anesthesia depth sequence is reconstructed through a small number of visible time window blocks to obtain effective common features of the three periods of anesthesia depth, so as to enhance the feature extraction capability of the model, improve the model performance and reduce the difficulty of model convergence. Thereby improving the prediction performance of the model for the induction period and the recovery period, and reducing the overfitting of the maintenance period.

[0043] As an exemplary description, this optional embodiment localizes the long-term sliding window into time window blocks by introducing causal convolution. The operation of the time window block enables the model to enhance the model's extraction of local features, thereby improving the model's ability to predict mutations.

[0044] As an illustrative example, the ability of Masked Signal Modeling (MSM) to learn representations from large-scale datasets was first used in natural language processing and later adapted to computer vision; successful applications to downstream tasks have shown that using MSM as a pre-training task to learn useful contextual knowledge; in essence, MSM is a generalized denoising autoencoder whose purpose is to recover the original data from the remaining data after masking; BIS (Bispectral Index, anesthesia depth index) data has implicit correlations with the effects of anesthetic drugs. Therefore, learning these implicit correlations by recovering the masked signal will equip the pre-trained model with a deep contextual understanding of the BIS data. The present invention divides the BIS data into time windows and then converts it into an embedding using a one-dimensional convolutional layer with a stride equal to the time window block size; it is then reconstructed through a Transformer model.

[0045] As an exemplary illustration, here, the idea of ​​quantifying the time series distribution is used. According to the maximum entropy principle, without making any prior assumptions about the segmentation of the time series data, in order to maximize the entropy of the total distribution, it is reasonable to make the distribution of each period as diverse as possible; this allows for more general and flexible modeling of future data; in addition, in order to maximize the use of shared knowledge under the time series, this can be achieved by finding the periods with the lowest similarity to each other; therefore, this problem can be expressed as follows:

[0046]

[0047] Where d is a distribution measure such as (Euclidean distance, cosine similarity), Δ1 and Δ2 are predefined parameters to avoid failure to capture distribution information, and K0 is a hyperparameter. The goal of this formula is to divide the input medication data into K periods, and the difference in the data distribution of medication in each period is the largest. We use a greedy algorithm to solve the K value of this formula.

[0048] As an example, each input univariate time series $x_{i}$ is first divided into overlapping or non-overlapping time window blocks. The time window block length is denoted as $P$ and the step length of the non-overlapping region between two consecutive time window blocks is denoted as $S$. After the time window block is generated, the time window block sequence is obtained. Where N is the number of time window blocks, Here, this chapter takes the last value before getting the time window block The number of repetitions is filled to the end of the original sequence. Using the time window block, the number of input tokens can be reduced from L to approximately L / S, which means that the memory usage and computational complexity of the attention map are quadratically reduced by S times. Therefore, the training time and GPU memory usage can be reduced. In addition, the time window block design allows the model to see a longer historical sequence, thereby improving prediction performance. Consistent with previous work, this chapter uses a random mask to shield the time window block with a shielding ratio of 75%. The other visible time window blocks are used as the Transformer encoder φ enc Finally, the hidden layer representation and learnable mask representation from the encoder and time window block are used as the Transformer decoder φ dec Input, get the reconstructed anesthesia depth signal y r , the loss function used here is the mean square error (MSE) loss function

[0049] As an example, using the time series distribution quantization method, assuming that the input data in Where n is the total length of the data, p is the number of features of the data, and m is the length of the data. i is the sequence length of each sampling point; if there is a teacher model and a student model, the number of features of the teacher model is 3, the number of features of the student model is 2, and the sequence length in the model is 120; first, the input data D is evenly divided into 10 segments, and the cosine similarity is used to measure the difference in data distribution between two time periods. We use a greedy algorithm to divide the 10 segments of input data into 5 time periods with the largest difference in data distribution between each two, which can be expressed as D = {D1, D2, D3, D4, D5}, where D1 represents the first time window subsequence.

[0050] As an exemplary illustration, the existing deep learning-based methods have poor prediction performance for the anesthesia depth trend in the induction period and the recovery period. In addition, the model is prone to overfitting for the maintenance period, which makes it easy to appear as a straight line during the maintenance period. The highlight of the present application is that the self-supervised strategy of self-mask pre-training uses a large amount of discarded data for self-mask reconstruction pre-training to obtain effective common features of the three periods of anesthesia depth (maintenance period, induction period and recovery period) to enhance the feature extraction ability of the model, improve model performance and reduce the difficulty of model convergence, thereby avoiding the problem of poor prediction performance of the model for the induction period and the recovery period, and the problem of overfitting for the maintenance period. The present application uses self-supervised pre-training to solve the problem of data imbalance in the three periods in the anesthesia depth sequence, thereby improving the feature extraction ability of the model.

[0051] In an optional embodiment, the step of training the anesthesia depth prediction model using the preprocessed training set includes:

[0052] The preprocessed training set is input into the anesthesia depth prediction model. The causal Transformer in the anesthesia depth prediction model extracts the dosage time series feature z based on the anesthetic drug dosage data and the anesthesia depth index value in the time window subsequence. (i) ;

[0053] The dosage time series characteristic x (i) The expressions include:

[0054]

[0055] In the formula, Attention(·) represents the attention mechanism, represents the query matrix corresponding to the time window subsequence input to the causal Transformer, Represents the key matrix corresponding to the time window subsequence of the input causal Transformer, represents the value matrix corresponding to the time window subsequence of the input causal Transformer, represents the scaling factor;

[0056] The bottleneck network transforms the dosage time series feature z (i) The static information corresponding to the dosage time series feature is spliced ​​into a spliced ​​feature, and the spliced ​​feature is passed through two fully connected neural networks and Relu activation functions in sequence to obtain the bottleneck network output T tout , the bottleneck network output T tout The predicted value of the anesthesia depth index of the historical anesthesia subject is obtained through a fully connected neural network with an output dimension of 1;

[0057] A loss function is set based on the predicted value of the anesthesia depth index to reduce the gap between the predicted value of the anesthesia depth index and the true value of the anesthesia depth index, and the loss function is iteratively solved. When the loss function value reaches a minimum or the number of iterations reaches a preset value, the iteration is terminated to obtain a trained anesthesia depth prediction model;

[0058] The medication dosage data of the anesthetized subject and the static information of the anesthetized subject are input into the trained anesthesia depth prediction model, and the trained anesthesia depth prediction model outputs the predicted value of the anesthesia depth index of the anesthetized subject.

[0059] As an example, the time series distribution is quantized to obtain D = D1, D2, D3, D4, D5, which is input into the causal Transformer. The input historical drug efficacy information x i First, the data in each time window is localized through causal convolution, so that the model can focus on multiple time points at the same time instead of a single time point. Each local time block obtained through causal convolution is used as the input of the Transformer encoder. Specifically, through the trainable linear projection W p ∈R D×P Map each local temporal block to a Transformer latent space of dimension D, with a learnable additional positional encoding W pos ∈R D×N , used to monitor the time sequence of local time blocks: Then each attention head of the multi-head attention, h=1,…,H, converts them into a query matrix key matrix Value Matrix in

[0060] , Afterwards, the scaled product is used to obtain the attention output Then the attention output generation is expressed as If there is a teacher model and a student model, the difference in the intermediate feature representation of the causal Transformer of the teacher model and the student model is calculated using cosine similarity and used as the regularization term of the student model loss function, denoted as L G (The fourth sub-loss function).

[0061] As an exemplary illustration, the causal Transformer refers to a Transformer model that enhances causal reasoning capabilities through pre-training and the introduction of causal convolutions. The use of the causal Transformer can capture the long-term dependency between drug injection history and anesthesia depth; the bottleneck network is a special neural network structure, which usually appears in the middle layer of a deep neural network and serves as an information bottleneck.

[0062] As an exemplary illustration, a causal Transformer model is proposed to solve the problem that the existing technology is unable to model long-term temporal relationships, while enhancing the model's ability to predict sudden changes in anesthesia depth.

[0063] As an exemplary illustration, the causal Transformer output and the corresponding patient's physiological characteristics are spliced ​​together as the output of the bottleneck network. The bottleneck network fully learns the relationship between the temporal feature information in the historical drug infusion data captured by the GRU and the corresponding human body characteristics. The bottleneck network structure is relatively simple, consisting of 2 fully connected layers, followed by a layer of Relu activation function. Finally, a fully connected layer is used to predict the anesthesia depth index (BIS), and finally the parameters required for training the network are obtained.

[0064] As an example, it is well known that RNN and its various variants are used to solve the problem of neural network's inability to capture long-term dependencies. However, studies have shown that the average effective context size of the language model using LSTM is about 200 tokens, but only 50 nearby tokens can be clearly distinguished, which shows that even LSTM has difficulty capturing long-term dependencies. At the same time, when predicting the depth of anesthesia from the history of drug injection, the long-term historical data of drug injection and the corresponding depth of anesthesia value are closely related to the depth of anesthesia at future moments; how to model long-term dependencies becomes a key step to achieve excellent performance; on the other hand, the current model aims to learn the relationship between historical drug injection data and depth of anesthesia within a time window, but the time points spanned in the time window are too long, which is not conducive to learning the mutation information in the three periods of anesthesia, so extracting local information is crucial to predict the local mutation of the depth of anesthesia; therefore, the present application uses the Transformer model to capture the long-term dependency between the history of drug injection and the depth of anesthesia. In addition, the present application also uses a time window block and a self-attention enhancement model through causal convolution to extract local features.

[0065] As an example, the causal transformer can learn the prediction knowledge of each period, but cannot reduce the distribution differences between different periods to utilize common knowledge. In order to match the time series distribution, the importance vector α is first introduced to evaluate the importance of each hidden state. Through α, the network can pay attention to the distribution differences between different periods, thereby reducing the distribution differences between different periods and achieving time series distribution matching. The output dimensions of the causal transformer model are merged into G teacher ∈R 256×7680 ,G student ∈R 256×7680 , passes through a fully connected layer with an output dimension of 120, then passes through a sigmoid activation function, and then is normalized by Softmax to obtain an importance vector α. Then we take the output of the GRU and α to calculate the distribution difference between different hidden states as the regular term L of the model's loss function t (Second loss function and first sub-loss function).

[0066] As an example, the intermediate features of the output of the causal Transformer model are represented as With the patient’s static information S∈R 256×4 (Height, weight, gender, age) can be expressed as T teacher ∈R 256×68 , T student ∈R 256×68 , and then pass through a fully connected neural network with an output dimension of 128, and then pass through a fully connected neural network with an output dimension of 128, and then pass through the Relu activation function to get T tout ,T sout ∈R 256×128 , including dynamic time series information features and static patient physiological characteristics information. Then use cosine similarity to calculate the bottleneck network output T of the two models tout ,T sout ∈R 256×128 The similarity of is also used as the regular term of the student network loss function, denoted by L S (Fifth sub-loss function).

[0067] In an optional embodiment, the loss function includes a first loss function, a second loss function and a third loss function;

[0068] The expression of the first loss function includes:

[0069]

[0070] Where, L tpred (θ) represents the first loss function, θ represents the parameters of the anesthesia depth prediction model, D jrepresents the j-th time window subsequence, Indicates D j The corresponding predicted value of anesthesia depth index, express The corresponding true value of the anesthesia depth index, l(·) represents the mean square error loss function, M t (·) represents the teacher network, K represents the total number of time window subsequences;

[0071] The expression of the second loss function includes:

[0072]

[0073] Where, L t (D i ,D j ; θ) represents the second loss function, D i represents the i-th time window subsequence, Indicates that D is in state t i and D j Importance parameter for the two time period distributions, and Respectively represent D i and D j The attention head of the causal Transformer corresponding to state t, d(·) represents the cosine similarity calculation function, and H represents the number of multi-head attention;

[0074] The third loss function includes:

[0075]

[0076] In the formula, represents the third loss function, Ω represents the set of masked time window sequences, x represents the information in the time window sequence input to the causal Transformer, and y r Represents the information reconstructed by the causal Transformer.

[0077] In an optional embodiment, the trained anesthesia depth prediction model is regarded as a teacher model;

[0078] Constructing a student model, wherein the student model includes a causal Transformer and a bottleneck network;

[0079] The medication dosage data corresponding to several historical anesthesia subjects in the training set and static information including the age, gender, height and / or weight of the historical anesthesia subjects are input into the student model as training data of the student model, and the student model outputs the predicted value of the anesthesia depth index of the historical anesthesia subjects;

[0080] A fourth loss function for reducing the gap between the predicted value of the anesthesia depth index output by the student model and the true value of the anesthesia depth index is set based on the predicted value of the anesthesia depth index, and a fifth loss function for reducing the gap between the predicted value of the anesthesia depth index output by the student model and the teacher model;

[0081] Iteratively solving the fourth loss function and the fifth loss function, and when the number of iterations reaches a preset value, ending the iteration to obtain a student model that has completed training;

[0082] The medication dosage data of the anesthetized subject and the static information of the anesthetized subject are input into the trained student model, and the trained student model outputs the predicted value of the anesthesia depth index of the anesthetized subject.

[0083] As an example, there may be a small number of missing values ​​in the sample data of the anesthetized subjects in the training set. Since PPF (propofol) and RFTN (Remifentanil) have a linear relationship with time, linear interpolation is used to fill in the small number of missing values ​​in the sample of the anesthetized subjects. The total dosage of PPF and RFNT is monotonically non-decreasing, so if there is a decrease in the total dosage of the anesthetized subjects' data samples due to problems with instrument recording or data export, we use the mean of the moment before and after that moment to fill in the total dosage.

[0084] As an example, the target-controlled injection pump records the cumulative injection volume (dose of medication). The injection history data retrieved from the target-controlled injection pump is updated every 10 seconds, so the cumulative drug dosage of PPF and RFTN within 10 seconds is used as a feature to establish a time window with a sequence length of 120; because the data changes of different anesthetized objects are too different, it is not conducive to network learning; at the same time, in order to save computing resources, the training data is downsampled, that is, a sampling point is selected every 10 seconds; and because the BIS values ​​of different anesthetized objects have different trends, and the BIS values ​​of each anesthetized object change frequently, we smoothed the BIS values ​​of the training set, using local weighted scatter plot smoothing (LOWESS) with a smoothing parameter of 0.03 to reduce the calculation error during training. In addition, we also introduced the BIS values ​​of historical moments as features in the teacher model, which can guide the teacher model to learn the trend of BIS value changes. The unprocessed BIS values ​​are used for verification and test data sets.

[0085] As an example, Figure 2 This is a schematic diagram of the framework of the anesthesia depth prediction method based on representation learning proposed in this embodiment. Figure 2 The distillation in represents knowledge distillation, such as Figure 2As shown, the two networks are trained as teacher model M t and Student Model M s , both models have the same framework, and the training process is performed according to the above process. The difference is that the teacher model not only inputs the history of drug infusion, but also inputs the BIS value at the historical moment to obtain a richer intermediate layer feature representation of temporal information; therefore, we perform knowledge distillation based on the intermediate layer feature representation. Specifically, we let the output of the student model imitate the output of the teacher model to obtain a more accurate feature representation of the BIS value change trend, and we make the distribution of the output feature representation of the bottleneck network of the student model and the distribution of the output feature representation of the bottleneck network of the teacher model as similar as possible. The teacher model has historical BIS data characteristics and can more accurately reflect the impact of static physiological information and dynamic drug information on BIS. Therefore, we make the feature representation of the output of the student bottleneck network imitate the feature representation of the output of the teacher bottleneck network to reduce the distribution difference between the two. In addition, the structural complexity and number of layers of the student model are lighter than those of the teacher model. Specifically, the number of GRU layers of the teacher model is 5, while the number of GRU network layers of the student model is only 1.

[0086] As an example, the missing data of propofol and remifentanil in the training set are filled with linear interpolation, and the mean of the previous and next moments of the recorded erroneous data is modified, and the BIS value is smoothed by local weighted scatter plot smoothing (LOWESS) with a smoothing parameter of 0.03. After the processing is completed, the cumulative dosage of propofol and remifentanil within 10s for each sampling point is used as the time series feature, especially for the teacher model, the BIS value of the previous moment of the prediction time is also used as the time series feature; according to the time series features of the teacher model and the student model, a time window with a sequence length of 120 is established as dynamic feature information.

[0087] In an optional embodiment, the fourth loss function includes a first sub-loss function, a second sub-loss function and a third sub-loss function;

[0088] The expression of the first sub-loss function includes:

[0089]

[0090] Where, L t (D i ,D j ; θ) represents the first sub-loss function, D i represents the i-th time window subsequence, Indicates that D is in state t i and D j Importance parameter for the two time period distributions, and Respectively represent D i and D j The attention head of the causal Transformer of the student model corresponding to state t, d(·) represents the cosine similarity calculation function, and H represents the number of multi-head attention heads;

[0091] The expression of the second sub-loss function includes:

[0092]

[0093] Where, L spred (θ1) represents the second sub-loss function, θ1 represents the parameters of the student model, D j represents the j-th time window subsequence, Indicates D j The corresponding predicted value of the anesthesia depth index output by the student model, express The corresponding true value of the anesthesia depth index, l(·) represents the mean square error loss function, M s (·) represents the student network, K represents the total number of time window subsequences;

[0094] The expression of the third sub-loss function includes:

[0095]

[0096] In the formula, represents the third sub-loss function, Ω represents the set of masked time window sequences, x represents the information in the time window sequence input to the causal Transformer, and Y r Information representing the causal Transformer reconstruction of the student model.

[0097] In an optional embodiment, the fifth loss function includes a fourth sub-loss function and a fifth sub-loss function;

[0098] The expression of the fourth sub-loss function includes:

[0099]

[0100] Where, L G represents the fourth sub-loss function, d(·) represents the cosine similarity calculation function, z (i) represents the time series features of medication dosage extracted by the teacher model, represents the time series characteristics of medication dosage extracted by the student model;

[0101] The expression of the fifth sub-loss function includes:

[0102] L S =d(Ttout ,T sout )

[0103] Where, L S represents the fifth sub-loss function, d(·) represents the cosine similarity calculation function, T tout , represents the bottleneck network output of the teacher model, T sout Represents the bottleneck network output of the student model.

[0104] As an example, the number of model iterations is set to 100, the number of early stopping iterations is set to 40, and the teacher model M t and Student Model M s Training is performed simultaneously, and the model parameters are updated according to the loss functions corresponding to the teacher model and the student model until the number of iterations is greater than 40 and the validation set loss no longer decreases or the number of iterations reaches 100. The student model parameters with the lowest validation set loss are taken as the final model, and then the student model is used to test the test set data.

[0105] Example 2

[0106] This embodiment proposes an anesthesia depth prediction system based on representation learning, which is used to implement an anesthesia depth prediction method based on representation learning proposed in Example 1.

[0107] Figure 3 This is the overall framework diagram of the anesthesia depth prediction system based on representation learning in this embodiment.

[0108] The anesthesia depth prediction system based on representation learning comprises:

[0109] A data acquisition module, used to acquire dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject;

[0110] A model building module, used to build an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract a dosage time series feature based on the dosage data, and the bottleneck network is used to combine the dosage time series feature and the static information corresponding to the dosage time series feature to predict the anesthesia depth index value;

[0111] The prediction module is used to input the dosage data of the anesthetized object and the static information of the anesthetized object into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia object.

[0112] It can be understood that the anesthesia depth prediction system based on representation learning in this embodiment improves the method of Example 1. The options in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0113] Example 3

[0114] This embodiment proposes a computer device for implementing an anesthesia depth prediction method based on representation learning proposed in Embodiment 1, comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the anesthesia depth prediction method based on representation learning described in Embodiment 1.

[0115] It can be understood that the anesthesia depth prediction system and computer device based on representation learning in this embodiment improves the method of Example 1. The options in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0116] The same or similar reference numerals correspond to the same or similar components;

[0117] The terms used to describe the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limiting the present embodiment;

[0118] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for predicting anesthesia depth based on representation learning, characterized in that: The following steps are involved: Obtaining medication dosage data of the subject to be anesthetized and static information including age, gender, height and / or weight of the subject to be anesthetized; Constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract the dosage time series characteristics based on the dosage data, and the bottleneck network is used to combine the dosage time series characteristics and the static information corresponding to the dosage time series characteristics to predict the anesthesia depth index value; The medication dosage data of the anesthetized subject and the static information of the anesthetized subject are input into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia subject.

2. The anesthesia depth prediction method based on representation learning according to claim 1, characterized in that: Before using the anesthesia depth prediction model to predict the anesthesia depth index value of the anesthesia subject, collecting a training set, preprocessing the training set, and using the preprocessed training set to train the anesthesia depth prediction model; The training set includes dosage data and anesthesia depth index values ​​corresponding to several historical anesthesia subjects, as well as static information including age, gender, height and / or weight of the historical anesthesia subjects.

3. The anesthesia depth prediction method based on representation learning according to claim 2, characterized in that: The step of preprocessing the training set includes: Removing invalid data from the training set, the invalid data including: data corresponding to historical anesthesia subjects whose anesthesia depth index value is lower than a preset value when the drug infusion starts, data corresponding to historical anesthesia subjects whose drug dosage data is lost for a time period exceeding a preset time, and data corresponding to historical anesthesia subjects whose drug dosage corresponding to the first anesthesia depth index value is not 0; In the training set after removing invalid data, the anesthetic drug dosage data with missing data are filled by linear interpolation; for the anesthetic drug dosage data with erroneous data, the erroneous data are replaced by the mean of the sum of the previous moment and the next moment of the erroneous data; the anesthesia depth index value is smoothed by local weighted scatter plot smoothing.

4. The anesthesia depth prediction method based on representation learning according to claim 3, characterized in that: The step of preprocessing the training set also includes: Sampling the anesthetic drug dosage data of several historical anesthesia subjects in the training set at preset time intervals, and recording the anesthesia depth index value corresponding to each sampling point, to form several time series corresponding to several historical anesthesia subjects, wherein the time series includes each sampling point, the cumulative dosage of each sampling point within a preset time period, and the anesthesia depth index value of each sampling point; Each time series is divided into a number of time window sequences using a number of time window blocks. The expressions of the time window sequences include: In the formula, R P×N represents a real number field with dimension P×N, P represents the time window block length, S represents the step size of the non-overlapping region between two consecutive time window blocks, N represents the number of time window blocks, and L represents the length of the time window sequence; The time window block sequence corresponding to the time window block of length P of the i-th time series; The time window sequence is shielded by a preset shielding number using a random mask method, and the unshielded time window sequence is divided into a time window subsequences in chronological order. The size of the data distribution difference between the two time window subsequences is measured, and the a time window subsequence is divided into a / 2 time window subsequences corresponding to the time periods with the largest data distribution differences between each two through a greedy algorithm. The anesthesia depth prediction model is trained using the time window subsequences.

5. The anesthesia depth prediction method based on representation learning according to claim 4, characterized in that: The step of training the anesthesia depth prediction model using the preprocessed training set includes: The preprocessed training set is input into the anesthesia depth prediction model. The causal Transformer in the anesthesia depth prediction model extracts the dosage time series feature z based on the anesthetic drug dosage data and the anesthesia depth index value in the time window subsequence. (i) ; The dosage time series characteristics z (i) The expressions include: In the formula, Attention(·) represents the attention mechanism, represents the query matrix corresponding to the time window subsequence input to the causal Transformer, Represents the key matrix corresponding to the time window subsequence of the input causal Transformer, represents the value matrix corresponding to the time window subsequence of the input causal Transformer, represents the scaling factor; The bottleneck network transforms the dosage time series feature z (i) The static information corresponding to the dosage time series feature is spliced ​​into a spliced ​​feature, and the spliced ​​feature is passed through two fully connected neural networks and Relu activation functions in sequence to obtain the bottleneck network output T tout , the bottleneck network output T tout The predicted value of the anesthesia depth index of the historical anesthesia subjects is obtained through a fully connected neural network with an output dimension of 1; A loss function is set based on the predicted value of the anesthesia depth index to reduce the gap between the predicted value of the anesthesia depth index and the true value of the anesthesia depth index, and the loss function is iteratively solved. When the loss function value reaches a minimum or the number of iterations reaches a preset value, the iteration is terminated to obtain a trained anesthesia depth prediction model; The medication dosage data of the anesthetized subject and the static information of the anesthetized subject are input into the trained anesthesia depth prediction model, and the trained anesthesia depth prediction model outputs the predicted value of the anesthesia depth index of the anesthetized subject.

6. The anesthesia depth prediction method based on representation learning according to claim 5, characterized in that: The loss function includes a first loss function, a second loss function and a third loss function; The expression of the first loss function includes: Where, L tpred (θ) represents the first loss function, θ represents the parameters of the anesthesia depth prediction model, D j represents the j-th time window subsequence, Indicates D j The corresponding predicted value of anesthesia depth index, express The corresponding true value of the anesthesia depth index, l(·) represents the mean square error loss function, M t (·) represents the teacher network, K represents the total number of time window subsequences; The expression of the second loss function includes: Where, L t (D i ,D j ; θ) represents the second loss function, D i represents the i-th time window subsequence, Indicates that D is in state t i and D j Importance parameter for the two time period distributions, and Respectively represent D i and D j The attention head of the causal Transformer corresponding to state t, d(·) represents the cosine similarity calculation function, and H represents the number of multi-head attention heads; The third loss function includes: In the formula, represents the third loss function, Ω represents the set of masked time window sequences, x represents the information in the time window sequence input to the causal Transformer, and y r Represents the information reconstructed by the causal Transformer.

7. The anesthesia depth prediction method based on representation learning according to any one of claims 2 to 6, characterized in that: The anesthesia depth prediction model that has completed the training is regarded as a teacher model; Constructing a student model, wherein the student model includes a causal Transformer and a bottleneck network; The medication dosage data corresponding to several historical anesthesia subjects in the training set and static information including the age, gender, height and / or weight of the historical anesthesia subjects are input into the student model as training data of the student model, and the student model outputs the predicted value of the anesthesia depth index of the historical anesthesia subjects; A fourth loss function for reducing the gap between the predicted value of the anesthesia depth index output by the student model and the true value of the anesthesia depth index is set based on the predicted value of the anesthesia depth index, and a fifth loss function for reducing the gap between the predicted value of the anesthesia depth index output by the student model and the teacher model; Iteratively solving the fourth loss function and the fifth loss function, and when the number of iterations reaches a preset value, ending the iteration to obtain a student model that has completed training; The medication dosage data of the anesthetized subject and the static information of the anesthetized subject are input into the trained student model, and the trained student model outputs the predicted value of the anesthesia depth index of the anesthetized subject.

8. The anesthesia depth prediction method based on representation learning according to claim 7, characterized in that: The fourth loss function includes a first sub-loss function, a second sub-loss function and a third sub-loss function; The expression of the first sub-loss function includes: Where, L t (D i , D j ; θ) represents the first sub-loss function, D i represents the i-th time window subsequence, Indicates that D is in state t i and D j Importance parameter for the two time period distributions, and Respectively represent D i and D j The attention head of the causal Transformer of the student model corresponding to state t, d(·) represents the cosine similarity calculation function, and H represents the number of multi-head attention heads; The expression of the second sub-loss function includes: Where, L spred (θ1) represents the second sub-loss function, θ1 represents the parameters of the student model, D j represents the j-th time window subsequence, Indicates D j The corresponding predicted value of the anesthesia depth index output by the student model, express The corresponding true value of the anesthesia depth index, l(·) represents the mean square error loss function, M s (·) represents the student network, K represents the total number of time window subsequences; The expression of the third sub-loss function includes: In the formula, represents the third sub-loss function, Ω represents the set of masked time window sequences, x represents the information in the time window sequence input to the causal Transformer, and Y r Information representing the causal Transformer reconstruction of the student model.

9. The anesthesia depth prediction method based on representation learning according to claim 7, characterized in that: The fifth loss function includes a fourth sub-loss function and a fifth sub-loss function; The expression of the fourth sub-loss function includes: Where, L G represents the fourth sub-loss function, d(·) represents the cosine similarity calculation function, z (i) represents the time series features of medication dosage extracted by the teacher model, represents the time series characteristics of medication dosage extracted by the student model; The expression of the fifth sub-loss function includes: L S =d(T tout, T sout ) Where, L S represents the fifth sub-loss function, d(·) represents the cosine similarity calculation function, T tout, represents the bottleneck network output of the teacher model, T sout Represents the bottleneck network output of the student model.

10. An anesthesia depth prediction system based on representation learning, used to implement the anesthesia depth prediction method based on representation learning according to any one of claims 1 to 9, characterized in that: include: A data acquisition module, used to acquire drug dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject; A model building module, used to build an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract a dosage time series feature based on the dosage data, and the bottleneck network is used to combine the dosage time series feature and the static information corresponding to the dosage time series feature to predict the anesthesia depth index value; The prediction module is used to input the dosage data of the anesthetized object and the static information of the anesthetized object into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia object.

Citation Information

Patent Citations

  • Anesthesia depth prediction method

    CN115444366A

  • Anesthesia depth prediction method based on recurrent neural network in combination with transfer learning and knowledge distillation

    CN115641966A

  • Artificial intelligence anesthesia management system

    CN116504413A

  • System and method for predicting depth of anesthesia

    WO2021110879A1

Cited By

  • Embolism and hemorrhage risk prediction method and system based on causal Transform model

    CN120183709A

  • Anesthesia depth accurate prediction system and method based on time sequence alignment

    CN121483484A