A method for predicting anesthesia depth based on representation learning

Abstract

The present invention relates to the technical field of anesthesia depth prediction, and proposes an anesthesia depth prediction method based on representation learning, comprising the following steps: obtaining medication dosage data of an anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject; constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model comprises a causal transformer and a bottleneck network; wherein the causal transformer is used to extract medication dosage time series features based on medication dosage data, and the bottleneck network is used to combine medication dosage time series features and static information corresponding to the medication dosage time series features to predict an anesthesia depth index value; the anesthesia depth index prediction value of the anesthesia subject obtained by using the anesthesia depth prediction model can accurately learn the time series association of medication dosage and the association between static information and medication dosage time series features, and the result has high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of anesthesia depth prediction, and more particularly, to an anesthesia depth prediction method based on representation learning. Background Art

[0002] In modern medicine, monitoring the depth of anesthesia is crucial for ensuring patient safety and comfort during surgery. Too shallow anesthesia can cause pain and discomfort during surgery, and even obscure the patient's memory of the procedure. Too deep anesthesia can lead to serious complications, such as respiratory and cardiovascular depression. Therefore, accurately predicting and controlling the depth of anesthesia is crucial for anesthesiologists.

[0003] Traditional pharmacokinetic-pharmacodynamic (PK-PD) models are widely used to predict anesthesia depth. PK-PD models predict and control anesthesia depth by establishing relationships between drug dose, plasma concentration, and drug efficacy. However, these models are typically based on simple mathematical relationships, such as linear models, Emax models, and sigmoid Emax models, and fail to fully account for drug distribution and mechanism of action at the effect site, as well as individual variability and dynamic changes in drug behavior. Although improved models, such as mechanistic PK-PD models and integrative PK-PD models, have been proposed, these models still have several shortcomings. For example, traditional PK-PD models primarily rely on empirical data and lack a detailed explanation of drug distribution and mechanism of action at the effect site. Second, drug absorption, distribution, metabolism, and excretion vary significantly between patients, making it difficult for traditional models to effectively capture these individual differences. Finally, drug concentration and pharmacodynamic responses during anesthesia vary dynamically, making it difficult for traditional models to accurately predict these dynamic changes. Summary of the Invention

[0004] In order to overcome the defect of the above-mentioned prior art that the depth of anesthesia cannot be accurately predicted, the present invention provides an anesthesia depth prediction method based on representation learning with high prediction accuracy.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0006] Obtaining medication dosage data and static information including age, gender, height and / or weight of the anesthetized subject;

[0007] Constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract medication dosage time series features based on the medication dosage data, and the bottleneck network is used to combine the medication dosage time series features and static information corresponding to the medication dosage time series features to predict the anesthesia depth index value;

[0008] The drug dosage data and static information of the anesthetized subject are input into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs an anesthesia depth index prediction value of the anesthesia subject.

[0009] The present invention also proposes a representation learning-based anesthesia depth prediction system for implementing the above-mentioned representation learning-based anesthesia depth prediction method, the system comprising:

[0010] A data acquisition module is used to obtain drug dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject;

[0011] A model construction module, configured to construct an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model comprises a causal transformer and a bottleneck network; wherein the causal transformer is configured to extract a dosage time series feature based on the dosage data, and the bottleneck network is configured to combine the dosage time series feature and static information corresponding to the dosage time series feature to predict the anesthesia depth index value;

[0012] The prediction module is used to input the drug dosage data and static information of the anesthetized subject into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia subject.

[0013] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0014] The present application extracts the time series characteristics of medication dosage, combines the said medication dosage time series characteristics and the static information corresponding to the said medication dosage time series characteristics to predict the anesthesia depth index value, accurately learns the time series association of medication dosage and the association between the static information and the medication dosage time series characteristics, thereby achieving the purpose of obtaining an accurate anesthesia depth index prediction value using the anesthesia depth prediction model by only obtaining the medication dosage data of the anesthetized subject and the static information including the age, gender, height and / or weight of the anesthetized subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of the anesthesia depth prediction method based on representation learning proposed in Example 1;

[0016] Figure 2 This is a schematic diagram of the framework of the anesthesia depth prediction method based on representation learning proposed in Example 1;

[0017] Figure 3 This is the overall framework diagram of the anesthesia depth prediction system based on representation learning proposed in Example 3. DETAILED DESCRIPTION

[0018] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present embodiment;

[0019] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0020] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0021] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0022] Example 1

[0023] This embodiment proposes a method for predicting anesthesia depth based on representation learning. Figure 1 Schematic diagram of the flow of the anesthesia depth prediction method based on representation learning in this embodiment;

[0024] A method for predicting anesthesia depth based on representation learning proposed in this embodiment includes the following steps:

[0025] S1: Obtaining medication dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject;

[0026] S2: constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract medication dosage time series features based on the medication dosage data, and the bottleneck network is used to combine the medication dosage time series features and static information corresponding to the medication dosage time series features to predict the anesthesia depth index value;

[0027] S3: Inputting the medication dosage data and static information of the anesthetized subject into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs a predicted value of the anesthesia depth index of the anesthetized subject.

[0028] During the specific implementation process, by extracting the time series characteristics of the medication dosage, combining the time series characteristics of the medication dosage and the static information corresponding to the time series characteristics of the medication dosage to predict the anesthesia depth index value, the time series association of the medication dosage and the association between the static information and the time series characteristics of the medication dosage are accurately learned, thereby achieving the purpose of obtaining the anesthesia depth index prediction value using the anesthesia depth prediction model by only obtaining the medication dosage data of the anesthetized subject and the static information including the age, gender, height and / or weight of the anesthetized subject.

[0029] As an example, the drugs used include a combination of propofol and remifentanil.

[0030] In an optional embodiment, before using the anesthesia depth prediction model to predict the anesthesia depth index value of the anesthesia subject, a training set is collected, the training set is preprocessed, and the anesthesia depth prediction model is trained using the preprocessed training set;

[0031] The training set includes medication dosage data, anesthesia depth index values, and static information including age, gender, height and / or weight of several historical anesthesia subjects.

[0032] As an example, we randomly selected data from an open-source database for 600 patients undergoing total intravenous anesthesia. We then processed this data from the start of propofol or remifentanil infusion to the end of BIS measurement. Data containing the following conditions were discarded: 1. BIS values ​​below 80 at the start of drug infusion; 2. Data exceeding 300 seconds were discarded; 3. The cumulative drug infusion volume was not zero when the first BIS was recorded. After this data processing, data from 332 patients remained and were randomly divided into training, validation, and test datasets, with 180 cases in the training set and 76 cases in both the validation and test sets.

[0033] In an optional embodiment, the step of preprocessing the training set includes:

[0034] Removing invalid data from the training set, the invalid data including: data corresponding to historical anesthesia subjects whose anesthesia depth index value was lower than a preset value when the drug infusion started, data corresponding to historical anesthesia subjects whose drug dosage data was missing for more than a preset time period, and data corresponding to historical anesthesia subjects whose drug dosage corresponding to the first anesthesia depth index value was not zero;

[0035] In the training set after removing invalid data, the anesthetic drug dosage data with missing data were filled by linear interpolation; the anesthetic drug dosage data with erroneous data were replaced by the mean of the sum of the previous moment and the next moment of the erroneous data; the anesthesia depth index value was smoothed by local weighted scatter plot smoothing.

[0036] In an optional embodiment, the step of preprocessing the training set further includes:

[0037] Sampling the anesthetic drug dosage data of several historical anesthesia subjects in the training set at preset time intervals, and recording the anesthesia depth index value corresponding to each sampling point to form a plurality of time series corresponding to the several historical anesthesia subjects, the time series including each sampling point, the cumulative dosage of each sampling point within a preset time period, and the anesthesia depth index value of each sampling point;

[0038] Each time series is divided into several time window sequences using several time window blocks. The expressions of the time window sequences include:

[0039]

[0040] Where R P×N represents a real number field with dimension P×N, P represents the time window block length, S represents the step size of the non-overlapping region between two consecutive time window blocks, N represents the number of time window blocks, and L represents the length of the time window sequence; Represents the time window block sequence corresponding to the time window block of length P of the i-th time series;

[0041] The time window sequence is shielded with a preset number of shielding numbers using a random mask method, and the unshielded time window sequence is divided into a time window subsequences in chronological order. The size of the data distribution difference between the two time window subsequences is measured, and the a time window subsequence is divided into a / 2 time window subsequences corresponding to the time periods with the largest data distribution differences between each two using a greedy algorithm. The anesthesia depth prediction model is trained using the time window subsequences.

[0042] As an example, in the prediction of anesthesia depth, the data of the maintenance period accounts for about 90% of the total data, while the data of the induction period and the recovery period only account for about 10%, resulting in poor prediction performance of the model for the induction period and the recovery period. Therefore, this application addresses this data imbalance problem by pre-training the anesthesia depth data through self-masking reconstruction. Specifically, after dividing the anesthesia depth sequence into time window blocks, the time window blocks are masked at a high ratio, and then the complete anesthesia depth sequence is reconstructed through a small number of visible time window blocks to obtain effective common features of the three periods of anesthesia depth, so as to enhance the feature extraction ability of the model, improve the model performance and reduce the difficulty of model convergence. Thereby improving the prediction performance of the model for the induction period and the recovery period, and reducing the overfitting of the maintenance period.

[0043] As an exemplary illustration, this optional embodiment localizes the long-term sliding window into a time window block by introducing causal convolution. The operation of the time window block enables the model to enhance the model's extraction of local features, thereby improving the model's ability to predict mutations.

[0044] As an illustrative example, the ability of Masked Signal Modeling (MSM) to learn representations from large-scale datasets was first used in natural language processing and later adapted to computer vision; successful applications to downstream tasks demonstrate the use of MSM as a pre-training task to learn useful contextual knowledge; in essence, MSM is a generalized denoising autoencoder whose purpose is to recover the original data from the remaining data after masking; BIS (Bispectral Index, anesthesia depth index) data has implicit correlations with the effects of anesthetic drugs. Therefore, learning these implicit correlations by recovering the masked signal will equip the pre-trained model with a deep contextual understanding of the BIS data. The present invention divides the BIS data into time windows and then converts it into an embedding using a one-dimensional convolutional layer with a stride equal to the time window block size; it is then reconstructed through a Transformer model.

[0045] As an example, the idea of ​​quantifying time series distribution is used here. According to the maximum entropy principle, without making any prior assumptions about the segmentation of time series data, in order to maximize the entropy of the total distribution, it is reasonable to make the distribution of each period as diverse as possible; this allows for more general and flexible modeling of future data; in addition, in order to maximize the use of shared knowledge under the time series, this can be achieved by finding the periods with the lowest similarity to each other; therefore, this problem can be expressed as follows:

[0046]

[0047] Where d is a distribution metric such as Euclidean distance or cosine similarity, Δ1 and Δ2 are predefined parameters to avoid losing distribution information, and K0 is a hyperparameter. The goal of this formula is to divide the input dosage data into K periods, each of which has the largest distribution difference in dosage data. We use a greedy algorithm to solve the K value in this formula.

[0048] As an example, each input univariate time series $x_{i}$ is first divided into overlapping or non-overlapping time window blocks. The time window block length is represented as $P$ and the step size of the non-overlapping region between two consecutive time window blocks is recorded as $S$. After the time window block is generated, the time window block sequence is obtained. Where N is the number of time window blocks, Here, this chapter takes the last value before getting the time window block. The number of repetitions is padded to the end of the original sequence. Using the time window block, the number of input tokens can be reduced from L to approximately L / S, which means that the memory usage and computational complexity of the attention map are quadratically reduced by S times. Therefore, the training time and GPU memory usage can be reduced. In addition, the time window block design allows the model to see a longer historical sequence, thereby improving prediction performance. Consistent with previous work, this chapter uses a random mask to shield the time window block with a shielding ratio of 75%. The other visible time window blocks serve as the Transformer encoder φ enc Finally, the hidden layer representation and learnable mask representation from the encoder and time window block are used as the Transformer decoder φ dec Input, get the reconstructed anesthesia depth signal y r , the loss function used here is the mean square error (MSE) loss function

[0049] As an example, using the time series distribution quantization method, assume that the input data in Where n is the total length of the data, p is the number of features of the data, m i is the sequence length of each sampling point; if there is a teacher model and a student model, the number of features of the teacher model is 3, the number of features of the student model is 2, and the sequence length in the model is 120; first, the input data D is evenly divided into 10 segments, and the cosine similarity is used to measure the size of the data distribution difference between the two time periods. We use the greedy algorithm to divide the 10-segment input data into 5 time periods with the largest data distribution difference between each two time periods, which can be expressed as D = {D1, D2, D3, D4, D5}, where D1 represents the first time window subsequence.

[0050] As an example, the existing deep learning-based methods have poor prediction performance for the anesthesia depth trend in the induction period and the recovery period. In addition, the model is prone to overfitting for the maintenance period, which makes it easy to appear as a straight line during the maintenance period. The highlight of this application is that the self-supervised strategy of self-mask pre-training uses a large amount of discarded data for self-mask reconstruction pre-training to obtain effective common features of the three periods of anesthesia depth (maintenance period, induction period and recovery period) to enhance the feature extraction ability of the model, improve the model performance and reduce the difficulty of model convergence, thereby avoiding the problem of poor prediction performance of the model for the induction period and the recovery period, and the problem of overfitting for the maintenance period. This application uses self-supervised pre-training to solve the problem of data imbalance in the three periods in the anesthesia depth sequence, thereby improving the feature extraction ability of the model.

[0051] In an optional embodiment, the step of training the anesthesia depth prediction model using the preprocessed training set includes:

[0052] The preprocessed training set is input into the anesthesia depth prediction model, and the causal transformer in the anesthesia depth prediction model extracts the dosage time series feature z based on the anesthetic drug dosage data and the anesthesia depth index value in the time window subsequence. (i) ;

[0053] The dosage time series characteristic x (i) The expressions include:

[0054]

[0055] Where Attention(·) represents the attention mechanism, represents the query matrix corresponding to the time window subsequence input to the causal Transformer, Represents the key matrix corresponding to the time window subsequence of the input causal Transformer, represents the value matrix corresponding to the time window subsequence of the input causal Transformer, represents the scaling factor;

[0056] The bottleneck network transforms the dosage time series feature z (i) The static information corresponding to the dosage time series feature is spliced ​​into a spliced ​​feature, and the spliced ​​feature is sequentially passed through two fully connected neural networks and ReLU activation functions to obtain the bottleneck network output T tout , the bottleneck network output T tout The predicted value of the anesthesia depth index of the historical anesthesia subjects is obtained through a fully connected neural network with an output dimension of 1;

[0057] A loss function is set based on the predicted value of the anesthesia depth index to reduce the gap between the predicted value of the anesthesia depth index and the true value of the anesthesia depth index, and the loss function is iteratively solved. When the loss function value reaches a minimum or the number of iterations reaches a preset value, the iteration is terminated to obtain a trained anesthesia depth prediction model;

[0058] The drug dosage data and static information of the anesthetized subject are input into the trained anesthesia depth prediction model, and the trained anesthesia depth prediction model outputs a predicted value of an anesthesia depth index of the anesthetized subject.

[0059] As an example, the time series distribution is quantized to obtain D = D1, D2, D3, D4, D5, which is input into the causal transformer. The input historical efficacy information x i First, the data in each time window is localized through causal convolution, so that the model can focus on multiple time points at the same time instead of a single time point. Each local time block obtained through causal convolution is used as the input of the Transformer encoder. Specifically, through the trainable linear projection W p ∈R D×P Map each local temporal block to a Transformer latent space of dimension D, and a learnable additional positional encoding W pos ∈R D×N , used to monitor the time sequence of local time blocks: Then each attention head of the multi-head attention, h=1,…,H, converts them into a query matrix key matrix Value Matrix in

[0060] , Afterwards, the scaled product is used to obtain the attention output The attention output generation is then represented as If there is a teacher model and a student model, the cosine similarity is used to calculate the difference in the intermediate feature representation of the causal Transformer of the teacher model and the student model and the difference is recorded as the regularization term of the student model loss function as L G (Fourth sub-loss function).

[0061] As an example, the causal Transformer refers to a Transformer model that enhances causal reasoning capabilities through pre-training and the introduction of causal convolution. The use of the causal Transformer can capture the long-term dependency between drug injection history and anesthesia depth; the bottleneck network is a special neural network structure, which usually appears in the middle layer of a deep neural network and serves as an information bottleneck.

[0062] As an exemplary illustration, a causal Transformer model is proposed to address the problem that existing technologies are unable to model long-term temporal relationships, while enhancing the model's ability to predict sudden changes in anesthesia depth.

[0063] As an example, the causal Transformer output and the corresponding patient's physiological characteristics are spliced ​​together as the output of the bottleneck network. The bottleneck network fully learns the temporal feature information in the historical drug infusion data captured by the GRU and the relationship between the corresponding human body characteristics; the bottleneck network structure is relatively simple, consisting of two fully connected layers, followed by a layer of ReLU activation function; finally, a fully connected layer is used to predict the anesthesia depth index (BIS), and finally the parameters required for training the network are obtained.

[0064] As an example, it is well known that RNN and its various variants are used to model the problem of neural networks being unable to capture long-term dependencies. However, studies have shown that the average effective context size of a language model using LSTM is approximately 200 tokens, but it can only clearly distinguish 50 nearby tokens, indicating that even LSTM has difficulty capturing long-term dependencies. At the same time, when predicting anesthesia depth from drug injection history, long-term drug injection history data and the corresponding anesthesia depth values ​​are closely related to the anesthesia depth at future moments. How to model long-term dependencies becomes a key step in achieving excellent performance. On the other hand, current models aim to learn the relationship between historical drug injection data and anesthesia depth within a time window. However, the time points spanned in the time window are too long, which is not conducive to learning the mutation information in the three periods of anesthesia. Therefore, extracting local information is crucial for predicting local mutations in anesthesia depth. Therefore, this application uses a Transformer model to capture the long-term dependency between drug injection history and anesthesia depth. In addition, this application also uses time window blocks and a self-attention enhancement model using causal convolution to extract local features.

[0065] As an example, the causal transformer can learn the prediction knowledge of each period, but cannot reduce the distribution differences between different periods to utilize common knowledge. In order to match the time series distribution, the importance vector α is first introduced to evaluate the importance of each hidden state. Through α, the network can pay attention to the distribution differences between different periods, thereby reducing the distribution differences between different periods and achieving time series distribution matching. The output dimensions of the causal transformer model are merged into G teacher ∈R 256×7680 ,G student ∈R 256×7680 , passes through a fully connected layer with an output dimension of 120, then passes through a sigmoid activation function, and then is normalized by Softmax to obtain the importance vector α. Then we take the output of the GRU and α to calculate the distribution difference between different hidden states as the model's loss function regularization term L t (Second loss function and first sub-loss function).

[0066] As an example, the intermediate features of the output of the causal Transformer model are represented as With the patient's static information S∈R 256×4 (Height, weight, gender, age) can be combined to form T teacher ∈R 256×68 , T student ∈R 256×68 , then passes through a fully connected neural network with an output dimension of 128, and then passes through a fully connected neural network with an output dimension of 128, and then passes through the Relu activation function to get T tout ,T sout ∈R 256×128 , including dynamic time series information features and static patient physiological characteristics information. Then use cosine similarity to calculate the bottleneck network output T of the two models tout ,T sout ∈R 256×128 The similarity of is also used as the regular term of the student network loss function, denoted as L S (Fifth sub-loss function).

[0067] In an optional embodiment, the loss function includes a first loss function, a second loss function and a third loss function;

[0068] The expression of the first loss function includes:

[0069]

[0070] Where, L tpred (θ) represents the first loss function, θ represents the parameters of the anesthesia depth prediction model, D jrepresents the j-th time window subsequence, Indicates D j The corresponding predicted value of anesthesia depth index, express The corresponding true value of the anesthesia depth index, l(·) represents the mean square error loss function, M t (·) represents the teacher network, K represents the total number of time window subsequences;

[0071] The expression of the second loss function includes:

[0072]

[0073] Where, L t (D i ,D j ; θ) represents the second loss function, D i represents the i-th time window subsequence, Indicates that D is in state t i and D j Importance parameter of the two time period distributions, and Respectively represent D i and D j The attention head of the causal Transformer corresponding to state t, d(·) represents the cosine similarity calculation function, and H represents the number of multi-head attention heads;

[0074] The third loss function includes:

[0075]

[0076] Where, represents the third loss function, Ω represents the set of masked time window sequences, x represents the information in the time window sequence of the input causal Transformer, and y r Represents the information reconstructed by the causal Transformer.

[0077] In an optional embodiment, the trained anesthesia depth prediction model is regarded as a teacher model;

[0078] Constructing a student model, wherein the student model includes a causal Transformer and a bottleneck network;

[0079] The medication dosage data corresponding to several historical anesthesia subjects in the training set, as well as static information including the age, gender, height and / or weight of the historical anesthesia subjects, are input into the student model as training data, and the student model outputs the predicted value of the anesthesia depth index of the historical anesthesia subjects;

[0080] A fourth loss function for reducing the gap between the anesthesia depth index predicted value output by the student model and the anesthesia depth index true value is set based on the anesthesia depth index predicted value, and a fifth loss function for reducing the gap between the anesthesia depth index predicted value output by the student model and the teacher model;

[0081] Iteratively solving the fourth loss function and the fifth loss function, and when the number of iterations reaches a preset value, ending the iteration to obtain a student model that has completed training;

[0082] The medication dosage data and static information of the anesthetized subject are input into the trained student model, and the trained student model outputs a predicted value of anesthesia depth index of the anesthetized subject.

[0083] As an example, the training set may contain a small number of missing values ​​in the sample data of the anesthetized subjects. Since PPF (propofol) and RFTN (remifentanil) have a linear relationship over time, linear interpolation is used to fill in the small number of missing values ​​in the anesthetized subject samples. The total dosage of PPF and RFTN is monotonically non-decreasing. Therefore, if the total dosage of the anesthetized subject data samples decreases due to instrument recording issues or data export problems, we use the mean of the previous and next moments to fill in the missing values.

[0084] As an example, the target-controlled syringe pump records the cumulative injection volume (dose). The injection history data retrieved from the target-controlled syringe pump is updated every 10 seconds. Therefore, a time window with a sequence length of 120 is established, using the cumulative drug dosage of PPF and RFTN within 10 seconds as features. Because the data changes significantly between different anesthetized subjects are not conducive to network learning, and to conserve computational resources, the training data is downsampled, with one sampling point selected every 10 seconds. Furthermore, because the BIS values ​​of different anesthetized subjects have different trends and fluctuate frequently for each subject, the BIS values ​​of the training set are smoothed using locally weighted scatter plot smoothing (LOWESS) with a smoothing parameter of 0.03 to reduce computational errors during training. Furthermore, we introduce historical BIS values ​​as features in the teacher model to guide the teacher model in learning BIS value trends. Unprocessed BIS values ​​are used in the validation and test datasets.

[0085] As an example, Figure 2 This is a schematic diagram of the framework of the anesthesia depth prediction method based on representation learning proposed in this embodiment. Figure 2 The distillation in represents knowledge distillation, such as Figure 2As shown, the two networks are trained as teacher models M t and student model M s Both models share the same framework and follow the training process described above. The difference is that the teacher model inputs not only the drug infusion history but also the BIS values ​​at historical moments to obtain a richer intermediate-layer feature representation of temporal information. Therefore, we perform knowledge distillation based on the intermediate-layer feature representation. Specifically, we have the student model's output mimic the teacher model's output to obtain a more accurate feature representation of the BIS value trend. We also ensure that the distribution of the output feature representation of the student model's bottleneck network is as similar as possible to that of the teacher model's bottleneck network. The teacher model incorporates historical BIS data features and can more accurately reflect the impact of static physiological and dynamic drug information on BIS. Therefore, we mimic the output feature representation of the student bottleneck network to reduce the distribution difference between the two. Furthermore, the student model's structural complexity and number of layers are both lighter than the teacher model. Specifically, the teacher model has five GRU layers, while the student model uses only one.

[0086] As an example, the missing data of propofol and remifentanil in the training set were filled using linear interpolation, and the mean of the previous and next moments of the recorded data was modified. The BIS value was smoothed using a local weighted scatter plot smoothing (LOWESS) with a smoothing parameter of 0.03. After processing, the cumulative dosage of propofol and remifentanil within 10 seconds for each sampling point was used as a time series feature. In particular, for the teacher model, the BIS value at the moment before the prediction time was also used as a time series feature. Based on the time series features of the teacher model and the student model, a time window with a sequence length of 120 was established as dynamic feature information.

[0087] In an optional embodiment, the fourth loss function includes a first sub-loss function, a second sub-loss function and a third sub-loss function;

[0088] The expression of the first sub-loss function includes:

[0089]

[0090] Where, L t (D i ,D j ; θ) represents the first sub-loss function, D i represents the i-th time window subsequence, Indicates that D is in state t i and D j Importance parameter of the two time period distributions, and Respectively represent D i and D j The attention head of the causal Transformer of the student model corresponding to state t, d(·) represents the cosine similarity calculation function, and H represents the number of multi-head attention heads;

[0091] The expression of the second sub-loss function includes:

[0092]

[0093] Where, L spred (θ1) represents the second sub-loss function, θ1 represents the parameters of the student model, D j represents the j-th time window subsequence, Indicates D j The corresponding student model outputs the predicted value of the anesthesia depth index, express The corresponding true value of the anesthesia depth index, l(·) represents the mean square error loss function, M s (·) represents the student network, K represents the total number of time window subsequences;

[0094] The expression of the third sub-loss function includes:

[0095]

[0096] Where, represents the third sub-loss function, Ω represents the set of masked time window sequences, x represents the information in the time window sequence of the input causal Transformer, and Y r Information representing the causal Transformer reconstruction of the student model.

[0097] In an optional embodiment, the fifth loss function includes a fourth sub-loss function and a fifth sub-loss function;

[0098] The expression of the fourth sub-loss function includes:

[0099]

[0100] Where, L G represents the fourth sub-loss function, d(·) represents the cosine similarity calculation function, z (i) represents the dosage time series features extracted by the teacher model, represents the time series features of medication dosage extracted by the student model;

[0101] The expression of the fifth sub-loss function includes:

[0102] L S =d(Ttout ,T sout )

[0103] Where, L S represents the fifth sub-loss function, d(·) represents the cosine similarity calculation function, T tout , represents the bottleneck network output of the teacher model, T sout Represents the bottleneck network output of the student model.

[0104] As an example, the number of model iterations is set to 100, the number of early stopping iterations is set to 40, and the teacher model M t and student model M s Training is performed simultaneously, and the model parameters are updated according to the loss functions corresponding to the teacher model and the student model until the number of iterations is greater than 40 and the validation set loss no longer decreases or the number of iterations reaches 100. The student model parameters with the lowest validation set loss are taken as the final model, and then the student model is used to test the test set data.

[0105] Example 2

[0106] This embodiment proposes an anesthesia depth prediction system based on representation learning, which is used to implement the anesthesia depth prediction method based on representation learning proposed in Example 1.

[0107] Figure 3 This is the overall framework diagram of the anesthesia depth prediction system based on representation learning in this embodiment.

[0108] The anesthesia depth prediction system based on representation learning includes:

[0109] A data acquisition module is used to obtain drug dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject;

[0110] A model construction module, configured to construct an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model comprises a causal transformer and a bottleneck network; wherein the causal transformer is configured to extract a dosage time series feature based on the dosage data, and the bottleneck network is configured to combine the dosage time series feature and static information corresponding to the dosage time series feature to predict the anesthesia depth index value;

[0111] The prediction module is used to input the drug dosage data and static information of the anesthetized subject into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia subject.

[0112] It can be understood that the anesthesia depth prediction system based on representation learning in this embodiment improves the method of Example 1. The options in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0113] Example 3

[0114] This embodiment proposes a computer device for implementing an anesthesia depth prediction method based on representation learning proposed in Example 1, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the anesthesia depth prediction method based on representation learning described in Example 1.

[0115] It can be understood that the anesthesia depth prediction system and computer device based on representation learning in this embodiment improves the method of Example 1. The options in the above-mentioned Example 1 are also applicable to this embodiment, so they will not be repeated here.

[0116] The same or similar reference numerals correspond to the same or similar components;

[0117] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting the present embodiment.

[0118] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for predicting anesthesia depth based on representation learning, characterized in that: The following steps are involved: Obtaining medication dosage data and static information including age, gender, height and / or weight of the anesthetized subject; Constructing an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model includes a causal transformer and a bottleneck network; wherein the causal transformer is used to extract medication dosage time series features based on the medication dosage data, and the bottleneck network is used to combine the medication dosage time series features and static information corresponding to the medication dosage time series features to predict the anesthesia depth index value; Inputting medication dosage data and static information of the anesthetized subject into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs a predicted value of an anesthesia depth index of the anesthetized subject; Before using the anesthesia depth prediction model to predict the anesthesia depth index value of the anesthesia subject, collecting a training set, preprocessing the training set, and using the preprocessed training set to train the anesthesia depth prediction model; The training set includes medication dosage data, anesthesia depth index values, and static information including age, gender, height, and / or weight of several historical anesthesia subjects; The step of preprocessing the training set includes: Removing invalid data from the training set, the invalid data including: data corresponding to historical anesthesia subjects whose anesthesia depth index value was lower than a preset value when the drug infusion started, data corresponding to historical anesthesia subjects whose drug dosage data was missing for more than a preset time period, and data corresponding to historical anesthesia subjects whose drug dosage corresponding to the first anesthesia depth index value was not zero; In the training set after removing invalid data, the anesthetic drug dosage data with missing data were filled by linear interpolation; the anesthetic drug dosage data with erroneous data were replaced by the mean of the sum of the previous and next moments of the erroneous data; the anesthesia depth index value was smoothed by local weighted scatter plot smoothing; The step of preprocessing the training set further includes: Sampling the anesthetic drug dosage data of several historical anesthesia subjects in the training set at preset time intervals, and recording the anesthesia depth index value corresponding to each sampling point to form a plurality of time series corresponding to the several historical anesthesia subjects, the time series including each sampling point, the cumulative dosage of each sampling point within a preset time period, and the anesthesia depth index value of each sampling point; Each time series is divided into several time window sequences using several time window blocks. The expressions of the time window sequences include: Where, Indicates the number of dimensions The field of real numbers, Indicates the time window block length, represents the step size of the non-overlapping area between two consecutive time window blocks, N is the number of time window blocks, Indicates the length of the time window sequence; Indicates the The length of a time series is A time window block sequence corresponding to the time window block of ; The time window sequence is shielded by a preset number of shielding numbers using a random mask method, and the unshielded time window sequence is divided into time window subsequences, measure the size of the data distribution difference between the two time window subsequences, and use the greedy algorithm to The time window subsequence is divided into A time window subsequence corresponding to a time period with the largest difference in pairwise data distribution is used to train the anesthesia depth prediction model.

2. The anesthesia depth prediction method based on representation learning according to claim 1, characterized in that: The steps of training the anesthesia depth prediction model using the preprocessed training set include: The preprocessed training set is input into the anesthesia depth prediction model, and the causal transformer in the anesthesia depth prediction model extracts the dosage time series features based on the anesthetic drug dosage data and the anesthesia depth index value in the time window subsequence. ; The dosage time series characteristics The expressions include: Where, represents the attention mechanism, represents the query matrix corresponding to the time window subsequence input to the causal Transformer, Represents the key matrix corresponding to the time window subsequence of the input causal Transformer, represents the value matrix corresponding to the time window subsequence of the input causal Transformer, represents the scaling factor; The bottleneck network takes the dosage time series characteristics The static information corresponding to the dosage time series feature is spliced ​​into a spliced ​​feature, and the spliced ​​feature is passed through two fully connected neural networks and Relu activation functions in sequence to obtain the bottleneck network output. , the bottleneck network output The predicted value of the anesthesia depth index of the historical anesthesia subjects is obtained through a fully connected neural network with an output dimension of 1; A loss function is set based on the predicted value of the anesthesia depth index to reduce the gap between the predicted value of the anesthesia depth index and the true value of the anesthesia depth index, and the loss function is iteratively solved. When the loss function value reaches a minimum or the number of iterations reaches a preset value, the iteration is terminated to obtain a trained anesthesia depth prediction model; The medication dosage data and static information of the anesthetized subject are input into the trained anesthesia depth prediction model, and the trained anesthesia depth prediction model outputs a predicted value of an anesthesia depth index of the anesthetized subject.

3. The anesthesia depth prediction method based on representation learning according to claim 2, characterized in that: The loss function includes a first loss function, a second loss function and a third loss function; The expression of the first loss function includes: Where, represents the first loss function, represents the parameters of the anesthesia depth prediction model, Indicates the time window subsequences, express The corresponding predicted value of anesthesia depth index, express The corresponding true value of the anesthesia depth index, represents the mean square error loss function, represents the teacher network, Represents the total number of subsequences in the time window; The expression of the second loss function includes: Where, represents the second loss function, Indicates the time window subsequences, Indicates that in state t and Importance parameter of the two time period distributions, and Respectively At the attention head of the causal Transformer corresponding to state t, represents the cosine similarity calculation function, Indicates the number of multi-head attention; The third loss function includes: Where, represents the third loss function, represents the set of masked time window sequences, Represents the information in the time window sequence of the input causal Transformer, Represents the information reconstructed by the causal Transformer.

4. The anesthesia depth prediction method based on representation learning according to any one of claims 1 to 3, characterized in that: The anesthesia depth prediction model that has completed training is regarded as a teacher model; Constructing a student model, wherein the student model includes a causal Transformer and a bottleneck network; The medication dosage data corresponding to several historical anesthesia subjects in the training set, as well as static information including the age, gender, height and / or weight of the historical anesthesia subjects, are input into the student model as training data, and the student model outputs the predicted value of the anesthesia depth index of the historical anesthesia subjects; A fourth loss function for reducing the gap between the anesthesia depth index predicted value output by the student model and the anesthesia depth index true value is set based on the anesthesia depth index predicted value, and a fifth loss function for reducing the gap between the anesthesia depth index predicted value output by the student model and the teacher model; Iteratively solving the fourth loss function and the fifth loss function, and when the number of iterations reaches a preset value, ending the iteration to obtain a student model that has completed training; The medication dosage data and static information of the anesthetized subject are input into the trained student model, and the trained student model outputs a predicted value of anesthesia depth index of the anesthetized subject.

5. The anesthesia depth prediction method based on representation learning according to claim 4, characterized in that: The fourth loss function includes a first sub-loss function, a second sub-loss function and a third sub-loss function; The expression of the first sub-loss function includes: Where, represents the first sub-loss function, Indicates the time window subsequences, Indicates that in state t and Importance parameter of the two time period distributions, and Respectively The attention head of the causal Transformer of the student model corresponding to state t, represents the cosine similarity calculation function, Indicates the number of multi-head attention; The expression of the second sub-loss function includes: Where, represents the second sub-loss function, represents the parameters of the student model, Indicates the time window subsequences, express The corresponding student model outputs the predicted value of the anesthesia depth index, express The corresponding true value of the anesthesia depth index, represents the mean square error loss function, represents the student network, Represents the total number of subsequences in the time window; The expression of the third sub-loss function includes: Where, represents the third sub-loss function, represents the set of masked time window sequences, Represents the information in the time window sequence of the input causal Transformer, Information representing the causal Transformer reconstruction of the student model.

6. The anesthesia depth prediction method based on representation learning according to claim 4, characterized in that: The fifth loss function includes a fourth sub-loss function and a fifth sub-loss function; The expression of the fourth sub-loss function includes: Where, represents the fourth sub-loss function, represents the cosine similarity calculation function, represents the dosage time series features extracted by the teacher model, represents the time series features of medication dosage extracted by the student model; The expression of the fifth sub-loss function includes: Where, represents the fifth sub-loss function, represents the cosine similarity calculation function, represents the bottleneck network output of the teacher model, Represents the bottleneck network output of the student model.

7. A system for predicting depth of anesthesia based on representation learning, for implementing the method for predicting depth of anesthesia based on representation learning according to any one of claims 1 to 6, characterized in that: include: A data acquisition module is used to obtain drug dosage data of the anesthetized subject and static information including the age, gender, height and / or weight of the anesthetized subject; A model construction module, configured to construct an anesthesia depth prediction model for predicting an anesthesia depth index value; the anesthesia depth prediction model comprises a causal transformer and a bottleneck network; wherein the causal transformer is configured to extract a dosage time series feature based on the dosage data, and the bottleneck network is configured to combine the dosage time series feature and static information corresponding to the dosage time series feature to predict the anesthesia depth index value; The prediction module is used to input the drug dosage data and static information of the anesthetized subject into the anesthesia depth prediction model, and the anesthesia depth prediction model outputs the anesthesia depth index prediction value of the anesthesia subject.

Images