Small sample anesthesia depth prediction method and system

Through feature distillation and label smoothing technology, combined with the balanced mean square error loss function, the overfitting problem of the anesthesia depth prediction model in the case of data imbalance is solved, and the generalization performance of the model and the prediction accuracy of small sample data are improved.

CN119049654APending Publication Date: 2024-11-29GUANGDONG UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411177077.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing anesthesia depth prediction models are prone to overfitting in the case of data imbalance and perform poorly on the new dataset, resulting in poor prediction results.

Method used

Feature distillation technology is used to migrate the features of the teacher model to the student model, and the student model is trained to improve its generalization performance and adaptability through the label smoothed reweighted loss function and balanced mean square error loss function.

Benefits of technology

It effectively prevents the model from overfitting in areas with large data volumes, improves the prediction accuracy and stability on small sample data sets, reduces the overfitting phenomenon, and enhances the generalization performance and adaptability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119049654A_ABST
    Figure CN119049654A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of anesthesia depth prediction, and provides a small sample anesthesia depth prediction method and system, and the method comprises the steps: collecting anesthesia operation record data as source domain data and small sample data, carrying out the preprocessing, building a teacher model and a student model, and inputting the source domain data into the teacher model for training; training a student model through feature distillation based on the trained teacher model; calculating the loss of the student model after the feature distillation training by using a label smoothing-based reweighted loss function, and updating the weight of the student model; inputting the small sample data into the student model for training, and taking a balanced mean square error as a loss function of the student model; and inputting anesthesia depth data to be predicted into the trained student model to obtain an anesthesia depth prediction result. Compared with the defect of poor training effect on unbalanced data in the prior art, the invention provides a more efficient and accurate solving method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of anesthesia depth prediction, and in particular relates to a small sample anesthesia depth prediction method. Background Art

[0002] Anesthesia plays a vital role in clinical surgery in hospitals. It can not only reduce surgical risks but also improve patient experience. However, excessive or insufficient anesthesia dosage may hinder the smooth progress of the operation, and in severe cases may even endanger the patient's life. Therefore, an accurate anesthetic efficacy prediction model is crucial to ensure the safety and effectiveness of the operation. Anesthesiologists usually adjust the dosage of drugs by monitoring anesthesia depth indicators such as the bispectral index (BIS) and the NI index (Narcotrend Index). In anesthesia prediction technology, data imbalance is a common problem, especially the BIS or NI index representing the depth of anesthesia in the data set is usually concentrated in the range of 40 to 60. This uneven distribution phenomenon brings two main problems: first, the model is prone to overfitting in the interval with a large sample size, that is, the interval of 40 to 60; second, the model performs poorly when fine-tuning training on a new data set with uneven sample distribution. Solving this data imbalance problem is crucial to improving the training effect and performance of deep learning models.

[0003] At present, the commonly used traditional anesthesia depth prediction method is based on the pharmacokinetic-pharmacodynamic (PK-PD) model. However, this method has high requirements on the quality of data. When the anesthesia depth varies greatly or is trained on unbalanced data, the trained model has poor prediction effect. Summary of the invention

[0004] In order to overcome the defect of poor model prediction effect caused by imbalanced training data in the above-mentioned prior art, the present invention proposes a small sample anesthesia depth prediction method.

[0005] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0006] A method for predicting anesthesia depth of a small sample comprises the following steps:

[0007] Anesthesia surgery record data is collected as source domain data and small sample data and preprocessed, a teacher model and a student model are established, and the source domain data is input into the teacher model for training;

[0008] Based on the trained teacher model, the student model is trained through feature distillation;

[0009] Calculate the loss of the student model that has completed feature distillation training using a label-smoothing-based reweighted loss function, and update the weight of the student model;

[0010] Inputting the small sample data into the student model for training, and using balanced mean square error as the loss function of the student model;

[0011] The anesthesia depth data to be predicted is input into the trained student model to obtain an anesthesia depth prediction result.

[0012] The present invention also proposes a small sample anesthesia depth prediction system, the system comprising:

[0013] Data preprocessing module: used to preprocess source domain data and small sample data;

[0014] Feature distillation module: used to input source domain data into the teacher model for training, and train the student model through feature distillation based on the trained teacher model; calculate the loss of the student model that has completed the feature distillation training using a label-smoothed reweighted loss function, and update the weight of the student model;

[0015] Student model training module: used to input small sample data into the student model for training, and use balanced mean square error as the loss function of the student model;

[0016] Prediction module: input the anesthesia depth data to be predicted into the trained student model to obtain the anesthesia depth prediction result.

[0017] The present invention also proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the small sample anesthesia depth prediction method as described in the present invention when executing the computer program.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for predicting the depth of anesthesia of a small sample as described in the present invention is implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The present invention transfers the feature modeling capability of the teacher model to the student model with smaller parameters by means of feature distillation, thereby improving the generalization performance of the model; by performing reweighted loss calculation for label distribution smoothing on the student model, the model can better focus on areas with less data in the data label distribution, thereby preventing it from overfitting in areas with more data; and using balanced mean square error as the loss function on a small sample data set helps to improve the training effect of the model under small sample conditions and reduce overfitting when the sample distribution is uneven. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the small sample anesthesia depth prediction method of Example 1.

[0022] Figure 2 This is the architecture diagram of the small sample anesthesia depth prediction system of Example 2. Figure 3 This is the prediction result of the test set in the small sample data of Example 3. DETAILED DESCRIPTION

[0023] The accompanying drawings are only used for illustrative purposes and are not to be construed as limiting the present invention;

[0024] It is understandable to those skilled in the art that some well-known descriptions may be omitted in the drawings.

[0025] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0026] Example 1

[0027] This embodiment proposes a small sample anesthesia depth prediction method, such as Figure 1 As shown, it is a flowchart of the small sample anesthesia depth prediction method of this embodiment.

[0028] The small sample anesthesia depth prediction method proposed in this embodiment includes the following steps:

[0029] Anesthesia surgery record data is collected as source domain data and small sample data and preprocessed, a teacher model and a student model are established, and the source domain data is input into the teacher model for training;

[0030] Based on the trained teacher model, the student model is trained through feature distillation;

[0031] Calculate the loss of the student model that has completed feature distillation training using a label-smoothing-based reweighted loss function, and update the weight of the student model;

[0032] Inputting the small sample data into the student model for training, and using balanced mean square error as the loss function of the student model;

[0033] The anesthesia depth data to be predicted is input into the trained student model to obtain an anesthesia depth prediction result.

[0034] In this embodiment, the domain data and small sample data are first preprocessed to reduce the impact of noise; the feature modeling capability of the teacher model is transferred to the student model with smaller parameters through feature distillation, so that the student model can maintain a high prediction performance even when the number of parameters is reduced; the reweighted loss calculation of the student model with smoothed label distribution is performed to adjust the model's attention to different labels, which helps the model to learn effective information in different areas of the data set and avoid overfitting in areas with large data volumes; and the balanced mean square error is used as the loss function on the small sample data set, which helps to improve the training effect of the model under small sample conditions and reduce overfitting when the sample distribution is uneven.

[0035] Specifically, the standard Transformer model is used as the base model. The Transformer model calculates each input vector x through the self-attention mechanism. i The self-attention operation first performs a weighted dot product based on the input Create three different linear weighted vectors, called query key value It is calculated as follows:

[0036] q i =W q x i ,k i =W k x i ,v i =W v x i

[0037] Where W q ,W k ,W v represents the learnable weight matrix. In the final output, the value vector v i The weight depends on the query vector q at position i i and the key vector k at position j j The mapping correlation between them is:

[0038]

[0039] In an optional embodiment, the preprocessing step includes: completing the null value parts in the source domain data and the small sample data by linear interpolation, and smoothing the source domain data and the small sample data.

[0040] Further optionally, the step of smoothing the source domain data and the small sample data includes: smoothing the source domain data and the small sample data through a local weighted scatter plot.

[0041] In this embodiment, linear interpolation is used to fill in the null value parts in the source domain data and small sample data, which can enhance the integrity of the data and make the data set more uniform, which helps the model better understand and learn the patterns and relationships in the data and improve the accuracy of data analysis; random fluctuations and outliers in the data can be effectively removed through smoothing, which is conducive to improving the stability of model training and the accuracy of prediction; local weighted scatter plot smoothing is a non-parametric regression method that can adapt the model to the local characteristics of the data, thereby more accurately reflecting the real trend and structure of the data. It reduces the risk of overfitting and improves its generalization ability.

[0042] As an exemplary illustration, the real anesthesia surgery record data in the vitalDB open source database is used as the source domain data, and the anesthesia surgery records collected by the hospital are used as small sample data.

[0043] In an optional embodiment, the step of calculating the loss of the student model that has completed feature distillation training using a reweighted loss function based on label smoothing includes:

[0044] Calculating the label density of the student model;

[0045] The inverse of the label density is used as the weight of the reweighted loss function, and the reweighted loss is calculated. The expression is as follows:

[0046]

[0047] Where N represents the maximum value of the anesthesia depth index BIS; θ is the parameter to be optimized in the model; y i The label representing the true value of the label i; Represents the model for label y i The predicted value of i is the inverse of the label density.

[0048] In this embodiment, the inverse of the label density is used as the weight. By adjusting the weight in the loss function to reflect the frequency of occurrence of each label, it can be ensured that the model pays appropriate attention to all labels during training, especially those that appear less frequently, thereby enhancing the model's predictive ability.

[0049] Further optionally, the step of calculating the label density of the student model includes:

[0050] The label density is calculated by convolving the empirical density of the corresponding label of the input data with the Gaussian kernel function; the calculation formula is as follows:

[0051]

[0052] Among them, p(y i ) indicates label yi The effective density of k(y,y i ) is the Gaussian kernel, which represents the target value y i The similarity between it and any y in the target space; p(y) represents the number of times label y appears in the input data.

[0053] In this embodiment, through kernel smoothing, the new distribution of the data can be smoother and more continuous, reducing the noise and excessive fluctuations in the data, which helps the model capture deeper data features; in addition, the negative Pearson correlation between the new distribution after kernel smoothing and the error shows that when the error decreases, the predicted distribution after kernel smoothing tends to be more accurate. The existence of this correlation shows that smoothing helps minimize the error in the prediction, which directly affects the overall performance and efficiency of the model.

[0054] In an optional embodiment, the step of using the balanced mean square error as the loss function of the student model includes: defining the loss function of the student model based on the model prediction value and the prior information of the label distribution, and the expression thereof is as follows:

[0055]

[0056] in, Indicates the temperature coefficient, B y Indicates the data batch value for each iteration during training.

[0057] Specifically, for the model’s prediction and training label distribution prior p train (y), BMSE (Bessel's Root Mean Square Error) loss is defined as

[0058]

[0059] In the above formula, Indicates that the mean is The variance is Under the condition of , y is normally distributed. As can be seen from the above formula, BMSE contains two parts: first, the standard mean square error loss, and second, the new balance term needs to be integrated. When the training label distribution p train When (y) is uniformly distributed, the equilibrium term can be calculated to be equal to a constant.

[0060] In this embodiment, the balanced mean square error loss function combines the standard mean square error and a balancing term based on the training label distribution. By introducing the balancing term, the loss function helps the model focus on a wider range of data distribution characteristics, effectively preventing overfitting; the BMSE loss introduces a dynamic coefficient based on data batch size and prior knowledge of label distribution to convert the actual distribution of the data set into a more balanced distribution, thereby improving the prediction performance of the model on different categories. In addition, when the training label distribution is close to a uniform distribution, the balancing term becomes a constant, which means that the loss function can simplify the calculation without sacrificing generalization performance, thereby improving calculation efficiency and helping to ensure that the model exhibits better adaptability and prediction accuracy when facing diverse data.

[0061] In an optional embodiment, when training the student model through feature distillation, the student model is trained based on a mean square error function.

[0062] In this embodiment, the mean square error function effectively guides the student model to learn and replicate the output features of the teacher model by minimizing the square difference between the student model prediction value and the teacher model prediction value; ensures that the student model with a reduced number of parameters and model complexity can simulate the behavior of the teacher model as accurately as possible, making the training process more efficient and easy to implement; and also helps the student model maintain performance consistency under different data inputs and reduce the risk of overfitting.

[0063] Example 2

[0064] This embodiment proposes a small sample anesthesia depth prediction system, which applies the small sample anesthesia depth prediction method proposed in Example 1. Figure 2 , which is an architecture diagram of a small sample anesthesia depth prediction system according to this embodiment.

[0065] This embodiment proposes a small sample anesthesia depth prediction system including:

[0066] Data preprocessing module: used to preprocess source domain data and small sample data;

[0067] Feature distillation module: used to input source domain data into the teacher model for training, and train the student model through feature distillation based on the trained teacher model; calculate the loss of the student model that has completed the feature distillation training using a label-smoothed reweighted loss function, and update the weight of the student model;

[0068] Student model training module: used to input small sample data into the student model for training, and use balanced mean square error as the loss function of the student model;

[0069] Prediction module: input the anesthesia depth data to be predicted into the trained student model to obtain the anesthesia depth prediction result.

[0070] It can be understood that the system of this embodiment corresponds to the method of the above-mentioned embodiment 1, and the options in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described repeatedly here.

[0071] Example 3

[0072] This embodiment applies the small sample anesthesia depth prediction method proposed in Embodiment 1 to public data sets and small data sets for training and prediction.

[0073] In this embodiment, in order to verify the validity of this patent, the public data set VitalDB is selected as the source domain data. After random selection, a total of 180 samples are used for training, 76 samples are used for verification, and another 76 samples are used for testing. The small sample data set comes from anesthesia surgery records collected in a hospital, containing a total of 44 samples; 20 samples are used for model fine-tuning, and the remaining 22 samples are used to test model performance. There are differences between the small sample data set and the VitalDB data set in population distribution, anesthesia depth index, and acquisition equipment. The data set of the small sample data set is mainly sampled from young people, so the average age of the samples in the small sample data set (39.9 years old) is smaller than the average age of the samples in the VitalDB data set (56.1 years old). In addition, the equipment used by the hospital is different from that used in the VitalDB data set. The VitalDB data set uses the EEG bispectral index, while the anesthesia depth index measured by the hospital is the NI index, but both are dimensionless indicators ranging from 0 to 100. For the same anesthesia depth, although there are differences in the specific values ​​of the two, the anesthesia depth indicated by the numerical range is roughly equal. Different anesthesia depth indicators can more effectively verify the generalization performance of this method.

[0074] like Figure 3 The figure shows the prediction results of the method for the test set part of the small sample data, where the gray line is the actual anesthesia depth NI value, the red line is the prediction result, and the horizontal axis unit is seconds. After learning on a small data set, this patent can make predictions on a small data set, and the prediction results are relatively accurate.

[0075] The small sample anesthesia depth prediction method proposed in this patent has shown significant prediction results on small data sets, proving that the method not only has strong learning ability, but can also accurately predict anesthesia depth when the sample size is limited. This shows that the method has good generalization performance and practicality, and can effectively adapt to changes under different data set conditions, especially when there are large differences in data distribution, measurement indicators and equipment, it can still provide stable and reliable prediction results.

[0076] Example 4

[0077] This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the processor executes the steps of the small sample anesthesia depth prediction method proposed in Example 1.

[0078] Example 5

[0079] This embodiment proposes a storage medium on which computer-readable instructions are stored, wherein the computer-readable instructions, when executed by a processor, implement the steps of the small sample anesthesia depth prediction method proposed in Embodiment 1.

[0080] Exemplarily, the storage medium includes, but is not limited to, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0081] Exemplarily, the instructions, programs, code sets or instruction sets may be implemented using conventional programming languages.

[0082] Exemplarily, the processor includes but is not limited to a smart phone, a personal computer, a server, a network device, etc., and is used to execute all or part of the steps of the small sample anesthesia depth prediction method described in Example 1.

Claims

1. A method for predicting anesthesia depth of a small sample, characterized in that: The following steps are involved: Anesthesia surgery record data is collected as source domain data and small sample data and preprocessed, a teacher model and a student model are established, and the source domain data is input into the teacher model for training; Based on the trained teacher model, the student model is trained through feature distillation; Calculate the loss of the student model that has completed feature distillation training using a label-smoothing-based reweighted loss function, and update the weight of the student model; Inputting the small sample data into the student model for training, and using balanced mean square error as the loss function of the student model; The anesthesia depth data to be predicted is input into the trained student model to obtain an anesthesia depth prediction result.

2. A method for predicting anesthesia depth of a small sample according to claim 1, characterized in that: The preprocessing step includes: completing the null value parts in the source domain data and the small sample data by linear interpolation, and performing smoothing on the source domain data and the small sample data.

3. A small sample anesthesia depth prediction method according to claim 2, characterized in that: The step of smoothing the source domain data and the small sample data includes: smoothing the source domain data and the small sample data through a local weighted scatter plot.

4. A method for predicting anesthesia depth of a small sample according to claim 1, characterized in that: The step of calculating the loss of the student model that has completed the feature distillation training using the label smoothing based reweighted loss function includes: Calculating the label density of the student model; The inverse of the label density is used as the weight of the reweighted loss function, and the reweighted loss is calculated. The expression is as follows: Where N represents the maximum value of the anesthesia depth index BIS; θ is the parameter to be optimized in the model; y i The label representing the true value of the label i; Represents the model for label y i The predicted value of i is the inverse of the label density.

5. A method for predicting anesthesia depth of a small sample according to claim 4, characterized in that: The step of calculating the label density of the student model comprises: The label density is calculated by convolving the empirical density of the corresponding label of the input data with the Gaussian kernel function; the calculation formula is as follows: Among them, p(y i ) indicates label y i The effective density of k(y,y i ) is the Gaussian kernel, which represents the target value y i The similarity between it and any y in the target space; p(y) represents the number of times label y appears in the input data.

6. A method for predicting anesthesia depth of a small sample according to claim 1, characterized in that: The step of using the balanced mean square error as the loss function of the student model includes: defining the loss function of the student model based on the model prediction value and the prior information of the label distribution, and the expression thereof is as follows: in, Indicates the temperature coefficient, B y Indicates the data batch value for each iteration during training.

7. A method for predicting anesthesia depth of a small sample according to claim 1, characterized in that: When the student model is trained by feature distillation, the student model is trained based on a mean square error function.

8. A system for predicting depth of anesthesia based on small sample data, applying the method according to any one of claims 1 to 7, characterized in that: include: Data preprocessing module: used to preprocess source domain data and small sample data; Feature distillation module: used to input source domain data into the teacher model for training, and train the student model through feature distillation based on the trained teacher model; calculate the loss of the student model that has completed the feature distillation training using a label-smoothed reweighted loss function, and update the weight of the student model; Student model training module: used to input small sample data into the student model for training, and use balanced mean square error as the loss function of the student model; Prediction module: input the anesthesia depth data to be predicted into the trained student model to obtain the anesthesia depth prediction result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, a small sample anesthesia depth prediction method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for predicting the depth of anesthesia of a small sample according to any one of claims 1 to 7 is implemented.