Anesthesia depth assessment method, device, electronic device and storage medium

By using multimodal data for in-depth evaluation of anesthesia, the problem of low evaluation accuracy when brain wave data is disturbed is solved, achieving higher evaluation accuracy.

CN118576160BActive Publication Date: 2024-11-29SHANGHAI FRAISI MEDICAL TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411067536.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-29
Estimated Expiration
2044-08-05

AI Technical Summary

Technical Problem

In the prior art, the accuracy of the in-depth evaluation of anesthesia based on brain wave data is poor, especially under the interference of high-frequency electrocutors and other devices.

Method used

These data are used for in-depth anesthesia assessment by obtaining trained anesthesia assessment models and multimodal data of the target object, including brain wave data, high-time dimension life data, and vital sign data.

Benefits of technology

When brain wave data is disturbed, in-depth anesthesia evaluation is performed through other data, which avoids the problem of reduced accuracy and improves the accuracy of in-depth anesthesia evaluation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118576160B_ABST
    Figure CN118576160B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses an anesthesia depth assessment method, device, electronic device and storage medium. The method comprises: obtaining a trained anesthesia depth assessment model and multimodal data of a target object; when the multimodal data comprises at least one of target brain wave data, target high time dimension life data and target vital sign data, the multimodal data is input into the anesthesia depth assessment model, and the anesthesia depth assessment result of the target object is obtained according to the output result of the anesthesia depth assessment model. The technical solution of the embodiment of the present invention improves the accuracy of anesthesia depth assessment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of medical technology, and in particular to an anesthesia depth assessment method, device, electronic device and storage medium. Background Art

[0002] Researchers have proposed the concept of Bispectral Index (BIS) in the EEG processing of anesthesia. Through BIS, doctors can more intuitively understand the patient's anesthesia depth, thereby improving the clinical effect of anesthesia to varying degrees. Therefore, anesthesia depth assessment is often performed based on electroencephalogram (EEG) data.

[0003] However, EEG data is easily interfered by equipment such as high-frequency electrosurgical units, which results in poor accuracy in anesthesia depth assessment based on EEG data, a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present invention provide a method, a device, an electronic device and a storage medium for evaluating the depth of anesthesia, which improve the accuracy of the evaluation of the depth of anesthesia.

[0005] According to one aspect of the present invention, a method for assessing the depth of anesthesia is provided, which may include: acquiring a trained anesthesia depth assessment model and multimodal data of a target object; when the multimodal data includes at least one of target brain wave data, target high time dimension life data, and target vital sign data, inputting the multimodal data into the anesthesia depth assessment model, and obtaining an anesthesia depth assessment result of the target object based on an output result of the anesthesia depth assessment model.

[0006] According to another aspect of the present invention, there is provided an anesthesia depth assessment device, which may include: a multimodal data acquisition module, used to acquire a trained anesthesia depth assessment model and multimodal data of a target object; an anesthesia depth assessment result acquisition module, used to input the multimodal data into the anesthesia depth assessment model when the multimodal data includes at least one of target brain wave data, target high time dimension life data and target vital sign data, and obtain the anesthesia depth assessment result of the target object based on the output result of the anesthesia depth assessment model.

[0007] According to another aspect of the present invention, an electronic device is provided, which may include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor implements the anesthesia depth assessment method provided by any embodiment of the present invention when executing the computer program.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a processor to implement the anesthesia depth assessment method provided by any embodiment of the present invention when executed.

[0009] The technical solution of the embodiment of the present invention obtains the trained anesthesia depth assessment model and the multimodal data of the target object; when the multimodal data includes at least one of the target brain wave data, the target high time dimension life data and the target vital sign data, the multimodal data is input into the anesthesia depth assessment model, and the anesthesia depth assessment result of the target object is obtained according to the output result of the anesthesia depth assessment model. The above technical solution can be implemented without being limited to the anesthesia depth assessment only through the brain wave data. By using any data among the target brain wave data, the target high time dimension life data and the target vital sign data as the determining factor of the anesthesia depth assessment, when the brain wave data is interfered by the high frequency electric knife and other equipment, it is not necessary to use the target brain wave data as the determining factor of the anesthesia depth assessment, and the anesthesia depth assessment can be performed through other data, thereby avoiding the problem of poor accuracy of anesthesia depth assessment based on brain wave data when the brain wave data is interfered by the high frequency electric knife and other equipment, thereby improving the accuracy of anesthesia depth assessment.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 The present invention provides a flowchart of an anesthesia depth assessment method.

[0013] Figure 2 The present invention is a flowchart of another method for assessing depth of anesthesia provided in accordance with an embodiment of the present invention.

[0014] Figure 3 The present invention is a flowchart of another method for assessing depth of anesthesia provided in accordance with an embodiment of the present invention.

[0015] Figure 4It is a structural diagram of a coding module in another anesthesia depth assessment method provided according to an embodiment of the present invention.

[0016] Figure 5 The flowchart is an optional example of another anesthesia depth assessment method provided according to an embodiment of the present invention.

[0017] Figure 6 It is a structural diagram of a GPT2 in an optional example of another anesthesia depth assessment method provided according to an embodiment of the present invention.

[0018] Figure 7 It is a structural diagram of another GPT2 in an optional example of another anesthesia depth assessment method provided according to an embodiment of the present invention.

[0019] Figure 8 The present invention is a block diagram of a device for evaluating the depth of anesthesia provided in accordance with an embodiment of the present invention.

[0020] Fig. 9 It is a structural schematic diagram of an electronic device for implementing the anesthesia depth assessment method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] Before introducing the embodiments of the present invention, an exemplary explanation is given of the implementation process of the current solution for anesthesia depth assessment and the reasons why the problem of poor accuracy in anesthesia depth assessment exists, so as to better understand why the solution proposed in the embodiments of the present invention improves the accuracy of anesthesia depth assessment.

[0024] For example, the depth of anesthesia is often assessed based on EEG data. The current understanding of the BIS assessment mechanism based on EEG data is limited. Although four predictors, including Relative Beta Ratio (RBR), SynchFastSlow, Burst Suppression Ratio (BSR), and QUAZI, are known to be associated with the original BIS algorithm, their underlying calculation mechanism is unclear. In addition, in some cases, EEG data is inaccurate, specifically because EEG data is sensitive to noise, so more complex algorithms and resources are required for noise filtering, and EEG signal EEG data itself is susceptible to interference from high-frequency electrosurgical equipment, so in some cases, the depth of anesthesia (DOA) monitoring method based on EEG data will fail, which leads to poor accuracy in depth of anesthesia assessment based on EEG data.

[0025] In addition, the use of models to implement anesthesia depth assessment is very challenging because the acquisition of EEG data is very challenging, and the labeling of EEG data usually requires a lot of effort from experts in the corresponding field, resulting in only a small labeled data set for specific tasks of the brain-computer interface. In addition, EEG data usually comes from a small number of participants and lasts for less than a few dozen hours. It is also affected by individual differences and noise in the data itself. Therefore, it is difficult to use a model to achieve accurate anesthesia depth assessment based on EEG data.

[0026] In view of this, the embodiments of the present invention do not need to be limited to the evaluation of anesthesia depth only through brain wave data. By using any data in the target brain wave data, the target high-time dimension life data, and the target vital sign data as the determining factor for anesthesia depth evaluation, when the brain wave data is interfered by equipment such as high-frequency electrosurgical unit, the anesthesia depth evaluation can also be performed through other data, thereby avoiding the problem of poor accuracy of anesthesia depth evaluation based on brain wave data when the brain wave data is interfered by equipment such as high-frequency electrosurgical unit, and improving the accuracy of anesthesia depth evaluation. This will be explained in detail below.

[0027] Figure 11 is a flow chart of an anesthesia depth assessment method provided in an embodiment of the present invention. This embodiment is applicable to the situation of anesthesia depth assessment. The method can be performed by an anesthesia depth assessment device provided in an embodiment of the present invention, and the device can be implemented by software and / or hardware. The device can be integrated in an electronic device, and the electronic device can be various user terminals or servers.

[0028] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps.

[0029] S110, obtaining the trained anesthesia depth assessment model and multimodal data of the target object.

[0030] The anesthesia depth assessment model is a model for assessing the anesthesia depth of a target object. The target object is an object for which anesthesia assessment is required, and the target object may be, for example, an object in an anesthetized state. Multimodal data is data having at least one form of expression.

[0031] In the embodiment of the present invention, for example, the multimodal data of the target object in an anesthetized state can be acquired through a medical instrument. In the embodiment of the present invention, the method of acquiring the multimodal data is not specifically limited.

[0032] S120. When the multimodal data includes at least one of target brain wave data, target high time dimension life data and target vital sign data, the multimodal data is input into an anesthesia depth assessment model, and an anesthesia depth assessment result of the target object is obtained based on the output result of the anesthesia depth assessment model.

[0033] Among them, the target brain wave data is the brain wave data of the target object. The target high time dimension life data is the vital sign data of the target object with high time dimension characteristics, and the target high time dimension life data is also the vital sign data of the target object with rich or complex information in the time dimension; the target high time dimension life data may include at least one of electrocardiogram (ECG) data, photoplethysmography (PPG) data, arterial pressure wave (ART), central venous pressure wave (CVP), airway pressure wave (APW) data, and carbon dioxide wave (CO2W), etc. The target vital sign data is the non-high time dimension vital sign data of the target object; the target vital sign data may include, for example, blood pressure (BP) data, heart rate (HR) data, body temperature data, repolarization process (ST segment) data of each part of the ventricle after depolarization in the electrocardiogram, and pulse oxygen saturation (SPO2), etc. The output result is the output result of the anesthesia depth assessment model; the output result may be, for example, expressed in the form of a bispectral index. The anesthesia depth assessment result is the assessment result of the anesthesia depth of the target object.

[0034] It is understandable that there is an unconscious similarity index (SI) related to heart rate variability (HRV), whose value is in the awake and anesthetized state during isoflurane anesthesia, and electrocardiogram data is a good alternative method for measuring HRV; and parameters derived from photoplethysmography can be used to monitor the sensory component in balanced general anesthesia, therefore, the multimodal data can include at least one of target brain wave data, target high time dimension life data, and target vital sign data.

[0035] In an embodiment of the present invention, when the multimodal data includes at least one of target brain wave data, target high time dimension life data, target vital sign data, and usage data such as drug flow rate of anesthetics such as propofol (PPF) and remifentanil (RFTN), the multimodal data can be input into an anesthesia depth assessment model, and an anesthesia depth assessment result of the target object can be obtained based on the output result of the anesthesia depth assessment model.

[0036] In an embodiment of the present invention, the anesthesia depth assessment model can process three types of data: target brain wave data, target high time dimension life data, and target vital sign data. Therefore, when the multimodal data includes at least one of the target brain wave data, the target high time dimension life data, and the target vital sign data, that is, when the multimodal data includes any one or more of the target brain wave data, the target high time dimension life data, and the target vital sign data, the anesthesia depth model can process the input multimodal data for anesthesia assessment to obtain an output result. The output result can be, for example, a bispectral index that can characterize the anesthesia depth of the target object. Therefore, the anesthesia depth assessment result of the target object can be obtained based on the output result. In an embodiment of the present invention, there is no specific limitation on the method of obtaining the anesthesia depth assessment result of the target object based on the output result of the anesthesia depth assessment model.

[0037] It is understandable that most modal data have no direct relationship with the bispectral index. The bispectral index is obtained by extracting feature information from EEG data and calculating nonlinear functions. Therefore, it is difficult to extract features related to the bispectral index through most modal data. However, using the model can extract features related to the bispectral index through most modal data.

[0038] The technical solution of the embodiment of the present invention obtains the trained anesthesia depth assessment model and the multimodal data of the target object; when the multimodal data includes at least one of the target brain wave data, the target high time dimension life data and the target vital sign data, the multimodal data is input into the anesthesia depth assessment model, and the anesthesia depth assessment result of the target object is obtained according to the output result of the anesthesia depth assessment model. The above technical solution can be implemented without being limited to the anesthesia depth assessment only through the brain wave data. By using any data among the target brain wave data, the target high time dimension life data and the target vital sign data as the determining factor of the anesthesia depth assessment, when the brain wave data is interfered by the high frequency electric knife and other equipment, it is not necessary to use the target brain wave data as the determining factor of the anesthesia depth assessment, and the anesthesia depth assessment can be performed through other data, thereby avoiding the problem of poor accuracy of anesthesia depth assessment based on brain wave data when the brain wave data is interfered by the high frequency electric knife and other equipment, thereby improving the accuracy of anesthesia depth assessment.

[0039] Figure 2It is a flow chart of another anesthesia depth assessment method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the anesthesia depth assessment model includes a brain wave feature extraction part, a high time dimension feature extraction part and a feature fusion part; the multimodal data is input into the anesthesia depth assessment model, and according to the output result of the anesthesia depth assessment model, the anesthesia depth assessment result of the target object is obtained, including: the multimodal data is input into the anesthesia depth assessment model, and the anesthesia depth assessment result of the target object is obtained by using the anesthesia depth assessment model, so as to perform an anesthesia depth assessment based on the multimodal data: when the multimodal data includes the target brain wave data, based on the brain wave feature extraction part, feature extraction is performed on the target brain wave data, and the brain wave features obtained by feature extraction are input into the feature fusion part; when the multimodal data includes In the case of target high-time dimension life data, based on the high-time dimension feature extraction part, feature extraction is performed on the target high-time dimension life data, and the high-time dimension features obtained by feature extraction are input into the feature fusion part; in the case where the multimodal data includes target vital signs data, the target vital signs data are input into the feature fusion part; based on the feature fusion part, feature fusion is performed on the input features to obtain fused features, wherein the input features include at least one of brain wave features, high-time dimension features and target vital signs data; based on the fused features, the model output result of the anesthesia depth assessment model is determined and output; according to the obtained model output result, the anesthesia depth assessment result of the target object is obtained. Among them, the explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0040] See also Figure 2 The method of this embodiment may specifically include the following steps.

[0041] S210, obtaining a trained anesthesia depth assessment model and multimodal data of a target object, wherein the anesthesia depth assessment model includes a brain wave feature extraction part, a high time dimension feature extraction part, and a feature fusion part.

[0042] Among them, the EEG feature extraction part is the part that extracts the features of the target EEG data. The high time dimension feature extraction part is the part that extracts the features of the target high time dimension life data. The feature fusion part is the part that fuses the features input therein.

[0043] In the embodiment of the present invention, the structures and parameters of the brain wave feature extraction part, the high time dimension feature extraction part and the feature fusion part are not specifically limited.

[0044] S220. When the multimodal data includes at least one of target brain wave data, target high time dimension life data and target vital sign data, the multimodal data is input into an anesthesia depth assessment model, and steps S2201 to S2205 are executed using the anesthesia depth assessment model to perform anesthesia depth assessment based on the multimodal data.

[0045] S2201. When the multimodal data includes target brain wave data, based on the brain wave feature extraction part, feature extraction is performed on the target brain wave data, and the brain wave features obtained by the feature extraction are input into the feature fusion part.

[0046] The brain wave feature is a feature obtained by extracting features from target brain wave data using a brain wave feature extraction part.

[0047] In the embodiment of the present invention, when the multimodal data includes the target brain wave data, that is, when the target brain wave data exists in the multimodal data, based on the brain wave feature extraction part, feature extraction is performed on the target brain wave data, that is, the target brain wave data is input into the brain wave feature extraction part to extract the features of the target brain wave data, and the brain wave features obtained by the feature extraction are input into the feature fusion part, that is, the target brain wave data can be involved in the anesthesia depth assessment of the target object; when the multimodal data does not include the target brain wave data, the above operation is not performed, that is, no target brain wave data is involved in the anesthesia depth assessment of the target object. In the embodiment of the present invention, the method of extracting features from the target brain wave data based on the brain wave feature extraction part is not specifically limited.

[0048] It should be noted that the EEG feature extraction part can also be understood as a EEG encoder, and the EEG feature extraction part can be designed based on the data setting of the target EEG data.

[0049] S2202. When the multimodal data includes target high time dimension life data, based on the high time dimension feature extraction part, feature extraction is performed on the target high time dimension life data, and the high time dimension features obtained by the feature extraction are input into the feature fusion part.

[0050] Among them, the high time dimension feature is a feature obtained by extracting features from the target high time dimension life data using the high time dimension feature extraction part.

[0051] In an embodiment of the present invention, when the multimodal data includes target high-time dimension life data, that is, when the target high-time dimension life data exists in the multimodal data, based on the high-time dimension feature extraction part, feature extraction is performed on the target high-time dimension life data, that is, the target high-time dimension life data is input into the high-time dimension feature extraction part to extract features from the target high-time dimension life data, and the high-time dimension features obtained by feature extraction are input into the feature fusion part, that is, the target high-time dimension life data can be involved in the anesthesia depth assessment of the target object; when the multimodal data does not include the target high-time dimension life data, the above operation is not performed, that is, no target high-time dimension life data participates in the anesthesia depth assessment of the target object. In an embodiment of the present invention, the method of extracting features from the target high-time dimension life data based on the high-time dimension feature extraction part is not specifically limited.

[0052] It should be noted that the high time dimension feature extraction part can also be understood as a high time dimension encoder, and the high time dimension feature extraction part can be designed according to the data setting of the target high time dimension life data.

[0053] S2203: When the multimodal data includes target vital sign data, the target vital sign data is input into the feature fusion part.

[0054] It can be understood that the target vital signs data can characterize the life status of the target object and does not need to be extracted through an encoder. Therefore, the target vital signs data can directly participate in feature fusion as feature data, so that the target vital signs data can participate in the anesthesia depth assessment of the target object, and there is no need for other parts to extract features from the target vital signs data; when the multimodal data does not include the target vital signs data, the above operation is not performed, that is, the target vital signs data does not participate in the anesthesia depth assessment of the target object.

[0055] In the embodiment of the present invention, when the multimodal data includes usage data such as the drug flow rate of anesthetics such as propofol and remifentanil, the usage data can also be input into the feature fusion part.

[0056] S2204. Based on the feature fusion part, feature fusion is performed on the input features to obtain fused features, wherein the input features include at least one of brain wave features, high time dimension features, and target vital sign data.

[0057] Among them, the fusion feature is a feature obtained by fusing the features of the input feature fusion part.

[0058] In the embodiment of the present invention, based on the feature fusion part, the input features are fused in series or other manners to obtain fused features. In the embodiment of the present invention, the manner of obtaining fused features by fusion of the input features based on the feature fusion part is not specifically limited.

[0059] In the embodiment of the present invention, the input features may also include usage data such as drug flow rate of anesthetics such as propofol and remifentanil.

[0060] S2205. Based on the fusion features, determine the model output result of the anesthesia depth assessment model and output it.

[0061] In an embodiment of the present invention, the anesthesia depth assessment model may further include a decoding part; the decoding part may be, for example, a fully connected layer capable of dimensionality reduction decoding, and the fused features may be decoded based on the decoding part to effectively fuse the information between the features; a regression relationship is established between the obtained decoding result and the BIS value and then output as a model output result of the anesthesia depth assessment model. In an embodiment of the present invention, there is no specific limitation on the way of outputting the model output result of the anesthesia depth assessment model based on the fused features.

[0062] S230: Obtain an anesthesia depth assessment result of the target object according to the obtained model output result.

[0063] The technical solution of the embodiment of the present invention is that the anesthesia depth assessment model includes a brain wave feature extraction part, a high time dimension feature extraction part and a feature fusion part; the multimodal data is input into the anesthesia depth assessment model, and the anesthesia depth assessment model is used to perform the following five steps to assess the anesthesia depth based on the multimodal data: when the multimodal data includes target brain wave data, based on the brain wave feature extraction part, feature extraction is performed on the target brain wave data, and the brain wave features obtained by feature extraction are input into the feature fusion part; when the multimodal data includes target high time dimension life data, based on the high time dimension feature extraction Part, feature extraction is performed on the target high time dimension life data, and the high time dimension features obtained by feature extraction are input into the feature fusion part; when the multimodal data includes target vital signs data, the target vital signs data is input into the feature fusion part; based on the feature fusion part, feature fusion is performed on the input features to obtain fusion features, wherein the input features include at least one of brain wave features, high time dimension features and target vital signs data; based on the fusion features, the model output results of the anesthesia depth assessment model are determined and output; according to the obtained model output results, the anesthesia depth assessment result of the target object is obtained. The above technical scheme can be performed by using the brain wave feature extraction part and the high time dimension feature extraction part to extract features of the target brain wave data and the target high time dimension life data respectively, so as to adopt different feature extraction methods for different multimodal data, thereby improving the accuracy of feature extraction, and by fusing different features, the information represented by different features is effectively fused, thereby further improving the accuracy of anesthesia depth assessment.

[0064] An optional technical solution, the brain wave feature extraction part includes a processing module, a space embedding module and a time embedding module; based on the brain wave feature extraction part, feature extraction is performed on the target brain wave data, and the brain wave features obtained by the feature extraction are input into the feature fusion part, including: using the brain wave feature extraction part, executing the following three steps to extract features from the target brain wave data: based on the processing module, processing the target brain wave data to obtain a dimension mapping result; based on the space embedding module, processing the dimension mapping result to obtain a space embedding result, and, based on the time embedding module, processing the dimension mapping result to obtain a time embedding result; fusing the dimension mapping result, the space embedding result and the time embedding result to obtain a fusion result, and outputting the brain wave features according to the fusion result; inputting the brain wave features into the feature fusion part.

[0065] Among them, the processing module is a module for performing dimension mapping and other processing on the target brain wave data. The spatial embedding module is a module for processing the dimension mapping result to obtain the spatial embedding result. The temporal embedding module is a module for processing the dimension mapping result to obtain the temporal embedding result. The dimension mapping result is the result of mapping the target brain wave data to the data of the dimension required to be used, for example, the dimension mapping result is the result of mapping the target brain wave data to the data of the input dimension of the encoding part. The spatial embedding result is the result of characterizing the spatial information of the target brain wave data. The temporal embedding result is the result of characterizing the temporal information of the target brain wave data.

[0066] In the embodiment of the present invention, the structures and parameters of the processing module, the spatial embedding module and the temporal embedding module are not specifically limited.

[0067] In an embodiment of the present invention, based on a processing module, dimension mapping and other processing are performed on the target brain wave data to obtain a dimension mapping result; based on a spatial embedding module, spatial information extraction and other processing are performed on the dimension mapping result to obtain a spatial embedding result, and, based on a temporal embedding module, temporal information extraction and other processing are performed on the dimension mapping result to obtain a temporal embedding result; the dimension mapping result, the spatial embedding result and the temporal embedding result are added and fused to obtain a fusion result, and brain wave features are obtained according to the fusion result for output; the brain wave features are input into a feature fusion part.

[0068] In an embodiment of the present invention, the high time dimension feature extraction part may include a high time dimension processing module and a high time dimension time embedding module; based on the high time dimension feature extraction part, feature extraction is performed on the target high time dimension life data, and the high time dimension features obtained by the feature extraction are input into the feature fusion part, including: using the high time dimension feature extraction part, executing the following three steps to extract features from the target high time dimension life data: based on the high time dimension processing module, processing the target high time dimension life data to obtain a high time dimension dimension mapping result; based on the high time dimension time embedding module, processing the high time dimension dimension mapping result to obtain a high time dimension time embedding result; fusing the high time dimension dimension mapping result and the high time dimension time embedding result to obtain a high time dimension fusion result, and outputting the high time dimension features according to the high time dimension fusion result; inputting the high time dimension features into the feature fusion part. In an embodiment of the present invention, a method of processing the target high time dimension life data based on the high time dimension processing module to obtain a high time dimension dimension mapping result may be the same as or different from a method of processing the target brain wave data based on the processing module to obtain a dimension mapping result; a method of processing the high time dimension dimension mapping result based on the high time dimension time embedding module to obtain a high time dimension time embedding result may be the same as or different from a method of processing the dimension mapping result based on the time embedding module to obtain a time embedding result; a method of fusing the high time dimension dimension mapping result and the high time dimension time embedding result to obtain a high time dimension fusion result may be the same as or different from a method of fusing the dimension mapping result, the spatial embedding result and the time embedding result to obtain a fusion result.

[0069] In an embodiment of the present invention, the EEG feature extraction part may further include an encoding module, and / or the high time dimension feature extraction part may further include a high time dimension encoding module to obtain spatiotemporal features; the large model not only performs well in natural language processing tasks, but can also be applied to other modal tasks. In the anesthesia assessment task, considering the performance of the model, resource consumption, the model's processing capabilities for different modal data, and the characteristics of the task, etc., the selection of a suitable large model and fine-tuning technology becomes a key issue. In the field of natural language processing, the generative pre-trained model (Generative Pre-trained Transformer, GPT) series that is pre-trained on a large amount of text corpus has shown excellent language generation capabilities and interactivity, and has shown strong task generalization capabilities and robustness. Therefore, the encoding module and / or the high time dimension encoding module can be, for example, based on the generative pre-trained transformer model 2 (Generative Pre-trained Transformer 2, GPT2); the fusion result can be encoded based on the encoding module to obtain the encoding result, and the encoding result can be output as the EEG feature, and / or the high time dimension fusion result can be encoded based on the high time dimension encoding module to obtain the high time dimension encoding result, and the high time dimension encoding result can be output as the high time dimension feature. It should be noted that the GPT2 large language model is composed of multiple transformer (Transform) blocks. In the process of training the anesthesia depth assessment model, in order to make the Transform more stable and efficient, layer normalization can be added to the query and keyword before the dot product attention mechanism, thereby avoiding excessive values ​​in the attention logic, that is, the formula can be used. To achieve, is the dimension of a head in multi-head attention, and LN represents the normalization layer (layerNorm); the bias term can be omitted in the calculation of Q, K, and V to accelerate training without reducing performance.

[0070] The above technical solution can process the dimension of the target EEG data more in line with the requirements through the processing module. By fusing the dimensional mapping results, spatial embedding results and temporal embedding results, the EEG features can be embedded with temporal and spatial information, thereby further improving the accuracy of anesthesia depth assessment.

[0071] Based on the above scheme, another optional technical scheme is that the processing module includes a slicing sub-module and a dimension mapping sub-module; based on the processing module, the target brain wave data is processed to obtain a dimension mapping result, including: based on the slicing sub-module, the target brain wave data is sliced ​​to obtain a slicing result; based on the dimension mapping sub-module, the slicing result is dimensionally mapped to obtain a dimensional mapping result.

[0072] The slicing submodule is a submodule for slicing the target EEG data. The dimension mapping submodule is a submodule for performing dimension mapping on the slicing result. The slicing result is the result obtained by slicing the target EEG data.

[0073] In the embodiment of the present invention, the structures and parameters of the slicing submodule and the dimension mapping submodule are not specifically limited.

[0074] It is understandable that each module or part after the processing module may have certain requirements for the length of the processed data, for example, may have certain requirements for the time length of the data. Therefore, based on the slicing sub-module, the target brain wave data can be sliced ​​in space or time according to the required spatial length or time length to obtain the slicing result; based on the dimension mapping sub-module, the slicing result can be dimensionally mapped to obtain the dimensional mapping result.

[0075] Exemplarily, based on the slicing submodule, the target EEG data is sliced ​​according to a set time interval to extract local semantic information and obtain a slicing result, which may include at least one target slice; for each target slice in the at least one target slice, based on the dimension mapping submodule, the slicing result may be dimensionally mapped, and the time series data in the slicing result may be projected to the required dimension of a specific pre-trained model to obtain a dimension mapping slice, and the dimension mapping slices corresponding to the at least one target slice are used as the dimension mapping result.

[0076] In an embodiment of the present invention, the processing module may further include a normalization layer, which may use the mean and variance to normalize the time series of the input data, that is, before slicing the target EEG data based on the slicing submodule and obtaining the slicing result, the target EEG data may be normalized based on the normalization layer, and the target EEG data may be updated according to the normalization result.

[0077] In the embodiment of the present invention, since data normalization is crucial for pre-trained models across various modes, in addition to the layer norms used in the pre-trained models, the processing module can also include a simple data normalization block, and the reverse instance norm is implemented through the data normalization block to further promote knowledge transfer, that is, before slicing the target EEG data based on the slicing submodule and obtaining the slicing result, the target EEG data can be subjected to data normalization processing based on the data normalization block, and the target EEG data can be updated according to the normalization result.

[0078] In an embodiment of the present invention, by slicing and dimensionally mapping the target EEG data through a slicing submodule and a dimension mapping submodule, the obtained dimensional mapping result can be more suitable for subsequent processing, thereby improving the efficiency of anesthesia depth assessment.

[0079] Figure 3 It is a flow chart of another anesthesia depth assessment method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the anesthesia depth assessment model is pre-trained by the following steps: obtaining sample data of the sample object and the anesthesia depth label of the sample data, and using the sample data and the anesthesia depth label as a group of training samples, wherein the sample data includes at least one of sample brain wave data, sample high time dimension life data and sample vital sign data; based on multiple groups of training samples, the original assessment model is trained to obtain an anesthesia depth assessment model. Among them, the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here.

[0080] See also Figure 3 The method of this embodiment may specifically include the following steps:

[0081] S310. Obtain sample data of the sample object and anesthesia depth labels of the sample data, and use the sample data and anesthesia depth labels as a group of training samples, wherein the sample data includes at least one of sample brain wave data, sample high time dimension life data, and sample vital sign data.

[0082] Among them, the sample data is an image used to train an anesthesia depth assessment model. The sample object is the object targeted when the sample data is collected. The anesthesia depth label is a label that can reflect the anesthesia depth of the sample object; the anesthesia depth label can be a bispectral index. The training sample can be understood as a sample used to train an anesthesia depth assessment model, and a group of training samples may include a group of sample data and its corresponding anesthesia depth label. The sample brain wave data is the brain wave data of the sample object. The sample high time dimension life data is the vital sign data of the sample object with high time dimension characteristics, and the sample high time dimension life data is also the vital sign data of the sample object with rich or complex information in the time dimension; the sample high time dimension life data may include, for example, at least one of electrocardiogram data, photoplethysmography data, arterial pressure wave, central venous pressure wave, airway pressure waveform data, and carbon dioxide waveform, etc. The sample vital signs data is non-high time dimension vital signs data of the sample object; the sample vital signs data may include, for example, at least one of blood pressure data, heart rate data, body temperature data, ST segment data in an electrocardiogram, pulse oximetry, drug flow rate of anesthetics such as propofol and remifentanil, and the like.

[0083] In an embodiment of the present invention, multiple groups of training samples can be selected from the vital signs database (VitalDB). For example, all data of 320 surgeries in the VitalDB database can be selected as multiple groups of training samples. In order to prevent individual differences and environmental differences from excessively affecting the performance of the model, so that the anesthesia depth assessment model obtained by the final training cannot represent the actual level of the model, the training samples in the VitalDB database can be screened according to the age, height, weight, surgical risk level and / or anesthetic drugs of the sample objects corresponding to each group of training samples. For example, 320 surgeries can be selected as multiple groups of training samples.

[0084] In the embodiment of the present invention, since the intraoperative data used to determine the sample data may be missing during the recording process, the start and end times of the intraoperative data between different modalities are different, and the sampling frequencies of the intraoperative data of different modalities are different, the intraoperative data can be preprocessed. Specifically, the intraoperative data with a higher sampling rate can be processed, and the induction period data, postoperative data, and data of the abnormal data time period can be removed. Then, according to the set sample time length, the intraoperative data can be preprocessed. Segmentation is performed, and at least one set of training samples is determined based on the segmentation results, and then multiple sets of training samples are obtained through multiple sets of data during surgery, where B is the number of data segmentations during surgery, C is the number of channels, except for EEG data and ECG data, which are two channels, and the remaining modal data are single channels, and F is the characteristic length of a single channel. The bispectral index during surgery can be used as an anesthesia depth label in the training samples It should be noted that, when the sample data includes sample vital signs data and sample high-time dimension life data, the sample vital signs data can be processed in parallel according to the data interception time point of the sample high-time dimension life data to update the sample vital signs data according to the processing result, thereby obtaining the updated sample vital signs data. In the embodiment of the present invention, the multimodal data may also be obtained through the above process, that is, the multimodal data is determined through the above process according to the data during the operation of the target object.

[0085] It should be noted that the anesthesia depth assessment model may have certain requirements for the input time length, so the training samples can be adjusted according to the input requirements.

[0086] S320: Based on multiple groups of training samples, the original evaluation model is trained to obtain an anesthesia depth evaluation model.

[0087] The original evaluation model is a model to be trained that can evaluate the depth of anesthesia.

[0088] In an embodiment of the present invention, the original evaluation model can be trained based on multiple groups of training samples to obtain an anesthesia depth evaluation model; the original evaluation model can also be trained based on some training samples in the multiple groups of training samples to obtain an anesthesia depth evaluation model, and then the remaining training samples in the multiple groups of training samples are used to evaluate the obtained anesthesia depth evaluation model. In this case, in order to evaluate the performance of the model under different situations, the multiple groups of training samples can be processed differently to meet the evaluation requirements.

[0089] In the embodiment of the present invention, the method of training the original evaluation model based on multiple groups of training samples to obtain the anesthesia depth evaluation model is not specifically limited.

[0090] S330, obtaining the trained anesthesia depth assessment model and multimodal data of the target object.

[0091] S340. When the multimodal data includes at least one of target brain wave data, target high time dimension life data and target vital sign data, the multimodal data is input into an anesthesia depth assessment model, and an anesthesia depth assessment result of the target object is obtained based on the output result of the anesthesia depth assessment model.

[0092] The technical solution of the embodiment of the present invention obtains sample data of a sample object and an anesthesia depth label of the sample data, and uses the sample data and the anesthesia depth label as a group of training samples, wherein the sample data includes at least one of sample brain wave data, sample high time dimension life data, and sample vital sign data; based on multiple groups of training samples, the original evaluation model is trained to obtain an anesthesia depth evaluation model. The above technical solution can improve the scope of application of the anesthesia depth evaluation model by training the original evaluation model with sample data including at least one of sample brain wave data, sample high time dimension life data, and sample vital sign data, so that the anesthesia evaluation model can realize anesthesia depth evaluation by inputting any one or more of the target brain wave data, the target high time dimension life data, and the target vital sign data.

[0093] An optional technical solution, based on multiple groups of training samples, trains the original evaluation model to obtain an anesthesia depth evaluation model, including: randomly selecting a preset proportion group of training samples from multiple groups of training samples as a proportion sample set; selecting a selected sample set from the proportion sample set, and performing noise processing on the selected sample set to obtain a noise sample set; based on multiple groups of training samples, the proportion sample set and at least one of the noise sample set, trains the original evaluation model to obtain an anesthesia depth evaluation model.

[0094] Among them, the proportional sample set is a sample set obtained by randomly selecting a preset proportion of training samples from multiple groups of training samples; the proportional sample set can also be called a small sample set (Small-shots). The preset proportion is the ratio of the preset selected proportional sample set to the number of multiple groups of training samples. The preset proportion can be, for example, 5%, 10% or 15%, etc. The selected sample set is a sample set composed of training samples selected from the proportional sample set. The noise sample set (Noise-shots) is a sample set composed of training samples with noise interference.

[0095] In the embodiment of the present invention, the selected sample set can be selected randomly or according to a preset selection rule from the proportional sample set, or all training samples in the proportional sample set can be selected to form the selected sample set. In the embodiment of the present invention, the method of selecting the selected sample set from the proportional sample set is not specifically limited.

[0096] In an embodiment of the present invention, noise processing such as adding Gaussian noise can be performed on the sample data in each training sample in the selected sample set to obtain a noise sample set. It should be noted that adding Gaussian noise to the sample data may, for example, be adding noise with a standard deviation of 1, 5, or 10; another example may be copying each training sample into three groups of training samples, and adding noise with standard deviations of 1, 5, and 10 to the sample data in the three groups of training samples, respectively, to obtain a noise sample set with a number of training samples that is three times that of the selected sample set. In an embodiment of the present invention, the method of performing noise processing on the selected sample set to obtain the noise sample set is not specifically limited.

[0097] In an embodiment of the present invention, multiple groups of training samples can also be used as normal sample sets (Normal-shots), and a preset number of groups of training samples are selected from the normal sample set as the initial sample set, and a fusion operation is performed on the initial sample set to obtain a difficult modal sample set (Different modal-shots). The fusion operation can, for example, be a fusion of sample high time dimension life data and sample vital sign data between the training samples in the initial sample set, and another example can be a fusion of sample high time dimension life data and sample vital sign data in the training samples in the initial sample set, and so on. In an embodiment of the present invention, the fusion method of the fusion operation is not specifically limited. In an embodiment of the present invention, the original evaluation model can be trained based on multiple groups of training samples, namely, at least one of a normal sample set, a proportional sample set, a noise sample set, and a difficult modal sample set to obtain an anesthesia depth evaluation model.

[0098] In an embodiment of the present invention, at least one of the multiple groups of training samples, proportional sample sets, noise sample sets, and difficult modal sample sets used to train the original evaluation model can be divided into training sets and test sets, respectively, so that the original evaluation model can be trained with the training set and the anesthesia depth evaluation model can be tested with the test set.

[0099] In an embodiment of the present invention, a method for evaluating the depth of anesthesia with the most advanced technology (State-of-the-art, SOTA) can be selected, for example, Interpretable Forecasts BIS, deep residual shrinkage convolutional network (DRSN-CW-CON) and hybrid network (CombineNET) can be selected to test the above three methods and the anesthesia depth evaluation model of the embodiment of the present invention, and verify the advance of the anesthesia depth evaluation model according to the test results; since the data sets used for the tests of the above three methods are not public, in order to make the test results credible, we can reproduce the above three methods and test them on the prepared data sets; since the above three methods only use EEG data, it is impossible to evaluate the depth of anesthesia of multimodal data; in order to compare the performance of the anesthesia depth evaluation model on multimodal data, a large model large brain model (Large Brain Model, LABRAM) trained by EEG data can be selected as a comparison model for multimodal data testing; multiple groups of training samples, proportional sample sets and noise sample sets can be used as test sets corresponding to Interpretable Forecasts. BIS, DRSN-CW-CON and CombineNET perform anesthesia depth assessment based on EEG data. The large model LABRAM trained by EEG data is selected as the comparison model for multimodal data testing. The test set corresponding to the difficult modal sample set can be used as the test LABRAM to evaluate the anesthesia depth based on multimodal data. During the test, the accuracy of each method can be analyzed by calculating the difference between the measured data and the predicted data. The evaluation indicators of each method can be the root mean square error (RMSE), the mean absolute error (MAE) and the Pearson correlation coefficient (PEARSON). The smaller the RMSE and MAE values ​​are, and the larger the PEARSON value is, the better the performance of the model is. Different indicators have different evaluation tendencies for the prediction ability of the model. The specific formula used for RMSE is as well as ,in, is the point of the prediction sequence, is the point of the real sequence; the specific formula used by PEARSON is ,in, is the predicted sequence, is a real sequence, and the correlation coefficient is a number with an absolute value less than 1. The closer the value is to 1, the greater the correlation between the two variables. The evaluation indicators of the test results of the above-mentioned models on the test sets corresponding to the four data sets are calculated, and it is concluded according to the results that the anesthesia depth assessment model of the embodiment of the present invention reaches an advanced level in Normal-shots and Small-shots, and reaches the most advanced level in Noise-shots and Different modal-shots. The anesthesia depth assessment model of the embodiment of the present invention can not only complete the anesthesia depth assessment task through EEG data or other modal data, but also has excellent performance in online rapid learning and noise resistance, and can complete the anesthesia depth assessment task through multimodal data. It should be noted that since the input time lengths of the three SOTA methods are not exactly the same, and the anesthesia depth assessment model and LABRAM of the embodiment of the present invention have no input length requirements, it is possible to choose to adjust the training set based on the input of each SOTA method as the basic input requirement. Interpretable Forecasts BIS and DRSN-CW-CON use 6s as a sample period, and the input shape of the model is (Bitch_size, 2,600). CombineNET uses 30s as a sample period, and the input shape of the model is (Bitch_size, 2,3000). The anesthesia depth assessment model and LABRAM of the embodiment of the present invention adjust the training set according to the input requirements.

[0100] It is understandable that since small sample data can complete the task well, large models have advantages when learning with small sample sets because they have more parameters and stronger representation capabilities, can better capture complex patterns in the data, and retain previously learned features. In the test of small sample sets under the anesthesia depth assessment task, the fine-tuning method of the anesthesia depth assessment model greatly exceeds all SOTA methods.

[0101] In an embodiment of the present invention, by determining a proportional sample set and a noise sample set, the original evaluation model can be trained based on at least one of multiple groups of training samples, proportional sample sets and noise sample sets to obtain an anesthesia depth evaluation model. This can achieve high efficiency in training the anesthesia depth evaluation model and can make the trained anesthesia depth evaluation model have good noise resistance.

[0102] Another optional technical solution is to train the original evaluation model based on multiple groups of training samples to obtain an anesthesia depth evaluation model, including: freezing the parameters of the self-attention layer in the original evaluation model and the parameters in the feedforward neural network; based on multiple groups of training samples, fine-tuning the parameters in the original evaluation model except the frozen parameters to obtain an anesthesia depth evaluation model.

[0103] It is understandable that comprehensive fine-tuning of large models usually requires a lot of computing resources. To alleviate this problem, the consumption of fine-tuning resources can be significantly reduced through the parameter-efficient fine-tuning (PEFT) method. Therefore, large-scale fine-tuning techniques such as parameter-efficient fine-tuning can be used to fine-tune the parameters of the original evaluation model except for the frozen parameters.

[0104] In an embodiment of the present invention, the self-attention layer and the feedforward neural network may be the self-attention layer and the feedforward neural network in the encoding module. Since the self-attention layer and the feedforward neural network may contain most of the learning knowledge from the pre-trained language model, the parameters in the self-attention layer and the feedforward neural network, that is, the learning knowledge in the self-attention layer and the feedforward neural network may be frozen, and the parameters in the original evaluation model other than the frozen parameters may be fine-tuned to retain the learning knowledge, provide assistance for the anesthesia depth assessment task, and thereby obtain the anesthesia depth assessment model. For example, for fine-tuning the encoding module, only the position embedding layer and the normalization layer in the encoding module may be fine-tuned.

[0105] In an embodiment of the present invention, based on multiple groups of training samples, the parameters other than the frozen parameters in the original evaluation model are fine-tuned to obtain an anesthesia depth evaluation model. For example, the parameters of the spatial embedding module, the temporal embedding module, the position embedding layer and the normalization layer in the encoding module and the regression module in the original evaluation model can be trained until the preset convergence conditions are reached, and the trained anesthesia depth evaluation model and model parameters based on the large model fine-tuning technology are obtained. The fine-tuning process can use the mean square error (MSE) as the loss function. The mean square error is the most commonly used error in the regression loss function. As the error decreases, the gradient also decreases, which is conducive to convergence. Using a fixed learning rate, it can also converge to the minimum value faster.

[0106] For example, see Figure 4 The encoding module may include a position embedding layer and a Transform block. The Transform block may include two normalization layers, a self-attention layer, and a feed-forward neural network. The parameters of the self-attention layer and the feed-forward neural network may be frozen, and the position embedding layer and the two normalization layers may be fine-tuned.

[0107] The above technical solution can retain the learning knowledge from the pre-trained language model by freezing the parameters of the self-attention layer and the parameters in the feedforward neural network, thereby improving the accuracy of anesthesia depth assessment.

[0108] In order to better understand the technical solution of the above embodiment of the present invention, an optional example is provided here. Figure 5 The anesthesia depth assessment model includes an EEG feature extraction part, a high time dimension feature extraction part, a feature fusion part, and a BIS decoding part for decoding to obtain the BIS value. The EEG feature extraction part includes a processing module, a spatial embedding module, a temporal embedding module, and GPT2. The high time dimension feature extraction part includes a processing module, a temporal embedding module, and GPT2. The processing module includes a slicing submodule and a dimensional mapping submodule. Obtain the trained anesthesia depth assessment model and the multimodal data of the target object during surgery, wherein the multimodal data includes target brain wave data, target high-time dimension life data and target vital signs data, the target high-time dimension life data includes ECG data, PPG data, APW data and CO2W, and the target vital signs data includes BP data, HR data, ST segment data in the electrocardiogram and SPO2; the anesthesia depth assessment model can extract features from the target brain wave data based on the brain wave feature extraction part, and input the brain wave features obtained by feature extraction into the feature fusion part; based on the high-time dimension feature extraction part, extract features from the target high-time dimension life data, and input the high-time dimension features obtained by feature extraction into the feature fusion part; input the target vital signs data into the feature fusion part; based on the feature fusion part, fuse the input features to obtain fused features; based on the BIS decoding part, decode the fused features to obtain the model output results of the anesthesia depth assessment model and output them. It should be noted that, refer to Figure 6 and Figure 7 , GPT2 can include a position embedding layer and a Transform block. In addition, the EEG feature extraction part and the high time dimension feature extraction part can also include a one-dimensional convolution layer and a linear transformation layer respectively; GPT2 can fuse the fusion result with the output result of the position embedding layer, and input the first fusion result into the one-dimensional convolution layer and the Transform block respectively, extract the input spatiotemporal information and save the main input feature information respectively, add and fuse the output result of the one-dimensional convolution layer and the output result of the Transform block, and obtain a second fusion result that represents the main input information and fuses the spatiotemporal information, and input the second fusion result into the linear transformation layer to obtain EEG features or high time dimension features.

[0109] Figure 8This is a structural block diagram of an anesthesia depth assessment device provided in an embodiment of the present invention. The device is used to execute the anesthesia depth assessment method provided in any of the above embodiments. The device and the anesthesia depth assessment method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the anesthesia depth assessment device, please refer to the embodiment of the anesthesia depth assessment method. Figure 8 The device may specifically include: a multimodal data acquisition module 410 and an anesthesia depth assessment result acquisition module 420.

[0110] Among them, the multimodal data acquisition module 410 is used to obtain the trained anesthesia depth assessment model and the multimodal data of the target object; the anesthesia depth assessment result acquisition module 420 is used to input the multimodal data into the anesthesia depth assessment model when the multimodal data includes at least one of the target brain wave data, the target high time dimension life data and the target vital sign data, and obtain the anesthesia depth assessment result of the target object according to the output result of the anesthesia depth assessment model.

[0111] Optionally, the anesthesia depth assessment model includes a brain wave feature extraction part, a high time dimension feature extraction part and a feature fusion part; the anesthesia depth assessment result obtaining module 420 includes: a multimodal data input submodule, used to input the multimodal data into the anesthesia depth assessment model, so as to utilize the anesthesia depth assessment model to perform anesthesia depth assessment based on the multimodal data through the following five units: a brain wave feature input unit, used to extract features from the target brain wave data based on the brain wave feature extraction part when the multimodal data includes target brain wave data, and input the brain wave features obtained by feature extraction into the feature fusion part; a high time dimension feature input unit, used to extract features from the target brain wave data based on the high time dimension feature extraction part when the multimodal data includes target high time dimension life data. Feature extraction is performed on the target high-time dimension life data, and the high-time dimension features obtained by the feature extraction are input into the feature fusion part; a target vital sign data input unit is used to input the target vital sign data into the feature fusion part when the multimodal data includes the target vital sign data; a fusion feature obtaining unit is used to perform feature fusion on the input features based on the feature fusion part to obtain fusion features, wherein the input features include at least one of brain wave features, high-time dimension features and target vital sign data; a model output result output unit is used to determine the model output result of the anesthesia depth assessment model based on the fusion feature and output it; an anesthesia depth assessment result obtaining submodule is used to obtain the anesthesia depth assessment result of the target object according to the obtained model output result.

[0112] Optionally, based on the above-mentioned device, the brain wave feature extraction part includes a processing module, a spatial embedding module and a temporal embedding module; the brain wave feature input unit includes: a target brain wave data feature extraction subunit, used to use the brain wave feature extraction part to execute the following three contents to extract features from the target brain wave data: based on the processing module, the target brain wave data is processed to obtain a dimension mapping result; based on the spatial embedding module, the dimension mapping result is processed to obtain a spatial embedding result, and, based on the temporal embedding module, the dimension mapping result is processed to obtain a temporal embedding result; the dimension mapping result, the spatial embedding result and the temporal embedding result are fused to obtain a fusion result, so as to obtain the brain wave feature according to the fusion result for output; a brain wave feature input subunit, used to input the brain wave feature into the feature fusion part.

[0113] Optionally, based on the above-mentioned device, the processing module includes a slicing sub-module and a dimension mapping sub-module; the target brain wave data feature extraction sub-unit can be specifically used to: based on the slicing sub-module, slice the target brain wave data to obtain a slicing result; based on the dimension mapping sub-module, dimensionally map the slicing result to obtain a dimensional mapping result.

[0114] Optionally, the device may also include the following modules for pre-training an anesthesia depth assessment model: a training sample as a module, used to obtain sample data of a sample object and an anesthesia depth label of the sample data, and use the sample data and the anesthesia depth label as a group of training samples, wherein the sample data includes at least one of sample brain wave data, sample high time dimension life data and sample vital sign data; an anesthesia depth assessment model acquisition module, used to train the original assessment model based on multiple groups of training samples to obtain an anesthesia depth assessment model.

[0115] Optionally, based on the above-mentioned device, an anesthesia depth assessment model obtaining module includes: a proportional sample set as a submodule, which is used to randomly select a preset proportional group of training samples from multiple groups of training samples as a proportional sample set; a noise sample set obtaining submodule, which is used to select a selected sample set from the proportional sample set, and perform noise processing on the selected sample set to obtain a noise sample set; a first anesthesia depth assessment model obtaining submodule, which is used to train the original assessment model based on at least one of multiple groups of training samples, proportional sample sets and noise sample sets to obtain an anesthesia depth assessment model.

[0116] Optionally, based on the above-mentioned device, an anesthesia depth assessment model is obtained, including: a parameter freezing sub-module, used to freeze the parameters of the self-attention layer in the original assessment model and the parameters in the feedforward neural network; a second anesthesia depth assessment model is obtained sub-module, used to fine-tune the parameters other than the frozen parameters in the original assessment model based on multiple groups of training samples to obtain an anesthesia depth assessment model.

[0117] The anesthesia depth assessment device provided by the embodiment of the present invention obtains the trained anesthesia depth assessment model and the multimodal data of the target object through the multimodal data acquisition module; through the anesthesia depth assessment result acquisition module, when the multimodal data includes at least one of the target brain wave data, the target high time dimension life data and the target vital sign data, the multimodal data is input into the anesthesia depth assessment model, and the anesthesia depth assessment result of the target object is obtained according to the output result of the anesthesia depth assessment model. The above device can be implemented without being limited to only through brain wave data to achieve anesthesia depth assessment. By using any data among the target brain wave data, the target high time dimension life data and the target vital sign data as the determining factor of anesthesia depth assessment, when the brain wave data is interfered by equipment such as high-frequency electrosurgery, it is not necessary to use the target brain wave data as the determining factor of anesthesia depth assessment, and other data can be used for anesthesia depth assessment, thereby avoiding the problem of poor accuracy of anesthesia depth assessment based on brain wave data when the brain wave data is interfered by equipment such as high-frequency electrosurgery, thereby improving the accuracy of anesthesia depth assessment.

[0118] The anesthesia depth assessment device provided in the embodiment of the present invention can execute the anesthesia depth assessment method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0119] It is worth noting that in the above-mentioned embodiment of the anesthesia depth assessment device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0120] Fig. 9A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0121] like Fig. 9 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0122] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0123] The processor 11 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as anesthesia depth assessment methods.

[0124] In some embodiments, the anesthesia depth assessment method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the anesthesia depth assessment method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the anesthesia depth assessment method in any other appropriate manner (e.g., by means of firmware).

[0125] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0127] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0128] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0129] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0130] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0131] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0132] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for assessing depth of anesthesia, characterized in that: include: Obtain the trained anesthesia depth assessment model and multimodal data of the target object; In the case where the multimodal data includes at least two of target brain wave data, target high-time dimension life data, target vital sign data, and drug flow rate of an anesthetic, the multimodal data is input into the anesthesia depth assessment model, and an anesthesia depth assessment result of the target object is obtained according to an output result of the anesthesia depth assessment model, wherein the target high-time dimension life data includes electrocardiogram data, photoplethysmography data, arterial pressure wave, central venous pressure wave, airway pressure waveform data, and carbon dioxide waveform, and the target vital sign data includes blood pressure data, heart rate data, body temperature data, repolarization process data of various parts of the ventricle in the electrocardiogram after depolarization, and pulse oxygen saturation; The anesthesia depth assessment model includes a brain wave feature extraction part, a high time dimension feature extraction part and a feature fusion part; The step of inputting the multimodal data into the anesthesia depth assessment model and obtaining an anesthesia depth assessment result of the target object according to an output result of the anesthesia depth assessment model comprises: The multimodal data is input into the anesthesia depth assessment model, and the anesthesia depth assessment model is used to perform the following five steps to perform anesthesia depth assessment based on the multimodal data: In the case where the multimodal data includes the target brain wave data, based on the brain wave feature extraction part, feature extraction is performed on the target brain wave data, and the brain wave features obtained by feature extraction are input into the feature fusion part; In the case where the multimodal data includes the target high time dimension life data, based on the high time dimension feature extraction part, feature extraction is performed on the target high time dimension life data, and the high time dimension features obtained by the feature extraction are input into the feature fusion part; In a case where the multimodal data includes the target vital sign data, inputting the target vital sign data into the feature fusion part; In the case where the multimodal data includes the drug flow rate, inputting the drug flow rate into the feature fusion part; Based on the feature fusion part, feature fusion is performed on the input features to obtain fusion features, wherein the input features include at least one of the brain wave features, the high time dimension features, the target vital sign data, and the drug flow rate; Based on the fusion feature, determining a model output result of the anesthesia depth assessment model for output; Obtaining anesthesia depth assessment results of the target object according to the obtained model output results; The EEG feature extraction part includes a processing module, a spatial embedding module, a temporal embedding module, GPT2, a one-dimensional convolutional layer and a linear transformation layer, and GPT2 includes a position embedding layer and a converter block; The step of extracting features of the target brain wave data based on the brain wave feature extraction part and inputting the brain wave features obtained by the feature extraction into the feature fusion part includes: Using the EEG feature extraction part, the following three steps are performed to extract features from the target EEG data: Based on the processing module, the target brain wave data is processed to obtain a dimension mapping result; Based on the spatial embedding module, the dimension mapping result is processed to obtain a spatial embedding result, and based on the temporal embedding module, the dimension mapping result is processed to obtain a temporal embedding result; The dimension mapping result, the spatial embedding result and the temporal embedding result are fused to obtain a fusion result, and the fusion result is fused with the output result of the position embedding layer through GPT2, and the first fusion result is respectively input into the one-dimensional convolution layer and the converter block, and the output result of the one-dimensional convolution layer and the output result of the converter block are added and fused to obtain a second fusion result that represents the main input information and fuses the spatiotemporal information, and the second fusion result is input into the linear transformation layer to obtain the brain wave feature for output; The EEG feature is input into the feature fusion part; the anesthesia depth assessment model is pre-trained by the following steps: Acquire sample data of a sample object and an anesthetic depth label of the sample data, and use the sample data and the anesthetic depth label as a set of training samples, wherein the sample data includes at least two of sample brain wave data, sample high time dimension life data, sample vital sign data, and drug flow rate of an anesthetic; Based on the multiple groups of training samples, the original evaluation model is trained to obtain the anesthesia depth evaluation model; The method further comprises: Taking multiple groups of the training samples as common sample sets, selecting a preset number of groups of training samples from the common sample sets as initial sample sets, fusing the sample high time dimension life data and sample vital sign data of the training samples in the initial sample sets, or fusing the sample high time dimension life data and sample vital sign data of the training samples in the initial sample sets, to obtain a difficult modality sample set; The method of training the original evaluation model based on the plurality of training samples to obtain the anesthesia depth evaluation model comprises: Randomly selecting a preset proportion group of training samples from the plurality of groups of training samples as a proportion sample set; Selecting a selected sample set from the proportional sample set, and performing noise processing on the selected sample set to obtain a noise sample set; Based on the common sample set, the proportional sample set, the noise sample set and the difficult modality sample set, the original evaluation model is trained to obtain the anesthesia depth evaluation model; The performing noise processing on the selected sample set to obtain a noise sample set includes: Adding Gaussian noise to the sample data of each training sample in the selected sample set to obtain a noise sample set; The method further comprises: In the case where the sample data includes sample vital signs data and sample high time dimension life data, the sample vital signs data are processed in parallel according to the data capture time point of the sample high time dimension life data to update the sample vital signs data according to the processing result.

2. The method according to claim 1, characterized in that The processing module includes a slicing submodule and a dimension mapping submodule; The processing module processes the target brain wave data to obtain a dimensional mapping result, including: Based on the slicing submodule, slicing the target EEG data to obtain a slicing result; Based on the dimension mapping submodule, dimension mapping is performed on the slicing result to obtain a dimension mapping result.

3. The method according to claim 1, characterized in that The method of training the original evaluation model based on the plurality of training samples to obtain the anesthesia depth evaluation model comprises: Freeze the parameters of the self-attention layer in the original evaluation model and the parameters in the feedforward neural network; Based on the multiple groups of training samples, the parameters other than the frozen parameters in the original evaluation model are fine-tuned to obtain the anesthesia depth evaluation model.

4. An anesthesia depth assessment device, characterized in that: include: A multimodal data acquisition module, used to acquire the trained anesthesia depth assessment model and multimodal data of the target object; anesthesia depth assessment result obtaining module, for inputting the multimodal data into the anesthesia depth assessment model when the multimodal data includes at least two of target brain wave data, target high time dimension life data, target vital sign data and drug flow rate of anesthetic, and obtaining the anesthesia depth assessment result of the target object according to the output result of the anesthesia depth assessment model, wherein the target high time dimension life data includes electrocardiogram data, photoplethysmography data, arterial pressure wave, central venous pressure wave, airway pressure waveform data and carbon dioxide waveform, and the target vital sign data includes blood pressure data, heart rate data, body temperature data, repolarization process data of each part of the ventricle in the electrocardiogram after depolarization, and pulse oxygen saturation; The anesthesia depth assessment model includes a brain wave feature extraction part, a high time dimension feature extraction part and a feature fusion part; The anesthesia depth assessment result obtaining module comprises: The multimodal data input submodule is used to input the multimodal data into the anesthesia depth assessment model, so as to utilize the anesthesia depth assessment model to perform anesthesia depth assessment based on the multimodal data through the following five units: a brain wave feature input unit, configured to extract features from the target brain wave data based on the brain wave feature extraction part when the multimodal data includes the target brain wave data, and input the brain wave features obtained by the feature extraction into the feature fusion part; a high time dimension feature input unit, configured to extract features from the target high time dimension life data based on the high time dimension feature extraction part when the multimodal data includes the target high time dimension life data, and input the high time dimension features obtained by the feature extraction into the feature fusion part; a target vital sign data input unit, configured to input the target vital sign data into the feature fusion part when the multimodal data includes the target vital sign data; a drug flow rate input unit, configured to input the drug flow rate into the feature fusion part when the multimodal data includes the drug flow rate; a fusion feature obtaining unit, configured to perform feature fusion on the input features based on the feature fusion part to obtain fusion features, wherein the input features include at least one of the brain wave features, the high time dimension features, the target vital sign data, and the drug flow rate; A model output result output unit, used to determine the model output result of the anesthesia depth assessment model based on the fusion feature and output it; An anesthesia depth assessment result obtaining submodule is used to obtain an anesthesia depth assessment result of the target object according to the obtained model output result; The EEG feature extraction part includes a processing module, a spatial embedding module, a temporal embedding module, GPT2, a one-dimensional convolutional layer and a linear transformation layer, and GPT2 includes a position embedding layer and a converter block; The brain wave feature input unit includes: a target brain wave data feature extraction subunit, which is used to use the brain wave feature extraction part to perform the following three contents to extract features from the target brain wave data: Based on the processing module, the target brain wave data is processed to obtain a dimension mapping result; Based on the spatial embedding module, the dimension mapping result is processed to obtain a spatial embedding result, and based on the temporal embedding module, the dimension mapping result is processed to obtain a temporal embedding result; The dimension mapping result, the spatial embedding result and the temporal embedding result are fused to obtain a fusion result, and the fusion result is fused with the output result of the position embedding layer through GPT2, and the first fusion result is respectively input into the one-dimensional convolution layer and the converter block, and the output result of the one-dimensional convolution layer and the output result of the converter block are added and fused to obtain a second fusion result that represents the main input information and fuses the spatiotemporal information, and the second fusion result is input into the linear transformation layer to obtain the brain wave feature for output; A brain wave feature input subunit, used for inputting the brain wave feature into the feature fusion part; The anesthesia depth assessment device further comprises the following modules which are pre-trained to obtain the anesthesia depth assessment model: The training sample is used as a module to obtain sample data of a sample object and an anesthetic depth label of the sample data, and use the sample data and the anesthetic depth label as a group of training samples, wherein the sample data includes at least two of sample brain wave data, sample high time dimension life data, sample vital sign data, and drug flow rate of an anesthetic agent; An anesthesia depth assessment model obtaining module is used to train the original assessment model based on multiple groups of training samples to obtain the anesthesia depth assessment model; The device is also used for: Taking multiple groups of the training samples as common sample sets, selecting a preset number of groups of training samples from the common sample sets as initial sample sets, fusing the sample high time dimension life data and sample vital sign data of the training samples in the initial sample sets, or fusing the sample high time dimension life data and sample vital sign data of the training samples in the initial sample sets, to obtain a difficult modality sample set; The anesthesia depth assessment model obtaining module includes: The proportion sample set is used as a submodule to randomly select a preset proportion group of training samples from the plurality of groups of training samples as the proportion sample set; A noise sample set obtaining submodule is used to select a selected sample set from the proportional sample set, and perform noise processing on the selected sample set to obtain a noise sample set; A first anesthesia depth assessment model obtaining submodule is used to train the original assessment model based on the common sample set, the proportional sample set, the noise sample set and the difficult modal sample set to obtain the anesthesia depth assessment model; The performing noise processing on the selected sample set to obtain a noise sample set includes: Adding Gaussian noise to the sample data of each training sample in the selected sample set to obtain a noise sample set; The device is also used for: In the case where the sample data includes sample vital signs data and sample high time dimension life data, the sample vital signs data are processed in parallel according to the data capture time point of the sample high time dimension life data to update the sample vital signs data according to the processing result.

5. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the anesthesia depth assessment method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the anesthesia depth assessment method as described in any one of claims 1-3 when executed.

Citation Information

Patent Citations

  • Analytical model training method and device, electronic equipment and storage medium

    CN112308238A

  • Anesthesia depth prediction method

    CN115444366A

  • Brain pattern recognition method and device, electronic equipment and storage medium

    CN115712885A

  • Anesthesia depth evaluation method, system and device and storage medium

    CN116636817A

  • Single-step adversarial training method and system based on data enhancement and step length adjustment

    CN117171762A