Sleep stage classification method, physical state stage method, and computer system

Abstract

Provided are a method and a system for training a sleep stage classification model of a self-supervised learning infrastructure using a small number of labels. A sleep stage classification method executed by a computer system according to an embodiment includes inputting sleep test data into a sleep stage classification model of a self-supervised learning infrastructure, and classifying a sleep stage from the sleep test data using the sleep stage classification model of the self-supervised learning infrastructure. The sleep classification model of the self-supervised learning infrastructure may be one in which a pattern for sleep stage classification is learned from new sleep data by transfer learning by finely adjusting a weight value based on a representation learning model in which a representation is learned using sleep signal data.

Description

Technical Field

[0001] The following description relates to a technique for classifying sleep stages.

Background Art

[0002] Sleep occupies a very large proportion in daily life and has an important impact on the body. Since sleep disorders and sleep deprivation cause problems in daily life such as decreased concentration, research on sleep has been actively conducted. Sleep stage classification is an essential element in sleep research, and it has become possible to predict efficiently and accurately by using artificial intelligence. Sleep stage classification based on deep learning mainly used a method based on a supervised learning model, but recently, there has been an increasing interest in models using self-supervised learning.

[0003] In the sleep stage classification method using a supervised learning model, rules are utilized using the given labels. Such a sleep stage classification method using a supervised learning model has several limitations. In order to improve the performance of the supervised learning model, it is realistically limited to produce labels every time for each dataset and proceed with learning, and there is a problem that the quality of rule learning also depends on the quality of the labels. In addition, the performance of the sleep stage classification method using a supervised learning model is limited to the dataset on which learning has been performed, so it is not appropriate.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Provided are a method and a system for classifying sleep stages with few labels by extracting a compressed representation that appropriately represents data from the signal itself by a self-supervised learning technique.

Means for Solving the Problems

[0005] A sleep stage classification method performed by a computer system includes the steps of inputting sleep test data into a sleep stage classification model on a self-supervised learning platform, and classifying sleep stages from the sleep test data using the sleep stage classification model on the self-supervised learning platform, wherein the sleep stage classification model on the self-supervised learning platform may have learned patterns for sleep stage classification from new sleep data by transfer learning, by fine-tuning weights based on a representation learning model whose representations have been learned using sleep signal data.

[0006] A sleep classification model on a self-supervised learning platform may be constructed as an adversarial generative model on an unsupervised learning platform in order to minimize the influence of class imbalances inherent in the sleep signal data.

[0007] An unsupervised generative adversarial neural network model may be trained to maximize the mutual information between the signals generated by the generators and the class codes, by receiving pre-defined time-unit electroencephalogram (EEG) and electrooculogram (EOG) data extracted from sleep signal data.

[0008] A generator may take a class-distinguishing code and noise as input and generate class-specific data.

[0009] A self-supervised learning-based sleep classification model may generate multiple positive pairs for sleep signal data using attention-based signal augmentation techniques and perform pairwise representation learning so that the distance between the generated positive pairs decreases.

[0010] A self-supervised learning-based sleep classification model may obtain attention scores for sleep signal data, and by applying a random ratio of masks to the obtained attention scores and randomly discarding results from the attention score matrix, it may obtain multiple positive pairs with different masks applied to them.

[0011] Pairwise representation learning may be performed by reducing the implicit cosine similarity between the generated positive pairs.

[0012] Pairwise representation learning may be performed by either constructing positive pairs for the overall sleep signal data to reduce the implicit cosine similarity, or by constructing positive pairs for sleep signal data for predefined signal units to reduce the average implicit cosine similarity.

[0013] A method for classifying body states performed by a computer system includes the steps of inputting biosignal data into a body state classification model on a self-supervised learning platform, and classifying body states from the biosignal data using the body state classification model on the self-supervised learning platform. The body state classification model on the self-supervised learning platform may be one in which patterns for classifying body states from new biosignal data have been learned through transfer learning by fine-tuning weights based on a representation learning model whose representations have been learned using biosignal data.

[0014] A computer system for sleep stage classification includes a data input unit that inputs sleep examination data into a sleep stage classification model on a self-supervised learning platform, and a sleep stage classification unit that uses the sleep stage classification model on the self-supervised learning platform to classify sleep stages from the sleep examination data. The sleep classification model on the self-supervised learning platform may be one in which patterns for sleep stage classification for new sleep data have been learned by transfer learning, by fine-tuning weight values ​​based on a representation learning model whose representations have been learned using sleep signal data.

[0015] The self-supervised learning-based sleep classification model is constructed using an unsupervised learning-based adversarial generative model to minimize the impact of class imbalances inherent in the sleep signal data. The unsupervised adversarial generative neural network model may receive pre-set time-unit electroencephalogram and electrooculogram data inputs extracted from the sleep signal data and learn in a manner that maximizes the mutual information between the signals generated by the generators and the class codes.

[0016] The self-supervised learning-based sleep classification model may generate a plurality of positive pairs for sleep signal data using an attention-based signal enhancement technique, and perform pair-wise representation learning so that the distances between the generated positive pairs approach each other.

[0017] The self-supervised learning-based sleep classification model may obtain an attention score for sleep signal data, and obtain a plurality of positive pairs with different masks by covering the obtained attention score with a random ratio of masks and randomly dropping out the result of the attention score matrix.

[0018] The pair-wise representation learning may be performed by reducing the negative cosine similarity between the generated positive pairs.

[0019] The pair-wise representation learning may be performed by constructing positive pairs for the overall sleep signal data to reduce the negative cosine similarity, and by constructing positive pairs for the sleep signal data of a preset signal unit to reduce the average of the negative cosine similarity.

Advantages of the Invention

[0020] By using the self-supervised learning-based sleep stage classification model, it is possible to accurately classify the sleep stage even with the use of a small number of labels.

Brief Description of the Drawings

[0021] [Figure 1] It is a diagram for explaining the operation of representation learning in one embodiment. [Figure 2] It is a diagram showing an example of data utilization when performing representation learning and transfer learning in one example. [Figure 3] It is a block diagram for explaining the configuration of a computer system in one embodiment. [[ID=三十五]] [Figure 4] It is a flowchart for explaining a method of classifying sleep stages in one embodiment. [Figure 5]It is a flowchart for explaining a method of classifying a physical state in one embodiment.

Embodiments for Carrying Out the Invention

[0022] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[0023] FIG. 1 is a diagram for explaining the operation of expression learning in one embodiment.

[0024] Sleep stage classification means classifying intervals defined by a sleep stage manual into specific stages based on polysomnography data. Polysomnography may include sleep examination data based on various electrical signals such as various electroencephalogram data, electrooculogram data, electrocardiogram data, electromyogram data, etc. during sleep. The computer system may automatically classify the sleep stage based on the sleep examination data. At this time, the computer system may be a sleep stage classification system for classifying the sleep stage. The computer system can predict the sleep stage with few labels by extracting a compressed expression that appropriately represents the data from the signal itself by means of a self-supervised learning technique.

[0025] The computer system may use an attention-based signal enhancement and a self-supervised learning model to learn effective expressions from electroencephalogram data (EEG) and electrooculogram data (EOG) among the sleep examination data, and classify the sleep stage using a small number of labels. The computer system may execute an expression learning stage of learning the overall characteristics of the sleep examination data using an expression learning model and a transfer learning stage of classifying the sleep stage with a small number of labels.

[0026] The representation learning model may consist of a self-supervised learning framework based on a generative model. The representation learning model may extract electroencephalogram (EEG) and electrooculogram (EOG) data in pre-defined time units (e.g., 30 seconds (1 epoch)) from the total sleep test data and use them as input data. In representation learning, to minimize the impact of class imbalances inherent in the sleep test data, an adversarial generative model on an unsupervised learning basis is used as the basic structure.

[0027] First, the generator may receive a class-distinguishing code c and noise z as input data and generate class-specific data. At this time, in order to learn the latent factors that distinguish classes, the generator will learn an adversarial generative model in a direction that maximizes the mutual information between the signal it generates and the class code. When learning an adversarial generative model, the objective function that maximizes the mutual information is as follows.

number

[0028] Next, in order to learn unique representations for each class, paired representation learning on a self-supervised learning platform may be performed. In paired representation learning, two positive pairs may be generated from the original signal data (electroencephalogram data, electrooculogram data), and learning may proceed so that the distance between the positive pairs decreases. Attention-based signal enhancement techniques may be used to generate two positive pairs from the original sleep signal data. First, when encoding the input signal, the attention score may be derived in the following way.

number

number

[0029] Algorithm 1: Positive Pair Representation Learning Algorithm JPEG0007837596000004.jpg105115

[0030] As can be seen in Algorithm 1, learning can be carried out by generating two positive pairs by randomly masking the attention score matrix and reducing the implicit cosine similarity between the positive pairs. The loss function for pairwise representation learning that reduces implicit cosine similarity is as follows.

number

[0031] The two positive pairs are encoded, then passed through a latent predictive model to obtain p1 and p2, and these obtained p1 and p2 are then passed through a latent factor model Q to obtain q1 and q2. sg is a "stop gradient" operation, which prevents the model's learning parameters from being updated. Consequently, when learning a latent representation that makes the two positive pairs similar, the model's learning parameters are updated only for the p1 portion, and q2 is used only to calculate cosine similarity.

[0032] Furthermore, since the prominent wave patterns observed within a single epoch differ depending on the type of sleep stage, positive pairs may be constructed for both the global loss function and the regional loss function, respectively, and learning may proceed accordingly. In the case of the global loss function, learning may proceed by constructing positive pairs for the overall portion of the signal and reducing the implicit cosine similarity. The global loss function is as follows:

number

[0033] In the case of a regional loss function, learning can be carried out by cutting the signal at predetermined time intervals (e.g., 5 seconds), constructing positive pairs for each interval, and lowering the average of the implicit cosine similarity. The regional loss function is as follows:

number

[0034] The objective function for pairwise representation learning, which includes both regional and global loss functions, is as follows:

number

number

[0035] After representation learning, transfer learning may be performed for sleep stage classification. Transfer learning may be carried out by adding a fully connected layer to the end of the latent factor model Q. Sleep stages may be classified by fine-tuning using labels restricted to the weights of the model learned in representation learning and the weights of the single fully connected layer.

[0036] In the embodiments described, the operation of representation learning and transfer learning for sleep stage classification was explained, but the invention is not limited to these embodiments, and the same methods can be applied to physical state classification in addition to sleep stage classification.

[0037] Figure 2 shows an example of data utilization when performing representation learning and transfer learning in one embodiment.

[0038] The computer system may perform representation learning and transfer learning based on a self-supervised learning model. In the representation learning stage, the computer system learns representations that compress data by performing asymmetric learning without using labels. In the representation learning stage, Sleep EDF, which is the most frequently used sleep data, may be used. In the transfer learning stage, the computer system learns sleep stage classification patterns using a small number of labels, by fine-tuning the weights based on the representation learning model learned in the representation learning stage. In the transfer learning stage, the system learns with new data that was not used in representation learning, so that it can be transferred and used with any of the data.

[0039] To evaluate the accuracy of the sleep stage classification according to the embodiment, Sleep EDFx data and ISRUC data may be used. In the case of Sleep EDFx data, 10% and 20% of the data and their labels that were not used for feature learning of the sleep data are used to classify the sleep stages. In the case of ISRUC data, 10% and 20% of the labels from ISRUC are used to fine-tune the weights of the model trained on 20% of the Sleep EDFx data to classify the sleep stages. Average accuracy and F1-macro may be used to compare performance.

[0040] The table below shows the performance of the models for each dataset. To compare the performance of sleep stage classification according to the embodiment with existing methods, a self-supervised learning-based model using a small number of labels was compared with a supervised learning model using all labels.

[0041] Performance comparison with self-supervised learning platform models [Table 1]

[0042] Performance comparison with supervised learning platform models [Table 2]

[0043] As the experimental results show, the results of the experiment according to the embodiment confirm that even with a small number of labels, it shows superior performance compared to existing methods. In particular, when using 10% labels, it was confirmed that it showed overwhelmingly superior performance compared to other self-supervised learning platform models. Furthermore, it was confirmed that it showed superior performance compared to existing self-supervised learning platform models and supervised learning platform models on ISRUC data that was not used during representation learning. This indicates that, in terms of generalization performance, it shows more consistent performance compared to existing learning models, and it can be confirmed that it is a model that can perform relatively accurate sleep stage classification even in limited label situations.

[0044] Figure 3 is a block diagram illustrating the configuration of a computer system in one embodiment, and Figure 4 is a flowchart illustrating a method for classifying sleep stages in one embodiment.

[0045] The processor of the computer system 100 may include a data input unit 310 and a sleep stage classification unit 320. The components of such a processor may be representations of different functions, executed by the processor according to control instructions provided by program code recorded in the computer system. The processor and its components may control the computer system to execute stages 410-420 included in the sleep stage classification method of Figure 4. In this case, the processor and its components may be implemented to execute instructions from operating system code contained in memory and code from at least one program.

[0046] The processor may load program code recorded in a program file for a sleep stage classification method into memory. For example, when a program is executed on a computer system, the processor may control the computer system to load program code from the program file into memory, in accordance with the control of the operating system. In this case, the data input unit 310 and the sleep stage classification unit 320 may be different functional representations of the processor for executing the corresponding parts of the program code loaded into memory to execute stages 410 to 420.

[0047] In step 410, the data input unit 310 may input the sleep test data into the sleep stage classification model of the self-supervised learning platform. At this time, different electrical signal data from the sleep (multi-component) test data may be input into the sleep stage classification model. For example, electroencephalogram (EEG) data and electrooculogram (EOG) data from the sleep test data may be input into the sleep stage classification model.

[0048] In stage 420, the sleep stage classification unit 320 may classify sleep stages from sleep test data using a sleep stage classification model based on self-supervised learning. For example, the sleep stage classification unit 320 may classify sleep into stages 1, 2, 3, 4, and 5. Alternatively, the sleep stage classification unit 320 may classify sleep into light sleep stages and deep sleep stages.

[0049] Figure 5 is a flowchart illustrating a method for classifying physical conditions in one embodiment.

[0050] In step 510, the computer system may input biosignal data into a self-supervised learning platform's physical condition stage classification model. In this case, the biosignal data may be health checkup results data or data measured in the user's daily life.

[0051] At stage 520, the computer system may classify a physical state from biosignal data using a self-supervised learning platform's physical state classification model. For example, a physical state may be defined by the degree to which the user is able to move their body. The computer system may classify the user's physical health based on the user's health checkup results for the heart, brain (stroke, cerebral hemorrhage), lungs, kidneys, etc. Alternatively, the computer system may classify the user's physical health based on (joint) movement data measured in the user's daily life. In this case, the computer system may classify the physical state based on the physical movements the user is able to perform according to their age.

[0052] The above-described apparatus may be implemented by hardware components, software components, and / or combinations of hardware and software components. For example, the apparatus and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as processors, controllers, ALUs (arithmetic logic units), digital signal processors, microcomputers, FPGAs (field programmable gate arrays), PLUs (programmable logic units), microprocessors, or various devices capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications running on the OS. The processing unit may also respond to software execution, access data, record, manipulate, process, and generate data. For convenience of understanding, it may be described as if a single processing unit is used, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Other processing configurations, such as a parallel processor, are also possible.

[0053] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired, or to instruct the processing unit independently or collectively. Software and / or data may be embodied in any kind of machine, component, physical device, virtual device, computer recording medium, or device in order to be interpreted based on the processing unit or to provide instructions or data to the processing unit. Software may be distributed across a network of computer systems, and may be recorded or executed in a distributed manner. Software and data may be recorded on one or more computer-readable recording media.

[0054] The methods according to the embodiment may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded on the medium may be specifically designed for the embodiment or may be available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to record and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code, such as that generated by a compiler, but also high-level language code that is executed by a computer using an interpreter or the like.

[0055] As described above, embodiments have been explained based on limited embodiments and drawings, but those skilled in the art will be able to make various modifications and variations from the above description. For example, the described technique may be performed in a different order than described, and / or the components of the described system, structure, apparatus, circuit, etc. may be combined or assembled in a different manner than described, or opposed or replaced by other components or equivalents, and still achieve suitable results.

[0056] Therefore, even if the embodiment is different, it falls within the scope of the attached claims if it is equivalent to the claims.

Claims

1. A sleep stage classification method performed by a computer system, The stage of inputting sleep test data into a self-supervised learning-based sleep stage classification model. A step of classifying sleep stages from sleep test data using the sleep stage classification model of the self-supervised learning platform. Includes, The sleep classification model on the self-supervised learning platform is one in which patterns for sleep stage classification have been learned from new sleep data through transfer learning, by fine-tuning weights based on a representation learning model in which representations have been learned using sleep signal data. The aforementioned sleep classification model on a self-supervised learning platform is Multiple positive pairs are generated for the sleep signal data using attention-based signal enhancement techniques, and pairwise representation learning is performed to reduce the distance between the generated multiple positive pairs. A sleep stage classification method characterized by the following:

2. The aforementioned sleep classification model on a self-supervised learning platform is To minimize the impact on class imbalances inherent in the aforementioned sleep signal data, it is configured as an adversarial generative model on an unsupervised learning platform. A sleep stage classification method according to claim 1, characterized by the above.

3. A sleep stage classification method performed by a computer system, The stage of inputting sleep test data into a self-supervised learning-based sleep stage classification model. A step of classifying sleep stages from sleep test data using the sleep stage classification model of the self-supervised learning platform. Includes, The sleep classification model on the self-supervised learning platform is one in which patterns for sleep stage classification have been learned from new sleep data through transfer learning, by fine-tuning weights based on a representation learning model in which representations have been learned using sleep signal data. The aforementioned sleep classification model on a self-supervised learning platform is In order to minimize the impact on class imbalances inherent in the sleep signal data, it is composed of an adversarial generative model on an unsupervised learning platform. The aforementioned unsupervised generative adversarial neural network model is, The system receives input of electroencephalogram (EEG) and electrooculogram (EOG) data extracted from the aforementioned sleep signal data in a predetermined time unit, and learns in a manner that maximizes the mutual information between the signals generated by the generator and the class code. A sleep stage classification method characterized by the following:

4. The aforementioned generator receives a class-distinguishing code and noise as input and generates class-specific data. A sleep stage classification method according to claim 3, characterized by the above.

5. The aforementioned sleep classification model on a self-supervised learning platform is The method involves obtaining attention scores for the sleep signal data, applying masks of random proportions to the obtained attention scores, and randomly removing results from the attention score matrix to obtain multiple positive pairs with different masks applied to each other. A sleep stage classification method according to claim 1, characterized by the above.

6. The aforementioned paired expression learning is, This is done by a method that reduces the implicit cosine similarity between the generated positive pairs. A sleep stage classification method according to claim 1, characterized by the above.

7. The aforementioned paired expression learning is, This is done by either forming positive pairs for the overall sleep signal data to reduce the negative cosine similarity, or by forming positive pairs for pre-defined sleep signal data units to reduce the average negative cosine similarity. A sleep stage classification method according to claim 1, characterized by the above.

8. A method for classifying physical conditions performed by a computer system, The stage of inputting biosignal data into a self-supervised learning-based body state stage classification model, and A step of classifying the physical state from the biosignal data using the physical state stage classification model of the self-supervised learning platform. Includes, The aforementioned self-supervised learning platform's body state stage classification model learns patterns for body state stage classification from new biodata through transfer learning, by fine-tuning weights based on a representation learning model whose representations were learned using biodata. The aforementioned sleep classification model on a self-supervised learning platform is Multiple positive pairs are generated for the biometric data using attention-based signal enhancement techniques, and pairwise representation learning is performed to reduce the distance between the generated multiple positive pairs. A method of grading physical states characterized by the following:

9. A computer system for classifying sleep stages, A data input unit for inputting sleep stage classification models on a self-supervised learning platform using sleep test data, and A sleep stage classification unit that classifies sleep stages from sleep test data using the sleep stage classification model of the self-supervised learning platform. Includes, The sleep classification model on the self-supervised learning platform is one in which patterns for sleep stage classification have been learned from new sleep data through transfer learning, by fine-tuning weights based on a representation learning model in which representations have been learned using sleep signal data. The aforementioned sleep classification model on a self-supervised learning platform is Multiple positive pairs are generated for the sleep signal data using attention-based signal enhancement techniques, and pairwise representation learning is performed to reduce the distance between the generated multiple positive pairs. A computer system characterized by the following:

10. The aforementioned sleep classification model on a self-supervised learning platform is In order to minimize the impact on class imbalances inherent in the aforementioned sleep signal data, it is constructed using an adversarial generative model on an unsupervised learning platform. The aforementioned unsupervised generative adversarial neural network model is, The system receives input of electroencephalogram (EEG) and electrooculogram (EOG) data extracted from the aforementioned sleep signal data in a predetermined time unit, and learns in a manner that maximizes the mutual information between the signals generated by the generator and the class code. The computer system according to claim 9, characterized by the following:

11. The aforementioned sleep classification model on a self-supervised learning platform is The method involves obtaining attention scores for the sleep signal data, applying masks of random proportions to the obtained attention scores, and randomly removing results from the attention score matrix to obtain multiple positive pairs with different masks applied to each other. The computer system according to claim 10, characterized by the above.

12. The aforementioned paired expression learning is, This is done by a method that reduces the implicit cosine similarity between the generated positive pairs. The computer system according to claim 10, characterized by the above.

13. The aforementioned paired expression learning is, This is done by either forming positive pairs for the overall sleep signal data to reduce the negative cosine similarity, or by forming positive pairs for pre-defined sleep signal data units to reduce the average negative cosine similarity. The computer system according to claim 10, characterized by the above.

Images