Automatic sleep staging method, system and device and storage medium

By combining the generative adversarial network and the hypergraph fusion module, the problem of modal information imbalance and insufficient information fusion is solved, and high-accurate sleep state detection and sleep quality analysis are realized, adapting to the data distribution of different groups and reducing the Misdiagnosis rates were obtained and accurate sleep quality reports were generated.

CN118986274AActive Publication Date: 2024-11-22SOUTH CHINA UNIV OF TECH

Patent Information

Application Number
CN202410964543.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-22
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

The existing automatic sleep staging method has the problem of modal information imbalance and insufficient information fusion, and ignores the high-order correlation between multimodal physiological signals, resulting in a high misdiagnosis rate and improper handling of data distribution differences in different groups.

Method used

The multimodal physiological signal is characterized by an adversarial enhancement method based on a generative adversarial network. The hypergraph fusion module learns the high-order correlation between multimodal physiological signals, and combines the channel feature mixing technology to achieve information balance and high Order features are fused and fine-tuned to adapt to the data distribution of different groups.

Benefits of technology

The prediction accuracy of sleep state detection is improved, the redundant information and related information of multimodal physiological signals are effectively utilized, the misdiagnosis rate is reduced, and accurate sleep quality reports and disease warnings are generated.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118986274A_ABST
    Figure CN118986274A_ABST
Patent Text Reader

Abstract

The invention discloses an automatic sleep staging method, an automatic sleep staging system, an automatic sleep staging device and a storage medium. The automatic sleep staging method comprises the following steps: acquiring a polysomnogram when a tested person sleeps; extracting physiological signal features according to the polysomnogram, and performing adversarial enhancement processing on the physiological signal features to obtain enhanced physiological signal features; and based on the enhanced physiological signal features, learning a high-order association relationship between the multi-modal physiological signals, performing multi-modal fusion according to the high-order association relationship, and obtaining a sleep stage classification result. According to the method, the physiological signal data after information amount balance is obtained through physiological signal confrontation enhancement, automatic sleep state detection is achieved through data fusion, and the prediction accuracy is improved. In addition, inter-modal high-order correlation capture is performed on the acquired multi-modal physiological signal features, and redundant information and correlation information of the multi-modal physiological signals are effectively utilized to perform feature fusion. The method can be widely applied to the technical field of sleep analysis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of sleep analysis, and in particular to an automatic sleep staging method, system, device and storage medium. Background Art

[0002] Sleep is one of the important physiological activities of human beings. Insufficient sleep and poor sleep quality can directly lead to various physiological and mental diseases, such as sleep apnea syndrome, depression, cognitive decline, etc. Sleep state staging can be used as an effective method to analyze and evaluate the sleep quality of patients. In clinical practice, polysomnography (PSG) is used to record multimodal physiological signals of patients during sleep. PSG contains a variety of multimodal physiological signals, such as electroencephalogram (EEG), electrooculogram (EOG), electromyogram (EMG), etc. The most commonly used staging method currently divides sleep states into five categories: rapid eye movement (REM), three types of non-rapid eye movement (NREM), and wakefulness (WAKE). The current manual staging method mainly calibrates the waveforms of various PSG physiological signals by sleep experts. This staging method requires sleep experts to spend a lot of time to make professional judgments on massive PSG segments, and there is also a subjective problem. Therefore, an automatic sleep stage staging scheme based on the fusion of multimodal physiological signal features is of great significance.

[0003] In practical applications, single-modal physiological signals often cannot fully reflect the physiological state of the human body, while the information carried by physiological signals of different modalities has certain complementarity. Multimodal physiological signal fusion can make full use of the complementarity of various signals and reduce the misdiagnosis rate. Common physiological signal fusion methods first extract various features of physiological signals, and then fuse the features of various physiological signals based on deep learning methods. The main methods include feature splicing, convolutional neural network (CNN), attention-based fusion method (Attention-based Fusion), bilinear pooling fusion, etc.

[0004] The existing automatic sleep staging scheme mainly collects multimodal human physiological signals, extracts the characteristics of physiological signals, and uses machine learning algorithms and deep learning algorithms to identify each sleep stage. However, the existing technical solutions have the following defects: 1) PSG mainly contains multiple EEG channels, while some modalities have only a small number of channels, resulting in an imbalance in modal information. 2) The high-order correlation between multimodal physiological signals, that is, the synergistic correlation between various physiological signals on sleep stages, is not considered, so there is a problem of insufficient information fusion. 3) The existing automatic sleep stage staging ignores the data distribution differences caused by different collection groups, resulting in errors in the staging effect of some groups. Summary of the invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the object of the present invention is to provide an automatic sleep staging method, system, device and storage medium based on a multimodal hypergraph fusion network.

[0006] The first technical solution adopted by the present invention is:

[0007] An automatic sleep staging method comprises the following steps:

[0008] Obtaining a polysomnogram while the subject is sleeping;

[0009] Extract physiological signal features according to the polysomnography, perform adversarial enhancement processing on the physiological signal features, and obtain enhanced physiological signal features;

[0010] Based on the enhanced physiological signal features, the high-order correlation between multimodal physiological signals is learned, multimodal fusion is performed according to the high-order correlation, and the sleep stage classification results are obtained.

[0011] Furthermore, the step of obtaining a polysomnogram of the subject during sleep includes:

[0012] Collecting multiple modal physiological signals of the subject, wherein the physiological signals include electroencephalogram (EEG), eye movement (EOG), electromyography (EMG), and electrocardiogram (ECG);

[0013] The collected physiological signals of multiple modes are amplified and filtered to obtain a polysomnogram.

[0014] Furthermore, the step of extracting physiological signal features according to the polysomnography and performing counter-enhancement processing on the physiological signal features to obtain enhanced physiological signal features includes:

[0015] After the polysomnogram is segmented into time segments according to a preset period, feature extraction is performed to obtain physiological signal features; wherein the extracted features include power spectrum density features, differential entropy features, and frequency domain statistical features;

[0016] According to the obtained physiological signal characteristics, cross-modal physiological signal data conversion is performed based on the generative adversarial network, and finally the characteristics of each physiological signal are mixed based on the channel feature mixing technology to obtain enhanced physiological signal characteristics.

[0017] Furthermore, the generative adversarial network includes a discriminator D and an encoder and a decoder corresponding to each modality;

[0018] The feature distribution of the encoded modality m1 is defined as:

[0019]

[0020] In the formula, is the encoder equation, θ m1 is the parameter of the equation; p(x m1 ) is the distribution of features of the encoding pre-modality m1; x m1 is the initial feature of mode m1, that is, the physiological signal feature of mode m1 without adversarial enhancement; is the feature of the encoded modality m1;

[0021] The discriminator D is used to determine whether the modal features generated by each encoder are the corresponding modality, and the discriminant loss function is defined as:

[0022]

[0023] The error adversarial loss function is defined as:

[0024]

[0025] The correct adversarial loss function is defined as:

[0026]

[0027] In the formula, is the feature of the encoded mode m2, is the feature of the encoded modality m3, w is a trainable parameter, and D is the discriminator;

[0028] The expression of the decoder is:

[0029]

[0030] In the formula, is the decoder equation, θ dm1 are the parameters of the equation; is the distribution of features of modality m1 after decoding; is the feature of modality m1 after being decoded by the decoder;

[0031] For the decoder, the reconstruction loss is defined as:

[0032]

[0033] In the formula, is the multimodal physiological signal feature after being decoded by the decoder, x f is the physiological signal feature that has not been encoded and decoded, |||| 2 is the L2 norm;

[0034] The features of each modality after encoding are mixed and enhanced, and the features of the enhanced modality m1 are defined as:

[0035]

[0036] In the formula, a and b are learnable parameters, M(x m1 ) is the physiological signal feature of the enhanced modality m1;

[0037] After channel mixing and enhancement, enhanced physiological signal characteristics are obtained.

[0038] Furthermore, based on enhancing physiological signal features, learning high-order correlations between multimodal physiological signals, performing multimodal fusion according to the high-order correlations, and obtaining sleep stage classification results include:

[0039] By building a hypergraph fusion module, multimodal fusion of enhanced physiological signal features is performed to output sleep stage classification results;

[0040] The hypergraph fusion module includes a hypergraph convolution layer, a graph feature reading layer, and a sleep stage classification layer; the hypergraph fusion module works as follows:

[0041] The enhanced physiological signal features of each time segment are used as the features of the hypergraph nodes and input into the hypergraph convolution layer;

[0042] After model training, the hypergraph convolutional layer adaptively learns the hyperedges between hypergraph nodes and the high-order correlation of physiological signals, reflecting the synergistic effects of various physiological organs during sleep; and updates the hypergraph node features based on the information transmission mechanism;

[0043] The updated hypergraph node features are input into the graph feature reading layer, the hypergraph features are read, and the hypergraph representation is classified through the sleep stage classification layer to obtain the corresponding sleep stage classification results.

[0044] Furthermore, the automatic sleep staging method further includes the step of fine-tuning the hypergraph fusion module:

[0045] For the multimodal physiological signals of the new group of subjects, some data are sampled and manually annotated, and compared with the classification results calculated by the hypergraph fusion module to calculate the classification accuracy;

[0046] If the classification accuracy is lower than the preset threshold, the hypergraph fusion module is fine-tuned and the classification result is obtained;

[0047] The sleep status data of the subject is stored in the database.

[0048] Furthermore, the automatic sleep staging method further comprises the following steps:

[0049] The sleep quality of the subject is analyzed based on the sleep stage classification results and the preset database.

[0050] The second technical solution adopted by the present invention is:

[0051] An automatic sleep staging system, comprising:

[0052] A polysomnography acquisition module, used for obtaining a polysomnography of the subject while he is sleeping;

[0053] A physiological signal confrontation and enhancement module is used to extract physiological signal features according to the polysomnogram, perform confrontation and enhancement processing on the physiological signal features, and obtain enhanced physiological signal features;

[0054] The hypergraph fusion module is used to learn the high-order correlation between multimodal physiological signals based on the enhanced physiological signal features, perform multimodal fusion according to the high-order correlation, and obtain the sleep stage classification results.

[0055] The third technical solution adopted by the present invention is:

[0056] An automatic sleep staging device, comprising:

[0057] at least one processor;

[0058] at least one memory for storing at least one program;

[0059] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0060] The fourth technical solution adopted by the present invention is:

[0061] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to execute the method described above when executed by the processor.

[0062] The beneficial effects of the present invention are as follows: the present invention obtains information-balanced physiological signal data by enhancing the physiological signal confrontation, and realizes automatic sleep state detection by data fusion, thereby improving the prediction accuracy. In addition, the high-order correlation between modalities is captured for the collected multimodal physiological signal features, and the redundant information and correlation information of the multimodal physiological signals are effectively used for feature fusion, which is an accurate and reliable multimodal data fusion method. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 It is a workflow diagram of an automatic sleep staging system based on a hypergraph fusion model in an embodiment of the present invention;

[0065] Figure 2 is a schematic diagram of a polysomnography acquisition module in an embodiment of the present invention;

[0066] Figure 3 is a schematic diagram of a physiological signal countermeasure enhancement module in an embodiment of the present invention;

[0067] Figure 4 is a working schematic diagram of a hypergraph fusion module in an embodiment of the present invention;

[0068] Figure 5 It is a flowchart of the steps of an automatic sleep staging method in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0070] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0071] In the description of the present invention, the meaning of "several" is one or more, the meaning of "more" is two or more, and the meanings of "greater than", "less than", "exceed" and the like are understood as not including the number itself, and the meanings of "above", "below", "within" and the like are understood as including the number itself. If there is a description of the first and the second, it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, "and / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0072] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0073] like Figure 5 As shown, this embodiment provides an automatic sleep staging method, comprising the following steps:

[0074] S1. Obtain a polysomnogram of the subject while he or she is sleeping.

[0075] First, the subject uses a polysomnogram acquisition module to collect a polysomnogram during sleep in a suitable measurement environment, and then sends it to a physiological signal countermeasure enhancement module.

[0076] S2. Extract physiological signal features according to the polysomnography, perform adversarial enhancement processing on the physiological signal features, and obtain enhanced physiological signal features.

[0077] The features of physiological signals are extracted through the physiological signal adversarial enhancement module, and the features are subjected to adversarial enhancement processing to obtain enhanced physiological signal features.

[0078] S3. Based on the enhanced physiological signal features, the high-order correlation between multimodal physiological signals is learned, multimodal fusion is performed according to the high-order correlation, and the sleep stage classification results are obtained.

[0079] The enhanced physiological signal features are input into the hypergraph fusion module to learn the high-order correlation between multimodal physiological signals, and multimodal fusion is performed based on this to obtain the sleep stage classification label.

[0080] S4, adapt fine-tuning to process the enhanced data.

[0081] For the multimodal physiological signals of the new group of subjects, some data are sampled and manually labeled by clinicians, and compared with the classification results calculated by the hypergraph fusion module to calculate the classification accuracy. If it is lower than the threshold, the model is fine-tuned through the data adaptation fine-tuning module to obtain the classification results. The sleep state data of the subjects are stored in the database.

[0082] S5. Analyze the sleep quality of the subject according to the sleep stage classification result and a preset database.

[0083] Based on the data in the database and the sleep stage classification results of the subject, a sleep quality report and warning are generated for the subject through the sleep quality analysis module.

[0084] Corresponding to the above method, Figure 1As shown, the present embodiment also provides an automatic sleep staging system based on a hypergraph fusion model, including a polysomnography acquisition module, a physiological signal confrontation enhancement module, a hypergraph fusion module, a data adaptation fine-tuning module, and a sleep quality analysis module. The subject undergoes polysomnography acquisition, and after physiological signal confrontation enhancement, the enhanced physiological signal characteristics are obtained, and then multimodal data fusion is performed. It is compared with some manual labels marked by clinicians. If the model labeling accuracy is lower than the threshold, data adaptation fine-tuning is performed, and the subject's sleep state data is stored in the database. Sleep quality analysis is performed based on the data in the database, and a sleep quality report and warning are generated for the subject. Specifically, the detailed working method of each module is as follows:

[0085] (1) Polysomnography acquisition module

[0086] The polysomnography acquisition module collects various physiological signals of the subjects, including electroencephalogram (EEG), eye movement (EOG), electromyography (EMG), and electrocardiogram (ECG), and performs operations such as filtering and power spectrum density feature extraction.

[0087] The flowchart of the polysomnography acquisition module is as follows Figure 2 The polysomnography acquisition module can use a polysomnography monitor for acquisition and processing. First, the electrode sheet is contacted to the scalp, skin, eyelids and other positions to make the electrode stably contact the corresponding acquisition position, and then the physiological signals of multiple channels are acquired. The acquired physiological signals are pre-amplified to obtain a relatively strong signal, and then post-amplification and filtering operations are performed to finally obtain a processed polysomnography.

[0088] (2) Physiological signal countermeasure enhancement module

[0089] Physiological signal countermeasure enhancement module such as Figure 3 As shown in the figure, the collected physiological signals are segmented into time segments with a period of time T, and features are extracted. Common features include power spectrum density features, differential entropy features, frequency domain statistical features, etc. Then, cross-modal physiological signal data conversion is performed based on the generative adversarial network, and finally, the features of each physiological signal are mixed based on the channel feature mixing technology to obtain enhanced physiological signal features.

[0090] The generative adversarial network includes a discriminator D and encoders and decoders corresponding to each modality. For the data of each modality, taking modality m1 as an example, m2 and m3 are the other two modalities, the feature distribution of modality m1 after encoding is defined as:

[0091]

[0092] in is the encoder equation, θ m1is the parameter of the equation, p(x m1 ) is the distribution of the features of the encoding pre-modality m1, x m1 is the initial feature of mode m1, that is, the physiological signal feature of mode m1 without adversarial enhancement. It is the feature of the encoded mode m1.

[0093] The discriminator D is used to determine whether the modal features generated by each encoder are the corresponding modality, and the discriminant loss function is defined as:

[0094]

[0095] The error adversarial loss function is defined as:

[0096]

[0097] The correct adversarial loss function is defined as:

[0098]

[0099] Where w is a trainable parameter.

[0100] The decoder is:

[0101]

[0102] in is the decoder equation, θ dm1 are the parameters of this equation. is the distribution of the features of modality m1 after decoding, is the feature of modality m1 after being encoded by the encoder, It is the feature of modality m1 after being decoded by the decoder.

[0103] For the decoder, the reconstruction loss is defined as:

[0104]

[0105] The discriminator, encoder, and decoder can be constructed by a variety of deep learning models, such as multi-layer perceptron, recurrent neural network, pulse neural network, etc.

[0106] The features of each modality after encoding are mixed and enhanced. Taking modality m1 as an example, the features of m1 after enhancement are defined as:

[0107]

[0108] Where a and b are learnable parameters, M(x m 1) is the physiological signal characteristics of m1 after enhancement.

[0109] After channel mixing and enhancement, enhanced physiological signal characteristics are obtained.

[0110] It should be noted that the above-mentioned physiological signal adversarial enhancement module generates enhanced physiological signal features, which solves the problem of unbalanced modal information. In addition to the above-mentioned technical means, variational autoencoders, diffusion models, etc. can also be used to convert data of each modality between modalities based on the different heterogeneous features between physiological signals of each modality.

[0111] (3) Hypergraph Fusion Module

[0112] See also Figure 4 , the hypergraph fusion module performs multimodal fusion on the enhanced physiological signal features. The hypergraph fusion module includes a hypergraph convolution layer, a graph feature reading layer, and a sleep stage classification layer. The enhanced physiological signal features of each time segment T are input into the hypergraph convolution layer as the features of the hypergraph nodes. After model training, the hypergraph convolution layer adaptively learns the hyperedges between the hypergraph nodes and the high-order correlation of the physiological signals, reflecting the synergy of various physiological organs during sleep. The hypergraph node features are updated based on the information transmission mechanism. The updated hypergraph node features are input into the graph feature reading layer, the hypergraph features are read, and the hypergraph representations are classified through the sleep stage classification layer to obtain the corresponding sleep stage prediction results.

[0113] Define the superpoint set as V = {v1, v2, ..., vn}, the hyperedge set as E = {E1, E2, ..., Em}, define W as the weight matrix of the hyperedge, w(e) as the weight of the hyperedge e, is the degree of the hyperedge, and the degree of the hyperpoint is defined as De and Dv represent the diagonal matrices of hyperedge degree and vertex degree respectively.

[0114] Define the mapping matrix H that the model can learn to represent the hypergraph structure, where each element h(v, e) in the matrix is ​​defined as:

[0115]

[0116] Define the characteristics of a superpoint:

[0117]

[0118] in Represents the initial features of the nth physiological signal channel.

[0119] The hypergraph convolutional layer is:

[0120]

[0121] in is the propagated representation of the first-layer hypergraph, and X(l) is the original representation of the first-layer hypergraph.

[0122] The residual connection layer is:

[0123]

[0124] Where α is a hyperparameter and σ is the activation function.

[0125] The activation function layer can use a linear rectifier function unit:

[0126] RELU(X)=max(0,X)

[0127] Where X represents the input vector of the activation function.

[0128] The graph feature reading layer is:

[0129]

[0130] Z represents the fused physiological signal features after reading. After the fused physiological signal features are passed through the sleep stage classification layer, the five-category sleep stage results are output. The classification function can be a Softmax function.

[0131] It should be noted that the above-mentioned hypergraph fusion module uses a hypergraph neural network to fuse multimodal data. In addition to this technical solution, other algorithms or models can also be used, such as convolutional neural network CNN, recurrent neural network RNN, bilinear pooling layer, integrated learning, etc.

[0132] (4) Data Adaptation Fine-tuning Module

[0133] For the data of the new group of subjects, polysomnography is first collected, and then clinicians manually mark a small part of the data into sleep states. The polysomnography is subjected to physiological signal adversarial enhancement and hypergraph fusion to obtain the model classification results, which are compared with the manual labels. If the accuracy is less than the threshold, a new data fine-tuning layer is created in the data adaptation fine-tuning module.

[0134] The data fine-tuning layer is:

[0135] Z′=(W a Z+b)

[0136] Where W a is a learnable weight matrix, and b is a learnable bias term. Z′ is the fused physiological signal feature obtained after data adaptation. Five-category sleep staging results are output based on the classification function. The classification function can be a Softmax function. The sleep staging statistics of each group are stored in the sleep information database of each group.

[0137] (5) Sleep quality analysis module

[0138] For each subject's physiological signal data, the system analyzes the subject's sleep quality based on the sleep quality analysis module and compares it with the data of each group in the database. If the subject's sleep stage distribution is similar to that of a certain group, and the group generally suffers from a certain disease, a sleep quality report is generated based on this and the subject is warned. For example: if the subject's sleep stage distribution is similar to that of the obstructive sleep apnea patient group in the database, the sleep quality analysis module generates a sleep quality report for the subject and warns the subject.

[0139] In summary, the present invention has at least the following advantages and beneficial effects compared to the prior art:

[0140] (1) The present invention proposes a physiological signal adversarial enhancement method to perform adversarial enhancement on physiological signal data of each modality. This method performs adversarial enhancement on multimodal physiological signals with unbalanced data volume and number of channels to obtain enhanced physiological signal characteristics, thereby balancing the data volume of different modalities, enhancing the learning ability of modal data with less information, and solving the problem of unbalanced information volume of physiological signals of each modality during polysomnography acquisition. Specifically, modality conversion is performed on data of each modality through generative adversarial networks, and modalities with larger information volume are converted into other modalities. Based on the method of channel feature mixing, physiological signal enhancement is further achieved to balance the information volume of each modality.

[0141] (2) The present invention proposes a hypergraph fusion method to perform feature fusion on multimodal physiological signal data. Commonly used multimodal fusion methods ignore the capture of high-order correlations between multimodal physiological signal data, making it impossible for the model to fully capture the complementary information of each modality and the modal coordination pattern with sleep stage classification. The hypergraph fusion method proposed in the present invention can effectively capture the high-order correlations between multimodal physiological signals and perform feature fusion, and based on this, achieve efficient and accurate automatic sleep staging.

[0142] (3) The present invention proposes to adapt to the physiological signal characteristics of different groups through data adaptation and fine-tuning, and to perform sleep quality analysis and disease warning on the subjects based on the data distribution of the sleep stages of different groups. The traditional automatic sleep stage staging method ignores the data adaptation for the data distribution of different groups. The data adaptation fine-tuning method proposed in the present invention realizes the mining and adaptation of the relevant patterns of the sleep stage staging of different groups, and based on this, analyzes the sleep quality of the subjects, realizes sleep quality reporting and disease warning.

[0143] (4) The system of the present invention integrates the physiological signal confrontation enhancement module, the hypergraph fusion module, and the data adaptive fine-tuning module into an automatic sleep staging method. The physiological signal confrontation enhancement is performed to obtain physiological signal data with balanced information quantity, and the enhanced data is processed and classified through data fusion and adaptive fine-tuning to realize automatic sleep state detection.

[0144] This embodiment also provides an automatic sleep staging device, comprising:

[0145] at least one processor;

[0146] at least one memory for storing at least one program;

[0147] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 5 The method described is shown.

[0148] An automatic sleep staging device of this embodiment can execute an automatic sleep staging method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0149] The present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 5 The method shown.

[0150] This embodiment also provides a storage medium, which stores instructions or programs that can execute an automatic sleep staging method provided by an embodiment of the method of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.

[0151] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0152] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0153] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0154] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0155] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0156] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0157] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0158] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0159] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. An automatic sleep staging method, characterized in that: The following steps are involved: Obtaining a polysomnogram of the subject; Extract physiological signal features according to the polysomnography, perform adversarial enhancement processing on the physiological signal features, and obtain enhanced physiological signal features; Based on the enhanced physiological signal features, the high-order correlation between multimodal physiological signals is learned, multimodal fusion is performed according to the high-order correlation, and the sleep stage classification results are obtained.

2. The automatic sleep staging method according to claim 1, characterized in that: The step of obtaining a polysomnogram of the subject comprises: Collecting multiple modal physiological signals of the subject, wherein the physiological signals include electroencephalogram signals, eye movement signals, electromyography signals, and electrocardiogram signals; The collected physiological signals of multiple modes are amplified and filtered to obtain a polysomnogram.

3. The automatic sleep staging method according to claim 1, characterized in that: The step of extracting physiological signal features according to the polysomnography and performing counter-enhancement processing on the physiological signal features to obtain enhanced physiological signal features includes: After dividing the polysomnogram into time segments according to a preset period, feature extraction is performed to obtain physiological signal features; Among them, the extracted features include power spectrum density features, differential entropy features, and frequency domain statistical features; According to the obtained physiological signal characteristics, cross-modal physiological signal data conversion is performed based on the generative adversarial network, and finally the characteristics of each physiological signal are mixed based on the channel feature mixing technology to obtain enhanced physiological signal characteristics.

4. The automatic sleep staging method according to claim 3, characterized in that: The generative adversarial network includes a discriminator D and an encoder and a decoder corresponding to each modality; The feature distribution of the encoded modality m1 is defined as: In the formula, is the encoder equation, θ m1 is the parameter of the equation; p(x m1 ) is the distribution of features of the encoding pre-modality m1; x m1 is the initial feature of mode m1, that is, the physiological signal feature of mode m1 without adversarial enhancement; is the feature of the encoded modality m1; The discriminator D is used to determine whether the modal features generated by each encoder are the corresponding modality, and the discriminant loss function is defined as: The error adversarial loss function is defined as: The correct adversarial loss function is defined as: In the formula, is the feature of the encoded mode m2, is the feature of the encoded modality m3, w is a trainable parameter, and D is the discriminator; The expression of the decoder is: In the formula, is the decoder equation, θ dm1 are the parameters of the equation; is the distribution of features of modality m1 after decoding; is the feature of modality m1 after being decoded by the decoder; For the decoder, the reconstruction loss is defined as: In the formula, is the multimodal physiological signal feature after being decoded by the decoder, x f is the physiological signal feature without encoding and decoding, ∥∥2 is the L2 norm; The features of each modality after encoding are mixed and enhanced, and the features of the enhanced modality m1 are defined as: In the formula, a and b are learnable parameters, M(x m1 ) is the physiological signal feature of the enhanced modality m1; After channel mixing and enhancement, enhanced physiological signal characteristics are obtained.

5. The automatic sleep staging method according to claim 1, characterized in that: The method of enhancing physiological signal features, learning high-order correlations between multimodal physiological signals, performing multimodal fusion according to the high-order correlations, and obtaining sleep stage classification results includes: By building a hypergraph fusion module, multimodal fusion of enhanced physiological signal features is performed to output sleep stage classification results; The hypergraph fusion module includes a hypergraph convolution layer, a graph feature reading layer, and a sleep stage classification layer; the hypergraph fusion module works as follows: The enhanced physiological signal features of each time segment are used as the features of the hypergraph nodes and input into the hypergraph convolution layer; After model training, the hypergraph convolution layer adaptively learns the hyperedges between hypergraph nodes, learns the high-order correlation relationship of physiological signals, reflects the synergy of various physiological organs during sleep, and updates the hypergraph node features based on the information transmission mechanism; The updated hypergraph node features are input into the graph feature reading layer, the hypergraph features are read, and the hypergraph representation is classified through the sleep stage classification layer to obtain the corresponding sleep stage classification results.

6. The automatic sleep staging method according to claim 5, characterized in that: The automatic sleep staging method further comprises the step of fine-tuning the hypergraph fusion module: For the multimodal physiological signals of the new group of subjects, some data are sampled and manually annotated, and compared with the classification results calculated by the hypergraph fusion module to calculate the classification accuracy; If the classification accuracy is lower than the preset threshold, the hypergraph fusion module is fine-tuned and the classification result is obtained; The sleep status data of the subject is stored in the database.

7. The automatic sleep staging method according to claim 1, characterized in that: The automatic sleep staging method further comprises the following steps: The sleep quality of the subject is analyzed based on the sleep stage classification results and the preset database.

8. An automatic sleep staging system, characterized in that: include: A polysomnography acquisition module, used for obtaining a polysomnography of the subject while he is sleeping; A physiological signal confrontation and enhancement module is used to extract physiological signal features according to the polysomnogram, perform confrontation and enhancement processing on the physiological signal features, and obtain enhanced physiological signal features; The hypergraph fusion module is used to learn the high-order correlation between multimodal physiological signals based on the enhanced physiological signal features, perform multimodal fusion according to the high-order correlation, and obtain the sleep stage classification results.

9. An automatic sleep staging device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 7 when executed by the processor.

Citation Information

Patent Citations

  • Sleep staging method and device

    CN112263218A

  • Multi-mode sleep staging method based on time sequence relevance driving

    CN112450881A

  • Driving behavior prediction method and device

    CN117195082A

  • Sleep staging method and device, storage medium and electronic equipment

    CN117322840A

  • Prediction system, method and apparatus for abnormal brain connectivity, and readable storage medium

    WO2023077603A1

Cited By

  • Self-attention sleep system and method based on polymorphic sleep data fusion

    CN119792762A

  • Self-attention Sleep System and Method for Polymorphic Sleep Data Fusion

    CN119792762B

  • Wireless sensing method for quantitative evaluation of obstruction degree of upper airway

    CN119837524A

  • Sleep staging analysis method and device based on electroencephalogram signals

    CN120241103A

  • A sleep stage analysis method and device based on electroencephalogram signals

    CN120241103B