Physiological signal self-supervised representation learning method and system based on time-frequency reconstruction
The self-supervised representation learning method for physiological signals based on time-frequency reconstruction solves the problem of ignoring frequency domain information in existing technologies, and improves the accuracy and generalization of physiological signal classification. It is applicable to the diagnosis of diseases based on physiological signals such as electroencephalograms and electrocardiograms.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2024-08-08
- Publication Date
- 2026-07-28
AI Technical Summary
Existing physiological signal classification methods ignore frequency domain information, and data augmentation methods may destroy the consistency labels of the original sequences, resulting in insufficient classification accuracy and generalization.
A self-supervised representation learning method for physiological signals based on time-frequency reconstruction is adopted. The time-domain and frequency-domain features of physiological signals are encoded by time-domain and frequency-domain encoders respectively. Combined with a time-frequency hybrid encoder, the accuracy and generalization ability of the model are improved by using pre-training and fine-tuning stages.
By effectively integrating the time and frequency domain information of physiological signals, the accuracy and generalization ability of physiological disease classification are improved, and the physiological signal data of different individuals are adapted.
Smart Images
Figure CN119089378B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of signal processing and pattern recognition technology, and in particular to a self-supervised representation learning method and system for physiological signals based on time-frequency reconstruction. Background Technology
[0002] Patterns in physiological signals can greatly help us understand the normal functioning of the human body and related disease mechanisms. There is a wealth of research on physiological signals, such as cognitive research, emotion recognition, and epilepsy detection based on electroencephalography (EEG); heart rhythm detection based on electrocardiography (ECG); and motor neuron disease monitoring based on electromyography (EMG). However, obtaining labels for this physiological signal data is often very difficult. Furthermore, due to individual differences and varying disease progression patterns among patients, models trained using physiological signal data from a subset of patients often perform poorly on other patients. As self-supervised learning has become the dominant learning paradigm in unlabeled environments, increasing research focuses on using self-supervised learning to learn effective general representations from massive amounts of unlabeled data. This not only reduces label dependence but also enhances the model's generalization ability.
[0003] Since physiological signals are essentially time-series data, general time-series self-supervised algorithms can also be used for this task. For example, TimeCLR [Yang, 2022] proposed a method for contrastive learning using DTW data augmentation on time-series data. TS2vec [Yue, 2022] designed a pruning and masking mechanism to perform contrastive learning in a hierarchical manner on an augmented context view, providing a contextual representation for each timestamp. TS-TCC [Eldele, 2021] learns robust and differential representations by applying time contrastive and context contrastive modules. Meanwhile, there are also some algorithms designed based on specific physiological signals; CLOCS [Kiyasseh, 2021] proposed a contrastive learning method using ECGs at the sample and patient levels. Mixing-up[
[2022] The consistency of samples is captured by a hybrid data augmentation scheme. TNC [Tonekaboni, 2021] compares neighboring samples in the same channel signal as positive sample pairs.
[0004] However, most of these methods focus only on the time-domain characteristics of time series, neglecting the importance of frequency information for physiological signals: the frequency characteristics of physiological signals are crucial for identifying different physiological and disease states. For example, in electroencephalography (EEG), different frequency bands are associated with different brain activity states. Alpha waves (8-13 Hz) are typically associated with relaxation and closed-eye states, while theta waves (4-8 Hz) may be associated with deep relaxation or certain types of epileptic activity. By identifying these specific frequency characteristics, doctors can diagnose whether a patient is in a specific mental state or suffers from certain neurological disorders. In electrocardiogram (ECG) analysis, arrhythmia can be detected by analyzing changes in heart rate frequency. Filtering and spectral analysis can help extract useful frequency information from noisy ECG signals. By identifying abnormal heart rate patterns, such as tachycardia or bradycardia, doctors can diagnose heart disease and take appropriate treatment measures. Furthermore, due to the sensitivity of physiological signals to the acquisition environment and data characteristics, using data augmentation methods for contrastive learning can disrupt the consistency labels of the original sequences, leading to label confusion and compromising the inherent characteristics of the data. Summary of the Invention
[0005] To address the problem that existing physiological signal classification methods neglect frequency domain information and destroy the original data, this invention proposes a self-supervised representation learning method and system for physiological signals based on time-frequency reconstruction. By fully integrating relevant information in the time and frequency domains of the original signal, it achieves accurate classification of physiological diseases.
[0006] The present invention adopts the following technical solution:
[0007] In a first aspect, the present invention provides a self-supervised representation learning method for physiological signals based on time-frequency reconstruction, comprising:
[0008] Acquire raw multi-channel physiological signal data and convert it into a time-frequency graph;
[0009] The raw multi-channel physiological signal data is segmented, one-dimensionally randomized, and feature-transformed to obtain a preliminary time-domain representation; a time-domain encoder is used to encode the data in both the time and channel dimensions to obtain the final time-domain representation.
[0010] The time-frequency graph is divided into blocks, subjected to two-dimensional random masking, and feature transformation to obtain a preliminary frequency domain representation; a frequency domain encoder is used to encode the time dimension and channel dimension respectively to obtain the final frequency domain representation.
[0011] A time-frequency hybrid encoder is used to fuse the final time-domain representation and the final frequency-domain representation to obtain a time-frequency hybrid representation. Based on the time-frequency hybrid representation, the original multi-channel physiological signal and its time-frequency map are reconstructed to obtain time-domain loss and frequency-domain loss. The time-domain encoder, frequency-domain encoder and time-frequency hybrid encoder are pre-trained by combining the time-domain loss and frequency-domain loss.
[0012] A multi-channel time-frequency representation of the original multi-channel physiological signal data to be processed is obtained by using a pre-trained time-domain encoder and a frequency-domain encoder. The multi-channel time-frequency representation includes a final time-domain representation and a final frequency-domain representation. A time-frequency hybrid representation is obtained by using a pre-trained time-frequency hybrid encoder. The time-frequency hybrid representation is used as a self-supervised representation of the learned physiological signal for physiological disease classification tasks.
[0013] Furthermore, the original multi-channel physiological signal data can be any one of electroencephalogram (EEG), electrocardiogram (ECG), electrooculogram (EOG), or electromyogram (EMG).
[0014] Furthermore, the aforementioned time-frequency diagram is obtained from the original multi-channel physiological signal data through short-time Fourier transform.
[0015] Furthermore, the segmentation, one-dimensional random masking, and feature transformation operations performed on the original multi-channel physiological signal data to obtain a preliminary time-domain representation include:
[0016] Given raw multi-channel physiological signal data, for any channel sequence, segment it into a series of overlapping segments and perform one-dimensional random masking; traverse all channels;
[0017] The fragments after one-dimensional random masking are linearly mapped to high-dimensional feature vectors, and positional encoding is added as a preliminary temporal representation.
[0018] The one-dimensional random mask is obtained by randomly sampling a certain proportion of fragments from the fragment combination and setting them to zero.
[0019] Furthermore, the aforementioned operations of segmenting the time-frequency graph, applying a two-dimensional random mask, and performing feature transformation to obtain a preliminary frequency domain representation include:
[0020] Given the time-frequency plot of any channel corresponding to the original multi-channel physiological signal data, divide it into blocks to obtain a frequency domain block sequence and perform a two-dimensional random mask; traverse all channels;
[0021] The frequency domain block sequence after two-dimensional random masking is linearly mapped to a high-dimensional feature vector, and position encoding is added as a preliminary frequency domain representation.
[0022] The two-dimensional random mask uses a masking method that combines time resolution and frequency resolution, that is, setting a certain row of frequency domain blocks to zero along the time axis and setting a certain column of frequency domain blocks to zero along the frequency axis.
[0023] Furthermore, the temporal encoder employs a Transformer encoder, which encodes in both the time and channel dimensions to obtain the final temporal representation, expressed as follows:
[0024] T′=Transformer(T)
[0025] T rep =Transformer(T time )
[0026] Where T is the initial time-domain representation, Transformer(.) is the Transformer encoder, and T′ is the representation after time-domain encoding. time It is the two-dimensional vector obtained by expanding T′, where T rep It is the final time-domain representation.
[0027] Furthermore, the frequency domain encoder employs a multilayer perceptron, and the frequency domain encoder encodes in both the time and channel dimensions to obtain the final frequency domain representation, expressed as:
[0028] H′=H+GA(MLP(H))
[0029] Z rep =H time +GA(MLP(H time ))
[0030] Where H is the initial frequency domain representation, MLP(.) is the multilayer perceptron, GA(.) is the gated attention mechanism, and H time Z is the frequency domain representation obtained by swapping the first and last dimensions of H′. rep It is the final frequency domain representation.
[0031] Furthermore, the time-frequency hybrid encoder employs pooling operations.
[0032] Furthermore, prior to the physiological disease classification task, a fine-tuning phase is included, comprising:
[0033] Acquire a fine-tuned training set containing multi-channel physiological signal data and their corresponding disease labels;
[0034] Multi-channel time-frequency fusion representations of multi-channel physiological signal data are obtained by using pre-trained time-domain encoders, frequency-domain encoders, and time-frequency hybrid encoders. Disease types are predicted by classifiers, and loss is calculated by combining disease labels. The classifiers and / or encoders are then fine-tuned.
[0035] Secondly, the present invention provides a self-supervised representation learning system for physiological signals based on time-frequency reconstruction, which is used to implement the above-mentioned self-supervised representation learning method for physiological signals based on time-frequency reconstruction.
[0036] Compared with the prior art, the beneficial effects of the present invention are: the present invention fully explores the characteristics of physiological signals in the time domain and frequency domain, effectively combines the time domain and frequency domain representations, and has higher accuracy, generalization ability and adaptability, providing an effective solution for physiological signal classification tasks. Attached Figure Description
[0037] Figure 1 This is a schematic diagram illustrating a self-supervised learning method for physiological signals based on time-frequency reconstruction, according to an exemplary embodiment.
[0038] Figure 2 This is a schematic diagram of a Transformer encoder according to an exemplary embodiment.
[0039] Figure 3 This is a schematic diagram of a multilayer perceptron encoder according to an exemplary embodiment.
[0040] Figure 4 This is a schematic diagram illustrating a gating attention mechanism according to an exemplary embodiment. Detailed Implementation
[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments. The accompanying drawings are merely illustrative diagrams of the present invention. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0042] The goal of this invention is to fully exploit the characteristics of physiological signals in the time and frequency domains, and to pre-train a model using data from these two domains to obtain an encoder with strong generalization performance. This, in turn, improves the accuracy and generalization of physiological disease classification during fine-tuning. Specifically, the model used in the self-supervised representation learning method for physiological signals is denoted as TFformer, a deep learning framework based on self-supervised learning, mask reconstruction, and Transformer. This invention enables the output of an effective representation for each channel from a given series of continuous, fixed-duration multi-channel physiological signal data. This representation, after aggregation, can be applied to downstream physiological disease classification tasks. Specifically, this invention first performs representation learning on the original physiological signal data; then, the learned representation is used for fine-tuning training for disease classification.
[0043] The overall framework of this invention (TFformer) consists of three main modules: a temporal feature learning module, a frequency domain feature learning module, and a time-frequency fusion module. The temporal feature learning module, based on the characteristics of physiological signals in the time domain, uses a Transformer encoder to perform encoding modeling in the channel and time dimensions, and learns temporal features through a mask reconstruction task. The frequency domain feature learning module, based on the characteristics of physiological signals in the frequency domain, uses a multilayer perceptron to perform encoding modeling in the channel and time dimensions. The time-frequency fusion module combines the temporal and frequency domain features to obtain the final time-frequency hybrid representation used to perform the reconstruction task.
[0044] Figure 1 The diagram shows the framework of the TFformer model proposed in this invention, including a pre-training stage and a fine-tuning stage. In the pre-training stage, the left side takes the original multi-channel physiological signal data as input, and performs a time-domain reconstruction task through a time-domain feature learning module; the right side takes the converted time-frequency plot data of the original multi-channel physiological signal data as input, and performs a frequency-domain reconstruction task through a frequency-domain feature learning module. By combining these two tasks, TFformer can learn effective representations from unlabeled physiological signal data.
[0045] The following sections will introduce each module in detail.
[0046] (1) Temporal feature learning module
[0047] For any channel of the original multi-channel physiological signal First, the original signal needs to be divided into a series of overlapping segments of length P according to a certain step size W, denoted as . Where N represents the number of segments after segmentation. This represents the floor function, which fills in any segments whose length is less than P with the last value in the segment.
[0048] For each time series x i Segmented fragment combination x pi It employs a learnable linear transformation. This is mapped to a set of high-dimensional feature vectors, where each segment corresponds to a feature vector, and D is the dimension of each feature vector; then, a learnable positional encoding is added to all segments. Right now in A single physiological signal sequence x i Transformation into preliminary time-domain representation t i .
[0049] At this time, for this multi-channel physiological signal A set of preliminary time-domain representations was generated. C is the number of channels for this physiological signal.
[0050] In one specific embodiment of the present invention, a Transformer encoder is used as a time-domain encoder to encode the time dimension and channel dimension:
[0051] T′=Transformer(T)
[0052] T rep =Transformer(T time )
[0053] Here, T′ is the representation after being encoded along the time dimension, which is unfolded into a two-dimensional vector and used as its own representation vector after being encoded by the Transformer encoder. Where d h =D×N. It is a representation after channel dimension encoding.
[0054] The structure of a Transformer encoder is as follows: Figure 2 As shown, it is composed of multiple layers. A single-layer Transformer encoder consists of a multi-head attention mechanism module and a feedforward neural network, accompanied by residual connections and batch regularization layers.
[0055] The multi-head attention mechanism consists of multiple attention heads. For each attention head h = 1, 2, ..., H, assume the input is x. i First, three learnable linear transformations are used to transform the feature matrix z. i Mapped to query matrix Q i,h Key matrix K i,h Value matrix V i,h Then, each query vector is pairwise multiplied with each key vector, and the results are normalized using Softmax. These results are then used as weights to perform a weighted summation of the value vectors, yielding the feature matrix z. i The result obtained after focusing on head h
[0056]
[0057] Here, the inner product of each query vector and each key vector needs to be divided by a factor. To prevent the gradient from vanishing in the Softmax function.
[0058] Multi-head attention mechanisms actually employ multiple independent attention mechanisms, extending the model's ability to focus on different locations and providing multiple representation subspaces for the attention layer. Finally, the results of these multiple attention mechanisms are concatenated to obtain z.i The final output after multi-head attention mechanism
[0059]
[0060] Here, Concat(.) represents the concatenation operation, and the default value is d. v =D / H, meaning that the dimension of each vector after concatenating the outputs of H attention heads is exactly reduced to D.
[0061] The feedforward network module consists of two fully connected layers, with the GeLU activation function and Dropout random deactivation operation in between.
[0062] (2) Frequency Domain Feature Learning Module
[0063] For any channel of the original multi-channel physiological signal Applying the short-time Fourier transform, it is converted into a time-frequency graph s. i .
[0064] The specific process of the short-time Fourier transform is as follows:
[0065]
[0066] Where, x i Let represent the physiological signal sequence of the i-th channel, and w(t-τ) represent the window function used to extract the signal. This represents the Fourier transform. i (ω,τ) represents the spectral result of the Fourier transform of the original physiological signal when the window function center is τ. The resulting time-frequency matrix is expressed as follows: m represents the frequency value corresponding to each point after the Fourier transform, and n represents the center time of the window.
[0067] Given a physiological signal sequence x i The corresponding time-frequency diagram s i The slice is then divided into blocks and subjected to feature transformation operations to obtain the preliminary frequency domain representation h of the corresponding slice. i :
[0068]
[0069] Among them, W f Let be a trainable linear projection matrix. For time-frequency graphs s i The frequency domain block sequence W after block division. f_posThis is a learnable positional encoding matrix; it's worth noting that the time-frequency plot here has an additional dimension compared to the original signal. When performing block segmentation, it can be similar to the block segmentation operation in Vision Transformer, which will not be elaborated further here. The preliminary frequency domain representation described above... D f Where is the dimension of each feature vector, and M is the number of frequency domain blocks. a and b represent the length and width of the defined frequency domain block, respectively, where a is divisible by m and b is divisible by n. A single physiological signal sequence x... i The corresponding time-frequency graph s i Convert to preliminary frequency domain characterization h i .
[0070] At this time, for this multi-channel physiological signal Corresponding time-frequency diagram A set of preliminary frequency domain representations was generated using the frequency domain feature learning module. C is the number of channels for this physiological signal.
[0071] In one specific embodiment of the present invention, a multilayer perceptron is used as a frequency domain encoder to encode the time dimension and channel dimension, ultimately obtaining the frequency domain representation Z. rep :
[0072] H′=H+GA(MLP(H))
[0073] Z rep =H time +GA(MLP(H time ))
[0074] Here, H′ is the frequency domain representation after time-dimensional modeling and encoding. By swapping its first and last dimensions, we obtain... Z rep It is a frequency domain representation after channel-dimensional encoding. MLP is a multilayer perceptron, and GA is a gated attention mechanism. Since the number of blocks in a two-dimensional time-frequency graph is much larger than that in a one-dimensional time-series graph, using a multilayer perceptron can reduce the complexity of modeling and learn the correlation between time-frequency blocks. At the same time, applying gated attention can allow the model to filter out unimportant features.
[0075] The structure of a multilayer perceptron (MLP) is as follows: Figure 3 As shown, it consists of multiple linear layers stacked together, accompanied by Dropout random deactivation and activation functions. The gated attention mechanism is as follows: Figure 4 As shown, it consists of a single linear layer, with attention weights calculated using a normalized exponential function, and the results are updated using a dot product operation. The gating attention operation process is as follows:
[0076] Assuming the output of the MLP is Y, then the attention weight WA The calculation is as follows:
[0077]
[0078] Where softmax is the normalization exponential function, If it is a linear layer mapping function, then the result GA(Y) is expressed as follows:
[0079] GA(Y)=W A ·Y
[0080] (3) Time-frequency fusion module
[0081] The time-frequency fusion module fuses the outputs of the time-domain feature learning module and the frequency-domain feature learning module to obtain the time-frequency hybrid feature H. rep Meanwhile, based on the time-frequency hybrid feature H rep Using a temporal linear layer as a temporal reconstruction head to predict the original time-series data. The original time-frequency map is predicted using a frequency domain linear layer as a frequency domain reconstruction head. The calculation process is as follows:
[0082] H bep =Pooling(T rep Z rep )
[0083]
[0084] in, It is a time-domain linear mapping function. It is a time-domain linear mapping function, and Pooling(.) is the pooling operation.
[0085] During the pre-training phase, each multi-channel physiological signal data X in the pre-training set... i Without labels, after segmenting / blocking the physiological signal data of a certain channel and its corresponding time-frequency plot, for each time series x i Segmented fragment combination x pi Further one-dimensional random masking is required, which involves randomly sampling a certain proportion of segments and setting them to 0. For the time-frequency graph s... i The frequency domain block sequence obtained after block division Further two-dimensional random masking is required, that is, for the frequency domain block sequence. A certain proportion of frequency domain blocks are randomly sampled and set to zero. In this embodiment, the two-dimensional random mask uses a masking method combining time resolution and frequency resolution, that is, setting a certain row of frequency domain blocks to zero along the time axis and a certain column of frequency domain blocks to zero along the frequency axis. The masked result is further processed to finally obtain the reconstructed result. The reconstruction error is calculated as follows:
[0086]
[0087] L=λL t +(1-λ)L f
[0088] Where L is the total loss, L t It is the temporal reconstruction loss, L f λ is the frequency domain reconstruction loss, M is the number of multi-channel physiological signal samples in the pre-training batch, and λ is the regularization parameter.
[0089] The model is trained by backpropagation using the two loss methods described above, and the model parameters are updated.
[0090] During the fine-tuning phase, each multi-channel physiological signal data X in the training set is fine-tuned. i For each disease label, X is obtained using the aforementioned time-domain feature learning module, frequency-domain feature learning module, and time-frequency fusion module. i The multi-channel time-frequency fusion representation is used to predict disease types through a classifier, and the loss is calculated by combining disease labels to fine-tune the classifier and / or encoder.
[0091] Finally, the fine-tuned encoder is used to obtain the multi-channel time-frequency representation of the original multi-channel physiological signal data to be processed, and the multi-channel time-frequency fusion representation is obtained. The multi-channel fusion representation is used as the learned physiological signal self-supervised representation for physiological disease classification tasks.
[0092] The above-mentioned fine-tuning stage can freeze the pre-trained encoder parameters and train only the classifier, or it can train the encoder and classifier simultaneously, or other fine-tuning methods known to those skilled in the art can be used. This invention does not limit these methods, and they all fall within the scope of protection of this invention.
[0093] Based on the same inventive concept, this embodiment also provides a self-supervised representation learning system for physiological signals based on time-frequency reconstruction. This system is used to implement the above embodiments, and details already described will not be repeated. The terms "module," "unit," etc., used below refer to combinations of software and / or hardware that implement a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible.
[0094] The system includes:
[0095] The data preprocessing module is used to acquire raw multi-channel physiological signal data and convert it into time-frequency graphs;
[0096] The temporal feature learning module is used to segment and transform the raw multi-channel physiological signal data to obtain a preliminary temporal representation; a temporal encoder is used to encode in the time dimension and channel dimension respectively to obtain the final temporal representation; the pre-training stage also includes the operation of one-dimensional random masking on the segmented sequence.
[0097] The frequency domain feature learning module is used to perform block segmentation and feature transformation operations on the time-frequency graph to obtain a preliminary frequency domain representation; the frequency domain encoder is used to encode in the time dimension and channel dimension respectively to obtain the final frequency domain representation; the pre-training stage also includes the operation of two-dimensional random masking on the block sequence;
[0098] The time-frequency fusion module is used to fuse the final time-domain representation and the final frequency-domain representation using a time-frequency hybrid encoder to obtain a time-frequency hybrid representation.
[0099] The reconstruction learning module is used to reconstruct the original multi-channel physiological signals and their time-frequency maps using time-frequency hybrid representations, calculate the loss for the time-domain reconstruction task and the frequency-domain reconstruction task, and train the encoder.
[0100] The self-supervised representation generation module is used to obtain a multi-channel time-frequency representation of the raw multi-channel physiological signal data to be processed using a pre-trained time-domain encoder and a frequency-domain encoder. The multi-channel time-frequency representation includes a final time-domain representation and a final frequency-domain representation. A time-frequency hybrid representation is obtained using a pre-trained time-frequency hybrid encoder. The time-frequency hybrid representation is used as the learned self-supervised representation of the physiological signal for physiological disease classification tasks.
[0101] The specific implementation process of the functions and roles of each module in the above system is detailed in the corresponding steps of the above method, and will not be repeated here. For the system embodiment, since it basically corresponds to the method embodiment, relevant parts can be referred to in the description of the method embodiment. The system embodiment described above is merely illustrative; the modules described as separate components may or may not be physically separated, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0102] The system embodiments of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The system embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution.
[0103] This embodiment uses a specific experiment to verify the effectiveness of the invention.
[0104] (1) Data Description
[0105] The AD dataset comprises EEG recordings from 12 Alzheimer's patients and 11 healthy controls. Each patient had an average of 30.0 ± 12.5 trials. Each trial corresponds to a 5-second interval with 1280 timestamps (sampled at 256 Hz) across 16 channels. During data preprocessing, each trial was split into nine semi-overlapping samples, each with a duration of 256 timestamps. Furthermore, the training, validation, and test sets were separated in a patient-independent manner. Samples from patients ID17 and ID18 were used as the validation set, and samples from ID19 and ID20 were used as the test set. All remaining samples were placed into the training set.
[0106] The PTB dataset consists of ECG recordings from 290 patients, with 15 channels sampled at 1000 Hz. Eight types of heart disease are represented in the dataset. This experiment focuses on binary classification using a subset of the dataset, comprising 198 myocardial infarction patients and healthy controls. These were first normalized after resampling to 250 Hz. Furthermore, the training, validation, and test sets were separated in a patient-independent manner. Samples from 28 patients (7 healthy and 21 positive) were used as the validation set, and samples from another 28 patients (7 healthy and 21 positive) were used as the test set. All remaining samples were placed in the training set.
[0107] (2) Baseline method.
[0108] This experiment compares the present invention with several existing benchmark methods.
[0109] TS2vec uses overlapping subsequences and a hierarchical loss function to introduce contextual consistency in order to capture data consistency at both the observation and sample levels.
[0110] Mixing-up proposes a method for augmentation by mixing two time series samples in proportion. This augmentation involves creating an augmented sample as a convex combination of two time series samples randomly selected from the dataset.
[0111] TF-C leverages the consistency between the time and frequency domains. This method assumes that time-based and frequency-based representations of the same instance exhibit proximity in the time-frequency space and designs a contrastive learning loss accordingly.
[0112] SimCLR is one of the earliest and most classic contrastive learning frameworks proposed in the field of computer vision. It applies data augmentation techniques to create augmented views of the input samples and constructs a contrastive loss based on these views.
[0113] (3) Physiological Disease Classification Experiment
[0114] In the physiological disease classification experiment, the model was first pre-trained on unlabeled data. After the pre-training phase, the multi-channel representations learned by the model were used for the downstream physiological disease classification task. Since there are interactions between physiological signal channels, this invention considers the correlation between channels. Therefore, the representation of each channel segment is first independently encoded using the method / system proposed in this invention, and finally, all channel segment representations are aggregated into a single representation, which is then used for the physiological disease classification task. The experimental results are as follows:
[0115] Table 1: Comparison of the effects of the model of this invention with other models
[0116]
[0117] On the AD (electroencephalogram) and PTB (electrocardiogram) datasets, the classification accuracy of the present invention and the baseline model were compared. The results are shown in Table 2. The validation metrics were the model's accuracy, precision, recall, and F1 score. The calculation methods are as follows:
[0118]
[0119] Wherein, TP: true positive, defined as the number of predicted results that are judged as positive samples and are actually positive samples; TN: true negative, defined as the number of predicted results that are judged as negative samples and are actually negative samples; FP: false positive, defined as the number of predicted results that are judged as positive samples but are actually negative samples; FN: false negative, defined as the number of predicted results that are judged as negative samples but are actually positive samples.
[0120] The method of this invention demonstrates superior performance on two datasets. Except for a slightly lower recall rate compared to the baseline model, it outperforms the baseline model in all other metrics, particularly the F1 score, a crucial indicator in disease surveillance tasks. This validates the effectiveness of this invention in physiological disease classification tasks and also verifies its generalization ability.
[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.
Claims
1. A self-supervised representation learning method for physiological signals based on time-frequency reconstruction, characterized in that, Includes the following steps: Acquire raw multi-channel physiological signal data and convert it into a time-frequency graph; The raw multi-channel physiological signal data is segmented, subjected to one-dimensional random masking, and feature transformation operations to obtain a preliminary time-domain representation. A time-domain encoder is then used to encode the data in both the time and channel dimensions to obtain the final time-domain representation. The time-domain encoder used is a Transformer encoder, and the encoding in both the time and channel dimensions to obtain the final time-domain representation is expressed as follows: ； ； in, This is a preliminary time-domain representation. It is a Transformer encoder. It is a representation encoded with the time dimension. It is The resulting two-dimensional vector is obtained by unfolding. It is the final time-domain representation; The time-frequency graph is divided into blocks, subjected to two-dimensional random masking, and feature transformation to obtain a preliminary frequency domain representation. A frequency domain encoder is then used to encode the representation in both the time and channel dimensions to obtain the final frequency domain representation. The frequency domain encoder employs a multilayer perceptron, and the encoding in both the time and channel dimensions to obtain the final frequency domain representation is expressed as follows: ； ； in, This is a preliminary frequency domain characterization. It is a multilayer perceptron. It is a gating attention mechanism. It is The frequency domain representation obtained by swapping its first and last dimensions It is the final frequency domain representation; A time-frequency hybrid encoder is used to fuse the final time-domain representation and the final frequency-domain representation to obtain a time-frequency hybrid representation; the time-frequency hybrid encoder employs pooling operations. Based on the time-frequency hybrid representation, the original multi-channel physiological signals and their time-frequency maps are reconstructed to obtain time-domain loss and frequency-domain loss; combined with the time-domain loss and frequency-domain loss, the time-domain encoder, frequency-domain encoder and time-frequency hybrid encoder are pre-trained. A multi-channel time-frequency representation of the original multi-channel physiological signal data to be processed is obtained by using a pre-trained time-domain encoder and a frequency-domain encoder. The multi-channel time-frequency representation includes a final time-domain representation and a final frequency-domain representation. A time-frequency hybrid representation is obtained by using a pre-trained time-frequency hybrid encoder. The time-frequency hybrid representation is used as a self-supervised representation of the learned physiological signal for physiological disease classification tasks.
2. The self-supervised representation learning method for physiological signals based on time-frequency reconstruction according to claim 1, characterized in that, The raw multi-channel physiological signal data mentioned above can be any one of electroencephalogram (EEG), electrocardiogram (ECG), electrooculogram (EOG), or electromyogram (EMG).
3. The self-supervised representation learning method for physiological signals based on time-frequency reconstruction according to claim 1, characterized in that, The aforementioned time-frequency diagram is obtained from the original multi-channel physiological signal data through short-time Fourier transform.
4. The self-supervised representation learning method for physiological signals based on time-frequency reconstruction according to claim 1, characterized in that, The aforementioned segmentation, one-dimensional random masking, and feature transformation operations on the raw multi-channel physiological signal data to obtain a preliminary time-domain representation include: Given raw multi-channel physiological signal data, for any channel sequence, segment it into a series of overlapping segments and perform one-dimensional random masking; traverse all channels; The fragments after one-dimensional random masking are linearly mapped to high-dimensional feature vectors, and positional encoding is added as a preliminary temporal representation. The one-dimensional random mask is obtained by randomly sampling a certain proportion of fragments from the fragment combination and setting them to zero.
5. The self-supervised representation learning method for physiological signals based on time-frequency reconstruction according to claim 1, characterized in that, The aforementioned operations of segmenting the time-frequency graph, applying a two-dimensional random mask, and performing feature transformation to obtain a preliminary frequency domain representation include: Given the time-frequency plot of any channel corresponding to the original multi-channel physiological signal data, divide it into blocks to obtain a frequency domain block sequence and perform a two-dimensional random mask; traverse all channels; The frequency domain block sequence after two-dimensional random masking is linearly mapped to a high-dimensional feature vector, and position encoding is added as a preliminary frequency domain representation. The two-dimensional random mask uses a masking method that combines time resolution and frequency resolution, that is, setting a certain row of frequency domain blocks to zero along the time axis and setting a certain column of frequency domain blocks to zero along the frequency axis.
6. The self-supervised representation learning method for physiological signals based on time-frequency reconstruction according to claim 1, characterized in that, Prior to the physiological disease classification task, a fine-tuning phase is also included, comprising: Acquire a fine-tuned training set containing multi-channel physiological signal data and their corresponding disease labels; Multi-channel time-frequency fusion representations of multi-channel physiological signal data are obtained using pre-trained time-domain encoders, frequency-domain encoders, and time-frequency hybrid encoders. Disease types are predicted by classifiers, and loss is calculated by combining disease labels. The classifiers and / or encoders are then fine-tuned.
7. A self-supervised representation learning system for physiological signals based on time-frequency reconstruction, used to implement the self-supervised representation learning method for physiological signals based on time-frequency reconstruction as described in claim 1, characterized in that, The system includes: The data preprocessing module is used to acquire raw multi-channel physiological signal data and convert it into time-frequency graphs; The temporal feature learning module is used to segment and transform the raw multi-channel physiological signal data to obtain a preliminary temporal representation; a temporal encoder is used to encode in the time dimension and channel dimension respectively to obtain the final temporal representation; the pre-training stage also includes the operation of one-dimensional random masking on the segmented sequence. The frequency domain feature learning module is used to perform block segmentation and feature transformation operations on the time-frequency graph to obtain a preliminary frequency domain representation; the frequency domain encoder is used to encode in the time dimension and channel dimension respectively to obtain the final frequency domain representation; the pre-training stage also includes the operation of two-dimensional random masking on the block sequence; The time-frequency fusion module is used to fuse the final time-domain representation and the final frequency-domain representation using a time-frequency hybrid encoder to obtain a time-frequency hybrid representation. The reconstruction learning module is used to reconstruct the original multi-channel physiological signals and their time-frequency maps using time-frequency hybrid representations, calculate the loss for the time-domain reconstruction task and the frequency-domain reconstruction task, and train the encoder. The self-supervised representation generation module is used to obtain a multi-channel time-frequency representation of the raw multi-channel physiological signal data to be processed using a pre-trained time-domain encoder and a frequency-domain encoder. The multi-channel time-frequency representation includes a final time-domain representation and a final frequency-domain representation. A time-frequency hybrid representation is obtained using a pre-trained time-frequency hybrid encoder. The time-frequency hybrid representation is used as the learned self-supervised representation of the physiological signal for physiological disease classification tasks.