Time sequence self-supervision pre-training method based on frequency mask modeling
By introducing frequency masks and self-supervised learning methods in the time series field, the problem of failure to effectively utilize frequency domain information in the prior art is solved, and the feature extraction capability of the model and the classification accuracy of downstream tasks are significantly improved.
Patent Information
- Application Number
- CN202510021234.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-07
- Publication Date
- 2025-06-06
AI Technical Summary
The prior art has failed to effectively utilize frequency domain information in the time series field, and the training objectives are relatively single, resulting in the model performance being limited by the data enhancement method and cannot adapt well to the needs of the time series field.
A time series self-supervised pre-training method based on frequency mask modeling is proposed. Time domain data is converted into frequency domain representation through Fourier transform, continuous frequency components are randomly selected for masking, and converted back to time domain through inverse Fourier transform to generate training samples. This method combines encoder and decoder for feature extraction and sequence reconstruction, and optimizes feature space through comparison learning.
Through frequency domain masking and self-supervised learning, the model can deeply understand the context information in different frequency components, extract valuable information, and significantly improve the model's feature extraction ability and classification accuracy of downstream tasks.
Smart Images

Figure CN120105137A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology in the field of neural network applications, in particular to a time series self-supervised pre-training method based on frequency mask modeling. Background Art
[0002] Time series analysis has a wide range of applications in many fields, such as epileptic seizure prediction, human activity recognition, and gesture recognition. However, since a large amount of time series data lacks labels or is too complex to be labeled, this poses a challenge to traditional supervised learning methods. Self-supervised learning, as an emerging learning paradigm, can learn useful feature representations by designing pre-training tasks without labels, thereby reducing the reliance on a large amount of labeled data. Summary of the invention
[0003] In view of the shortcomings of the existing time-frequency consistency (TFC) technology and time series representation learning through time and context (TS-TCC) technology, which do not construct data enhancement methods only for the frequency domain, ignore the effective information in the frequency domain and have a relatively single training objective, resulting in the model performance being severely limited by the data enhancement method and unable to adapt well to the time series field, this paper proposes a time series self-supervised pre-training method based on frequency mask modeling, which performs data enhancement through frequency domain masks, can deeply understand the context within different frequency components, and help to extract valuable information that may be overlooked.
[0004] The present invention is achieved through the following technical solutions:
[0005] The present invention relates to a time series self-supervised pre-training method based on frequency mask modeling, comprising:
[0006] Process 1: Collect time series data: Use the accelerometer on the wearable device to collect raw time domain data.
[0007] Process 2: Construct training samples through frequency domain masking, including:
[0008] 2.1 Use Fourier transform to convert the input time domain data into frequency domain representation, then randomly select continuous frequency components for masking, set these frequency domain components to zero, thereby removing the corresponding frequency domain information;
[0009] 2.2 The masked frequency domain data is converted back to the time domain through inverse Fourier transform to complete the frequency domain masking process. For each time domain input data, applying different frequency interval masks can generate two samples, which are regarded as positive sample pairs and the combination with other samples in the same batch is regarded as negative sample pairs.
[0010] Process 3: Use the encoder to extract features and the decoder to reconstruct the sequence: The encoder receives the masked data, gradually extracts feature information through multiple neural network layers, and finally generates a feature vector; the decoder reconstructs the original data before masking based on the features extracted by the encoder.
[0011] The encoder uses but is not limited to a convolutional neural network (CNN), a Transformer or a hybrid structure as a feature extractor to extract effective feature representations from a time series.
[0012] The decoder uses, but is not limited to, a linear layer to reconstruct the original input sequence using the features extracted by the encoder.
[0013] Process 4, self-supervised learning: Use contrastive learning to shape the feature distribution in the feature space, maximize the similarity between positive samples, and minimize the similarity between negative samples; use sequence reconstruction to enhance the ability to capture frequency domain information.
[0014] Process 5: Fine-tuning and application of downstream tasks: Use the trained encoder to extract effective feature representations from the time series and use its output as the input of downstream tasks.
[0015] The downstream tasks include: epileptic seizure prediction, human behavior detection, and human posture detection. Technical Effects
[0016] In the field of time series, the present invention effectively combines contrastive learning and sequence reconstruction through frequency domain masking, so that these two pre-training methods complement each other, thereby enhancing the feature extraction ability of the model. Frequency domain masking has a dual purpose. First, it aims to promote model learning to obtain contextual information in different frequency components through masking and subsequent reconstruction processes. In visual images, low-frequency components usually contain smooth structural elements of objects, such as color and style, while high-frequency components mainly contain contour information of objects. Similarly, the frequency domain of time series data presents similar characteristics, and each frequency band contains different semantic information. Therefore, by removing specific frequency components to reconstruct time series data, the present invention can extract valuable information. In addition, frequency domain masking can be effectively used as a data enhancement method, and through contrastive learning, the extracted features are made more stable and discriminative. This enhancement strategy significantly improves the representation learning ability of the model. Compared with the prior art, the pre-trained encoder of the present invention can effectively extract time series features in tasks such as epilepsy detection, human activity recognition and gesture recognition. By fine-tuning with only a small amount of labeled data in downstream tasks, a significant improvement in classification accuracy can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a schematic diagram of the process of the present invention;
[0018] Figure 2 It is a schematic diagram of the model structure of the present invention. DETAILED DESCRIPTION
[0019] like Figure 1 As shown, this embodiment relates to a time series self-supervised pre-training method based on frequency mask modeling, including:
[0020] S11: Perform frequency domain masking on the input time series to construct input samples. By maximizing the similarity between positive samples, minimizing the similarity between negative samples and reconstructing the original input sequence, the encoder and decoder are trained in a self-supervised pre-training manner, and the trained encoder is used as a feature extractor.
[0021] The frequency domain mask processing layer is used to perform frequency domain masking on the input time series, eliminating some frequency domain components to construct training samples. The encoder acts as a feature extractor to extract effective features from the time domain sequence. The nonlinear mapping head is used to enhance the learned representation to make it more suitable for similarity calculation in contrastive learning tasks. The decoder uses the features extracted by the encoder to reconstruct the original input sequence.
[0022] The self-supervised pre-training includes:
[0023] Process 1) Frequency masking: Use Fourier transform to convert the input time domain data into frequency domain representation, then randomly select continuous frequency components for masking, set these frequency domain components to zero, and thus remove the corresponding frequency domain information. Then, use inverse Fourier transform to convert the masked frequency domain data back to the time domain, completing the frequency domain masking process. For each time domain input data x, applying different frequency interval masks can generate two samples x i and x j , these two samples are regarded as positive sample pairs, and the combination with other samples in the same batch is regarded as negative sample pairs.
[0024] Process 2) Feature extraction: The encoder receives the masked data x i , through multiple neural network layers, feature information is gradually extracted, and finally a feature vector h is generated i .
[0025] Process 3) Sequence reconstruction: By extracting the feature vector h from the encoder i Input the decoder, predict the missing values in the frequency domain, and calculate the loss function and iteratively update the network parameters to ensure that the reconstructed spectrum is as similar as possible to the spectrum of the original data, thereby achieving the training of the encoder and decoder.
[0026] The loss function Where: frequency distance metric f o and f r The tables represent the reconstructed time series and the original time series, R(x) and I(x) represent the real and imaginary parts of F(x), respectively. The exponent γ controls the sharpness of the distance function and is set to 1 by default. For each input time series, this embodiment calculates the average of the distances at each frequency and uses it as the reconstruction loss.
[0027] Process 4) Similarity learning: Use contrastive learning to shape the feature distribution in the feature space, maximize the similarity between positive samples, and minimize the similarity between negative samples. Use the NT-Xent loss function to calculate the distance between positive sample pairs, i.e., Figure 2 Medium i and h j The output obtained by inputting the nonlinear mapping head. For a pair of positive samples (i, j), the distance is expressed as Where sim(u, v) represents the cosine similarity, 1 [k≠i] ∈{0, 1} is an indicator function, which is 1 if and only if k=i, otherwise it is 0, and τ represents a temperature parameter. The contrast loss within a batch is
[0028] Self-supervised loss It consists of two parts, reconstruction loss and contrast loss, and end-to-end training is performed by adjusting the weights of these two parts. Through this method, this embodiment can not only learn the frequency context information in the time series data, but also learn stable and discriminative feature representations, thereby improving the performance of downstream tasks.
[0029] S12: Use the trained encoder for time series feature extraction and train a classifier for downstream tasks.
[0030] S13: The trained encoder and classifier together constitute an end-to-end time series classification model, which inputs the original time series and outputs the predicted category.
[0031] This embodiment uses a pre-trained encoder as a feature extractor and is applied to a variety of downstream tasks, including epileptic seizure prediction, human behavior detection, and human posture estimation. In step S12, for different downstream tasks, labeled data sets are used to train corresponding classifiers; then, in step S13, the trained encoder and classifier are combined to perform specific downstream tasks. For different downstream tasks, the difference in the operation process is the collection and labeling of data. The following briefly introduces the three data sets:
[0032] A) Epileptic seizure prediction: The epilepsy dataset contains 100 single-channel EEG signals from 500 subjects, which are classified into 5 categories and collected at a sampling rate of 174 Hz for 23.6 seconds per subject. Since only one class contains epileptic seizure events, it is considered as a binary classification task.
[0033] B) Human Activity Recognition: The HAR dataset contains 10,299 records of 30 participants performing six selected activities (including lying down, sitting, walking, climbing stairs, and descending stairs). The original data are three-axis linear acceleration and angular velocity signals collected by accelerometers and gyroscopes with a sampling rate of 50Hz.
[0034] C) Gesture Recognition: The Gesture dataset is recorded by 8 users through accelerometers and contains 440 samples of eight gesture patterns, including waving the palm to the left, right, up, and down, drawing a circle clockwise or counterclockwise, drawing a square, and waving the right arrow. According to the corresponding gesture, the data segment is marked with 8 different labels.
[0035] Taking gesture recognition as an example, in step S12, the trained encoder is used to extract features from the acceleration data. Subsequently, these extracted features are input into the classifier to predict the gesture category. In order to optimize the performance of the classifier, the cross entropy loss function is used to guide the learning process of the classifier to ensure that the model can accurately distinguish different gesture categories. Step S13 combines the trained encoder and classifier into an end-to-end gesture recognition system that can identify the corresponding gesture based on the data collected by the acceleration sensor.
[0036] After specific practical experiments, the specific environment and parameter settings are as follows: A convolutional neural network (CNN) is used as the encoder, a two-layer multilayer perceptron (MLP) is used as the nonlinear mapping head, and another set of two-layer multilayer perceptrons (MLP) are used for the reconstruction process. A low-frequency mask is used as the masking strategy, and a positive hyperparameter S is introduced. For each sample, the frequency domain mask interval is set to [0, s], where s is a randomly selected positive number less than S. The batch size of the epileptic seizure prediction and human activity recognition tasks is set to 128, while the batch size of the gesture recognition task is set to 64. To optimize the model, the Adam optimizer is used, the learning rate is set to 3e-4, the weight decay is set to 3e-4, β1=0.9, β2=0.99. For the contrastive learning component, the temperature parameter τ is set to 0.2. Tables 1 and 2 show the experimental results under this setting.
[0037] Table 1 [Comparison results with the baseline model]
[0038] As shown in Table 1. This embodiment performs outstandingly in all three tasks, leading in both accuracy and AUPRC values. Although there is a small gap of 0.25% between this embodiment and TS-TCC in AUPRC on the epilepsy dataset, this embodiment performs better in the remaining five indicators, confirming the robustness and versatility of this embodiment. Compared with MoCo and SimCLR using contrastive learning, this embodiment shows significant performance advantages, highlighting the effectiveness of the frequency domain mask of this embodiment as a data augmentation technique and the performance improvement brought by reconstruction. TS-TCC performs well, while the performance of T-FC is less significant, probably because it is tailored for cross-domain tasks. Overall, these results emphasize the efficient feature learning capabilities of this embodiment and its beneficial impact on downstream tasks.
[0039] Table 2 [Ablation experiment results]
[0040] The results of the ablation experiment are shown in Table 2. Both the encoder and the classifier of this embodiment produce good results. The encoder contributes significantly to the performance improvement, and the average values of various indicators have increased. For example, the accuracy of the epilepsy dataset has increased by 4.83%. In addition, the classifier effectively reduces the standard deviation, indicating that the constraints imposed by contrastive learning lead to more stable and discriminative features.
[0041] Compared with the prior art, this method effectively combines contrastive learning and sequence reconstruction through frequency domain masking, so that these two pre-training methods complement each other, thereby enhancing the feature extraction ability of the model. Frequency domain masking has a dual purpose. First, it aims to promote model learning to obtain contextual information in different frequency components through masking and subsequent reconstruction processes. In visual images, low-frequency components usually contain smooth structural elements of objects, such as color and style, while high-frequency components mainly contain contour information of objects. Similarly, the frequency domain of time series data presents similar characteristics, and each frequency band contains different semantic information. Therefore, by removing specific frequency components to reconstruct time series data, the present invention is able to extract valuable information. In addition, frequency domain masking can be effectively used as a data enhancement method, making the extracted features more stable and discriminative through contrastive learning. The comparison between Table 1 and the baseline method and the ablation experiment in Table 2 confirm that this enhancement strategy significantly improves the representation learning ability of the model.
[0042] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.
Claims
1. A time series self-supervised pre-training method based on frequency mask modeling, characterized in that: The input time series is masked in the frequency domain to construct input samples. The encoder and decoder are trained in a self-supervised pre-training manner by maximizing the similarity between positive samples, minimizing the similarity between negative samples and reconstructing the original input sequence. The trained encoder is used as a feature extractor, and then the trained encoder is used for time series feature extraction. The classifier is trained for downstream tasks. Finally, the trained encoder and classifier are combined to form an end-to-end time series classification model. The original time series is input and the predicted category is output.
2. The time series self-supervised pre-training method based on frequency mask modeling according to claim 1 is characterized in that: The frequency domain mask constructs the input sample, specifically including: 2.1 Use Fourier transform to convert the input time domain data into frequency domain representation, then randomly select continuous frequency components for masking, set these frequency domain components to zero, thereby removing the corresponding frequency domain information; 2.2 The masked frequency domain data is converted back to the time domain through inverse Fourier transform to complete the frequency domain masking process. For each time domain input data, two samples can be generated by applying different frequency interval masks. These two samples are regarded as positive sample pairs, and the combination with other samples in the same batch is regarded as negative sample pairs.
3. The time series self-supervised pre-training method based on frequency mask modeling according to claim 1 is characterized in that: The self-supervised pre-training includes: Process 1) Frequency masking: Use Fourier transform to convert the input time domain data into frequency domain representation, then randomly select continuous frequency components for masking, set these frequency domain components to zero, thereby removing the corresponding frequency domain information, and then use inverse Fourier transform to convert the masked frequency domain data back to the time domain to complete the frequency domain masking process. For each time domain input data x, applying different frequency interval masks can generate two samples x i and x j , these two samples are regarded as positive sample pairs, and the combination with other samples in the same batch is regarded as negative sample pairs; Process 2) Feature extraction: The encoder receives the masked data x i , through multiple neural network layers, feature information is gradually extracted, and finally a feature vector h is generated i ; Process 3) Sequence reconstruction: By extracting the feature vector h from the encoder i Input the decoder, predict the missing values in the frequency domain, and ensure that the reconstructed spectrum is as similar as possible to the spectrum of the original data by calculating the loss function and iteratively updating the network parameters, that is, training the encoder and decoder; Process 4) Similarity learning: Contrastive learning is used to shape the feature distribution in the feature space, maximizing the similarity between positive samples while minimizing the similarity between negative samples. The NT-Xent loss function is used to calculate the distance between positive sample pairs. For a pair of positive sample pairs (i, j), the distance is expressed as Where sim(u,v) represents the cosine similarity, 1 [k≠i] ∈{0,1} is an indicator function. The value of this function is 1 if and only if k=i, otherwise it is 0. τ represents a temperature parameter. The contrast loss within a batch is 4. The time series self-supervised pre-training method based on frequency mask modeling according to claim 3 is characterized in that: The loss function Where: frequency distance metric f o and f r The subtables represent the reconstructed time series and the original time series. R(x) and I(x) represent the real and imaginary parts of F(x), respectively. The exponent γ controls the sharpness of the distance function. For each input time series, the average of the distances at each frequency is calculated and used as the reconstruction loss; The self-supervised loss It consists of two parts: reconstruction loss and contrast loss. End-to-end training is performed by adjusting the weights of these two parts.
