A supervised contrastive learning method and device based on spatio-temporal frequency consistency

Abstract

The application provides a kind of supervised contrast learning method and equipment based on space-time frequency consistency, it is related to machine learning field, including: constructing TSF-SCL model;Through multi-source data enhancement module, the original electroencephalogram signal set is carried out multi-source data enhancement, obtains time domain enhanced data set, space domain enhanced data set and frequency domain enhanced data set;Through the original electroencephalogram signal set, time domain enhanced data set, space domain enhanced data set and frequency domain enhanced data set, each module in TSF-SCL model is synchronously trained;Through the TSF-SCL model trained well, the original electroencephalogram signal set to be measured is classified, and classification prediction result is obtained.The application extracts discriminative features from three enhanced views through cross-view contrast learning module, and the features of different views respectively constitute positive and negative sample pairs to train the model, to enhance the consistency of features;Cross-model contrast learning module further improves the consistency of features extracted by different networks (such as feature extractor and encoder).

Description

Technical Field

[0001] This invention relates to the field of machine learning, and in particular to a supervised contrastive learning method and device based on spatiotemporal frequency consistency. Background Technology

[0002] Brain-computer interface (BCI) technology provides a direct communication pathway between the human brain and external devices, playing a significant role in enhancing human cognitive and sensorimotor functions in the medical field. Non-invasive electroencephalogram (EEG) signals are widely researched and applied due to their convenience and low cost. EEG-based BCI systems can assist, enhance, and even potentially restore human cognitive or sensorimotor functions, and are widely used in cognitive behavioral assessment, emotion regulation, and other fields. Motor imagery (MI) signals are one of the effective control signals for BCI. Users can imagine the movement of specific body parts (such as the left hand, right hand, feet, or tongue) to regulate different areas of the brain's motor cortex, thereby generating corresponding types of EEG signals. Current challenges of BCI systems include limited training data, non-stationary EEG signals, and significant individual user variability.

[0003] Collecting labeled calibration data from new users is time-consuming and labor-intensive; therefore, it is highly meaningful to fully utilize the information in existing training data and design effective EEG data augmentation strategies. Currently, EEG data augmentation mainly includes time-domain, frequency-domain, spatial-domain, and model-based data augmentation. Time-domain data augmentation includes adding noise and signal inversion; frequency-domain data augmentation generally involves first transforming the signal to the frequency domain and then performing band filtering and phase transformation; spatial-domain data augmentation typically involves randomly swapping channels of the EEG signal or swapping locations according to different brain regions to create new samples. Traditional EEG signal processing and classification methods usually rely on pre-defined network architectures, but these architectures struggle to comprehensively learn the characteristics of EEG signals. For example, the EEGNet architecture, inspired by the co-spatial patterns of filter banks, focuses on spatial filtering to maximize the variance ratio between classes, but it has limitations in time- and frequency-domain feature learning, thus restricting the quality of the extracted features.

[0004] The following three problems exist in existing motor imagery brain-computer interface systems.

[0005] First, the training data volume is small. Existing EEG datasets are typically much smaller than datasets in image or natural language processing, which significantly limits the performance of deep learning models. Some studies have used data augmentation techniques to alleviate the problem of small sample sizes during training; however, a single data augmentation technique cannot fully exploit the overall characteristics of EEG signals, such as time, frequency, and spatial domains. Existing work pays little attention to multi-source data augmentation and fails to fully utilize multi-source knowledge for optimization.

[0006] Second, there is a lack of effective data augmentation techniques. Currently, some studies have used data augmentation techniques in classification tasks of brain-computer interface systems; however, these methods have limited performance and are not effective across multiple paradigms and scenarios. Furthermore, parameter tuning is often required, leading to high uncertainty in their effectiveness. Using only one data augmentation technique cannot fully exploit the overall time, frequency, and spatial characteristics of EEG signals. Therefore, improving data augmentation strategies and achieving multi-source data augmentation fusion is crucial.

[0007] Third, there is a lack of network structures for extracting multi-source features. Compared to image and natural language processing, EEG data features focus on the time, spatial, and frequency domains. From the perspective of the time-series signal itself, its temporal, channel, and frequency information are crucial for decoding EEG signals. However, there is currently a lack of research on simultaneously considering multi-source features across time, space, and frequency and designing network structures to improve feature consistency. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a supervised contrastive learning method and device based on spatiotemporal frequency consistency, which solves the technical problems of existing motor imagery brain-computer interface systems, such as small training data volume, lack of effective data augmentation technology, and lack of network structure for extracting multi-source features.

[0009] This invention provides a supervised contrastive learning method based on spatiotemporal frequency consistency, comprising:

[0010] S1: Construct the TSF-SCL model, which includes: a multi-source data augmentation module, a cross-view contrastive learning module, a cross-model contrastive learning module, and a supervised classification module;

[0011] S2: Obtain the raw EEG signal set, and perform multi-source data enhancement on the raw EEG signal set through the multi-source data enhancement module to obtain the time domain enhanced data set, the spatial domain enhanced data set, and the frequency domain enhanced data set;

[0012] S3: Synchronously train each module in the TSF-SCL model using the original EEG signal set, time-domain enhanced data set, spatial-domain enhanced data set, and frequency-domain enhanced data set to obtain a trained TSF-SCL model;

[0013] S4: Obtain the set of raw EEG signals to be tested, and classify the set of raw EEG signals to be tested using the trained TSF-SCL model to obtain the classification prediction results.

[0014] Preferably, step S2 specifically includes:

[0015] S21: The multi-source data enhancement module includes: a time-domain enhancement module, a spatial-domain enhancement module, and a frequency-domain enhancement module;

[0016] S22: Input the original set of EEG signals into the time domain enhancement module, spatial domain enhancement module and frequency domain enhancement module respectively for data enhancement, and obtain the time domain enhanced data set T, spatial domain enhanced data set S and frequency domain enhanced data set F respectively.

[0017] Preferably, step S3 specifically includes:

[0018] S31: The acquired raw EEG signal set includes N EEG signals x i The time-domain augmentation dataset includes N time-domain augmentation datasets. The airspace augmentation dataset includes N airspace augmentation datasets. The frequency domain enhancement dataset includes N frequency domain enhancement datasets. i is the number of the EEG signal, and N is the total number of EEG signals;

[0019] S32: Construct the first positive sample pair set and the first negative sample pair set using the time-domain enhanced data set, the spatial-domain enhanced data set, and the frequency-domain enhanced data set;

[0020] S33: Calculate the cross-view contrast learning loss of the cross-view contrast learning module using the first set of positive sample pairs and the first set of negative sample pairs.

[0021] S34: The cross-model comparison learning module includes an upper branch network and a lower branch network; the time-domain enhanced data set, the spatial domain enhanced data set, and the frequency domain enhanced data set are input into the upper branch network to obtain the upper branch enhanced feature set, and the original EEG signal set is input into the lower branch network to obtain the original feature set. The second positive sample pair set and the second negative sample pair set are constructed through the upper branch enhanced feature set and the original feature set.

[0022] S35: Calculate the cross-model contrastive learning loss of the cross-model contrastive learning module using the second set of positive sample pairs and the second set of negative sample pairs.

[0023] S36: The supervised cross-entropy classification loss of the supervised classification module is obtained by calculating the raw EEG signal set.

[0024] S37: Calculate the total loss of the TSF-SCL model for:

[0025]

[0026] Where λ and γ are adjustable hyperparameters greater than 0;

[0027] S38: Repeat steps S31 to S37 for a total of P times, and use the TSF-SCL model with the minimum total loss as the trained TSF-SCL model.

[0028] Preferably, step S32 specifically includes:

[0029] S321: Get x i corresponding and and x j corresponding and Composition of time-domain and frequency-domain sample pairs Time-space sample pairs Spatial-frequency sample pairs

[0030] S322: If i = j, then... and Both are taken as the first positive sample pair; if i ≠ j, then... and All of them are used as the first negative sample pair;

[0031] S323: Repeat steps S321 to S322 a total of N times to obtain the first set of positive sample pairs and the first set of negative sample pairs, where j is the number of the EEG signal.

[0032] Preferably, step S33 specifically includes:

[0033] S331: Calculate the distance between each first positive and negative sample pair using the following formula:

[0034]

[0035] Where i and j are the numbers of the EEG signals, and N is the total number of EEG signals; For the first positive sample pair, v,k∈{T,S,F} and v≠k; For the first negative sample pair, i≠j, v≠Vneg; d(·) represents the distance; sim() is the cosine similarity function; I [i≠j] The indicator function is 1 if and only if i ≠ j; τ is an adjustable hyperparameter.

[0036] S332: Calculate the cross-view contrastive learning loss by using the distance between each first positive and negative sample pair. The calculation formula is:

[0037]

[0038] Where V represents the total number of views.

[0039] Preferably, step S34 specifically includes:

[0040] S341: x i corresponding and Input the upper branch network to obtain x i The corresponding upper branch enhancement features include: temporal features airspace features and frequency domain features x j Input the lower branch network to obtain the original feature z j ;

[0041] S342: Composing the original time-domain sample pair (z j , ), original-spatial sample pairs (z j , ) and original-frequency domain sample pairs (z j , );

[0042] S343: If i = j, then (z) j , ), (z j , ) and (z j , All of them are taken as the second positive sample pair. If i ≠ j, then (z) j , ), (z j , ) and (z j , All of them are used as the second negative sample pair;

[0043] S344: Repeat steps S341 to S343 for a total of N times to obtain the second set of positive sample pairs and the second set of negative sample pairs, where j is the number of the EEG signal.

[0044] Preferably, step S35 specifically includes:

[0045] S351: Calculate the distance between each second positive and negative sample pair using the following formula:

[0046]

[0047] Where i and j are the numbers of the EEG signals, and N is the total number of EEG signals; (z i , (z) represents the second positive sample pair, v∈{T,S,F}; i , ) represents the second negative sample pair, Mneg∈{T,S,F}, and i≠j; d(·) represents the distance; sim() is the cosine similarity function; I [i≠j] The indicator function is 1 if and only if i ≠ j; τ is an adjustable hyperparameter.

[0048] S352: Calculate the cross-model contrastive learning loss by measuring the distance between each second positive and negative sample pair. The calculation formula is:

[0049]

[0050] Where V represents the total number of views.

[0051] A storage medium storing instructions and data for implementing the supervised contrastive learning method based on spatiotemporal frequency consistency.

[0052] A supervised contrastive learning device based on spatiotemporal frequency consistency includes: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the supervised contrastive learning method based on spatiotemporal frequency consistency.

[0053] The present invention has the following beneficial effects:

[0054] At the data processing level, positive and negative sample pairs are formed by augmenting data in the time, spatial, and frequency domains. This fully utilizes multi-source data for TSF-SCL model training, thereby improving the classification accuracy of the TSF-SCL model. At the model design level, a cross-view contrastive learning module and a cross-model contrastive learning module are designed. The cross-view contrastive learning module extracts discriminative features from three augmented views, and features from different views are used to form positive and negative sample pairs to train the model, enhancing feature consistency. The cross-model contrastive learning module further improves the consistency of features extracted by different networks (such as feature extractors and encoders). These methods effectively achieve the fusion of multi-source (temporal, spatial, and frequency domain) knowledge, improving the decoding accuracy and practicality of the motor imagery brain-computer interface system. Attached Figure Description

[0055] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of the workflow of the TSF-SCL model;

[0057] Figure 3A comparison image of EEG signals before and after different types of data enhancement;

[0058] Figure 4 Performance comparison of combinations of different types of data augmentation on four motor imagery EEG datasets;

[0059] Figure 5 Results of ablation experiments on four motor imagery EEG datasets;

[0060] Figure 6 For the parameter sensitivity analysis experiment on four motor imagery EEG datasets;

[0061] Figure 7 For feature visualization plots on the MI1 dataset, each subplot represents a data visualization of a single participant;

[0062] Figure 8 This is a structural diagram of the device of the present invention;

[0063] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0064] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0065] Reference Figure 1 This invention provides a supervised contrastive learning method based on spatiotemporal frequency consistency, comprising:

[0066] S1: Construct the TSF-SCL model, which includes: a multi-source data augmentation module, a cross-view contrastive learning module, a cross-model contrastive learning module, and a supervised classification module;

[0067] Specifically, the workflow of the TSF-SCL model is as follows: Figure 2 As shown, the cross-view comparative learning module, the cross-model comparative learning module, and the supervised classification module are arranged in a parallel structure.

[0068] S2: Obtain the raw EEG signal set, and perform multi-source data enhancement on the raw EEG signal set through the multi-source data enhancement module to obtain the time domain enhanced data set, the spatial domain enhanced data set, and the frequency domain enhanced data set;

[0069] Furthermore, step S2 specifically involves:

[0070] S21: The multi-source data enhancement module includes: a time-domain enhancement module, a spatial-domain enhancement module, and a frequency-domain enhancement module;

[0071] S22: Input the original set of EEG signals into the time domain enhancement module, spatial domain enhancement module and frequency domain enhancement module respectively for data enhancement, and obtain the time domain enhanced data set T, spatial domain enhanced data set S and frequency domain enhanced data set F respectively.

[0072] Specifically, in the spatiotemporal-frequency multi-source data enhancement module, three branches were designed to perform EEG data enhancement in the time domain, frequency domain, and spatial domain of the original signal, respectively. The specific strategies are shown in Table 2.

[0073] Table 2 Implementation methods and parameters of different types of data augmentation strategies

[0074]

[0075] This invention implements the aforementioned seven data augmentation techniques, including three time-domain augmentations, two frequency-domain augmentations, and two spatial-domain augmentations. Visualization of the signal before and after data augmentation using these seven strategies is shown below. Figure 3 As shown, after multi-source data augmentation, the augmented signals and the original signals are input into the proposed supervised contrastive learning training framework TSF-SCL model.

[0076] S3: Synchronously train each module in the TSF-SCL model using the original EEG signal set, time-domain enhanced data set, spatial-domain enhanced data set, and frequency-domain enhanced data set to obtain a trained TSF-SCL model;

[0077] Furthermore, step S3 specifically includes:

[0078] S31: The acquired raw EEG signal set includes N EEG signals x i The time-domain augmentation dataset includes N time-domain augmentation datasets. The airspace augmentation dataset includes N airspace augmentation datasets. The frequency domain enhancement dataset includes N frequency domain enhancement datasets. i is the number of the EEG signal, and N is the total number of EEG signals;

[0079] S32: Construct the first positive sample pair set and the first negative sample pair set using the time-domain enhanced data set, the spatial-domain enhanced data set, and the frequency-domain enhanced data set;

[0080] Furthermore, these three types of augmented data are paired into time-frequency, time-space, and space-frequency sample pairs, respectively. Augmented samples from the same original sample are defined as positive sample pairs, and augmented samples from different original samples are defined as negative sample pairs. Alignment between different views is achieved using the NT-Xent (normalized temperature-scaled cross-entropy) loss commonly used in contrastive learning. Assuming each training batch has N samples, for each sample, 3N augmented samples can be generated, of which 3(N-1) are cross-view negative samples of the original sample. The rest are positive samples.

[0081] Step S32 is as follows:

[0082] S321: Get x i corresponding and and x j corresponding and Composition of time-domain and frequency-domain sample pairs Time-space sample pairs Spatial-frequency sample pairs

[0083] S322: If i = j, then... and Both are taken as the first positive sample pair; if i ≠ j, then... and All of them are used as the first negative sample pair;

[0084] S323: Repeat steps S321 to S322 a total of N times to obtain the first set of positive sample pairs and the first set of negative sample pairs, where j is the number of the EEG signal.

[0085] S33: Calculate the cross-view contrast learning loss of the cross-view contrast learning module using the first set of positive sample pairs and the first set of negative sample pairs.

[0086] Furthermore, the cross-view contrastive learning module is trained in a self-supervised manner. By designing a cross-view contrastive learning loss, it achieves alignment of different types of augmented data, thereby improving the consistency of the extracted features.

[0087] Step S33 is as follows:

[0088] S331: Calculate the distance between each first positive and negative sample pair using the following formula:

[0089]

[0090] Where i and j are the numbers of the EEG signals, and N is the total number of EEG signals; For the first positive sample pair, v,k∈{T,S,F} and v≠k; For the first negative sample pair, i≠j, v≠Vneg; d(·) represents the distance; sim() is the cosine similarity function; I [i≠j] The indicator function is 1 if and only if i ≠ j; τ is an adjustable hyperparameter.

[0091] S332: Calculate the cross-view contrastive learning loss by using the distance between each first positive and negative sample pair. The calculation formula is:

[0092]

[0093] Where V is the total number of views, and V is set to 3.

[0094] S34: The cross-model comparison learning module includes an upper branch network and a lower branch network; the time-domain enhanced data set, the spatial domain enhanced data set, and the frequency domain enhanced data set are input into the upper branch network to obtain the upper branch enhanced feature set, and the original EEG signal set is input into the lower branch network to obtain the original feature set. The second positive sample pair set and the second negative sample pair set are constructed through the upper branch enhanced feature set and the original feature set.

[0095] Furthermore, the enhanced features extracted from the upper branch network and the features extracted from the original data through the lower branch network are combined in pairs to form positive and negative sample pairs. Samples from the same original sample before and after enhancement are defined as positive sample pairs, and samples from different original samples before and after enhancement are defined as negative sample pairs. Similarly, alignment between output features of different models is achieved based on NT-Xent loss. Assuming each training batch has N samples, with a total of 4N training samples including original samples and enhanced samples, there are 4(N-1) cross-model negative samples. The rest are positive samples.

[0096] Step S34 is as follows:

[0097] S341: x i corresponding and Input the upper branch network to obtain x i The corresponding upper branch enhancement features include: temporal features airspace features and frequency domain features x j Input the lower branch network to obtain the original feature z j ;

[0098] S342: Composing the original time-domain sample pair (z j , ), original-spatial sample pairs (z j , ) and original-frequency domain sample pairs (z j , );

[0099] S343: If i = j, then (z) j , ), (z j , ) and (z j , All of them are taken as the second positive sample pair. If i ≠ j, then (z) j , ), (z j , ) and (z j , All of them are used as the second negative sample pair;

[0100] S344: Repeat steps S341 to S343 for a total of N times to obtain the second set of positive sample pairs and the second set of negative sample pairs, where j is the number of the EEG signal.

[0101] S35: Calculate the cross-model contrastive learning loss of the cross-model contrastive learning module using the second set of positive sample pairs and the second set of negative sample pairs.

[0102] Furthermore, the cross-model comparative learning module consists of two parts: the encoder and projector in the upper branch are trained in a self-supervised manner, while the backbone network and classifier in the lower branch are trained in a supervised manner. This cross-model comparative learning module further improves the consistency of features extracted by different models.

[0103] Step S35 is as follows:

[0104] S351: Calculate the distance between each second positive and negative sample pair using the following formula:

[0105]

[0106] Where i and j are the numbers of the EEG signals, and N is the total number of EEG signals; (z i , (z) represents the second positive sample pair, v∈{T,S,F}; i , ) represents the second negative sample pair, Mneg∈{T,S,F}, and i≠j; d(·) represents the distance; sim() is the cosine similarity function; I [i≠j]The indicator function is 1 if and only if i ≠ j; τ is an adjustable hyperparameter.

[0107] S352: Calculate the cross-model contrastive learning loss by measuring the distance between each second positive and negative sample pair. The calculation formula is:

[0108]

[0109] Where V represents the total number of views.

[0110] S36: The supervised cross-entropy classification loss of the supervised classification module is obtained by calculating the raw EEG signal set.

[0111] S37: Calculate the total loss of the TSF-SCL model for:

[0112]

[0113] Where λ and γ are adjustable hyperparameters greater than 0; λ and γ are set to 0.1;

[0114] S38: Repeat steps S31 to S37 for a total of P times, and use the TSF-SCL model with the minimum total loss as the trained TSF-SCL model.

[0115] S4: Obtain the set of raw EEG signals to be tested, and classify the set of raw EEG signals to be tested using the trained TSF-SCL model to obtain the classification prediction results.

[0116] Specifically, the steps for training the TSF-SCL model are as follows:

[0117] Operation S1: Input the raw training data Test data Initialize the networks E and P of the upper branch, the backbone network E of the lower branch, and the classifier F;

[0118] Operation S2: Perform different types of data augmentation on the i-th input data according to the batch, and define A. T (·) represents temporal enhancement, A s (·) represents spatial enhancement, A F (·) represents frequency domain enhancement, and the enhanced data can be represented as:

[0119]

[0120] Furthermore, by enhancing the data input to the upper branch of the network E and P, we obtain:

[0121]

[0122] Operation S3: Input the original data into the lower branch network E and F to obtain z. i =E(x) i ),

[0123] Operation S4: Calculate the cross-view contrastive learning loss, the cross-model contrastive learning loss, and the supervised cross-entropy classification loss respectively;

[0124] Operation S5: Use the trained backbone network E and classifier F to classify the test data and obtain the predicted labels.

[0125]

[0126] Experimental comparison:

[0127] Table 2 presents the experimental results of this patented solution on four motor imagery EEG datasets: Zhou2016, MI1, BNCI2014002, and BNCI2015001, using three different model architectures: EEGNet, ShallowCNN, and DeepCNN. The results show that TSF-SCL significantly outperforms all baseline methods, including those without augmentation, those using seven different data augmentations, and those using two self-supervised methods.

[0128] Table 2 Performance Comparison of TSF-SCL and Baseline Methods

[0129]

[0130] Figure 4 The results of ablation experiments show that the combination of different enhancement methods of this patented solution has significant effects within the proposed TSF-SCL framework.

[0131] Table 3 shows the performance of the spatiotemporal-frequency-spatial supervised contrastive learning framework of this patented solution. It is still an improvement compared to the benchmark method, and the effect is not much different from that of TSF-SCL.

[0132] Table 3 compares the performance of three-view augmentation and spatiotemporal-frequency view augmentation on four datasets.

[0133]

[0134] Figure 5-6 The results of ablation experiments and parameter sensitivity analysis are presented, showing that the different modules proposed all bring performance improvements and demonstrate the robustness of the method. Figure 7The results of feature visualization are shown: (a) the baseline method, where the feature distribution is uneven across different views; and (b) the TSF-SCL method, where the proposed scheme ensures that all data are evenly distributed in the feature space and the classification interface is clearly visible. The uneven feature distribution across different views in the baseline method, compared to the uniform distribution of features extracted by TSF-SCL and the clear visibility of the classification interface, demonstrates that the proposed scheme significantly improves the consistency of the features extracted by the model and achieves better classification performance.

[0135] Please see Figure 8 , Figure 8 This is a schematic diagram of the hardware device in operation according to an embodiment of the present invention. The hardware device specifically includes: a supervised contrastive learning device 401 based on spatiotemporal frequency consistency, a processor 402, and a storage medium 403.

[0136] A supervised contrastive learning device 401 based on spatiotemporal frequency consistency: The supervised contrastive learning device 401 based on spatiotemporal frequency consistency implements the supervised contrastive learning method based on spatiotemporal frequency consistency.

[0137] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the supervised contrastive learning method based on spatiotemporal frequency consistency.

[0138] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the supervised contrastive learning method based on spatiotemporal frequency consistency.

[0139] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0140] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as identifiers.

[0141] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A supervised contrastive learning method based on spatiotemporal frequency consistency, characterized in that, include: S1: Construct the TSF-SCL model, which includes: a multi-source data augmentation module, a cross-view contrastive learning module, a cross-model contrastive learning module, and a supervised classification module; S2: Obtain the raw EEG signal set, and perform multi-source data enhancement on the raw EEG signal set through the multi-source data enhancement module to obtain the time domain enhanced data set, the spatial domain enhanced data set, and the frequency domain enhanced data set; S3: Synchronously train each module in the TSF-SCL model using the original EEG signal set, time-domain enhanced data set, spatial-domain enhanced data set, and frequency-domain enhanced data set to obtain a trained TSF-SCL model; Step S3 is as follows: S31: The acquired raw EEG signal set includes N EEG signals. The time-domain augmentation dataset includes N time-domain augmentation datasets. The airspace augmentation dataset includes N airspace augmentation datasets. The frequency domain enhancement dataset includes N frequency domain enhancement datasets. , i is the number of the EEG signal, and N is the total number of EEG signals; S32: Construct the first positive sample pair set and the first negative sample pair set using the time-domain enhanced data set, the spatial-domain enhanced data set, and the frequency-domain enhanced data set; S33: Calculate the cross-view contrast learning loss of the cross-view contrast learning module using the first set of positive sample pairs and the first set of negative sample pairs. ; S34: The cross-model comparison learning module includes an upper branch network and a lower branch network; the time-domain enhanced data set, the spatial domain enhanced data set, and the frequency domain enhanced data set are input into the upper branch network to obtain the upper branch enhanced feature set, and the original EEG signal set is input into the lower branch network to obtain the original feature set. The second positive sample pair set and the second negative sample pair set are constructed through the upper branch enhanced feature set and the original feature set. S35: Calculate the cross-model contrastive learning loss of the cross-model contrastive learning module using the second set of positive sample pairs and the second set of negative sample pairs. ; S36: The supervised cross-entropy classification loss of the supervised classification module is obtained by calculating the raw EEG signal set. ; S37: Calculate the total loss of the TSF-SCL model for: in, and For adjustable hyperparameters greater than 0; S38: Repeat steps S31 to S37 for a total of P times, and take the TSF-SCL model with the minimum total loss as the trained TSF-SCL model. S4: Obtain the set of raw EEG signals to be tested, and classify the set of raw EEG signals to be tested using the trained TSF-SCL model to obtain the classification prediction results.

2. The supervised contrastive learning method based on spatiotemporal frequency consistency according to claim 1, characterized in that, Step S2 is as follows: S21: The multi-source data enhancement module includes: a time-domain enhancement module, a spatial-domain enhancement module, and a frequency-domain enhancement module; S22: Input the original set of EEG signals into the time domain enhancement module, spatial domain enhancement module and frequency domain enhancement module respectively for data enhancement, and obtain the time domain enhanced data set T, spatial domain enhanced data set S and frequency domain enhanced data set F respectively.

3. The supervised contrastive learning method based on spatiotemporal frequency consistency according to claim 1, characterized in that, Step S32 is as follows: S321: Obtain corresponding , and ,as well as corresponding , and Forming time-domain and frequency-domain sample pairs ( , ), time-space sample pairs ( , ) and spatial-frequency sample pairs ( , ); S322: If i=j, then ( , ), ( , )and( , All of them are taken as the first positive sample pair. If i ≠ j, then ( , ), ( , )and( , All of them are used as the first negative sample pair; S323: Repeat steps S321 to S322 a total of N times to obtain the first set of positive sample pairs and the first set of negative sample pairs, where j is the number of the EEG signal.

4. The supervised contrastive learning method based on spatiotemporal frequency consistency according to claim 1, characterized in that, Step S33 is as follows: S331: Calculate the distance between each first positive and negative sample pair using the following formula: Where i and j are the numbers of the EEG signals, and N is the total number of EEG signals; This is the first positive sample pair. ; For the first negative sample pair, i ≠ j, ;d Indicates distance; sim() is the cosine similarity function; For an indicator function, if and only if The time is 1; It is an adjustable hyperparameter; S332: Calculate the cross-view contrastive learning loss by using the distance between each first positive and negative sample pair. The calculation formula is: Where V represents the total number of views.

5. The supervised contrastive learning method based on spatiotemporal frequency consistency according to claim 1, characterized in that, Step S34 is as follows: S341: Will corresponding , and Input the upper branch network to obtain The corresponding upper branch enhancement features include: temporal features airspace characteristics and frequency domain features ;Will Input the lower branch network to obtain the original features ; S342: Composing original-time domain sample pairs ( , ), original-spatial sample pairs ( , ) and original-frequency domain sample pairs ( , ); S343: If i=j, then ( , ), ( , )and( , All of them are taken as the second positive sample pair. If i ≠ j, then ( , ), ( , )and( , All of them are used as the second negative sample pair; S344: Repeat steps S341 to S343 for a total of N times to obtain the second set of positive sample pairs and the second set of negative sample pairs, where j is the number of the EEG signal.

6. The supervised contrastive learning method based on spatiotemporal frequency consistency according to claim 1, characterized in that, Step S35 is as follows: S351: Calculate the distance between each second positive and negative sample pair using the following formula: Where i and j are the numbers of the EEG signals, and N is the total number of EEG signals; This is the second positive sample pair. ; This is the second negative sample pair. And i≠j; d Indicates distance; sim() is the cosine similarity function; For an indicator function, if and only if The time is 1; It is an adjustable hyperparameter; S352: Calculate the cross-model contrastive learning loss by measuring the distance between each second positive and negative sample pair. The calculation formula is: Where V represents the total number of views.

7. A storage medium, characterized in that: The storage medium stores instructions and data to implement the supervised contrastive learning method based on spatiotemporal frequency consistency as described in any one of claims 1 to 6.

8. A supervised contrastive learning device based on spatiotemporal frequency consistency, characterized in that: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a supervised contrastive learning method based on spatiotemporal frequency consistency as described in any one of claims 1 to 6.

Images