A Dual-Domain, Dual-Scale, Cross-Subject SSVEP Decoding Method and Device Based on Transfer Learning

Abstract

This invention discloses a dual-domain, dual-scale cross-subject SSVEP decoding method and apparatus based on transfer learning, belonging to the field of brain-computer interface technology. The method includes: acquiring source domain SSVEP EEG data and target domain SSVEP EEG data to be decoded; subband filtering and generating sine and cosine reference matrix templates; for the source domain data of each subband in the multi-subband data of the source domain, constructing source domain dual-scale template pairs for each subband for each candidate stimulus frequency, and extracting single-cycle cross-subject spatial filters; applying temporal local constraints to the target domain data of each subband, and extracting adaptive spatial filters; calculating multi-path evidence scores, and performing weighted fusion based on subband weights to output the decoding result. This invention does not require target subject training data, and achieves high-precision, robust cross-subject decoding through dual-domain, dual-scale feature fusion and adaptive adaptation. It has low computational cost and supports real-time operation, significantly improving the practicality and generalization ability of brain-computer interfaces.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, specifically a dual-domain, dual-scale cross-subject SSVEP (steady-state visual evoked potential) decoding method and device based on transfer learning. Background Technology

[0002] Brain-computer interfaces (BCIs) based on steady-state visually evoked potentials (SSVEPs) have attracted widespread attention in the field of BCIs due to their advantages such as high information transmission rate and multiple control targets. When a user gazes at a stimulus source flashing at a stable frequency, the visual cortex in the posterior occipital lobe of the brain generates an electroencephalogram (EEG) response corresponding to the stimulus frequency and its harmonics. By analyzing the steady-state response in the scalp EEG signal, the user's gaze target can be identified, thereby enabling command decoding. SSVEP-based BCIs have demonstrated high performance in BCI applications such as spelling, device control, and games.

[0003] Designing an efficient SSVEP feature decoding method is the core issue of SSVEP-BCI. Classical SSVEP decoding methods such as Canonical Correlation Analysis (CCA) and Filter Bank Canonical Correlation Analysis (FBCCA) have limited recognition accuracy due to signal noise. Task-Related Component Analysis (TRCA) and Periodically Repeated Component Analysis (PRCA) have trained recognition algorithms that perform well among subjects, but they generally rely on a large amount of training data from the target subjects. In real-world applications, the tedious data collection process increases user fatigue and seriously affects the practicality of the BCI system. At the same time, the individual differences that are common among subjects exacerbate the instability of cross-subject decoding. Directly transferring filters or templates will limit generalization ability and affect decoding performance and recognition accuracy. Furthermore, algorithms such as Transfer Template-based Canonical Correlation Analysis (TTCCA) and Cross-subject Spatial Filter Transfer (CSSFT) mostly utilize spatial filters obtained from CCA for transfer, only considering the overall steady-state response, resulting in weak feature extraction capabilities. Moreover, the template scale is singular during transfer, which limits the final recognition performance of the algorithms.

[0004] Therefore, there is an urgent need for a decoding method that can directly utilize the group prior formed by the historical data of other subjects to carry out transfer learning without any calibration or training for the target subjects, and can still maintain high recognition accuracy and stability under different subject conditions, so as to reduce deployment costs, shorten deployment cycle and improve the universality and robustness of practical applications. Summary of the Invention

[0005] The technical objective of this invention is to address the problems of existing SSVEP brain-computer interface decoding methods, such as reliance on individual training, insufficient cross-subject generalization ability, and a single template scale during transfer learning. This invention proposes a dual-domain, dual-scale cross-subject SSVEP decoding method and device based on transfer learning for SSVEP frequency recognition.

[0006] To achieve the above technical objectives, the embodiments of this application adopt the following technical solutions.

[0007] In a first aspect, the embodiment provides a dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning, including: acquiring source domain SSVEP EEG data and target domain SSVEP EEG data to be decoded;

[0008] Based on the SSVEP EEG data from the source and target domains, multi-subband filtering is performed to obtain source domain multi-subband data and target domain multi-subband data; corresponding sine and cosine reference matrix templates are generated for all candidate stimulus frequencies decoded by SSVEP.

[0009] For the source domain data of each sub-band in the source domain multi-sub-band data, a source domain overall average template and a source domain single-cycle template are constructed for each candidate stimulus frequency, forming a source domain dual-scale template pair exclusive to each sub-band.

[0010] For each sub-band, a single-cycle cross-subject space filter is extracted from the source domain single-cycle template.

[0011] Temporal local constraints are applied to the target domain data of each sub-band in the multi-sub-band data of the target domain to obtain the target domain data with time constraints specific to each sub-band.

[0012] For each sub-band and its corresponding candidate stimulus frequency, the target domain adaptive filter for each sub-band is extracted by combining the target domain data of that sub-band with the sine and cosine reference matrix template.

[0013] For each subband, based on the corresponding time-constrained target domain data, single-cycle cross-subject space filter, target domain adaptive filter, and the source domain dual-scale template pair, three-way evidence scores are calculated, and the total evidence score is calculated. Combined with the preset weight coefficients of each subband, the total evidence scores of all subbands are weighted and fused to obtain the discrimination score of each candidate stimulus frequency. The candidate stimulus frequency with the largest discrimination score is selected as the decoding result output.

[0014] On the other hand, the embodiments also provide a dual-domain, dual-scale cross-subject SSVEP decoding device based on transfer learning, comprising:

[0015] The data acquisition module is used to acquire source domain SSVEP EEG data and target domain SSVEP EEG data to be decoded;

[0016] The sub-band data determination module is used to perform multi-sub-band filtering on SSVEP EEG data based on the source domain and the target domain to obtain source domain multi-sub-band data and target domain multi-sub-band data respectively.

[0017] The reference matrix template generation module is used to generate corresponding sine and cosine reference matrix templates for all candidate stimulus frequencies of SSVEP decoding.

[0018] The template pair generation module is used to construct an overall average template and a single-period template of the source domain for each sub-band of the source domain multi-sub-band data, corresponding to each candidate stimulus frequency, to form a unique dual-scale template pair of the source domain for each sub-band.

[0019] The spatial filter generation module is used to extract the corresponding single-period cross-subject spatial filter for each sub-band's source domain single-period template.

[0020] The time-constrained target domain data generation module is used to apply local time constraints to the target domain data of each sub-band in the multi-sub-band data of the target domain, so as to obtain time-constrained target domain data for each sub-band.

[0021] The adaptive filter extraction module is used to extract a target domain adaptive filter for each sub-band and its corresponding candidate stimulus frequency, by combining the target domain data of the sub-band with the sine and cosine reference matrix template.

[0022] The decoding module is used to calculate the three-way evidence score for each subband based on the corresponding target domain data or time-constrained target domain data, single-cycle cross-subject space filter, target domain adaptive filter, and the source domain dual-scale template pair, and to calculate the total evidence score; combined with the preset weight coefficients of each subband, the total evidence scores of all subbands are weighted and fused to obtain the discrimination score of each candidate stimulus frequency, and the candidate stimulus frequency with the largest discrimination score is selected as the decoding result output.

[0023] Compared with the prior art, the beneficial technical effects achieved by the dual-domain dual-scale cross-subject SSVEP decoding method and apparatus based on transfer learning provided in this application include:

[0024] 1. This invention does not require individual training data from the target subjects and does not require calibration time, which greatly improves the practicality of the BCI system;

[0025] 2. In the process of constructing transfer features, two templates are used simultaneously: the overall average template and the single-period template of the source domain data (training data) for each sub-band in the multi-sub-band data of the source domain. The overall average template reflects the overall steady-state law of SSVEP, while the single-period template captures the detailed changes in a single stimulus cycle. By fusing these two types of scale features during the transfer process, this invention can maintain overall robustness while taking into account subtle differences at the cycle level, thereby improving transfer capability.

[0026] 3. By extracting stable single-cycle cross-subject spatial filters from the source domain and transferring them to the target domain, and combining them with the specific spatial distribution characteristics of the target domain's EEG, the synergistic utilization of cross-subject knowledge and individual characteristics is achieved, significantly improving the accuracy and robustness of cross-subject training-free decoding.

[0027] It should be understood that the summary section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this application in any way.

[0029] In the attached diagram:

[0030] Figure 1 A schematic diagram of the process of the dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning provided for an embodiment;

[0031] Figure 2 This is a schematic diagram of the process for preprocessing EEG signals in the embodiment;

[0032] Figure 3 This is a schematic diagram of the process for obtaining standardized sub-band signals and sine and cosine reference matrix templates based on preprocessed standardized signals in the embodiment.

[0033] Figure 4 This is a schematic diagram of the process for obtaining dual-scale template pairs based on standardized subband signals in the embodiment.

[0034] Figure 5 This is a schematic diagram of the process for obtaining a single-cycle cross-subject space filter based on a single-cycle stacked tensor in the embodiment.

[0035] Figure 6 This is a flowchart illustrating the process of obtaining local temporal constraints based on target domain data and a sine / cosine reference matrix template in an embodiment.

[0036] Figure 7 This is a flowchart illustrating the adaptive filter for the target domain in the embodiment.

[0037] Figure 8 This is a schematic diagram illustrating the process of performing three-way correlation calculations, outputting recognition results, and applying them in this embodiment.

[0038] Figure 9 This is a schematic diagram of the technical route of the dual-domain dual-scale cross-subject SSVEP decoding method based on transfer learning provided in the embodiment. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0040] A dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning, such as... Figure 1 and Figure 9 As shown, it includes the following steps:

[0041] Step 1: Acquire source domain SSVEP EEG data and target domain SSVEP EEG data to be decoded.

[0042] In some embodiments, such as Figure 2 As shown, in step 1, a publicly available benchmark dataset was used. The collected EEG signals were recorded at relevant electrode sites Pz, POz, PO3, PO4, PO5, PO6, Oz, O1, and O2 in the occipital and parietal lobes according to the international 10-10 system, with a sampling frequency f. s =250 Hz.

[0043] In some embodiments, step 1 further includes standardization preprocessing of the source domain SSVEP EEG data and the target domain SSVEP EEG data to be decoded, thereby obtaining standardized signal input. Taking a publicly available benchmark dataset as an example, the standardization preprocessing includes: using a 50 Hz comb notch filter for noise reduction to suppress power frequency interference; and aligning all trials uniformly at the stimulus start time.

[0044] Some implementations use a specified time point (e.g., 0.5s) after the stimulus start time as the starting point for data analysis, while using a uniform latency fixed compensation time (e.g., 140 ms) for early transients, for example, starting from 0.5s + 0.14s, i.e., 0.64s, as the final starting point.

[0045] Step 2: As Figure 3 As shown, based on the (standardized preprocessed) source domain and target domain SSVEP EEG data, multi-subband filtering is performed to obtain source domain multi-subband data and target domain multi-subband data; corresponding sine and cosine reference matrix templates (also known as standard sine and cosine templates) are generated for all candidate stimulus frequencies of SSVEP decoding.

[0046] To improve bandwidth resolution and anti-interference performance, after preprocessing, each segment of EEG from the source and target domains is synchronously fed into a subband filter bank consisting of K bandpass filters (K is a positive integer, and in some embodiments K is 5) for decomposition.

[0047] As an example, multi-subband filtering is implemented using a Chebyshev Type I bandpass filter bank, which divides the data into K subbands. The Chebyshev Type I bandpass filter bank contains K Chebyshev Type I bandpass filters, where K is a positive integer.

[0048] The k-th subband uses a Chebyshev Type I bandpass filter with passband and stopband set to [8k, 90] Hz and [8k−2, 100] Hz respectively, with passband ripple not greater than 0.5dB and stopband attenuation not less than 40dB.

[0049] Within the source domain, let the frequency be f, the subband number be k∈{1,…,K}, the number of channels be C, and the sampling frequency be f. s The signal length of the standard analysis window is T (number of sample points), N tr Number of trials for source domain data ， Let f be the set of real numbers. Then, for the same candidate stimulus frequency and subband, let f be the m-th candidate stimulus frequency after subband filtering. m The test signal for the h-th test in the k-th subband is: , .

[0050] Within the target domain, let the sub-band signal of the target domain trial be denoted as... The target domain trial signal in the k-th subband after subband filtering is: .

[0051] In some embodiments, a sine and cosine reference matrix template is generated for each candidate stimulus frequency in SSVEP decoding; this is the most commonly used reference signal in SSVEP decoding research. s and f m N represents the sampling frequency and the frequency of the m-th SSVEP candidate stimulus, respectively. h Represents the harmonic number (N in the experiment) h (Can be set to 5, can be set):

[0052]

[0053] For the m-th SSVEP candidate stimulus frequency f m The generated sine and cosine reference matrix template.

[0054] Step 3: For the source domain data of each sub-band in the multi-sub-band source domain data, construct the overall average template and the single-period template of the source domain for each candidate stimulus frequency, forming a unique dual-scale template pair for each sub-band.

[0055] Step 3 involves overlaying and averaging the source domain data to obtain the "overall scale" template at each frequency. After upsampling, the template is sliced ​​according to the period length, overlaid and averaged, and then copied and expanded to obtain the "single period scale" template, forming a "dual scale" representation of the source domain template.

[0056] In some embodiments, a source domain overall average template and a source domain single-period template are constructed, such as... Figure 4 As shown, this includes: for each sub-band, averaging the source domain data with the same candidate stimulus frequency within that sub-band trial by trial to obtain the overall source domain average template for the corresponding candidate stimulus frequency of that sub-band, as expressed below:

[0057] ,

[0058] This is the source domain average template for the candidate stimulus frequencies corresponding to the k-th subband.

[0059] Overall average template of the source domain Upsampling is performed along the time axis using two-dimensional interpolation (upsampling factor can be up = 10). The upsampled overall average template of the source domain is divided into several period segments according to the period length (i.e., window length) of the candidate stimulus frequency, resulting in stacked tensors for each period. In the embodiment, the test signal for each training trial of the source domain is... Click on the window along the timeline Slicing yields periodically stacked tensors: , f is the frequency of the m-th SSVEP candidate stimulus. m The number of sampling points per cycle (i.e., the cycle length), p m This is a periodic segment index, with a total of P. m (One cycle segment).

[0060] In the embodiment, the frequency f of the m-th SSVEP candidate stimulus m Number of sampling points per cycle With the divisible period P m as follows:

[0061] ;

[0062] The round(·) function rounds the integer to the nearest whole number. This indicates rounding down, and up is the upsampling factor (set to 10 in the experiment).

[0063] The single-period component (PRC) is obtained by averaging the stacked tensors across all periods:

[0064] ;

[0065] All single-period components Repeating along the time dimension and truncating to the same length as the analysis window of the source domain SSVEP EEG data, we obtain the source domain single-cycle template for the candidate stimulus frequencies corresponding to that subband:

[0066] .

[0067] This results in a "dual-scale" template pair: The former describes the overall steady-state morphology, while the latter emphasizes the periodic-level transferable details. It retains the phase-locked SSVEP components in a statistical sense and suppresses incoherent noise, making the resulting template more morphologically stable and more usable in cross-subject transfer.

[0068] Step 4: Extract the corresponding single-cycle cross-subject space filter for the source domain single-cycle template of each sub-band.

[0069] In some embodiments, such as Figure 5 As shown, step 4 includes single-cycle cross-subject space filter extraction, and the extraction process includes:

[0070] For the m-th SSVEP candidate stimulus frequency f m With subband k, take the single-cycle stacked tensor obtained in step 3. Construct the task-related covariance matrix With the global covariance matrix ;

[0071] Among them, the task-related covariance matrix for:

[0072] ;

[0073] Global covariance matrix for:

[0074] .

[0075] The generalized feature problem is solved using the two constructed covariance matrices, and the eigenvector corresponding to the largest eigenvalue is taken as the initial single-cycle cross-subject space filter:

[0076] ,in w represents the spatial filter to be solved in the generalized characteristic problem. The eigenvector corresponding to the largest eigenvalue is then used as the single-cycle cross-subject space filter for subsequent transfer.

[0077] The spatial responses of the brain are similar across different SSVEP stimuli. Therefore, spatial filters of all candidate stimulus frequencies within the same subband are cascaded to further enhance feature extraction capabilities, resulting in a single-cycle cross-subject spatial filter, expressed as: .

[0078] The single-cycle cross-subject space filter is learned entirely from the source domain template and is independent of the target subject data. In subsequent steps, it serves as a fixed projection, mapping the target trial and the cycle template to the same single-cycle subspace for the correlation assessment of single-cycle detail similarity.

[0079] This invention mainly consists of small-scale generalized feature decomposition and correlation calculation, which has low computational complexity and small memory footprint. Moreover, it adopts the construction form of average template, which compresses a large amount of source domain data into an average template of one category, greatly reducing the amount of computation and storage requirements. It can complete a frequency discrimination in milliseconds on a general-purpose CPU without the need for dedicated acceleration hardware, making it easy to implement in embedded or low-power systems, and has high real-time performance and computational efficiency.

[0080] Step 5: Apply time local constraints to the target domain data of each sub-band in the multi-sub-band data of the target domain to obtain the target domain data with time constraints for each sub-band.

[0081] Step 5 constructs a time-local Laplace matrix for each sub-band of the target domain, applies time-local constraints to the target trial signal, enhances the consistency between adjacent time points and suppresses transient noise, thereby reflecting the time structure characteristics of the target domain. At the same time, the same time-local constraints are applied to the sine and cosine reference signals to maintain consistent constraint conditions.

[0082] To enhance temporal consistency within a short time window and suppress unsteady disturbances, some embodiments, such as Figure 6 As shown, step 5 specifically includes: setting the time scale τ for the k-th sub-band. k (Unit: sampling points), that is, embedding the Laplacian matrix L into the covariance matrix to extract the temporal local information of the data, thereby improving the recognition performance of SSVEP. Let the window length used for correlation calculation be T, and the temporal local transformation matrix, i.e., the Laplacian matrix L, be constructed as follows:

[0083] Firstly, based on the time scale τ k The adjacency matrix is ​​defined using the Tukey weighting function. :

[0084] ;

[0085] in: ;

[0086] Where i and j are time point indices; τ represents the connection weight between time points i and j; k This represents the time scale parameter set for the k-th sub-band, which determines the temporal neighborhood range of the local constraints; r is the polynomial order, typically taken as r = 3. That is, the closer two time points are in time, the greater their weights.

[0087] Based on the time scale τ k Determine the diagonal matrix :

[0088] ;

[0089] diagonal matrix The diagonal element represents the sum of the connection weights between the i-th time point and all other time points.

[0090] This leads to the Graph Laplace matrix. : .

[0091] The graph Laplacian matrix is ​​used as a local time matrix and simultaneously right-multiplied by the target domain trial signal in the k-th subband before correlation calculation. With respect to the frequency f of the m-th SSVEP candidate stimulus m Generated sine and cosine reference matrix template Transform it in the time dimension to obtain the signal after time local constraint correction. With reference matrix :

[0092] ;

[0093] ;

[0094] This strengthens coherent components and weakens long-distance correlations and slow drift interference within the local temporal neighborhood, thereby enhancing the dynamic information in time and reflecting the temporal structure characteristics of the target domain.

[0095] Step 6: For each subband and its corresponding candidate stimulus frequency, combine the target domain data of that subband with the sine and cosine reference matrix template to extract the target domain adaptive filter for each subband.

[0096] like Figure 7 As shown, step 6 combines the current trial data of the target subject and uses CCA to obtain its own target domain adaptive spatial filter. The target trial is then projected in the same direction as the overall template of the source domain to calculate the consistency of the "overall scale" in order to improve individual fit.

[0097] In this embodiment, step 6 involves extracting an adaptive filter through target domain adaptive projection.

[0098] As an example, the steps for extracting the adaptive filter include: for the m-th SSVEP candidate stimulus frequency f m With sub-band k, the target domain trial signal on the k-th sub-band With respect to the frequency f of the m-th SSVEP candidate stimulus m Generated sine and cosine reference matrix template (including N) h Perform canonical correlation analysis (CCA) on the target domain using (sine and cosine) of order:

[0099] ;

[0100] Take the EEG side solution of the first canonical vector As a target domain adaptive filter :

[0101] ;

[0102] Adaptive Filter The corresponding canonical correlation coefficient is denoted as .

[0103] vector As a spatial projection of the target domain adaptation, it is subsequently used to project the target trial and the overall average template of the source domain onto the same one-dimensional subspace of the EEG side for consistency comparison, thereby compensating for individual differences and improving the adaptability of cross-subject decoding.

[0104] Step 7: For each subband, calculate the three-way evidence score based on the corresponding target domain data or time-constrained target domain data, single-cycle cross-subject space filter, target domain adaptive filter, and source domain dual-scale template pair, and calculate the total evidence score; combine the weight coefficients of each subband, perform weighted fusion of the total evidence scores of all subbands to obtain the discrimination score of each candidate stimulus frequency, and select the candidate stimulus frequency with the largest discrimination score as the decoding result output.

[0105] In the embodiments, such as Figure 8 As shown, step 7 calculates the three-way evidence score, including the FBTCCA path score, the single-cycle migration path score, and the target domain adaptive path score, specifically including:

[0106] (a) FBTCCA (Filter Bank Time-weighted Canonical Correlation Analysis) path: FBTCCA correlation with time local constraints (individual's own SSVEP response characteristics).

[0107] After time constraints, a time-weighted canonical correlation analysis is performed between the target domain data and the corresponding sine and cosine reference matrix template for each subband. The first canonical correlation coefficient is taken, and the band sign square of the first canonical correlation coefficient is used as the FBTCCA path score for that subband.

[0108] In this embodiment, the temporal local constraint result obtained in step 5 is used. Perform CCA calculation and take the first canonical correlation coefficient. The signed square is used as the reference match for the FBTCCA path score. : .

[0109] (b) Single-cycle migration path: "Single-cycle scale" consistency (target domain detail variation features). Using the single-cycle cross-subject space filter of each sub-band, the target domain data and the source domain single-cycle template of the sub-band are projected onto the single-cycle feature space respectively. The single-cycle correlation coefficient of the projected data is calculated, and the band sign square of the single-cycle correlation coefficient is used as the single-cycle migration path score of the sub-band.

[0110] In this embodiment, the single-cycle cross-subject space filter learned in step 4 is used. The target trial and the source domain single-period template are simultaneously projected onto the same single-period feature space, and the value at frequency f is calculated. m Below, the single-period correlation coefficient under a single-period migration path :

[0111] ;

[0112] Where corr(·) is the Pearson correlation coefficient.

[0113] Obtain consistent single-cycle migration path score :

[0114] .

[0115] (c) Target Domain Adaptive Path: "Overall Scale" Consistency (Overall Change Characteristics of the Target Domain). Using the target domain adaptive filter of each sub-band, the target domain data of the sub-band and the overall average template of the source domain are projected to the same feature subspace. The adaptive correlation coefficient of the projected data is calculated, and the band-sign square of the adaptive correlation coefficient is taken as the target domain adaptive path score of the sub-band.

[0116] In this embodiment, the target domain adaptive filter obtained from step 6 Project the target trial and the overall average template of the source domain onto the same one-dimensional subspace of the EEG, and calculate the frequency f. m Adaptive correlation coefficient under the target domain adaptive path :

[0117] ;

[0118] Obtain the consistent target domain adaptive path score based on the "target domain - global scale". :

[0119] .

[0120] The scores from the three channels are then processed through a multi-channel fusion and sub-band weighting process for the final frequency decision.

[0121] In this embodiment, the multi-path fusion and sub-band weighting process includes: adjusting the frequency f of the m-th SSVEP candidate stimulus. m In each subband k=1,…,K, we obtain the three correlation scores from step 7. , , To achieve joint utilization of "dual-domain × dual-scale" consistency within the sub-band dimension, this embodiment linearly synthesizes the scores of the three evidence streams mentioned above to obtain the total evidence score corresponding to each sub-band:

[0122] ;

[0123] Where w k is the weight coefficient of the k-th sub-band.

[0124] In some embodiments, weighting coefficients are pre-configured for each subband for subsequent subband weighted fusion.

[0125] As an example, the outputs of each sub-band are weighted and fused according to the following power-law weights based on the corresponding sub-band index k during subsequent fusion: .

[0126] Then, the total evidence scores of all subbands are summed over the frequency dimension to form the discrimination score for the candidate stimulus frequency:

[0127] ;

[0128] This fusion structure is based on time-constrained FBTCCA matching, and introduces two transfer evidences: the "source domain overall average template" and the "source domain single-cycle template". It not only captures the overall steady-state law of SSVEP, but also enhances the detailed changes in a single stimulus cycle. At the same time, it combines the specific spatial distribution characteristics of the target domain EEG to achieve the synergistic utilization of cross-subject knowledge and individual characteristics.

[0129] The system selects the frequency corresponding to the highest discrimination score calculated from all candidate stimulus frequencies as the recognition output: ;

[0130] In the embodiments, the results can be used for brain-controlled device control or human-computer interaction interface display, thereby completing robust decoding across subjects without training.

[0131] In the embodiments, the classification accuracy of different models was evaluated, as shown in Tables 1 and 2.

[0132] Table 1. Classification accuracy results of different models

[0133]

[0134] Table 2. Schematic diagram of the average ablation experimental results for the entire time window of 0.6-1.5s in the examples.

[0135]

[0136] In this example, accuracy refers to the number of correctly identified trials (T). correct ) and total number of trials (T) total The proportion of ) can be expressed mathematically as:

[0137] .

[0138] Table 1 shows the classification accuracy of each algorithm under different time window lengths (0.6s–1.5s). The horizontal axis represents the time window (s), and the vertical axis represents the accuracy (%).

[0139] Trans-eCCA is a statistical learning method that deeply integrates transfer learning and extended canonical correlation analysis (eCCA).

[0140] CIRCST is a cross-stimulus transfer method using common impulse response.

[0141] As shown in Table 1, the classification accuracy of all algorithms increases with the length of the time window, indicating that longer signals provide a more stable steady-state response, thus improving recognition stability. The dual-domain, dual-scale cross-subject SSVEP decoding method (3DSCST) based on transfer learning provided in this application performs best under all time windows, with a more pronounced effect under shorter windows. This result demonstrates that the proposed method maintains high accuracy even under short time windows, indicating its strong temporal robustness and generalization ability; it also fully verifies the superiority of 3DSCST in multi-domain feature fusion.

[0142] Table 2 shows the average ablation experiment results for the entire time window from 0.6 to 1.5 s, i.e., the impact of progressively removing each feature module (single-cycle detail features, target domain global features, temporal local constraints, and individual features) on the final classification performance, and provides the average accuracy and significance test results (p-value) for each condition.

[0143] Table 2 shows the average classification accuracy and significance of each feature module after independent removal within a time window of 0.6–1.5 s, validating the effectiveness of different functional modules in this method. In the experiment, four modules—single-cycle detail change features, overall target domain change features, temporal local constraints, and individual features—were removed individually and compared with the complete model. The results show that the complete method (Cond5) achieved the highest average accuracy of 77.94% under all conditions, significantly outperforming the other feature-missing cases (p<0.001). The accuracy dropped most significantly when individual features (Cond4) were removed, reaching only 64.89%, indicating that the individual's SSVEP response features had the greatest impact on model performance. Removing temporal local constraints (Cond3) reduced the accuracy to 76.59%, demonstrating the positive effect of local consistency in suppressing transient noise. Removing single-cycle detail change features (Cond1) and overall target domain change features (Cond2) resulted in accuracies of 75.97% and 74.34%, respectively, both significantly lower than the complete model. It is evident that the multi-scale, multi-domain fusion structure proposed in this invention plays a crucial role at the temporal, spatial, and individual levels. The absence of any module will weaken the overall recognition performance, verifying the effectiveness of the proposed method in maintaining high robustness and adaptability under complex cross-subject conditions.

[0144] The terms "Step 1", "Step 2", etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance, limiting the order of steps, or implicitly specifying the indicated technical features.

[0145] Based on the same inventive concept as the dual-domain dual-scale cross-subject SSVEP decoding method based on transfer learning provided in the above embodiments, this application also provides a dual-domain dual-scale cross-subject SSVEP decoding device based on transfer learning, including a data acquisition module, a sub-band data determination module, a reference matrix template generation module, a template pair generation module, a spatial filter generation module, a time-constrained target domain data generation module, an adaptive filter extraction module, and a decoding module.

[0146] The data acquisition module is used to acquire source domain SSVEP EEG data and target domain SSVEP EEG data to be decoded.

[0147] The sub-band data determination module is used to perform multi-sub-band filtering on SSVEP EEG data based on the source domain and target domain to obtain source domain multi-sub-band data and target domain multi-sub-band data respectively.

[0148] The reference matrix template generation module is used to generate corresponding sine and cosine reference matrix templates for all candidate stimulus frequencies of SSVEP decoding.

[0149] The template pair generation module is used to construct an overall average template and a single-period template for the source domain for each sub-band in the multi-sub-band source domain data, corresponding to each candidate stimulus frequency, forming a unique dual-scale template pair for each sub-band.

[0150] The spatial filter generation module is used to extract the corresponding single-period cross-subject spatial filter for each sub-band's source domain single-period template.

[0151] The time-constrained target domain data generation module is used to apply time local constraints to the target domain data of each sub-band in the multi-sub-band data of the target domain, so as to obtain the time-constrained target domain data of each sub-band.

[0152] The adaptive filter extraction module is used to extract a target domain adaptive filter for each sub-band and its corresponding candidate stimulus frequency, combining the target domain data of that sub-band with the sine and cosine reference matrix template.

[0153] The decoding module is used to calculate the three-way evidence score for each subband based on the corresponding target domain data or time-constrained target domain data, single-cycle cross-subject space filter, target domain adaptive filter, and source domain dual-scale template pair, and to calculate the total evidence score. Combined with the preset weight coefficients of each subband, the total evidence scores of all subbands are weighted and fused to obtain the discrimination score of each candidate stimulus frequency. The candidate stimulus frequency with the largest discrimination score is selected as the decoding result output.

[0154] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, or a tablet computer, or any combination of these devices.

[0155] The above provides a detailed description of the dual-domain dual-scale cross-subject SSVEP decoding method and apparatus based on transfer learning provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the concept of this application and should not be construed as limiting the scope of protection of this application.

Claims

1. A dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning, characterized in that, include: The source domain SSVEP EEG data and the target domain SSVEP EEG data to be decoded were acquired. The acquired EEG signals were recorded at relevant electrode sites in the occipital and parietal lobes according to the international 10-10 system. Based on the SSVEP EEG data from the source and target domains, multi-subband filtering is performed to obtain multi-subband data from the source and target domains. The multi-subband filtering is implemented using a Chebyshev Type I bandpass filter bank, which divides the data into K subbands. The Chebyshev Type I bandpass filter bank contains K Chebyshev Type I bandpass filters, where K is a positive integer. Corresponding sine and cosine reference matrix templates are generated for all candidate stimulus frequencies decoded by SSVEP. For the source domain data of each sub-band in the source domain multi-sub-band data, a source domain overall average template and a source domain single-period template are constructed for each candidate stimulus frequency to form a source domain dual-scale template pair exclusive to each sub-band. For each subband, a single-cycle cross-subject space filter is extracted from the source domain single-cycle template. Temporal local constraints are applied to the target domain data of each sub-band in the multi-sub-band data of the target domain to obtain the target domain data with time constraints specific to each sub-band. For each subband and its corresponding candidate stimulus frequency, the target domain adaptive filter for each subband is extracted by combining the target domain data of that subband with the sine and cosine reference matrix template. For each subband, based on the corresponding target domain data or time-constrained target domain data, single-cycle cross-subject space filter, target domain adaptive filter, and the source domain dual-scale template pair, three-way evidence scores are calculated, and the total evidence score is calculated. Combining the preset weight coefficients of each subband, the total evidence scores of all subbands are weighted and fused to obtain the discrimination score of each candidate stimulus frequency. The candidate stimulus frequency with the largest discrimination score is selected as the decoding result output.

2. The dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning according to claim 1, characterized in that, The method further includes standardization preprocessing of the source domain SSVEP EEG data and the target domain SSVEP EEG data to be decoded, the standardization preprocessing including: A 50Hz comb-shaped notch filter is used to suppress power frequency interference. Using the start time of SSVEP stimulation as a reference, the timelines of all trials were aligned uniformly. The data analysis started at a time point specified after the start of SSVEP stimulation, and all data were uniformly compensated for a latency of a preset duration.

3. The dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning according to claim 1, characterized in that, The weighting coefficients for each sub-band are power-law weights determined based on the corresponding sub-band number.

4. The dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning according to claim 1, characterized in that, Constructing the overall average template and the single-period template of the source domain, including: For each subband, the source domain data with the same candidate stimulus frequency within that subband are averaged in each trial to obtain the overall average template of the source domain for the corresponding candidate stimulus frequency in that subband. The source domain overall average template is upsampled along the time axis using two-dimensional interpolation. The upsampled source domain overall average template is divided into several period segments according to the period length of the candidate stimulus frequency to obtain the stacked tensor of each period. The single-cycle component is obtained by averaging the stacked tensors across all cycles. All single-cycle components are repeated along the time dimension and truncated to the same length as the analysis window of the source domain SSVEP EEG data to obtain the source domain single-cycle template corresponding to the candidate stimulus frequency of the subband.

5. The dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning according to claim 4, characterized in that, The specific extraction process of the single-period cross-subject space filter includes: The task-related covariance matrix and the global covariance matrix are calculated based on the periodic stacked tensors described above; Solve the generalized feature problem formed by the task-related covariance matrix and the global covariance matrix, obtain the eigenvector corresponding to the largest eigenvalue, and use it as the initial single-cycle cross-subject space filter for the candidate stimulus frequency corresponding to the subband; cascade the initial single-cycle cross-subject space filters for all candidate stimulus frequencies in the subband to obtain the single-cycle cross-subject space filter for the subband.

6. The dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning according to claim 1, characterized in that, The process of applying local temporal constraints includes: defining the adjacency matrix A for each sub-band using the Tukey weighting function; Construct a diagonal matrix D based on the adjacency matrix A; The graph Laplacian matrix L is calculated using the formula L=DA; the graph Laplacian matrix L is then right-multiplied by the target domain data of the sub-band to obtain the time-constrained target domain data of the sub-band.

7. The dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning according to claim 1, characterized in that, The three-way evidence scores include the FBTCCA path score, the single-cycle migration path score, and the target domain adaptive path score. Among them, after time constraints, the target domain data of each sub-band is subjected to time-weighted canonical correlation analysis with the sine and cosine reference matrix template corresponding to the sub-band. The first canonical correlation coefficient is taken, and the band sign square of the first canonical correlation coefficient is used as the FBTCCA path score of the sub-band. Using the single-cycle cross-subject space filter of each sub-band, the target domain data and source domain single-cycle template of the sub-band are projected onto the single-cycle feature space respectively. The single-cycle correlation coefficient of the projected data is calculated, and the band sign square of the single-cycle correlation coefficient is used as the single-cycle migration path score of the sub-band. Using the target domain adaptive filter of each sub-band, the target domain data of the sub-band and the overall average template of the source domain are projected to the same feature subspace, and the adaptive correlation coefficient of the projected data is calculated. The band sign square of the adaptive correlation coefficient is used as the target domain adaptive path score of the sub-band.

8. The dual-domain, dual-scale cross-subject SSVEP decoding method based on transfer learning according to claim 1, characterized in that, The decoding results are used for brain-controlled device control or human-computer interaction interface display to achieve robust decoding of SSVEP across subjects without training.

9. A dual-domain, dual-scale cross-subject SSVEP decoding device based on transfer learning, characterized in that, include: The data acquisition module is used to acquire source domain SSVEP EEG data and target domain SSVEP EEG data to be decoded. The acquired EEG signals are recorded at relevant electrode sites in the occipital and parietal lobes according to the international 10-10 system. The sub-band data determination module is used to perform multi-sub-band filtering on SSVEP EEG data from the source and target domains respectively to obtain multi-sub-band data from the source domain and multi-sub-band data from the target domain. The multi-sub-band filtering is implemented using a Chebyshev Type I bandpass filter bank, which divides the data into K sub-bands. The Chebyshev Type I bandpass filter bank contains K Chebyshev Type I bandpass filters, where K is a positive integer. The reference matrix template generation module is used to generate corresponding sine and cosine reference matrix templates for all candidate stimulus frequencies of SSVEP decoding. The template pair generation module is used to construct an overall average template and a single-period template of the source domain for each sub-band of the source domain multi-sub-band data, corresponding to each candidate stimulus frequency, to form a unique dual-scale template pair of the source domain for each sub-band. The spatial filter generation module is used to extract the corresponding single-period cross-subject spatial filter for each sub-band's source domain single-period template. The time-constrained target domain data generation module is used to apply local time constraints to the target domain data of each sub-band in the multi-sub-band data of the target domain, so as to obtain time-constrained target domain data for each sub-band. The adaptive filter extraction module is used to extract a target domain adaptive filter for each sub-band and its corresponding candidate stimulus frequency, by combining the target domain data of the sub-band with the sine and cosine reference matrix template. The decoding module is used to calculate the three-way evidence score for each subband based on the corresponding target domain data or time-constrained target domain data, single-cycle cross-subject space filter, target domain adaptive filter, and the source domain dual-scale template pair, and to calculate the total evidence score; combined with the preset weight coefficients of each subband, the total evidence scores of all subbands are weighted and fused to obtain the discrimination score of each candidate stimulus frequency, and the candidate stimulus frequency with the largest discrimination score is selected as the decoding result output.

Images