Electroencephalogram signal general decoding method and device based on double-scale convolution and cross-branch attention interaction

Abstract

This application provides a general decoding method and apparatus for EEG signals based on dual-scale convolution and cross-branch attention interaction, relating to the field of EEG signal recognition technology. The method includes: inputting a shared spatiotemporal token sequence obtained after converting the original EEG trial signal into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch; performing intra-branch attention refinement on the obtained fine-scale and coarse-scale temporal features respectively; and using an attention mechanism to perform bidirectional information exchange on the refined fine-scale and coarse-scale features; performing adaptive token aggregation on the obtained interactive fine-scale and coarse-scale features respectively; and concatenating the generated fine-scale compact representation and coarse-scale compact representation before inputting them into a classifier to obtain the EEG decoding prediction result. Thus, under a unified architecture, through the clear division of labor between convolution for multi-scale modeling and attention for scale interaction, compact and efficient decoding is achieved in independent scenarios for subjects across tasks.

Description

Technical Field

[0001] This application relates to the field of EEG signal recognition technology, and in particular to a universal decoding method and apparatus for EEG signals based on dual-scale convolution and cross-branch attention interaction. Background Technology

[0002] Electroencephalography (EEG) signals are widely used in brain-computer interfaces, mental state recognition, and auxiliary assessment of neurological diseases due to their non-invasiveness and high temporal resolution. With the expansion of application scenarios, EEG decoding models not only need to handle single tasks but also maintain stable decoding performance under different tasks, different subjects, and different channel configurations. Therefore, constructing a universal decoding architecture that can adapt to multiple EEG tasks under independent subject conditions has become an important research direction in this field.

[0003] However, the solutions in related technologies achieve explicit collaborative interaction between features at multiple time scales under a unified and compact architecture, but the attention mechanism is overburdened, making it difficult to balance the model's generalization ability and computational efficiency in cross-task and independent subject scenarios.

[0004] Therefore, there is an urgent need to provide a new general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction, so as to achieve multi-scale collaborative modeling, explicit scale interaction, compact representation aggregation and strong generalization ability under a unified architecture. Summary of the Invention

[0005] The purpose of this application is to provide a general decoding method and device for EEG signals based on dual-scale convolution and cross-branch attention interaction. Under a unified architecture, convolution is responsible for multi-scale modeling and attention is dedicated to scale interaction, which clearly divides the work and achieves high-precision, high-generalization and compact and efficient decoding in independent scenarios of subjects across tasks.

[0006] This application provides a general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction, including: The raw EEG trial signals are acquired and converted into a shared spatiotemporal token sequence. This shared spatiotemporal token sequence is simultaneously input into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features representing local short-term details and coarse-scale temporal features representing long-term context, respectively. Intra-branch attention refinement is performed on the fine-scale and coarse-scale temporal features to obtain refined fine-scale and refined coarse-scale features. A bidirectional information exchange mechanism is then used between the refined fine-scale and refined coarse-scale features to obtain interactive fine-scale and interactive coarse-scale features. Adaptive token aggregation is then performed on the interactive fine-scale and interactive coarse-scale features to generate a fine-scale compact representation and a coarse-scale compact representation. The concatenated fine-scale and coarse-scale compact representations are then input into a classifier to obtain the EEG decoding prediction result.

[0007] Optionally, converting the original EEG trial signal into a shared spatiotemporal token sequence includes: performing temporal convolution and spatial convolution on the original EEG trial signal sequentially to obtain a shared spatiotemporal feature map; compressing the shared spatiotemporal feature map spatially and rearranging it into a token sequence form; projecting it onto a preset embedding dimension via linear mapping; and superimposing positional encoding to obtain the shared spatiotemporal token sequence.

[0008] Optionally, the intra-branch attention refinement is implemented through a multi-head self-attention mechanism and a feedforward network; the intra-branch attention refinement of the fine-scale temporal features and the coarse-scale temporal features respectively to obtain refined fine-scale features and refined coarse-scale features includes: performing multi-head self-attention calculation, residual connection, layer normalization, feedforward network processing, residual connection and layer normalization processing on the input of the target branch in sequence to obtain the target scale features output by the target branch; wherein, the target branch is either the fine-scale temporal convolution branch or the coarse-scale temporal convolution branch.

[0009] Optionally, the step of using an attention mechanism to perform bidirectional information exchange between the refined fine-scale features and the refined coarse-scale features to obtain interactive fine-scale features includes: using the refined fine-scale features as queries and the refined coarse-scale features as keys and values, performing multi-head cross-attention calculation with learnable interaction strength parameters to enable the fine-scale branches to absorb coarse-scale context information, and then processing the results through a feedforward network and layer normalization to obtain the interactive fine-scale features.

[0010] Optionally, the step of using an attention mechanism to perform bidirectional information exchange between the refined fine-scale features and the refined coarse-scale features to obtain interactive coarse-scale features includes: using the refined coarse-scale features as a query and the refined fine-scale features as keys and values, performing multi-head cross-attention calculation with learnable interaction strength parameters to enable the coarse-scale branch to absorb local detail information of the fine scale, and then processing it through a feedforward network and layer normalization to obtain the interactive coarse-scale features.

[0011] Optionally, performing adaptive token aggregation on the fine-scale interaction features and the coarse-scale interaction features to generate fine-scale compact representations and coarse-scale compact representations includes: for each token in the interaction features of any target branch, calculating an attention score using a learnable projection vector, and normalizing the attention score of each token to obtain the aggregation weight corresponding to each token; and performing a weighted summation of all tokens in the target branch according to the aggregation weight to obtain a compact representation of the target branch.

[0012] This application also provides a universal decoding device for EEG signals based on dual-scale convolution and cross-branch attention interaction, comprising: A shared spatiotemporal tokenization module is used to acquire the original EEG trial signals and convert them into a shared spatiotemporal token sequence. A dual-scale temporal convolution module is used to simultaneously input the shared spatiotemporal token sequence into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features representing local short-term details and coarse-scale temporal features representing long-term context, respectively. An intra-branch attention refinement module is used to perform intra-branch attention refinement on the fine-scale temporal features and the coarse-scale temporal features to obtain refined fine-scale features and refined coarse-scale features. An inter-branch attention interaction module is used to perform bidirectional information exchange on the refined fine-scale features and the refined coarse-scale features using an attention mechanism to obtain interactive fine-scale features and interactive coarse-scale features. An adaptive token aggregation module is used to perform adaptive token aggregation on the interactive fine-scale features and the interactive coarse-scale features to generate a fine-scale compact representation and a coarse-scale compact representation. A prediction module is used to concatenate the fine-scale compact representation and the coarse-scale compact representation and input them into a classifier to obtain the EEG decoding prediction result.

[0013] Optionally, the shared spatiotemporal tokenization module is specifically used to sequentially perform temporal and spatial convolution processing on the original EEG trial signals to obtain a shared spatiotemporal feature map; the shared spatiotemporal tokenization module is further used to spatially compress the shared spatiotemporal feature map and rearrange it into a token sequence, project it onto a preset embedding dimension via linear mapping, and then superimpose positional encoding to obtain the shared spatiotemporal token sequence. Optionally, the dual-scale temporal convolution module is specifically used to sequentially perform multi-head self-attention calculation, residual connection, layer normalization, feedforward network processing, residual connection and layer normalization processing on the input of the target branch to obtain the target scale features output by the target branch; wherein, the target branch is either the fine-scale temporal convolution branch or the coarse-scale temporal convolution branch.

[0014] Optionally, the inter-branch attention interaction module is specifically used to use the refined fine-scale features as queries and the refined coarse-scale features as keys and values, and through multi-head cross-attention calculation with learnable interaction strength parameters, enable the fine-scale branches to absorb coarse-scale context information, and obtain the interaction fine-scale features after processing by a feedforward network and layer normalization.

[0015] Optionally, the inter-branch attention interaction module is specifically used to use the refined coarse-scale features as queries and the refined fine-scale features as keys and values, and through multi-head cross-attention calculation with learnable interaction strength parameters, enable the coarse-scale branches to absorb fine-scale local detail information, and obtain the interaction coarse-scale features after processing by a feedforward network and layer normalization.

[0016] Optionally, the adaptive token aggregation module is specifically used to calculate the attention score for each token in the interaction features of any target branch using a learnable projection vector, and to normalize the attention score of each token to obtain the aggregation weight corresponding to each token; the adaptive token aggregation module is also specifically used to perform a weighted summation of all tokens in the target branch according to the aggregation weight to obtain a compact representation of the target branch.

[0017] This application also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction as described above.

[0018] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the above-described general decoding methods for EEG signals based on dual-scale convolution and cross-branch attention interaction.

[0019] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction as described above.

[0020] The general decoding method and apparatus for EEG signals based on dual-scale convolution and cross-branch attention interaction provided in this application first acquires the original EEG trial signal and converts it into a shared spatiotemporal token sequence. Then, the shared spatiotemporal token sequence is simultaneously input into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features representing local short-term details and coarse-scale temporal features representing long-term context, respectively. Intra-branch attention refinement is performed on the fine-scale and coarse-scale temporal features to obtain refined fine-scale and refined coarse-scale features. A bidirectional information exchange is then performed on the refined fine-scale and refined coarse-scale features using an attention mechanism to obtain interactive fine-scale and interactive coarse-scale features. Finally, adaptive token aggregation is performed on the interactive fine-scale and interactive coarse-scale features to generate a fine-scale compact representation and a coarse-scale compact representation. The fine-scale compact representation and the coarse-scale compact representation are concatenated and input into a classifier to obtain the EEG decoding prediction result. In this way, under a unified architecture, the clear division of labor between convolution for multi-scale modeling and attention for scale interaction enables high precision, high generalization, and compact and efficient decoding in independent scenarios for subjects across tasks. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is the overall architecture diagram of the general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction provided in this application; Figure 2 This is a flowchart illustrating the general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction provided in this application. Figure 3This is a diagram of the dual-scale temporal convolution branch structure provided in this application; Figure 4 This is a schematic diagram of the structure of the universal EEG signal decoding device based on dual-scale convolution and cross-branch attention interaction provided in this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. All actions involving the acquisition of signal information or data in this application are performed in accordance with the relevant data protection laws and policies of the country where the application is located and with authorization from the owner of the relevant device.

[0025] In the pursuit of versatility and high performance, EEG decoding solutions in related technologies still generally suffer from the following technical shortcomings: 1. Existing multi-scale modeling schemes mostly adopt a paradigm of first extracting features independently and then fusing them at the end, lacking the ability to model the dynamic collaboration and explicit interaction between features at different time scales. The models have difficulty in effectively determining whether to focus on short-term local patterns or long-term contextual dependencies in a specific EEG state, which limits the accuracy of heterogeneous temporal dynamics representation.

[0026] 2. While some methods incorporating attention mechanisms enhance the ability to capture global dependencies, they often cause the attention unit to bear an excessive burden of multiple responsibilities, such as long-range modeling, feature fusion, and relationship enhancement. This design leads to a bloated model structure, increased computational overhead, and, given the limited amount of EEG data and significant individual differences, the excessive attention burden exacerbates the optimization difficulty and weakens the model's generalization robustness under cross-task and subject-independent conditions.

[0027] 3. Existing solutions typically require targeted adjustments to the model structure or architecture-related hyperparameters when facing different types of EEG tasks. For example, the discriminant feature distributions differ significantly between tasks such as motor imagery, mental illness assessment, and cognitive load monitoring. If the decoding model needs to frequently change its structure as the task switches, it will severely limit its standardization and practical deployment value.

[0028] 4. Existing high-performance solutions struggle to balance accuracy, efficiency, and compactness. Lightweight models lack sufficient expressive power for complex heterogeneous tasks, while complex hybrid models face a significant increase in parameter and computational costs. Maintaining high decoding accuracy while avoiding the introduction of heavy classification heads or deep attention stacking remains a pressing issue in current technology.

[0029] This application provides a general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction. This method converts raw EEG signals into a unified representation through a shared spatiotemporal tokenization unit, and then extracts complementary temporal features through fine-scale and coarse-scale branches with different temporal receptive fields. Based on this, an intra-branch attention refinement unit enhances important responses within each scale, and an inter-branch attention interaction unit explicitly achieves information exchange between different time scales. Finally, an adaptive token aggregation unit outputs a more compact feature representation while maintaining discriminative ability, thereby reducing the number of parameters and computational overhead. The technical solution in this application overcomes the problems of insufficient multi-scale collaboration, excessive attention burden, and high model complexity in existing technologies by structurally dividing the responsibilities of different units. The convolution unit primarily undertakes multi-scale temporal modeling, the attention unit primarily undertakes intra-scale refinement and inter-scale interaction, and the aggregation unit primarily undertakes compact representation generation.

[0030] like Figure 1 The diagram shows the overall architecture of a general EEG signal decoding method based on dual-scale convolution and cross-branch attention interaction provided in this application. This method takes the original EEG trial signal as input. First, it uses a shared spatiotemporal tokenization unit to extract shared spatiotemporal representations and organizes them into a token sequence embedded in a unified space. Then, the shared tokens are simultaneously input into fine-scale and coarse-scale temporal convolution branches to model local short-term details and longer-term contexts, respectively. Next, intra-branch attention refinement is performed on the outputs of the two branches to enhance more discriminative token relationships within their respective scales. Based on this, bidirectional information exchange is performed between the two scale branches through an inter-branch attention interaction unit. Finally, adaptive token aggregation is performed on the outputs of the two branches, and the aggregated dual-branch features are concatenated and input into a classifier to obtain the final prediction result.

[0031] From the perspective of the overall information flow, this solution forms a complete pipeline of "shared representation construction—dual-scale temporal modeling—intra-scale refinement—inter-scale interaction—compact aggregation classification". This pipeline is not simply a superposition of multiple existing units, but is organically designed around the goal of "universal EEG decoding": the shared front end ensures a unified representation foundation, the dual-scale convolutional branches provide receptive fields at multiple temporal scales, the two types of attention within and between branches serve different functions, and the aggregation unit controls the output dimension and overall complexity.

[0032] The following description, in conjunction with the accompanying drawings, details the general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction provided in this application, through specific embodiments and application scenarios.

[0033] like Figure 2 As shown in the embodiment of this application, a general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction is provided. This method may include the following steps 201 to 204: Step 201: Obtain the original EEG trial signal and convert the original EEG trial signal into a shared spatiotemporal token sequence.

[0034] For example, the above-mentioned original EEG test signal is multi-channel EEG time series data after preprocessing such as bandpass filtering, and its form is usually a two-dimensional matrix of the number of channels multiplied by the time sampling points.

[0035] Specifically, step 201 above, which involves converting the original EEG trial signal into a shared spatiotemporal token sequence, may further include steps 201a1 and 201a2: Step 201a1: Perform temporal convolution and spatial convolution on the original EEG test signals in sequence to obtain a shared spatiotemporal feature map.

[0036] For example, the aforementioned temporal convolutional layer is used to extract local dynamic change patterns along the time axis, and the spatial convolutional layer (or depthwise separable spatial convolution) is used to fuse spatial topological information between different electrode channels. Through this shared front-end processing, the original signal is compressed and encoded into a discriminative low-level spatiotemporal feature map.

[0037] Step 201a2: Compress the shared spatiotemporal feature map spatially and rearrange it into a token sequence. After linear mapping and projection onto a preset embedding dimension, position encoding is superimposed to obtain the shared spatiotemporal token sequence.

[0038] For example, for the feature map described above, a compression operation (such as global average pooling or spatial downsampling achieved through convolution stride) is performed along the spatial dimension to obtain a one-dimensional feature sequence distributed along the time axis. Subsequently, this one-dimensional feature sequence is rearranged according to the order of time steps to form an initial token sequence. In this process, the feature vector corresponding to each time step constitutes an initial token, representing an abstract snapshot of the global spatial information at that moment.

[0039] For example, the preset embedding dimension is the number of feature channels that all subsequent modules of the model work uniformly. If the output token feature dimension is consistent with this embedding dimension, the linear mapping can be regarded as an identity mapping; otherwise, dimension alignment is achieved through a learnable fully connected layer. In order to preserve the crucial temporal sequence relationship in the token sequence, a set of learnable positional encoding vectors are superimposed on the projected token sequence. The final output shared spatiotemporal token sequence serves as the common input basis for both the fine-scale and coarse-scale branches.

[0040] For example, the step of converting the original EEG trial signal into a shared spatiotemporal token sequence described above can be represented by the following formula: ; ; in, C For the number of EEG channels, T This represents the number of sampling points; Represents a shared spatiotemporal tokenized mapping. Indicates the output channel dimension. This represents the number of tokens formed after time-domain downsampling. Subsequently, the feature map... Perform spatial dimension compression and rearrange the data into a token sequence: ; To ensure that subsequent units operate under a unified embedding dimension, a linear mapping is then used. Project it into the embedding space of dimension d: ; Among them, when and When they are equal, It can be an identity mapping; otherwise, This can be achieved using a learnable linear layer. To preserve the order information of the tokens, the projected tokens are scaled and superimposed with learnable positional codes to obtain a shared token sequence: ; in, This represents the location encoding function. Through the above steps, the raw EEG signal is converted into a compact, uniform, and shared spatiotemporal token representation suitable for subsequent two-branch processing.

[0041] It should be noted that the reason for adopting the above-mentioned shared tokenization process instead of independent front-end encoding for each branch is that, on the one hand, both time-scale branches are built on the same unified representation, which is conducive to subsequent explicit inter-scale information interaction; on the other hand, the shared front-end can significantly reduce parameter redundancy and repetitive feature extraction calculations, thereby improving the overall model's compactness and training efficiency.

[0042] Step 202: Input the shared spatiotemporal token sequence into both the fine-scale temporal convolution branch and the coarse-scale temporal convolution branch to extract fine-scale temporal features that represent local short-term details and coarse-scale temporal features that represent long-term context, respectively.

[0043] It should be noted that the fine-scale temporal convolution branches and the coarse-scale temporal convolution branches have the same topology, but the convolution kernel size configuration parameters are not exactly the same.

[0044] For example, to adapt to the characteristic that discriminative information is distributed across different time scales in different EEG tasks, this embodiment of the application sets up two parallel temporal convolution branches: a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch. The two branches share the same input. However, different convolutional kernel sizes are used to create different temporal receptive fields. Among them, the fine-scale branch emphasizes local, short-term, and rapidly changing temporal patterns, while the coarse-scale branch emphasizes dynamic dependencies within a longer temporal context and a wider time window.

[0045] For example, both the fine-scale temporal convolution branch and the coarse-scale temporal convolution branch contain multiple cascaded temporal convolutional blocks, and the temporal scale of the convolutional kernel in the fine-scale temporal convolution branch is smaller than the temporal scale of the convolutional kernel in the coarse-scale temporal convolution branch; the temporal convolutional block includes: a deep temporal convolutional layer for channel-independent convolution along the temporal dimension, a grouped pointwise transformation layer for channel expansion and channel projection, and a residual connection structure; the residual connection structure includes: a learnable scaling factor for controlling the update intensity of the current convolutional block output to the branch state.

[0046] For example, before entering the convolution process, the shared token sequence is first converted into channel-first form: ; In such Figure 3In the dual-scale temporal convolution branch structure diagram shown, for any branch b∈{f,c} (f is the fine-scale temporal convolution branch, c is the coarse-scale temporal convolution branch, and each branch contains multiple cascaded lightweight temporal convolutional blocks), its th... The output of a lightweight temporal convolutional block can be written as: ; in, DW This represents a depthwise temporal convolution performed independently per channel, with a kernel size of [size missing]. ; and These represent the point-by-point transformations used for channel expansion and channel projection, respectively; represents the GELU activation function; BN represents the batch normalization operation. The advantage of this design is that time dependencies are mainly achieved through compact convolutions, rather than stacking a large number of attention layers, thereby reducing model complexity.

[0047] For example, to enhance optimization stability and preserve information from previous layers, each convolutional block employs a residual connection with a learnable scaling factor, updated as follows: in, This is a learnable scalar used to control the strength of the impact of the current convolutional block output on the branch state update. After several stacked layers, the outputs of the two branches are reconverted into token form: ; ; in, and These represent the output token sequences of the fine-scale temporal convolution branch and the coarse-scale temporal convolution branch, respectively. and These are the final results output by the convolutional layers for the fine-scale temporal convolutional branch and the coarse-scale temporal convolutional branch, respectively.

[0048] It should be noted that the lightweight temporal convolutional blocks described above are, in order: depthwise temporal convolution, grouped pointwise expand, grouped pointwise project, BN, and residuals and learnable coefficients.

[0049] In one possible implementation, such as Figure 3As shown, the fine-scale temporal convolution branch and the coarse-scale temporal convolution branch further re-inject the shared spatiotemporal token sequence into their respective branch outputs through learnable residual coefficients, in order to retain common basic information and enhance the specialization capability for different receptive fields. That is: ; ; Here, α1 and α2 are learnable scalars.

[0050] Step 203: Perform intra-branch attention refinement on the fine-scale time features and the coarse-scale time features respectively to obtain refined fine-scale features and refined coarse-scale features. Then, use the attention mechanism to perform bidirectional information exchange on the refined fine-scale features and the refined coarse-scale features to obtain interactive fine-scale features and interactive coarse-scale features.

[0051] The intra-branch attention refinement is used to enhance the discriminative token relationships within each scale; the bidirectional information exchange is used to enable fine-scale branches to integrate coarse-scale contextual information, and to enable coarse-scale branches to integrate fine-scale local detail information.

[0052] For example, step 203 above includes two stages: first, performing intra-scale feature refinement, and then performing explicit interactions between scales. After the dual-scale temporal convolution, this scheme does not directly perform fusion, but instead performs intra-branch attention refinement on each of the two scale branches. The purpose is not to replace the aforementioned temporal convolution modeling, but to further highlight the most important token relationships within each scale based on the already formed feature representations at each scale, thereby improving the discriminativeness of the intra-scale representation.

[0053] Specifically, step 203 above may also include the following step 203a: Step 203a: Perform multi-head self-attention calculation, residual connection, layer normalization, feedforward network processing, residual connection and layer normalization processing on the input of the target branch in sequence to obtain the target scale features output by the target branch.

[0054] The target branch is either the fine-scale temporal convolution branch or the coarse-scale temporal convolution branch.

[0055] For example, regarding the intra-branch attention refinement step, intra-branch attention refinement is used to enhance the discriminative token relationships within each scale. This stage is implemented through a multi-head self-attention mechanism and a feedforward network. Taking a fine-scale branch as an example, the specific refinement process is as follows: First, the obtained fine-scale temporal features are used as input to the target branch. Multi-head self-attention computation is then performed on the input features sequentially: that is, query vector, key vector, and value vector are generated using the input features themselves, the internal correlation weights between tokens are calculated, and the value vectors are weighted and aggregated according to the weights, thereby highlighting more critical moments or patterns in the sequence.

[0056] Subsequently, the output of the multi-head self-attention system is residually connected to the original input and then subjected to layer normalization to stabilize the feature distribution. Next, the normalized features are fed into a feedforward network consisting of two fully connected layers (with a non-linear activation function in between) to further enhance the non-linear expressive power of the features. Finally, the output of the feedforward network is again subjected to residual connection and layer normalization to obtain refined fine-scale features. The coarse-scale branch follows the exact same processing flow to obtain refined coarse-scale features.

[0057] It's important to note that the intra-branch attention mechanism here plays a "refining" role in the architecture, rather than a "core modeling" role. In many existing Transformer-like solutions, the attention module typically needs to handle multiple tasks simultaneously, such as long-range dependency capture and feature fusion, leading to an excessive optimization burden. In this embodiment, however, the basic temporal dependency extraction is already accomplished by a lightweight deep temporal convolution, and the attention mechanism only enhances the discriminative power of the formed features and further strengthens the relationships between key tokens within its own scale. This clear division of labor effectively reduces the training difficulty and computational redundancy of the model.

[0058] For example, for any branch , No. The attention layer within each branch can be represented as: ; ; in, This indicates the bulls' self-attention. Indicates the belt Nonlinear two-layer feedforward network Representation layer normalization. This structure allows the model to adaptively reinforce more important token dependencies within a single timescale. For example, in fine-scale branches, key connections between short-term local responses are more likely to be highlighted; while in coarse-scale branches, contextual organization across a wider time window is more likely to be emphasized.

[0059] For example, after completing the intra-scale refinement, this scheme further introduces an inter-branch attention interaction unit to explicitly realize bidirectional information exchange between fine-scale and coarse-scale features. This step is the key difference between this scheme and many schemes that "simplely fuse after multi-scale parallel extraction".

[0060] Specifically, step 203 above, the step of obtaining the interactive fine-scale features, may further include the following step 203b1: Step 203b1: Using the refined fine-scale features as the query and the refined coarse-scale features as the key and value, the fine-scale branch absorbs the coarse-scale context information through multi-head cross-attention calculation with learnable interaction strength parameters, and after processing by a feedforward network and layer normalization, the interactive fine-scale features are obtained.

[0061] Specifically, step 203 above, the step of obtaining the interactive coarse-scale features, may further include the following step 203b2: Step 203b2: Using the refined coarse-scale features as the query and the refined fine-scale features as the key and value, the coarse-scale branch absorbs local detail information of the fine-scale branch through multi-head cross-attention calculation with learnable interaction strength parameters, and after processing by feedforward network and layer normalization, the interactive coarse-scale features are obtained.

[0062] For example, given two refined branch features and This scheme employs a symmetrical bidirectional cross-attention mechanism. The fine-scale branch absorbs broader contextual information provided by the coarse-scale branch, and its interaction process can be represented as follows: ; ; The coarse-scale branch absorbs more local and finer-grained discriminative information provided by the fine-scale branch, and their interaction process can be represented as follows: ; ; in, and This is a learnable interaction intensity parameter. Through this bidirectional cross-attention design, fine-scale branches no longer rely solely on local short-term patterns but can selectively integrate broader temporal contexts; coarse-scale branches no longer retain only long-term overviews but can absorb discriminative cues from local details.

[0063] It should be noted that this approach does not simply concatenate information from two scales, but rather explicitly learns "what information one scale should borrow from the other scale." This enables the model to not only possess multi-scale features, but also to truly learn multi-scale collaboration.

[0064] Step 204: Perform adaptive token aggregation on the interaction fine-scale features and the interaction coarse-scale features respectively to generate fine-scale compact representation and coarse-scale compact representation. Concatenate the fine-scale compact representation and the coarse-scale compact representation and input them into the classifier to obtain the EEG decoding prediction result.

[0065] For example, after obtaining the dual-branch features after interaction and Subsequently, this scheme does not use simple average pooling, nor does it directly flatten all tokens and send them into the heavy classification head. Instead, it introduces an adaptive token aggregation unit to assign different weights to different tokens in order to obtain a more compact and discriminative branch-level representation.

[0066] Specifically, step 204 above, the step of performing adaptive token aggregation, may further include the following steps 204a1 and 204a2: Step 204a1: For each token in the interaction features of any target branch, calculate the attention score using the learnable projection vector, and normalize the attention score of each token to obtain the aggregate weight corresponding to each token.

[0067] Step 204a2: Perform a weighted summation of all tokens within the target branch according to the aggregation weight to obtain a compact representation of the target branch.

[0068] For example, for any branch Let the first branch be... Each token is represented as This scheme first uses learnable projection vectors Calculate the attention score for each token: ; Then, the corresponding weights are obtained through softmax normalization: ; After obtaining the corresponding weights, a weighted sum is performed on all tokens according to their weights to obtain the aggregate representation of this branch: ; in, Each can represent a compact representation of fine-scale branches. Compact representation of coarse-scale branches Finally, combine the two: The input is a linear classifier to obtain the predicted output: ; in, and These represent the classification layer weights and biases, respectively.

[0069] For example, compared to directly flattening all tokens, this aggregation strategy has two distinct advantages: first, it can highlight tokens that are truly important for classification through weight learning; second, it significantly reduces the feature dimension entering the classifier, avoiding the formation of an overly heavy classification head and maintaining the compactness of the overall model.

[0070] In summary, the overall execution flow of the general EEG signal decoding method based on dual-scale convolution and cross-branch attention interaction provided in this application is as follows: Step S1: Acquire the raw EEG test signal and input it into the shared spatiotemporal tokenization unit.

[0071] Step S2: Obtain the shared spatiotemporal feature map through two layers of temporal convolution, one cross-channel depth spatial convolution, and two temporal downsampling.

[0072] Step S3: Perform compression, transpose, linear mapping, and positional encoding on the feature map to obtain a shared token sequence.

[0073] Step S4: Input the shared token sequence into both the fine-scale and coarse-scale temporal convolution branches simultaneously, and perform multi-layer lightweight temporal convolution processing on each branch to obtain... and .

[0074] Step S5: Inject the shared token into both branches as a residual to preserve common underlying information and enhance branch specialization.

[0075] Step S6: Perform intra-branch attention refinement on the two branches respectively to obtain the enhanced feature representations within the scale.

[0076] Step S7: Perform bidirectional inter-branch attention interaction on the two branches to obtain the dual-scale features after interaction. and .

[0077] Step S8: For each and Perform adaptive token aggregation to obtain a compact branch representation. and .

[0078] Step S9: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] and splicing as joint features h The data is input into a classifier and the final prediction result is output. .

[0079] Compared with the technical solutions in related technologies, the general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction provided in this application has at least the following effects.

[0080] 1. This solution utilizes a complete, division-of-labor structure—"shared tokenization + dual-scale temporal convolution + intra-branch refinement + inter-branch interaction + adaptive aggregation"—to effectively extract, enhance, and coordinate features at different time scales. This enables the model to not only "see multiple scales" but also to truly selectively organize and integrate information at different scales. Therefore, this solution is better suited to situations where discrimination information is distributed across different time ranges in EEG tasks.

[0081] 2. Under strict independent evaluation by participants, this approach achieved the best ACC and weighted F1-score on ten publicly available datasets across five types of EEG decoding tasks. This demonstrates that the approach is not only applicable to motor imagery tasks but also to various types of EEG decoding tasks, such as mental illness recognition, neurodegenerative disease recognition, mental workload recognition, and cognitive attention recognition, exhibiting strong cross-task generalization ability. DSAINet maintained the best ACC and weighted F1-score on all ten datasets.

[0082] 3. The proposed solution demonstrates significant performance improvements on multiple representative datasets. For example, on the Shin2018 dataset, compared to the suboptimal method, the proposed solution improves ACC by 2.46% and weighted F1-score by 2.63%; on the ADFTD dataset, ACC improves by 1.91% and weighted F1-score by 1.80%; and significant improvements are also achieved on challenging motion visualization datasets such as BCIC-IV-2a and Zhou2016. This indicates that the proposed solution is not only effective on a few tasks, but maintains a stable advantage under different task attributes, data sizes, and channel configurations.

[0083] 4. This scheme balances accuracy and efficiency. While maintaining excellent performance, this scheme only requires about 77K trainable parameters; at the same time, in the accuracy-efficiency analysis on PhysioNet-MI, this scheme achieves the highest ACC while significantly reducing the number of parameters and computational cost compared to Deformer, demonstrating a good balance between accuracy and efficiency.

[0084] 5. Ablation experiments show that each key unit in this scheme contributes positively to the final performance. Removing positional encoding, switching to a single branch, removing intra-branch attention, removing inter-branch attention, and removing adaptive token aggregation all resulted in a decrease in the model's ACC on representative tasks. Taking the three representative tasks BCIC-IV-2a, BCIC-IV-2b, and ADFTD as examples, the complete model achieved ACCs of 61.79±0.68, 76.41±0.31, and 59.65±0.55, respectively, while removing the corresponding units all led to varying degrees of performance degradation. This indicates that the improvement in this scheme stems from the overall structural synergy, rather than relying on a single isolated unit.

[0085] 6. This solution also possesses a certain degree of interpretability. This solution further provides cross-task saliency maps and attention pattern visualizations, demonstrating that the spatial attention regions and attention patterns learned by this solution are task-related and do not rely on a fixed bias for classification. Instead, they can adaptively perform intra-branch refinement and cross-branch interaction based on the temporal organization characteristics of different EEG tasks.

[0086] The general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction provided in this application first acquires the original EEG trial signal and converts it into a shared spatiotemporal token sequence. Then, the shared spatiotemporal token sequence is simultaneously input into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features representing local short-term details and coarse-scale temporal features representing long-term context, respectively. Intra-branch attention refinement is performed on the fine-scale and coarse-scale temporal features to obtain refined fine-scale and refined coarse-scale features. A bidirectional information exchange is then performed on the refined fine-scale and refined coarse-scale features using an attention mechanism to obtain interactive fine-scale and interactive coarse-scale features. Finally, adaptive token aggregation is performed on the interactive fine-scale and interactive coarse-scale features to generate a fine-scale compact representation and a coarse-scale compact representation. The fine-scale compact representation and the coarse-scale compact representation are concatenated and input into a classifier to obtain the EEG decoding prediction result. In this way, under a unified architecture, the clear division of labor between convolution for multi-scale modeling and attention for scale interaction enables high precision, high generalization, and compact and efficient decoding in independent scenarios for subjects across tasks.

[0087] It should be noted that the general EEG signal decoding method based on dual-scale convolution and cross-branch attention interaction provided in this application embodiment can be executed by a general EEG signal decoding device based on dual-scale convolution and cross-branch attention interaction, or by a control module within that device for executing the general EEG signal decoding method based on dual-scale convolution and cross-branch attention interaction. This application embodiment uses the execution of the general EEG signal decoding method based on dual-scale convolution and cross-branch attention interaction by a general EEG signal decoding device based on dual-scale convolution and cross-branch attention interaction as an example to illustrate the general EEG signal decoding device based on dual-scale convolution and cross-branch attention interaction provided in this application embodiment.

[0088] It should be noted that, in the embodiments of this application, the accompanying drawings of the various methods described above illustrate the general decoding method of EEG signals based on dual-scale convolution and cross-branch attention interaction, all of which are exemplified by referring to one of the accompanying drawings in the embodiments of this application. In specific implementation, the general decoding method of EEG signals based on dual-scale convolution and cross-branch attention interaction shown in the accompanying drawings of the various methods described above can also be implemented in conjunction with any other accompanying drawings that can be combined as illustrated in the above embodiments, which will not be elaborated here.

[0089] The following describes the universal EEG signal decoding device based on dual-scale convolution and cross-branch attention interaction provided in this application. The description below can be referred to in correspondence with the universal EEG signal decoding method based on dual-scale convolution and cross-branch attention interaction described above.

[0090] Figure 4 A schematic diagram of the structure of the universal EEG signal decoding device based on dual-scale convolution and cross-branch attention interaction provided in the embodiments of this application is shown below. Figure 4 As shown, it specifically includes: The shared spatiotemporal tokenization module 401 is used to acquire the original EEG trial signal and convert it into a shared spatiotemporal token sequence; the dual-scale temporal convolution module 402 is used to simultaneously input the shared spatiotemporal token sequence into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch, respectively extracting fine-scale temporal features representing local short-term details and coarse-scale temporal features representing long-term context; the intra-branch attention refinement module 403 is used to perform intra-branch attention refinement on the fine-scale temporal features and the coarse-scale temporal features respectively to obtain refined fine-scale features. The system comprises: fine-scale features and refined coarse-scale features; an inter-branch attention interaction module 404, used to perform bidirectional information exchange between the refined fine-scale features and the refined coarse-scale features using an attention mechanism to obtain interactive fine-scale features and interactive coarse-scale features; an adaptive token aggregation module 405, used to perform adaptive token aggregation on the interactive fine-scale features and the interactive coarse-scale features respectively to generate fine-scale compact representations and coarse-scale compact representations; and a prediction module 406, used to concatenate the fine-scale compact representations and the coarse-scale compact representations and input them into the classifier to obtain the EEG decoding prediction result.

[0091] Optionally, the shared spatiotemporal tokenization module 401 is specifically used to sequentially perform temporal convolution and spatial convolution processing on the original EEG trial signal to obtain a shared spatiotemporal feature map; the shared spatiotemporal tokenization module 401 is also specifically used to spatially compress the shared spatiotemporal feature map and rearrange it into a token sequence, project it onto a preset embedding dimension through linear mapping, and then superimpose position encoding to obtain the shared spatiotemporal token sequence.

[0092] Optionally, the dual-scale temporal convolution module 402 is specifically used to sequentially perform multi-head self-attention calculation, residual connection, layer normalization, feedforward network processing, residual connection and layer normalization processing on the input of the target branch to obtain the target scale features output by the target branch; wherein, the target branch is either the fine-scale temporal convolution branch or the coarse-scale temporal convolution branch.

[0093] Optionally, the inter-branch attention interaction module 404 is specifically used to use the refined fine-scale features as queries and the refined coarse-scale features as keys and values, and through multi-head cross-attention calculation with learnable interaction strength parameters, enable the fine-scale branches to absorb coarse-scale context information, and obtain the interaction fine-scale features after processing by a feedforward network and layer normalization.

[0094] Optionally, the inter-branch attention interaction module 404 is specifically used to use the refined coarse-scale features as queries and the refined fine-scale features as keys and values, and through multi-head cross-attention calculation with learnable interaction strength parameters, enable the coarse-scale branches to absorb fine-scale local detail information, and obtain the interaction coarse-scale features after processing by a feedforward network and layer normalization.

[0095] Optionally, the adaptive token aggregation module 405 is specifically used to calculate the attention score for each token in the interaction features of any target branch using a learnable projection vector, and to normalize the attention score of each token to obtain the aggregation weight corresponding to each token; the adaptive token aggregation module 405 is further used to perform a weighted summation of all tokens in the target branch according to the aggregation weight to obtain a compact representation of the target branch.

[0096] The general EEG signal decoding device based on dual-scale convolution and cross-branch attention interaction provided in this application first acquires the original EEG trial signal and converts it into a shared spatiotemporal token sequence. Then, the shared spatiotemporal token sequence is simultaneously input into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features representing local short-term details and coarse-scale temporal features representing long-term context, respectively. Intra-branch attention refinement is performed on the fine-scale and coarse-scale temporal features to obtain refined fine-scale and refined coarse-scale features. A bidirectional information exchange is then performed on the refined fine-scale and refined coarse-scale features using an attention mechanism to obtain interactive fine-scale and interactive coarse-scale features. Finally, adaptive token aggregation is performed on the interactive fine-scale and interactive coarse-scale features to generate a fine-scale compact representation and a coarse-scale compact representation. The fine-scale compact representation and the coarse-scale compact representation are concatenated and input into a classifier to obtain the EEG decoding prediction result. In this way, under a unified architecture, the clear division of labor between convolution for multi-scale modeling and attention for scale interaction enables high precision, high generalization, and compact and efficient decoding in independent scenarios for subjects across tasks.

[0097] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logic instructions in the memory 530 to execute a general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction. This method includes: first, acquiring raw EEG trial signals and converting them into a shared spatiotemporal token sequence; then, simultaneously inputting the shared spatiotemporal token sequence into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features representing local short-term details and coarse-scale temporal features representing long-term context, respectively; performing intra-branch attention refinement on the fine-scale and coarse-scale temporal features to obtain refined fine-scale features and refined coarse-scale features, and using an attention mechanism to perform bidirectional information exchange on the refined fine-scale and refined coarse-scale features to obtain interactive fine-scale features and interactive coarse-scale features; finally, performing adaptive token aggregation on the interactive fine-scale and interactive coarse-scale features to generate a fine-scale compact representation and a coarse-scale compact representation, concatenating the fine-scale compact representation and the coarse-scale compact representation, and inputting the concatenated fine-scale compact representation into a classifier to obtain the EEG decoding prediction result. In this way, under a unified architecture, the clear division of labor between convolution for multi-scale modeling and attention for scale interaction enables high precision, high generalization, and compact and efficient decoding in independent scenarios for subjects across tasks.

[0098] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, this application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the general decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction provided by the above methods. The method includes: first, acquiring raw EEG trial signals and converting the raw EEG trial signals into a shared spatiotemporal token sequence; then, simultaneously inputting the shared spatiotemporal token sequence into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal tokens representing local short-term details, respectively. The system employs a combination of fine-scale temporal features and coarse-scale temporal features representing long-term context. Intra-branch attention refinement is performed on both the fine-scale and coarse-scale temporal features to obtain refined fine-scale and coarse-scale features. A bidirectional information exchange mechanism is then used between these refined fine-scale and coarse-scale features to obtain interactive fine-scale and coarse-scale features. Finally, adaptive token aggregation is performed on both interactive fine-scale and coarse-scale features to generate compact fine-scale and coarse-scale representations. These compact fine-scale and coarse-scale representations are concatenated and input into a classifier to obtain the EEG decoding prediction result. Thus, under a unified architecture, a clear division of labor—with convolution handling multi-scale modeling and attention dedicated to scale interaction—achieves high-precision, high-generalization, and compact and efficient decoding across independent scenarios involving subjects in various tasks.

[0100] Furthermore, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned general decoding methods for EEG signals based on dual-scale convolution and cross-branch attention interaction. The method includes: first, acquiring raw EEG trial signals and converting the raw EEG trial signals into a shared spatiotemporal token sequence; then, simultaneously inputting the shared spatiotemporal token sequence into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features representing local short-term details and coarse-scale features representing long-term context, respectively. The system first identifies time features at each scale. Then, it performs intra-branch attention refinement on both the fine-scale and coarse-scale time features to obtain refined fine-scale and coarse-scale features. A bidirectional information exchange mechanism is then used between these refined fine-scale and coarse-scale features to obtain interactive fine-scale and coarse-scale features. Finally, adaptive token aggregation is performed on both interactive fine-scale and coarse-scale features to generate compact fine-scale and coarse-scale representations. These compact fine-scale and coarse-scale representations are concatenated and input into a classifier to obtain the EEG decoding prediction result. Thus, under a unified architecture, a clear division of labor—with convolution handling multi-scale modeling and attention dedicated to scale interaction—achieves high accuracy, high generalization, and compact, efficient decoding across independent scenarios involving subjects in various tasks.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A universal decoding method for EEG signals based on dual-scale convolution and cross-branch attention interaction, characterized in that, include: The raw EEG trial signals are acquired and converted into a shared spatiotemporal token sequence. The shared spatiotemporal token sequence is simultaneously input into a fine-scale temporal convolution branch and a coarse-scale temporal convolution branch to extract fine-scale temporal features that characterize local short-term details and coarse-scale temporal features that characterize long-term context, respectively. Intra-branch attention refinement is performed on the fine-scale temporal features and the coarse-scale temporal features respectively to obtain refined fine-scale features and refined coarse-scale features. Then, bidirectional information exchange is performed on the refined fine-scale features and the refined coarse-scale features using an attention mechanism to obtain interactive fine-scale features and interactive coarse-scale features. Adaptive token aggregation is performed on the fine-scale interaction features and the coarse-scale interaction features respectively to generate a fine-scale compact representation and a coarse-scale compact representation. The fine-scale compact representation and the coarse-scale compact representation are concatenated and input into the classifier to obtain the EEG decoding prediction result.

2. The method according to claim 1, characterized in that, The intra-branch attention refinement is used to enhance the discriminative token relationships within each scale; the bidirectional information exchange is used to enable fine-scale branches to integrate coarse-scale contextual information, and to enable coarse-scale branches to integrate fine-scale local detail information.

3. The method according to claim 1, characterized in that, The step of converting the original EEG trial signal into a shared spatiotemporal token sequence includes: The original EEG test signals were sequentially processed by temporal convolution and spatial convolution to obtain a shared spatiotemporal feature map. The shared spatiotemporal feature map is spatially compressed and rearranged into a token sequence. After being linearly mapped and projected onto a preset embedding dimension, positional encoding is superimposed to obtain the shared spatiotemporal token sequence.

4. The method according to claim 1, characterized in that, Both the fine-scale temporal convolution branch and the coarse-scale temporal convolution branch contain multiple cascaded temporal convolutional blocks, and the temporal scale of the convolutional kernel in the fine-scale temporal convolution branch is smaller than that in the coarse-scale temporal convolution branch; the temporal convolutional block includes: a deep temporal convolutional layer for channel-independent convolution along the temporal dimension, a grouped pointwise transformation layer for channel expansion and channel projection, and a residual connection structure; the residual connection structure includes: a learnable scaling factor for controlling the update intensity of the current convolutional block output to the branch state.

5. The method according to claim 4, characterized in that, The fine-scale temporal convolution branch and the coarse-scale temporal convolution branch also re-inject the shared spatiotemporal token sequence into their respective branch outputs through learnable residual coefficients, so as to retain common basic information and enhance the specialization ability for different receptive fields.

6. The method according to claim 1, characterized in that, The intra-branch attention refinement is achieved through a multi-head self-attention mechanism and a feedforward network; The step of performing intra-branch attention refinement on the fine-scale temporal features and the coarse-scale temporal features respectively to obtain refined fine-scale features and refined coarse-scale features includes: The input of the target branch is sequentially processed by multi-head self-attention calculation, residual connection, layer normalization, feedforward network processing, residual connection and layer normalization to obtain the target scale features output by the target branch; The target branch is either the fine-scale temporal convolution branch or the coarse-scale temporal convolution branch.

7. The method according to claim 1, characterized in that, The step of using an attention mechanism to perform bidirectional information exchange between the refined fine-scale features and the refined coarse-scale features to obtain interactive fine-scale features includes: Using the refined fine-scale features as queries and the refined coarse-scale features as keys and values, the fine-scale branches absorb coarse-scale contextual information through multi-head cross-attention computation with learnable interaction strength parameters, and are then processed by a feedforward network and layer normalization to obtain the interactive fine-scale features.

8. The method according to claim 1, characterized in that, The step of using an attention mechanism to perform bidirectional information exchange between the refined fine-scale features and the refined coarse-scale features to obtain interactive coarse-scale features includes: Using the refined coarse-scale features as queries and the refined fine-scale features as keys and values, the coarse-scale branches absorb local detail information from the fine-scale branches through multi-head cross-attention computation with learnable interaction strength parameters. After processing by a feedforward network and layer normalization, the interactive coarse-scale features are obtained.

9. The method according to claim 1, characterized in that, The step of performing adaptive token aggregation on the interaction fine-scale features and the interaction coarse-scale features respectively to generate fine-scale compact representations and coarse-scale compact representations includes: For each token in the interaction features of any target branch, the attention score is calculated using the learnable projection vector, and the attention score of each token is normalized to obtain the aggregate weight corresponding to each token. The target branch is obtained by weighted summation of all tokens within the target branch based on the aggregation weight.

10. A universal decoding device for EEG signals based on dual-scale convolution and cross-branch attention interaction, characterized in that, The device includes: A shared spatiotemporal tokenization module is used to acquire raw EEG trial signals and convert the raw EEG trial signals into a shared spatiotemporal token sequence. The dual-scale temporal convolution module is used to simultaneously input the shared spatiotemporal token sequence into the fine-scale temporal convolution branch and the coarse-scale temporal convolution branch, respectively extracting fine-scale temporal features that represent local short-term details and coarse-scale temporal features that represent long-term context. The intra-branch attention refinement module is used to perform intra-branch attention refinement on the fine-scale temporal features and the coarse-scale temporal features respectively, to obtain refined fine-scale features and refined coarse-scale features; The inter-branch attention interaction module is used to perform bidirectional information exchange between the refined fine-scale features and the refined coarse-scale features using an attention mechanism to obtain interactive fine-scale features and interactive coarse-scale features. An adaptive token aggregation module is used to perform adaptive token aggregation on the fine-scale interaction features and the coarse-scale interaction features respectively, to generate a fine-scale compact representation and a coarse-scale compact representation. The prediction module is used to concatenate the fine-scale compact representation and the coarse-scale compact representation and input them into the classifier to obtain the EEG decoding prediction result.

Images