Stereo image quality evaluation method based on multi-mode brain visual perception
Through the multimodal brain visual perception method, combined with EEG and visual information, the problem of inconsistent with human subjective evaluation is solved, and efficient and accurate stereoscopic image quality evaluation is achieved.
Patent Information
- Application Number
- CN202411242055.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-20
AI Technical Summary
The existing stereoscopic image quality evaluation methods cannot be highly consistent with human subjective evaluation results, and methods based on convolutional neural networks are prone to problems of insufficient overfitting and generalization capabilities.
Using a stereoscopic image quality evaluation method based on multimodal brain visual perception, the EEG features are extracted through the ventral and dorsal EEG encoder and the image encoder based on ventral and dorsal cognition are extracted, and the adversarial network and flow characteristics are generated in combination with the ventral and dorsal lateral side. Cross attention module, integrating EEG and visual information to generate evaluation results.
It achieves evaluation results that are more consistent with human subjective evaluation results without the need for EEG signals, which improves the accuracy and reliability of stereoscopic image quality evaluation.
Smart Images
Figure CN119168978A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of stereoscopic image quality evaluation, and in particular to a stereoscopic image quality evaluation method based on multimodal brain visual perception. Background Art
[0002] Stereoscopic images have become the cornerstone of the development of technologies such as virtual reality (VR), augmented reality (AR), 3D movies, and medical imaging by providing a more realistic sense of depth and visual experience, and have promoted progress in many fields such as entertainment, medical care, and education. However, stereoscopic images will inevitably suffer from quality degradation during the process of acquisition, storage, compression, and transmission. Problems in any link may lead to a decrease in the quality of stereoscopic images, which in turn causes visual discomfort to users. Efficient stereoscopic image quality assessment (SIQA) methods play a key role in these processes, optimizing the processing flow by providing feedback, ensuring the quality of stereoscopic images, and maintaining the user's visual immersion experience. Therefore, the development of efficient and reliable evaluation algorithms has become a key focus of researchers.
[0003] Generally, SIQA methods are divided into subjective evaluation and objective evaluation. The results of subjective evaluation are considered to be the most accurate reference labels because they directly rely on human visual perception. However, because it relies on manual scoring, the process is cumbersome and time-consuming, making it difficult to promote in large-scale applications. In addition, the additional depth dimension of stereoscopic images may have a negative impact on the viewer's eyes, causing visual discomfort or even fatigue, further increasing the complexity of subjective evaluation. In order to address these limitations of subjective evaluation, researchers are more inclined to develop and apply objective evaluation methods, which can efficiently evaluate the quality of stereoscopic images without human intervention, and therefore have been more widely favored in practical application scenarios.
[0004] At present, the development of objective stereoscopic image quality assessment methods has achieved remarkable results by simulating the human visual system using convolutional neural networks. Unlike traditional planar image quality assessment, modeling binocular vision characteristics in objective SIQA methods is crucial. However, the experimental results of convolutional neural network-based methods rely heavily on training datasets. As the complexity of the model increases, the limited SIQA dataset may lead to problems of overfitting and insufficient generalization. Given the complexity of the human visual system, it is almost impossible to fully simulate the entire cognitive process from visual stimulus reception to stereoscopic image quality perception. Therefore, the SIQA results obtained by modeling will be difficult to fully match human evaluation criteria. In order to achieve an evaluation method consistent with human perception, some studies have attempted to directly introduce and decode information related to visual perception to assist the model in obtaining more accurate results. Among them, electroencephalogram (EEG), as a non-invasive and low-cost technology, has been widely used to capture brain activity. Studies have shown that electroencephalogram (EEG) signals can reveal high-level semantic information and low-level image quality perception by recording neural activity in the cerebral cortex. Some methods map image features to EEG waveforms through reverse engineering to apply to image classification and emotion recognition tasks, thereby simulating the visual and emotional perception process of the human brain. Therefore, this paper proposes to introduce EEG information into the SIQA task, develops a brain-inspired SIQA method, and establishes a brain-visual multimodal SIQA dataset. Experimental results show that the proposed brain-inspired SIQA method performs well in practical applications without EEG signals. Summary of the invention
[0005] The purpose of the present invention is to solve the problem that the evaluation results of the prior art cannot be highly consistent with the human subjective evaluation results when evaluating stereoscopic images, and further proposes a stereoscopic image quality evaluation method based on multimodal brain visual perception.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] A method for evaluating the quality of a stereoscopic image based on multimodal brain visual perception comprises the following steps:
[0008] S1, pre-process the collected EEG signals and send them to the ventral and dorsal EEG encoder to obtain EEG features;
[0009] S2, sending the distorted image to an image encoder based on ventral-dorsal cognition to extract image visual features, wherein the image encoder based on ventral-dorsal cognition is designed according to human visual characteristics;
[0010] S3, taking the extracted EEG features as the true value, generating ventral and dorsal EEG features respectively through the ventral-dorsal generative adversarial network, and applying the quality-aware convolution module to retain the low-level quality features;
[0011] S4. The features of the ventral and dorsal pathways are integrated through the stream feature cross-attention module and sent to the classification layer to obtain the final evaluation result.
[0012] Preferably, the preprocessing workflow in S1 is specifically as follows:
[0013] The acquired EEG signals were downsampled, filtered, baseline processed, artifacts removed, and sliced.
[0014] Preferably, the workflow of the ventral and dorsal EEG encoder in S1 is specifically as follows:
[0015] The EEG signals were divided into ventral and dorsal regions according to the electrode distribution, with 23 ventral EEG channels and 18 dorsal channels;
[0016] The EEG signal passes through the channel attention module, temporal feature extractor, deep attention module and spatial feature extractor in sequence to gradually extract and integrate temporal and spatial information;
[0017] In the temporal feature extractor, multi-scale one-dimensional temporal convolution kernels are used, and the lengths of the convolution kernels are 0.5, 0.25, and 0.125 times the length of the corresponding EEG slices, respectively, to process the temporal context information.
[0018] Preferably, the workflow of the image encoder based on ventral and dorsal cognition in S2 is specifically as follows:
[0019] For the input image, local and abstract semantic information are extracted through shallow and deep convolution blocks respectively to obtain shallow and deep left and right view features;
[0020] The shallow fusion features are obtained by calculating the sum and difference of the shallow left and right view features; while the deep left and right view features are integrated through the stereo cross attention module to obtain the deep fusion features;
[0021] After the shallow and deep features are cascaded in the channel dimension, they are passed to the temporal lobe module and the parietal lobe module respectively; the parietal lobe module uses 1×3 and 3×1 asymmetric convolution kernels to extract spatial position information and obtain the dorsal visual pathway features; while the ventral module serializes the image features and adds classification labels, passes through the self-attention module and performs residual addition, and finally obtains the ventral visual pathway features and classification labels.
[0022] Preferably, the workflow of the ventral-dorsal generative adversarial network in S3 is specifically as follows:
[0023] The ventral generative adversarial network receives ventral visual features and classification labels, generates ventral EEG features, and enhances quality details through the quality-aware convolution module;
[0024] The dorsal generative adversarial network receives dorsal visual features and generates dorsal EEG features. The dorsal generative adversarial network uses a simple upsampling and downsampling structure to transfer spatial information.
[0025] Preferably, the quality-aware convolution module in S3 is used to improve the quality of image features, and its specific workflow is as follows:
[0026] The convolutional feature maps are upsampled to restore spatial details;
[0027] The feature maps are adjusted using quality-conditional batch normalization, which uses modulation parameters (γ and β) from the classification labels to enhance the quality information of the features;
[0028] The adjusted feature map is combined with the original feature map by residual addition, thereby enhancing the consistency and quality perception of the features.
[0029] Preferably, the workflow of the stream feature cross attention module in S4 is specifically as follows:
[0030] The ventral-dorsal EEG features are matrix multiplied in the stream feature cross-attention module to obtain the ventral-dorsal cross-attention matrix;
[0031] Apply the softmax operation to extract the ventral-dorsal attention weight matrix from the cross attention matrix;
[0032] Perform matrix multiplication and element-by-element addition on the weight matrix and the corresponding ventral and dorsal features to obtain ventral and dorsal fusion features respectively;
[0033] The ventral and dorsal fusion features are added element by element to obtain the final fusion feature.
[0034] Compared with the prior art, the present invention provides a stereoscopic image quality evaluation method based on multimodal brain visual perception, which has the following beneficial effects:
[0035] (1) The present invention constructs a ventral-dorsal EEG encoder to extract important information about stereoscopic visual perception contained in the EEG in the ventral and dorsal visual processing pathways; the electroencephalogram (EEG) encoder combining these two pathways can comprehensively extract information from different visual processing information angles and provide richer cognitive features.
[0036] (2) The present invention constructs an image encoder based on ventral-dorsal cognition to obtain spatial and detailed visual information of stereoscopic images by simulating the characteristics of visual cells. It can more accurately simulate the complex perception process of the human visual system and help improve the accuracy of quality assessment of stereoscopic images.
[0037] (3) The ventral-dorsal generative adversarial network projects visual information into the brain-inspired fusion representation to obtain brain-inspired multimodal perceptual features. Finally, the stream feature cross-attention module integrates these perceptual features into brain-inspired multimodal information for SIQA. In this way, in practical applications, no electroencephalogram (EEG) signal is required to obtain evaluation results that are relatively consistent with human subjective evaluation, ensuring that it can be used in technical practice. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 A flowchart of a stereoscopic image quality evaluation method based on multimodal brain visual perception proposed by the present invention;
[0039] Figure 2 This is a structural diagram of an image encoder module based on ventral and dorsal cognition mentioned in Example 1 of the present invention;
[0040] Figure 3 This is a structural diagram of the ventral and dorsal generative adversarial network module mentioned in Example 1 of the present invention. DETAILED DESCRIPTION
[0041] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.
[0042] Embodiment 1:
[0043] See also Figure 1 The present invention provides a method for evaluating the quality of a stereoscopic image based on multimodal brain visual perception, the method comprising the following steps:
[0044] 101: Preprocessing
[0045] The collected electroencephalogram (EEG) signals are subjected to downsampling, filtering, baseline processing, artifact removal and slicing.
[0046] 102: EEG encoder extracts EEG features
[0047] The EEG signals are divided into ventral and dorsal regions according to the electrode distribution, with 23 ventral EEG channels and 18 dorsal channels. The EEG signals are sequentially passed through the channel attention module, temporal feature extractor, deep attention module and spatial feature extractor to gradually extract and integrate temporal and spatial information. In the temporal feature extractor, a multi-scale one-dimensional temporal convolution kernel is used to process rich temporal context information, with the length of the convolution kernel being 0.5, 0.25 and 0.125 times the length of the corresponding EEG slice.
[0048] 103: Image encoder based on ventral-dorsal cognition to extract image visual features
[0049] For the input image, the network extracts local and abstract semantic information through shallow and deep convolution blocks respectively, and obtains shallow and deep left and right view features. Then, by calculating the sum and difference of the shallow left and right view features, the shallow fusion features are obtained; while the deep left and right view features are integrated through the stereo cross attention module to obtain the deep fusion features. After the shallow and deep features are cascaded in the channel dimension, they are passed to the temporal lobe module and the parietal lobe module respectively. The parietal lobe module uses 1×3 and 3×1 asymmetric convolution kernels to extract spatial position information and obtain the dorsal visual pathway features; while the ventral module serializes the image features and adds classification tags, passes through the self-attention module and performs residual addition, and finally obtains the ventral visual pathway features and classification tags.
[0050] 104: Ventral and dorsal generative adversarial networks generate ventral and dorsal EEG features respectively
[0051] The ventral GAN receives ventral visual features and classification labels, generates ventral EEG features, and enhances quality details through the quality-aware convolution module. The dorsal GAN receives dorsal visual features and generates dorsal EEG features. The dorsal GAN uses a simple upsampling and downsampling structure to transfer spatial information.
[0052] 105: Technology Application
[0053] The present invention aims to more accurately reflect the subjective quality perception of the image on the basis of ensuring the accuracy of the evaluation results. As an important direction in visual perception research, stereoscopic image quality evaluation is showing its application potential in many fields. In reality, this method has been applied to a variety of products and services, such as virtual reality and augmented reality applications, by introducing the analysis results of EEG signals to optimize image quality and user experience. Stereoscopic companies can also use this method to improve the quality assessment capabilities of their products to ensure the quality of stereoscopic images and thus improve the user experience. Stereoscopic image quality evaluation is a key technology in the current field of digital images, and its research results have important scientific significance and application value.
[0054] Embodiment 2:
[0055] See also Figure 2-3 , based on Example 1 but different in that the solution in Example 1 is further introduced below in combination with specific calculation formulas and example data, as described below:
[0056] 201: Preprocessing
[0057] (1) Use the EEGlab tool in MATLAB to load the collected raw EEG data.
[0058] (2) Since the original EEG signal is mixed with a lot of high-frequency environmental noise, the original electroencephalogram (EEG) signal must be passed through a bandpass filter to eliminate noise interference.
[0059] (3) Independent component analysis (ICA) is used to remove artifact signals from the original electroencephalogram (EEG) signal.
[0060] (4) Electrode selection: remove eye electrode data and remove electrode data with interference.
[0061] 202: EEG encoder extracts EEG features
[0062] The electroencephalogram (EEG) signal is divided into the ventral region and the dorsal region according to the electrode distribution, where the number of ventral EEG channels is 23 and the number of dorsal channels is 18. The EEG signal passes through the channel attention module, the temporal feature extractor, the deep attention module and the spatial feature extractor in sequence to gradually extract and integrate temporal and spatial information. In the temporal feature extractor, a multi-scale one-dimensional temporal convolution kernel is used to process rich temporal context information, with the length of the convolution kernel being 0.5, 0.25 and 0.125 times the length of the corresponding EEG slice.
[0063] 203: Image encoder based on ventral-dorsal cognition to extract image visual features
[0064] Inspired by the dual-stream hypothesis, the present invention proposes an image encoder based on ventral-dorsal cognition (e.g. Figure 2 First, in order to simulate the binocular information fusion mechanism in the occipital lobe of the brain, feature interaction is carried out between the shallow and deep layers of the encoder network. Specifically, the shallow visual features containing specific information and It is obtained through two convolutional blocks. Subsequently, three residual blocks are used to extract deep features. and To obtain abstract semantic information. Shallow and deep features are input into the shallow binocular fusion module and the deep binocular fusion module respectively. In the shallow binocular fusion module, the sum and difference of the left and right shallow features are calculated, and the obtained features are then concatenated into shallow binocular features. In the deep binocular fusion module, the stereo cross attention module is used to integrate the deep features. and Emphasize the importance of parallax through the feature correlation of the horizontal line. The process can be expressed as:
[0065]
[0066] in, is the shallow feature map, is the deep feature map, H, W and C i Represent the height, width and number of channels of the feature map respectively.
[0067] Secondly, in order to simulate two-stream stereo information processing, the shallow feature map extracted from the occipital lobe and deep feature maps It is forwarded to the temporal lobe module to obtain abstract semantic information and passed to the parietal lobe module to capture spatial location information. In the temporal lobe module, these two feature maps are processed simultaneously. The shallow and deep features are processed by the serialization module, and then through the hierarchical self-attention, the final linear mapping generates the ventral visual feature F I-ventral In the serialization module, multiple layers of features are concatenated and reshaped along the channel dimension, ensuring that each embedding contains information from different levels. The layer-based self-attention strategy aims to capture the hierarchical dependencies between multiple layers of visual information and highlight significant structural details to obtain quality-related features. Specifically, and Concatenate along the channel dimension to generate I c , and then reshaped into an embedding sequence Where N = H × W represents the total number of embeddings and the embedding length is C1 + C2. Learnable class label embedding T CLS is added to the beginning of the embedding sequence. The modified sequence T' is fed into the LSA module and combined with skip connections to alleviate gradient vanishing. The process can be expressed as:
[0068]
[0069] Similarly, the parietal module aims to simulate the information flow of the dorsal visual pathway, which is often characterized by its direction selectivity. Therefore, asymmetric convolution kernels of 1×3 and 3×1 are used to simulate this feature. Finally, the classes are labeled Ventral visual feature F I-ventral and dorsal visual feature F I-dorsal It will be used as input to the multimodal generative adversarial network.
[0070] 204: Ventral and dorsal generative adversarial networks generate ventral and dorsal EEG features respectively
[0071] In order to utilize information from different visual areas, this paper proposes a multimodal generative adversarial network architecture, including a ventral generative adversarial network, a dorsal generative adversarial network and a stream feature cross-attention module, such as Figure 3 This architecture enables the generative network to reconstruct features across different visual areas and integrate them into realistic brain-inspired information, thereby emphasizing the visual properties that are most relevant to these EEG features.
[0072] The ventral-dorsal generative adversarial network integrates two different networks, one responsible for fusing semantic visual perception information and the other responsible for generating brain-inspired spatial localization information. The ventral generative adversarial network receives the visual features F I-ventral and class markers As the EEG features To ensure that quality-related information is focused on, the present invention introduces a quality-related convolution module to emphasize low-level features (such as shape and texture) that may be lost during the information transfer process of the generator. In the quality-related convolution module, the reshaped intermediate features are transferred through the upsampling module to form richer spatial details. Subsequently, quality regularization is applied to improve the quality of the features. Finally, these features are integrated with the overall features through residual connections.
[0073] The backside generative adversarial network receives the main features F I-dorsal As the dorsal EEG features Considering the spatial localization visual information of the dorsal pathway, a simple up-sampling and down-sampling structure is applied to effectively extract features while reducing the computational complexity.
[0074] Generated dorsal EEG features is used as the input of the stream feature cross-attention module. First, matrix multiplication is performed on the linearly mapped and transposed features to generate the ventral-dorsal cross-attention matrix. By calculating the cross-attention matrix, we can capture the spatial and temporal correlations between the ventral and dorsal features. Subsequently, a softmax operation is applied to obtain the dorsal attention weight matrix W d→v and the ventral attention weight matrix W v→d The dorsal fusion feature is recorded as By inputting features and W v→d The matrix multiplication is obtained and then combined with the feature Perform element-by-element addition. Similarly, the ventral fusion feature can be obtained Finally, the fusion features Through and It is calculated by adding each element. The calculation formula is:
[0075]
[0076] in, represents element-by-element addition, Represents matrix multiplication.
[0077] Embodiment 3:
[0078] Based on Examples 1-2, but with the difference that the feasibility of the solutions in Examples 1 and 2 is verified in combination with specific experiments, as described below:
[0079] The experiment designed in this embodiment tests the performance based on the constructed brain-visual multi-modality (BVMM) stereo image quality library, which contains electroencephalogram (EEG) data of 6 subjects (4 males and 2 females). The data records the electroencephalogram (EEG) signals of four distortion levels when the subjects watch 812 different distortion types. The EEG signals are collected at 1000 Hz through 64 EEG electrodes, including 8 sessions, each session contains 101 or 103 trials, so each subject has a total of 812 EEG trials.
[0080] In order to more comprehensively measure the classification effect of the method, three evaluation indicators including accuracy, F1 score and Kappa value are used. The evaluation indicators used in the present invention can be calculated using the following formula:
[0081]
[0082]
[0083] Among them, in the classification task, TP (True Positive) represents the number of positive samples correctly identified as positive samples, FP (False Positive) represents the number of negative samples incorrectly identified as positive samples, TN (True Negative) represents the number of negative samples correctly identified as negative samples, and FN (False Negative) represents the number of positive samples incorrectly identified as negative samples. p0 represents the actual classification accuracy, p e Indicates the classification accuracy of the model under random conditions, such as p for a binary classification problem e It is 50%.
[0084] In order to verify the performance of the present invention, this embodiment selects five publicly available stereoscopic image quality evaluation methods for comparison, which are methods from Fang, Yan, Shen, Bourbia, and Messai2022. The results are shown in Tables 2 and 3.
[0085] Table 2 compares the overall performance of different methods on the BVMM stereo image quality evaluation library.
[0086]
[0087] Table 3 Performance comparison of symmetric and asymmetric distortion on the BVMM stereo image quality evaluation library
[0088]
[0089] The present invention compares and analyzes five single-modality stereoscopic image quality evaluation methods. The results are shown in Table 2. The method proposed by the present invention can achieve the best results in all test groups. Table 3 shows the comparison results of symmetric and asymmetric distortion. It is obvious from the content of Table 3 that the method proposed by the present invention can still achieve the best results in all test groups. This further proves the effectiveness of EEG information integration.
[0090] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for evaluating the quality of a stereoscopic image based on multimodal brain visual perception, characterized in that: The following steps are involved: S1, pre-process the collected EEG signals and send them to the ventral and dorsal EEG encoder to obtain EEG features; S2, sending the distorted image to an image encoder based on ventral-dorsal cognition to extract image visual features, wherein the image encoder based on ventral-dorsal cognition is designed according to human visual characteristics; S3, taking the extracted EEG features as the true value, generating ventral and dorsal EEG features respectively through the ventral-dorsal generative adversarial network, and applying the quality-aware convolution module to retain the low-level quality features; S4. The features of the ventral and dorsal pathways are integrated through the stream feature cross-attention module and sent to the classification layer to obtain the final evaluation result.
2. The method for evaluating the quality of a stereoscopic image based on multimodal brain visual perception according to claim 1, characterized in that: The specific workflow of the preprocessing described in S1 is: The acquired EEG signals were downsampled, filtered, baseline processed, artifacts removed, and sliced.
3. The method for evaluating the quality of a stereoscopic image based on multimodal brain visual perception according to claim 1, characterized in that: The workflow of the ventral and dorsal EEG encoder described in S1 is as follows: The EEG signals were divided into ventral and dorsal regions according to the electrode distribution, with 23 ventral EEG channels and 18 dorsal channels; The EEG signal passes through the channel attention module, temporal feature extractor, deep attention module and spatial feature extractor in sequence to gradually extract and integrate temporal and spatial information; In the temporal feature extractor, multi-scale one-dimensional temporal convolution kernels are used, and the lengths of the convolution kernels are 0.5, 0.25, and 0.125 times the length of the corresponding EEG slices, respectively, to process the temporal context information.
4. The method for evaluating stereoscopic image quality based on multimodal brain visual perception according to claim 1, characterized in that: The workflow of the image encoder based on ventral-dorsal cognition described in S2 is specifically as follows: For the input image, local and abstract semantic information are extracted through shallow and deep convolution blocks respectively to obtain shallow and deep left and right view features; The shallow fusion features are obtained by calculating the sum and difference of the shallow left and right view features; while the deep left and right view features are integrated through the stereo cross attention module to obtain the deep fusion features; After the shallow and deep features are cascaded in the channel dimension, they are transmitted to the temporal lobe module and the parietal lobe module respectively; the parietal lobe module uses 1×3 and 3×1 asymmetric convolution kernels to extract spatial position information and obtain the dorsal visual pathway features; The ventral module serializes the image features and adds classification labels, passes through the self-attention module and adds residuals to finally obtain the ventral visual pathway features and classification labels.
5. The method for evaluating stereoscopic image quality based on multimodal brain visual perception according to claim 1, characterized in that: The workflow of the ventral-dorsal generative adversarial network described in S3 is as follows: The ventral generative adversarial network receives ventral visual features and classification labels, generates ventral EEG features, and enhances quality details through the quality-aware convolution module; The dorsal generative adversarial network receives dorsal visual features and generates dorsal EEG features. The dorsal generative adversarial network uses a simple upsampling and downsampling structure to transfer spatial information.
6. The method for evaluating the quality of a stereoscopic image based on multimodal brain visual perception according to claim 1, characterized in that: The quality-aware convolution module described in S3 is used to improve the quality of image features. Its specific workflow is as follows: The convolutional feature maps are upsampled to restore spatial details; The feature maps are adjusted using quality-conditional batch normalization, which uses modulation parameters from classification labels to enhance the quality information of the features; The adjusted feature map is combined with the original feature map by residual addition, thereby enhancing the consistency and quality perception of the features.
7. The method for evaluating stereoscopic image quality based on multimodal brain visual perception according to claim 1, characterized in that: The workflow of the stream feature cross attention module described in S4 is as follows: The ventral-dorsal EEG features are matrix multiplied in the stream feature cross-attention module to obtain the ventral-dorsal cross-attention matrix; Apply the softmax operation to extract the ventral-dorsal attention weight matrix from the cross attention matrix; Perform matrix multiplication and element-by-element addition on the weight matrix and the corresponding ventral and dorsal features to obtain ventral and dorsal fusion features respectively; The ventral and dorsal fusion features are added element by element to obtain the final fusion feature.