Automatic epilepsy detection method and system based on electroencephalogram and deep learning

Abstract

The invention discloses an automatic epilepsy detection method and system based on an electroencephalogram and deep learning, and the method comprises the following steps: inputting a preprocessed EEG time sequence signal of the electroencephalogram and a nuclear magnetic resonance MRI voxel image into a double-branch encoder in parallel, and obtaining shared features and specific features; sequentially executing local graph attention and global graph attention operations on each branch to obtain a local convergence vector and a global convergence vector; output is discriminated through MLP, main task loss is calculated, reconstruction and contrast supervision are introduced in parallel after each modal branch is output, end-to-end training of the whole network is achieved, and finally an epilepsy detection prediction result is output. According to the method, the problem of information splitting caused by a traditional later fusion strategy is solved, deep collaboration of EEG time sequence information and MRI space information is realized, and the accuracy of epilepsy detection is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an automated epilepsy detection method and system based on electroencephalography (EEG) and deep learning. Background Technology

[0002] Deep learning, through its multi-layered neural network structure, can automatically extract complex spatiotemporal features from electroencephalogram (EEG) signals. Many deep learning-based epilepsy detection methods have made significant progress in feature extraction and integration. For example, Samee et al. proposed an EEG seizure detection method combining a bidirectional long short-term memory network (BiLSTM), which improves the prediction accuracy of seizures by deeply mining the temporal dependencies of EEG signals. Kaestner et al. proposed an MRI image analysis method based on a three-dimensional convolutional neural network (3D-CNN) to identify epilepsy-related brain regions, improving the accuracy of capturing lesion regions in brain structures. While these methods have made significant progress in improving the accuracy of epilepsy detection, the design of a single module still has certain limitations when dealing with complex seizure patterns and cross-patient variability.

[0003] In the area of ​​multimodal data fusion, Hosseini et al. proposed a multimodal data analysis system for epilepsy EEG and rs-fMRI based on deep learning and edge computing. This approach inputs independently acquired and processed EEG and fMRI data into a Long Short-Term Memory (LSTM) network for late-stage feature fusion to achieve lesion localization and seizure prediction. However, this approach processes independently acquired (asynchronous) EEG and fMRI data. This asynchrony makes it impossible to align neurophysiological activity (EEG) and blood oxygen dynamics response (fMRI) at precisely matched time scales, causing the fusion analysis to remain at a coarse correlation level. It fails to capture the subtle and dynamic causal relationships between the two modalities, limiting the accuracy and reliability of dynamic functional connectivity analysis. Furthermore, this approach employs a typical late-stage fusion strategy, where each modality is first deeply processed and features extracted separately before fusion is performed in the final stage. This strategy ignores the deep interactions and complementarities of different modal data at intermediate feature levels, failing to achieve true cross-modal feature enhancement, resulting in insufficient information fusion and a low upper limit for model performance.

[0004] Furthermore, Shih-Cheng Huang et al. (ICCV 2021) proposed the GLoRIA framework, a multimodal global-local representation learning model for medical images. This technique utilizes radiological reports as supervision signals and learns both global and local representations of images simultaneously through contrastive learning. However, its local alignment relies on text supervision, resulting in insufficient flexibility: its excellent local representation capabilities heavily depend on high-quality text reports. For application scenarios lacking detailed textual annotations (such as those with only keyword tags), its attention mechanism struggles to learn effectively. Moreover, the GLoRIA framework is essentially a dual-encoder (image encoder + text encoder) framework plus contrastive learning, designed for paired data of medical images and their corresponding report texts. It cannot be directly applied to the fusion problem of two medical imaging modalities (such as EEG and MRI), and it does not employ graph neural networks (GNNs) to model the complex topological and interactive relationships within and between modalities. Its representational capabilities are limited by the expressive power of the encoder and attention mechanism. Summary of the Invention

[0005] To overcome the shortcomings and deficiencies of existing technologies, this invention provides an automated epilepsy detection method and system based on electroencephalography (EEG) and deep learning. By combining multiple deep learning modules and different biomedical signals, and through bi-branch feature factorization, local-global two-level graph attention, and multiple supervision mechanisms, this invention achieves deep synergy between EEG temporal details and MRI spatial structures. This allows for a more comprehensive capture of the spatiotemporal characteristics of epileptic seizures and the effective fusion of information from different signal sources, thereby significantly improving the accuracy and robustness of multimodal diagnosis and prediction.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides an automated epilepsy detection method based on electroencephalography (EEG) and deep learning, comprising the following steps:

[0008] Acquire EEG time-series signals and MRI voxel images and perform data preprocessing;

[0009] Temporal vector sequences are extracted from EEG time-series signals, and spatial vector sequences are extracted from nuclear magnetic resonance MRI voxel images.

[0010] The temporal vector sequence and the spatial vector sequence are respectively processed by the same shared encoder to obtain shared EEG features and shared MRI features, and the temporal vector sequence and the spatial vector sequence are respectively processed by their respective unique encoders to obtain unique EEG features and unique MRI features;

[0011] A binary adjacency matrix is ​​constructed based on the nearest time frame and the electrode spatial topology, and an adjacency matrix is ​​constructed based on the spatial block adjacency.

[0012] The binary adjacency matrix is ​​input into the local graph attention network along with shared and unique EEG features to obtain local convergence vectors. Local convergence vector The adjacency matrix is ​​combined with shared MRI features and unique MRI features, respectively, to input a local graph attention network to obtain a local convergence vector. Local convergence vector ;

[0013] Global input node set Vectors are obtained from the shared fully connected adjacency matrix using the Global-GAT operator. sum vector Unique branch node set Vectors are obtained from the unique fully connected adjacency matrix using the Global-GAT operator. ,vector ;

[0014] Based on four vectors Construct a fusion representation, discriminate the output using an MLP, calculate the main task loss, and introduce reconstruction and contrastive supervision in parallel after the output of each modal branch, and globally converge the vectors. The InfoNCE global contrast loss is applied to the global auxiliary vector, and the entire network is trained end-to-end, ultimately outputting the epilepsy detection prediction result.

[0015] As a preferred technical solution, a binary adjacency matrix is ​​constructed based on the nearest time frame and the electrode spatial topology, specifically represented as follows:

[0016] ;

[0017] in, For window size, Indicates temporal adjacency. Indicates airspace adjacency. This represents the set threshold for the Euclidean distance between electrodes.

[0018] As a preferred technical solution, an adjacency matrix is ​​constructed based on spatial block adjacency, specifically as follows:

[0019] Based on the relative positions of each spatial block in the 3D grid, two blocks are considered adjacent if they are adjacent to each other on the voxel grid. The adjacency matrix is ​​represented as follows:

[0020] ;

[0021] in, This represents the adjacency matrix.

[0022] As a preferred technical solution, the binary adjacency matrix is ​​input into a local graph attention network with shared EEG features and unique EEG features respectively to obtain a local convergence vector. Local convergence vector The adjacency matrix is ​​combined with shared MRI features and unique MRI features, respectively, to input a local graph attention network to obtain a local convergence vector. Local convergence vector Specifically, it includes:

[0023] The binary adjacency matrix is ​​input into the local graph attention network with shared EEG features and unique EEG features respectively. For each pair of nodes, attention weights are calculated using linear transformation of the attention vector and the LeakyReLU activation function. After normalization with the neighbors, the nodes are concatenated to obtain a new node representation. The new node representation is then subjected to average or attention-weighted pooling to obtain a local convergence vector.

[0024] As a preferred technical solution, based on four vectors Constructing a fusion representation specifically includes:

[0025] Four vectors As four nodes in the fusion graph, a fusion adjacency matrix is ​​constructed based on principles such as shared↔shared, specific↔specific, and weak connections across branches. The fusion representation is obtained by updating the vectors of the four nodes and concatenating and pooling them using the Fusion-GAT operator.

[0026] As a preferred technical solution, reconstruction and comparison supervision are introduced in parallel after the output of each modal branch, specifically including:

[0027] The shared vector at each position will be decoded. and unique vector decoding After splicing, the result is Input lightweight decoder to generate reconstructed features ;

[0028] With original features Calculate the mean square error reconstruction loss;

[0029] Node representation after local graph attention network Calculate the InfoNCE local contrast loss using the corresponding positive sample auxiliary representation;

[0030] For global convergence vector With global auxiliary vector Calculate the global contrast loss for InfoNCE;

[0031] Calculate CMD alignment loss for shared EEG and shared MRI features;

[0032] Calculate orthogonal differential loss for shared EEG features and unique EEG features.

[0033] This invention also provides an automatic epilepsy detection system based on electroencephalography (EEG) and deep learning, for implementing an automatic epilepsy detection method based on EEG and deep learning, including: a data acquisition module, a data preprocessing module, a spatiotemporal sequence extraction module, a feature extraction module, a binary adjacency matrix construction module, an adjacency matrix construction module, a local feature generation module, a global feature generation module, and a fusion discrimination module;

[0034] The data acquisition module is used to acquire EEG time-series signals and MRI voxel images;

[0035] The data preprocessing is used to preprocess the EEG time-series signals and MRI voxel images.

[0036] The spatiotemporal sequence extraction module is used to extract time-series vector sequences based on EEG time-series signals and to extract spatial vector sequences based on nuclear magnetic resonance (MRI) voxel images.

[0037] The feature extraction module is used to pass the temporal vector sequence and the spatial vector sequence through the same shared encoder to obtain shared EEG features and shared MRI features, and to pass the temporal vector sequence and the spatial vector sequence through their respective unique encoders to obtain unique EEG features and unique MRI features.

[0038] The binary adjacency matrix construction module is used to construct a binary adjacency matrix based on adjacent time frames and electrode spatial topology;

[0039] The adjacency matrix construction module is used to construct an adjacency matrix based on spatial block adjacency;

[0040] The local feature generation module is used to input the binary adjacency matrix and the shared EEG features and the unique EEG features into the local graph attention network to obtain the local convergence vector. Local convergence vector The adjacency matrix is ​​input into the local graph attention network along with shared MRI features and unique MRI features to obtain the local convergence vector. Local convergence vector ;

[0041] The global feature generation module is used to process the global input node set. Vectors are obtained from the shared fully connected adjacency matrix using the Global-GAT operator. sum vector , set of unique branch nodes Vectors are obtained from the unique fully connected adjacency matrix using the Global-GAT operator. ,vector ;

[0042] The fusion discrimination module is used based on four vectors Construct a fusion representation, discriminate the output using an MLP, calculate the main task loss, and introduce reconstruction and contrastive supervision in parallel after the output of each modal branch, and globally converge the vectors. The InfoNCE global contrast loss is applied to the global auxiliary vector, and the entire network is trained end-to-end, ultimately outputting the epilepsy detection prediction result.

[0043] The present invention also provides a computer-readable storage medium storing a program that, when executed by a processor, implements the above-described automatic epilepsy detection method based on electroencephalography and deep learning.

[0044] The present invention also provides a computer device, including a processor and a memory for storing processor-executable programs, wherein when the processor executes the program stored in the memory, it implements the automatic epilepsy detection method based on electroencephalography and deep learning as described above.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] (1) The spatiotemporal information fragmentation and feature coupling problems existing in the current multimodal analysis of EEG and MRI make it difficult to take into account the complementary characteristics of EEG and MRI in time and space. This invention addresses the characteristics of multimodal features that contain both commonalities between modalities and their own specificities. By using a dual-branch coding structure, multimodal features are explicitly decoupled into two subspaces: shared and specific. This effectively decouples and optimizes the two types of information, can separate common biomarkers and isolate modality-specific noise artifacts, significantly enhances the robustness and generalization ability of the model when facing different patient data, and achieves deep coupling and synergy of information.

[0047] (2) Existing Transformer-based fusion methods ignore proximity priors and cannot distinguish between local and global interactions. This invention uses a hierarchical graph attention network to achieve deep modal collaboration and information complementarity. First, a local graph is constructed based on spatial adjacency. The spatiotemporal dependence within a modality is captured through the local graph. Then, cross-modal information interaction is carried out through the global graph to achieve structured priors. This overcomes the information fragmentation problem caused by traditional late-stage fusion strategies and achieves deep collaboration between EEG temporal information and MRI spatial information, thereby improving the accuracy of epilepsy detection.

[0048] (3) Relying solely on the loss of the discrimination task can easily lead to information loss. This invention constructs a multi-supervision mechanism through parallel reconstruction loss, local and global comparison loss, shared alignment loss and orthogonal differentiation loss. Among them, the reconstruction loss ensures that the coding branch can align with the common features and retain the private features, so as not to lose the key information of the original signal during the feature extraction process, thus preserving the original image information to the greatest extent and avoiding information bottlenecks. The orthogonal differentiation loss ensures the effective decoupling of shared and unique features.

[0049] (4) The present invention adopts a dynamic alignment technology based on feature domain. By explicitly modeling the interaction and correlation between cross-modal features in the local and global two-level graph attention network, it realizes the deep fusion of EEG time series signals and MRI spatial structure at the semantic level. It has strong inclusiveness and robustness for multimodal data acquired asynchronously and non-aligned in clinical practice. This allows the present invention to make full use of a large amount of existing historical medical record data acquired asynchronously and relaxes the stringent requirements on future data acquisition equipment.

[0050] (5) In response to the common problem of limited computing resources in medical scenarios, this invention introduces lightweight BERT and lightweight 3D-CNN as the core of feature extraction, and optimizes the node dimension and number of attention heads in the graph attention network to achieve the best balance between model complexity and performance. While maintaining the accuracy of epilepsy detection, it significantly reduces the computing load and storage requirements, and has the ability to run efficiently on edge devices and local hospital servers, providing a practical solution for low-power, real-time deployment in clinical environments.

[0051] (6) The present invention adopts a modular and end-to-end training integrated technical solution, which solves the problems of process redundancy and poor adaptability caused by phased optimization of traditional multimodal models, realizes the unity of high performance and high flexibility, supports independent adjustment of functional modules according to actual data conditions, and significantly improves deployment efficiency and maintainability in real medical environments. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall process of the automatic epilepsy detection method based on electroencephalography and deep learning of the present invention.

[0053] Figure 2 This is a schematic diagram of the overall network structure of the dual-branch feature factorization module of the present invention;

[0054] Figure 3 This is a schematic diagram of the overall network structure of the local graph attention operation of the hierarchical graph attention fusion module of the present invention.

[0055] Figure 4This is a schematic diagram of the overall network structure of the global graph attention operation of the hierarchical graph attention fusion module of the present invention;

[0056] Figure 5 This is a schematic diagram of the overall network structure of the multi-supervision and total loss function module of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0058] Example 1

[0059] like Figure 1 As shown, this embodiment provides an automated epilepsy detection method based on electroencephalography (EEG) and deep learning. The overall process begins with the original signal input and sequentially proceeds through steps such as corresponding feature extractor, dual-branch sharing and unique feature factorization encoding, hierarchical (local and global) graph attention fusion, fusion discrimination and reconstruction under multiple supervision, ultimately outputting a multimodal fusion-based diagnostic or predictive result. Specific steps include:

[0060] S1: The preprocessed EEG time-series signal and MRI voxel image are fed into the feature extractor, where the EEG time-series signal is used to extract time-series vector sequences through a bidirectional recurrent network (BiLSTM) or a lightweight BERT model. MRI voxel images are segmented into several spatial blocks using 3D-CNN or visual Transformer, and spatial vector sequences are extracted. Where T is the number of EEG time frames and P is the number of MRI spatial blocks;

[0061] S2: Feature factorization is performed on the preprocessed EEG and MRI data sequences. For example... Figure 2 As shown, this embodiment constructs two parallel encoding branches: a shared encoder. With unique encoder . Specifically, right and Extracting shared features using the same parameter set , To ensure that different modalities are distributed consistently in the shared subspace; and , Then, EEG and MRI are encoded independently to obtain unique features. and Each modality retains its private information;

[0062] S3: As Figure 3 As shown, for shared feature sequences and and unique sequences and Local graph attention operations are applied separately. Within the shared branch, for each... For each node, there is one node corresponding to each time frame; temporal adjacency is defined as... , Define the window size (e.g., taking 1 or 2 frames before and after); define spatial adjacency as... Based on the geometric layout (2D / 3D coordinates) of the EEG electrodes on the scalp, the Euclidean distance between each pair of electrodes is pre-calculated. To set a distance threshold, a binary adjacency matrix is ​​constructed based on nearby time frames and the electrode spatial topology. :

[0063] ;

[0064] The matrix size is .

[0065] All and Input a Local Graph Attention Network (Local-GAT) and perform a process for each pair of nodes. (Only) Attention weights are calculated using a linear transformation of the attention vector and the LeakyReLU activation function, and then normalized and concatenated with neighbor data to obtain a new node representation. :

[0066] ;

[0067] in This indicates H-head splicing. For the normalized weights, For learnable linear transformations, The activation function outputs a new representation for all nodes. Then, average or attention-weighted pooling is used to obtain the local convergence vector. Similarly, with each For nodes, construct a spatial block adjacency-based system. ,get Based on the relative positions of each block in the 3D grid, two blocks are considered adjacent if they are adjacent to each other on the voxel grid (faces or edges are adjacent). Applying the same multi-head graph attention mechanism as the EEG branch to the input feature sequence and matrix yields new node representations. After pooling and aggregation, the following is obtained In the specific branches, respectively... , Applying the same process, the adjacency matrix and Reuse and Each node is input into the Local Graph Attention Network (Local-GAT) along with its respective modal adjacency matrix. After obtaining the new node representations, the convergence vectors are generated. and The aforementioned local steps effectively capture the fine-grained spatiotemporal dependencies within each mode;

[0068] S4: As Figure 4 As shown, the local convergence vectors of the two modalities in each branch are used as global graph nodes, and a second graph attention is performed to uncover cross-modal and cross-branch relationships. In the shared branch, the global input node set... With shared fully connected adjacency matrix Right now The updated node representation is calculated using the Global-GAT operator through linear transformation attention scoring, LeakyReLU activation function, normalized attention coefficients, and multi-head concatenation. and And thus for each Perform attention-weighted pooling for , Similarly, for the unique branch, the set of unique branch nodes is processed using the Global-GAT operator. With unique fully connected adjacency matrix Input, get , Through the layered design of the two-layer GAT, both physical and spatial priors are preserved, and semantic complementarity between modalities and branches is enhanced.

[0069] S5: This embodiment provides a fusion discrimination strategy, which combines the above four vectors Reconsidering them as four nodes in the fusion graph, a fusion adjacency matrix is ​​constructed based on principles such as shared↔shared, specific↔specific, and weak connections across branches. That is, there are "strong edges" connecting nodes 1 and 2, "strong edges" connecting nodes 3 and 4, and "weak edges" connecting nodes 1 and 2 with nodes 3 and 4 respectively.

[0070] ;

[0071] The fused representation is then obtained by updating the four node vectors using the Fusion-GAT operator and concatenating and pooling them. Finally, the MLP is used to determine the output, which is a sigmoid binary classification value {0 (Non-seizure), 1 (Seizure)}. The cross-entropy loss is then calculated. The main task loss for classification. The fusion of GAT and MLP methods provides a flexible trade-off between computational efficiency and interpretability / controllability.

[0072] S6: To ensure that the encoding and fusion process does not lose original information and to improve feature discrimination capabilities, such as... Figure 5 As shown, this invention introduces reconstruction and contrastive supervision in parallel after the output of each modal branch. Specifically, the shared vector at each position is decoded. and unique vector decoding After splicing, the result is The input is a lightweight decoder D, typically consisting of two fully connected layers, which generates reconstructed features. and with original features Calculate the mean square error reconstruction loss Meanwhile, to enhance the consistency and distinctiveness of local node representations, the node representation after Local-GAT is... Auxiliary representations of corresponding positive samples (e.g., cross-modal text or label vectors) Calculate the InfoNCE local contrast loss ; for the global convergence vector With global auxiliary vector Perform global comparison loss in InfoNCE Furthermore, in shared branch pairs and Apply Central Moment Discrepancy (CMD) alignment loss Constrain the shared feature distribution to be consistent; apply orthogonal differentiation loss to shared and unique features. This ensures that the information from both sources is complementary and not redundant.

[0073] The total loss function in this embodiment is defined as follows:

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] in, , , , , These are the model hyperparameters, which are used to adjust the intensity of cross-modal alignment loss, control the strength of shared / specific decoupling, weight of reconstruction consistency, coefficients of local comparison terms, and coefficients of global comparison terms. Further parameter tuning is needed during the experiment. The main task labels and prediction output; It is an m-th order center distance function; It is the order; For all shared vectors at N positions A matrix assembled from parts; For the corresponding unique vector matrix; It is a unit matrix of the same dimension; It is the Frobenius norm; It is a similarity function; For comparison temperature; This represents the number of local nodes. This refers to the batch size.

[0082] Through the modular design and multiple supervision described above, this invention can achieve comprehensive modeling and efficient fusion of local / global and shared / unique features in EEG and MRI multimodal fusion tasks, significantly improving the accuracy, robustness and interpretability of the model.

[0083] Example 2

[0084] This embodiment provides an automatic epilepsy detection system based on electroencephalography (EEG) and deep learning, used to implement the automatic epilepsy detection method based on EEG and deep learning in Embodiment 1 above. The system includes: a data acquisition module, a data preprocessing module, a spatiotemporal sequence extraction module, a feature extraction module, a binary adjacency matrix construction module, an adjacency matrix construction module, a local feature generation module (Local-GAT), a global feature generation module (Global-GAT), and a fusion discrimination module (Fusion-GAT).

[0085] The data acquisition module is used to acquire EEG time-series signals and MRI voxel images;

[0086] Data preprocessing is used to preprocess the EEG temporal signals and MRI voxel images. The preprocessed EEG temporal data and MRI spatial block data are first input in parallel into a dual-branch encoder to obtain shared and unique features. Subsequently, local graph attention and global graph attention operations are performed sequentially on each branch to produce local convergence vectors and global convergence vectors. Finally, the diagnostic or predictive results are output through a fusion discriminant module. The entire network is trained end-to-end using a single total loss function.

[0087] The spatiotemporal sequence extraction module is used to extract time-series vector sequences based on EEG time-series signals and spatial vector sequences based on nuclear magnetic resonance (MRI) voxel images.

[0088] The feature extraction module is used to pass the temporal vector sequence and spatial vector sequence through the same shared encoder to obtain shared EEG features and shared MRI features, and to pass the temporal vector sequence and spatial vector sequence through their respective unique encoders to obtain unique EEG features and unique MRI features.

[0089] The binary adjacency matrix construction module is used to construct a binary adjacency matrix based on adjacent time frames and electrode spatial topology;

[0090] The adjacency matrix construction module is used to construct adjacency matrices based on spatial block adjacency.

[0091] The local feature generation module is used to input the binary adjacency matrix and shared EEG features and unique EEG features into the local graph attention network to obtain the local convergence vector. Local convergence vector The adjacency matrix is ​​input into the local graph attention network along with shared MRI features and unique MRI features to obtain the local convergence vector. Local convergence vector The local generation module is constructed independently for each branch (Shared / Specific) and each modality (EEG / MRI), mainly used to capture spatiotemporal proximity information. First, for nodes and adjacent parts, for the EEG branch, its shared and specific features are used as a set of nodes, and each node has a dimension d1=128; a sparse adjacency matrix based on temporal adjacency and electrode physical topology is constructed, with the edge weight initially set to 1; for the MRI branch, its shared and specific features are used as a set of nodes, and each node has a dimension d1=128.

[0092] The Graph Attention structure uses two layers of GAT, each with 4 heads and an output dimension of 32; the activation function is LeakyReLU (α=0.2). The first layer receives node features and the adjacency matrix, and outputs an intermediate representation; the second layer applies graph attention to the intermediate representation again to obtain the final node representation.

[0093] Pooling and Output Structure: Average pooling is used to generate local convergence vectors with a dimension of 128 for both modal branches. Four local convergence vectors for EEG and MRI are output in the Shared and Specific branches, respectively.

[0094] The global feature generation module is used to generate the global input node set. Vectors are obtained from the shared fully connected adjacency matrix using the Global-GAT operator. sum vector , set of unique branch nodes Vectors are obtained from the unique fully connected adjacency matrix using the Global-GAT operator. ,vector ;

[0095] The global feature generation module captures macroscopic complementary information within each branch and between different modalities. Two local convergence vectors are used as nodes in the global graph, with 2 nodes and a node dimension of 128. A fully connected adjacency matrix A = ,] is constructed to ensure bidirectional flow of cross-modal information;

[0096] The Graph Attention structure uses a three-layer GAT, with 8 heads per layer and 16 dimensions per head; it also employs LeakyReLU (α=0.2), stacking the three layers to fully exploit cross-modal semantics, and concatenating the outputs of each head between layers. The node representation of the last layer is directly used as the global convergence vector.

[0097] The fusion discrimination module is used to determine the identity based on four vectors. Construct a fusion representation, discriminate the output using an MLP, calculate the main task loss, and introduce reconstruction and contrastive supervision in parallel after the output of each modal branch, and globally converge the vectors. The InfoNCE global contrast loss is applied to the global auxiliary vector, and the entire network is trained end-to-end, ultimately outputting the epilepsy detection prediction result.

[0098] In this embodiment, the fusion discrimination module uses the above four vectors as nodes to construct a fully connected or strongly connected adjacency matrix Afussion based on shared↔shared and specific↔specific. Two layers of GAT (number of heads = 4, head dimension = 64) are applied to this fusion graph to output the fusion node representation. Then, average pooling is used to obtain the fusion representation (dimension 256). After passing through a fully connected layer, ŷ is output.

[0099] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An automatic epilepsy detection method based on electroencephalogram and deep learning, characterized in that, Includes the following steps: Acquire EEG time-series signals and MRI voxel images and perform data preprocessing; Temporal vector sequences are extracted from EEG time-series signals, and spatial vector sequences are extracted from nuclear magnetic resonance MRI voxel images. The temporal vector sequence and the spatial vector sequence are respectively processed by the same shared encoder to obtain shared EEG features and shared MRI features, and the temporal vector sequence and the spatial vector sequence are respectively processed by their respective unique encoders to obtain unique EEG features and unique MRI features; A binary adjacency matrix is ​​constructed based on the nearest time frame and the electrode spatial topology, and an adjacency matrix is ​​constructed based on the spatial block adjacency. The binary adjacency matrix is respectively input into a local graph attention network with shared EEG features and unique EEG features to obtain a local aggregation vector , the local aggregation vector , the adjacency matrix is respectively input into a local graph attention network with shared MRI features and unique MRI features to obtain a local aggregation vector , the local aggregation vector ; Global input node set Vectors are obtained from the shared fully connected adjacency matrix using the Global-GAT operator. sum vector Unique branch node set Vectors are obtained from the unique fully connected adjacency matrix using the Global-GAT operator. ,vector ; Based on four vectors Construct a fusion representation, discriminate the output using an MLP, calculate the main task loss, and introduce reconstruction and contrastive supervision in parallel after the output of each modal branch, and globally converge the vectors. The InfoNCE global contrast loss is applied to the global auxiliary vector, and the entire network is trained end-to-end, ultimately outputting the epilepsy detection prediction result.

2. The automated epilepsy detection method based on electroencephalography and deep learning according to claim 1, characterized in that, A binary adjacency matrix is ​​constructed based on the nearest time frame and the electrode spatial topology, specifically represented as follows: ； in, For window size, Indicates temporal adjacency. Indicates airspace adjacency. This represents the set threshold for the Euclidean distance between electrodes.

3. The automated epilepsy detection method based on electroencephalography and deep learning according to claim 1, characterized in that, The adjacency matrix is ​​constructed based on spatial block adjacency, specifically as follows: Based on the relative positions of each spatial block in the 3D grid, two blocks are considered adjacent if they are adjacent to each other on the voxel grid. The adjacency matrix is ​​represented as follows: ； in, This represents the adjacency matrix.

4. The automated epilepsy detection method based on electroencephalography and deep learning according to claim 1, characterized in that, The binary adjacency matrix is ​​input into the local graph attention network along with shared and unique EEG features to obtain local convergence vectors. Local convergence vector The adjacency matrix is ​​combined with shared MRI features and unique MRI features, respectively, to input a local graph attention network to obtain a local convergence vector. Local convergence vector Specifically, it includes: The binary adjacency matrix is ​​input into the local graph attention network with shared EEG features and unique EEG features respectively. For each pair of nodes, attention weights are calculated using linear transformation of the attention vector and the LeakyReLU activation function. After normalization with the neighbors, the nodes are concatenated to obtain a new node representation. The new node representation is then subjected to average or attention-weighted pooling to obtain a local convergence vector.

5. The automated epilepsy detection method based on electroencephalography and deep learning according to claim 1, characterized in that, Based on four vectors Constructing a fusion representation specifically includes: Four vectors As four nodes in the fusion graph, a fusion adjacency matrix is ​​constructed based on principles such as shared↔shared, specific↔specific, and weak connections across branches. The fusion representation is obtained by updating the vectors of the four nodes and concatenating and pooling them using the Fusion-GAT operator.

6. The automated epilepsy detection method based on electroencephalography and deep learning according to claim 1, characterized in that, Reconstruction and comparison supervision are introduced in parallel after the output of each modal branch, specifically including: The shared vector at each position will be decoded. and unique vector decoding After splicing, the result is Input lightweight decoder to generate reconstructed features ; With original features Calculate the mean square error reconstruction loss; Node representation after local graph attention network Calculate the InfoNCE local contrast loss using the corresponding positive sample auxiliary representation; For global convergence vector With global auxiliary vector Calculate the global contrast loss for InfoNCE; Calculate CMD alignment loss for shared EEG and shared MRI features; Calculate orthogonal differential loss for shared EEG features and unique EEG features.

7. An automated epilepsy detection system based on electroencephalography and deep learning, characterized in that, The automatic epilepsy detection method based on electroencephalography and deep learning according to any one of claims 1-6 includes: a data acquisition module, a data preprocessing module, a spatiotemporal sequence extraction module, a feature extraction module, a binary adjacency matrix construction module, an adjacency matrix construction module, a local feature generation module, a global feature generation module, and a fusion discrimination module. The data acquisition module is used to acquire EEG time-series signals and MRI voxel images; The data preprocessing is used to preprocess the EEG time-series signals and MRI voxel images. The spatiotemporal sequence extraction module is used to extract time-series vector sequences based on EEG time-series signals and to extract spatial vector sequences based on nuclear magnetic resonance (MRI) voxel images. The feature extraction module is used to pass the temporal vector sequence and the spatial vector sequence through the same shared encoder to obtain shared EEG features and shared MRI features, and to pass the temporal vector sequence and the spatial vector sequence through their respective unique encoders to obtain unique EEG features and unique MRI features. The binary adjacency matrix construction module is used to construct a binary adjacency matrix based on adjacent time frames and electrode spatial topology; The adjacency matrix construction module is used to construct an adjacency matrix based on spatial block adjacency; The local feature generation module is used to input the binary adjacency matrix and the shared EEG features and the unique EEG features into the local graph attention network to obtain the local convergence vector. Local convergence vector The adjacency matrix is ​​input into the local graph attention network along with shared MRI features and unique MRI features to obtain the local convergence vector. Local convergence vector ; The global feature generation module is used to process the global input node set. Vectors are obtained from the shared fully connected adjacency matrix using the Global-GAT operator. sum vector , set of unique branch nodes Vectors are obtained from the unique fully connected adjacency matrix using the Global-GAT operator. ,vector ; The fusion discrimination module is used based on four vectors Construct a fusion representation, discriminate the output using an MLP, calculate the main task loss, and introduce reconstruction and contrastive supervision in parallel after the output of each modal branch, and globally converge the vectors. The InfoNCE global contrast loss is applied to the global auxiliary vector, and the entire network is trained end-to-end, ultimately outputting the epilepsy detection prediction result.

8. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the automatic epilepsy detection method based on electroencephalography and deep learning as described in any one of claims 1-6.

9. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the automatic epilepsy detection method based on electroencephalography and deep learning as described in any one of claims 1-6.

Images