Brain-computer interface method and system based on multi-modal biological signals

Abstract

The brain-computer interface method and system based on multi-modal biological signals can efficiently read brain signals, realize adaptive brain-computer interaction, and bring more efficient and accurate diagnosis and treatment solutions to the medical field. The method includes: (1) collecting multi-modal biological signals; (2) processing and analyzing signals; (3) multi-treatment mode selection and feedback.

Description

Technical Field

[0001] This invention relates to the field of biomedical technology, and more particularly to a brain-computer interface method based on multimodal biological signals, and a brain-computer interface system based on multimodal biological signals. Background Technology

[0002] Brain-computer interface (BCI) is a direct communication connection established between the brain and external devices. It collects bioelectrical signals generated by the brain, such as electroencephalograms (EEGs), and converts them into commands that can be understood by a computer, thereby enabling the brain to control external devices or for external devices to monitor and provide feedback on the brain's state. In the field of BCI technology, existing biosignal acquisition methods are mostly limited to a single modality, exhibiting significant shortcomings and failing to comprehensively and accurately reflect the physiological and psychological states of the brain and body. Taking single EEG acquisition as an example, due to its weak signal, it is highly susceptible to interference from complex external electromagnetic environments, leading to data deviations and failing to stably and reliably present the true electrical activity state of the brain. While electrocardiogram (ECG) signal analysis can reflect the body's physiological condition to some extent, it has significant limitations in accurately capturing complex emotional changes and their close correlation with neural activity, failing to clearly reveal the deep-seated neurophysiological mechanisms behind emotional fluctuations. Micro-expression recognition, as an important window into psychological activity, currently suffers from a lack of effective integration with other physiological signals. This prevents a comprehensive and in-depth assessment of the rich psychological information carried by micro-expressions within the context of overall physiological state, resulting in an insufficient interpretation of an individual's psychological state. Synchronous decoding of multimodal biosignals can significantly improve the performance of brain-computer interfaces. The information carried by different modalities complements each other; through precise synchronous acquisition and decoding, the accuracy and comprehensiveness of signal analysis can be significantly improved. Simultaneously, multimodal biosignal decoding offers the possibility of personalized and precise treatment. By integrating multimodal information such as EEG, ECG, and micro-expressions, it is possible to more accurately assess a patient's condition and develop targeted treatment plans.

[0003] Brain-computer interface technology has been gradually applied to scenarios such as brain function monitoring, neuromodulation, damage repair, and prosthetic control.

[0004] Currently, there is a Chinese patent CN202410554538.0 for neurorehabilitation treatment based on brain-computer interface technology, entitled "A functional electrical stimulation system control method, device, equipment, and storage medium". This invention decodes the patient's motor intentions through electroencephalogram (EEG) signals and outputs stimulation to the patient's limbs in a closed-loop control manner to assist in the limb rehabilitation of stroke or paralyzed patients.

[0005] Currently, there is a Chinese patent CN202411965261.7 for brain-computer interface therapy of nervous system diseases based on non-invasive neuromodulation, entitled "Multi-channel signal acquisition closed-loop ultrasound stimulation system and method based on fuzzy PID control". This scheme acquires EEG and electromyography (EMG) signals and performs transcranial ultrasound stimulation.

[0006] In the field of biosignal acquisition, EEG acquisition technology is mature, collecting brain electrical activity through scalp electrodes. Currently, there is a Chinese patent CN202510309096, entitled "Method and System for Grading and Early Warning of Depressive Mood State Based on Brain-Computer Interface Technology," which proposes a brain-computer interface system and method based on EEG to determine emotions. However, this solution is only for the auxiliary diagnosis of depression and does not involve treatment, and it only uses EEG as the sole signal source.

[0007] Electrocardiogram (ECG) signal acquisition can reflect cardiac electrical and metabolic levels, overall nervous system excitability, etc. Micro-expression recognition technology analyzes subtle facial muscle movements through computer vision, which can assist in judging emotions and psychological states. Currently, there are Chinese patents such as CN202411993952, entitled "Fully Automatic Pain Measurement System Based on Artificial Intelligence Integrated Vital Sign Monitoring," and CN202510322350, entitled "A Real-Time Interactive Method and System Applicable to Multimodal Rehabilitation Medicine Information," which are mainly for auxiliary diagnosis and do not involve closed-loop system control of treatment output.

[0008] Inputting information into the brain is also a crucial step in the practical application of brain-computer interfaces (BCIs). Commonly used neuromodulation techniques directly stimulate the brain through sound, light, electricity, and magnetism, or indirectly stimulate peripheral nerves such as the vagus nerve to achieve therapeutic or neurological health maintenance goals for various neurological disorders such as depression and epilepsy. However, the integration of current neuromodulation techniques with BCIs is not yet high; there are no technologies or systems that can decode brain signals using multimodal signals and then adaptively modulate brain function using various physical methods. Summary of the Invention

[0009] To overcome the shortcomings of existing technologies, the technical problem to be solved by this invention is to provide a brain-computer interface method based on multimodal biological signals, which can efficiently read brain signals, realize adaptive brain-computer interaction, and bring more efficient and accurate diagnosis and treatment solutions to the medical field.

[0010] The technical solution of this invention is: a brain-computer interface method based on multimodal biological signals, which includes the following steps: (1) Acquisition of multimodal biological signals: EEG signals are acquired through disc electrodes or an integrated EEG cap; ECG signals are acquired by attaching electrode pads to the skin of the chest; micro-expression signals are acquired using a high frame rate camera, and facial key point detection algorithm is used to capture subtle changes in facial muscle movements; each acquisition device acquires multimodal signals synchronously through a synchronous clock and communication module. (2) Signal processing and analysis: The acquired multimodal signals are preprocessed. For EEG signals, bandpass filtering is used to remove power frequency interference, and wavelet transform is used to remove motion artifacts. For ECG signals, baseline drift and noise are eliminated by adaptive filtering algorithm. For micro-expression signals, image enhancement and normalization are performed. Then, signal features are extracted. For EEG signals, energy features and entropy features of different frequency bands are extracted. For ECG signals, features of R-R interval and heart rate variability are extracted. For micro-expression signals, features of facial key point displacement and muscle contraction intensity are extracted. A deep neural network fusion model is used to learn the spatiotemporal correlation between different modal signals and perform deep fusion analysis. (3) Selection and feedback of multiple treatment methods: Based on the results of the fusion analysis, the patient's disease status and treatment needs are judged by the machine learning model, and the parameters of multiple physical therapy devices are adjusted; based on the treatment process and the patient's multimodal biosignal feedback, the parameters of stimulation intensity and frequency are adjusted in real time.

[0011] This invention, based on multimodal biosignal acquisition using EEG, ECG, and micro-expression data, comprehensively and accurately reflects the patient's physiological and psychological state, overcoming the limitations of single-modal signal acquisition. Deep fusion analysis of multimodal signals improves the accuracy of disease state assessment. Personalized multi-physical therapy plans, tailored to individual patient differences, enhance treatment efficacy and reduce side effects and risks. Acoustic feedback and treatment synergy mechanisms enhance the patient's treatment experience and effectiveness. This invention integrates biosignal acquisition, analysis, and treatment, providing a new and effective approach for the treatment of neurological diseases and neurological health maintenance. It efficiently reads brain signals, enabling adaptive brain-computer interaction, bringing more efficient and precise diagnostic and treatment solutions to the medical field.

[0012] A brain-computer interface system based on multimodal biological signals is also provided, which includes: The multimodal biosignal acquisition module is configured to acquire electroencephalogram (EEG) signals via disc electrodes or an integrated EEG cap; acquire electrocardiogram (ECG) signals by attaching electrode pads to the chest skin; acquire micro-expression signals using a high frame rate camera, and capture subtle changes in facial muscle movements using a facial key point detection algorithm; and synchronize the acquisition of multimodal signals through a synchronous clock and communication module. The signal processing and analysis module is configured to preprocess the acquired multimodal signals. For EEG signals, bandpass filtering is used to remove power frequency interference, and wavelet transform is combined to remove motion artifacts. For ECG signals, an adaptive filtering algorithm is used to eliminate baseline drift and noise. Micro-expression signals undergo image enhancement and normalization. Then, signal features are extracted: energy and entropy features of different frequency bands are extracted from EEG signals; R-R interval and heart rate variability features are extracted from ECG signals; and facial key point displacement and muscle contraction intensity features are extracted from micro-expression signals. A deep neural network fusion model is used to learn the spatiotemporal correlations between different modal signals and perform deep fusion analysis. The selection and feedback module is configured to use machine learning models to determine the patient's disease status and treatment needs based on the fusion analysis results, and adjust the parameters of the multi-physical therapy devices; based on the treatment progress and the patient's multimodal biosignal feedback, it adjusts the parameters of stimulation intensity and frequency in real time. Attached Figure Description

[0013] Figure 1 A flowchart of a brain-computer interface method based on multimodal biological signals according to the present invention is shown.

[0014] Figure 2 A schematic diagram of the structure of a brain-computer interface system based on multimodal biosignals according to the present invention is shown. Detailed Implementation

[0015] like Figure 1 As shown, a brain-computer interface method based on multimodal biological signals includes the following steps: (1) Acquisition of multimodal biological signals: EEG signals are acquired through disc electrodes or an integrated EEG cap; ECG signals are acquired by attaching electrode pads to the skin of the chest; micro-expression signals are acquired using a high frame rate camera, and facial key point detection algorithm is used to capture subtle changes in facial muscle movements; each acquisition device acquires multimodal signals synchronously through a synchronous clock and communication module. (2) Signal processing and analysis: The acquired multimodal signals are preprocessed. For EEG signals, bandpass filtering is used to remove power frequency interference, and wavelet transform is used to remove motion artifacts. For ECG signals, baseline drift and noise are eliminated by adaptive filtering algorithm. For micro-expression signals, image enhancement and normalization are performed. Then, signal features are extracted. For EEG signals, energy features and entropy features of different frequency bands are extracted. For ECG signals, features of R-R interval and heart rate variability are extracted. For micro-expression signals, features of facial key point displacement and muscle contraction intensity are extracted. A deep neural network fusion model is used to learn the spatiotemporal correlation between different modal signals and perform deep fusion analysis. (3) Selection and feedback of multiple treatment methods: Based on the results of the fusion analysis, the patient's disease status and treatment needs are judged by the machine learning model, and the parameters of multiple physical therapy devices are adjusted; based on the treatment process and the patient's multimodal biosignal feedback, the parameters of stimulation intensity and frequency are adjusted in real time.

[0016] This invention, based on multimodal biosignal acquisition using EEG, ECG, and micro-expression data, comprehensively and accurately reflects the patient's physiological and psychological state, overcoming the limitations of single-modal signal acquisition. Deep fusion analysis of multimodal signals improves the accuracy of disease state assessment. Personalized multi-physical therapy plans, tailored to individual patient differences, enhance treatment efficacy and reduce side effects and risks. Acoustic feedback and treatment synergy mechanisms enhance the patient's treatment experience and effectiveness. This invention integrates biosignal acquisition, analysis, and treatment, providing a new and effective approach for the treatment of neurological diseases and neurological health maintenance. It efficiently reads brain signals, enabling adaptive brain-computer interaction, bringing more efficient and precise diagnostic and treatment solutions to the medical field.

[0017] Preferably, in step (1), the EEG signal acquisition device, ECG signal acquisition device, and high frame rate camera are all connected to the same synchronous clock source through a physical synchronization interface, so that each device uses the same time reference during the acquisition process; before acquisition, clock calibration is completed; the main control terminal issues acquisition instructions simultaneously through a unified acquisition control program, and each device starts the acquisition task at the same time point. While acquiring data, each device timestamps the signal stream and transmits it to the data central server in real time through a wired or wireless local area network; the server splices and caches the multimodal data in real time according to the timestamp information, thereby ensuring the timing alignment of multi-source signals.

[0018] Preferably, in step (2), the deep neural network fusion model is a Transformer architecture. The method is as follows: for the EEG signal, ECG signal and micro-expression signal after preprocessing and feature extraction, independent embedding layers are constructed respectively. The feature vectors of each modality are uniformly mapped to a high-dimensional space, and learnable modality embedding positions or codes are set for different modalities to distinguish different signal sources and their temporal features. Subsequently, the feature vectors of each modality are concatenated and input into the multi-layer Transformer encoder module. This module includes a multi-head self-attention mechanism and a feedforward neural network layer. The self-attention mechanism dynamically captures the spatiotemporal dependencies between and within each modality, realizing cross-modal and cross-time slice feature deep fusion. The network training stage adopts a supervised learning method and performs end-to-end optimization according to the labels of disease state or treatment needs. The optimization objectives include classification loss and regression loss. The model output layer outputs preset parameter combinations or recommended values ​​of treatment parameters using a fully connected layer according to task requirements.

[0019] Preferably, in step (3), for epilepsy treatment, ultrasound stimulation is used, set to a low-frequency pulse mode, and the stimulation site is via the vagus nerve, transcranial, or a combination of multiple sites; for depression treatment, magnetic stimulation is used, set to a high-frequency pulse mode, and the site is transcranial, using ultrasound via the vagus nerve for combined treatment; for health care treatment to improve sleep disorders and enhance learning ability, direct current stimulation is used, set to a low-frequency mode, and the stimulation site is transcranial, using ultrasound via the vagus nerve for combined treatment.

[0020] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium. When executed, the program includes the steps of the methods of the above embodiments. The storage medium can be ROM / RAM, magnetic disk, optical disk, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes a brain-computer interface system based on multimodal biosignals. This system is typically represented in the form of functional modules corresponding to the steps of the method. The system includes: The multimodal biosignal acquisition module is configured to acquire electroencephalogram (EEG) signals via disc electrodes or an integrated EEG cap; acquire electrocardiogram (ECG) signals by attaching electrode pads to the chest skin; acquire micro-expression signals using a high frame rate camera, and capture subtle changes in facial muscle movements using a facial key point detection algorithm; and synchronize the acquisition of multimodal signals through a synchronous clock and communication module. The signal processing and analysis module is configured to preprocess the acquired multimodal signals. For EEG signals, bandpass filtering is used to remove power frequency interference, and wavelet transform is combined to remove motion artifacts. For ECG signals, an adaptive filtering algorithm is used to eliminate baseline drift and noise. Micro-expression signals undergo image enhancement and normalization. Then, signal features are extracted: energy and entropy features of different frequency bands are extracted from EEG signals; R-R interval and heart rate variability features are extracted from ECG signals; and facial key point displacement and muscle contraction intensity features are extracted from micro-expression signals. A deep neural network fusion model is used to learn the spatiotemporal correlations between different modal signals and perform deep fusion analysis. The selection and feedback module is configured to use machine learning models to determine the patient's disease status and treatment needs based on the fusion analysis results, and adjust the parameters of the multi-physical therapy devices; based on the treatment progress and the patient's multimodal biosignal feedback, it adjusts the parameters of stimulation intensity and frequency in real time.

[0021] Preferably, in the multimodal biosignal acquisition module, the EEG signal acquisition device, ECG signal acquisition device, and high frame rate camera are all connected to the same synchronous clock source through a physical synchronization interface, so that each device uses the same time reference during the acquisition process; before acquisition, clock calibration is completed; the main control terminal issues acquisition commands simultaneously through a unified acquisition control program, and each device starts the acquisition task at the same time. While acquiring data, each device timestamps the signal stream and transmits it to the data central server in real time through a wired or wireless local area network; the server performs real-time splicing and caching of multimodal data according to the timestamp information, thereby ensuring the timing alignment of multi-source signals.

[0022] Preferably, in the multimodal biosignal acquisition module, flexible and stretchable electrodes are used for EEG and ECG acquisition; and infrared micro-expression recognition method is used to acquire micro-expression signals.

[0023] Preferably, in the signal processing and analysis module, the deep neural network fusion model is a Transformer architecture. The method is as follows: for the preprocessed and feature-extracted EEG signals, ECG signals, and micro-expression signals, independent embedding layers are constructed respectively. The feature vectors of each modality are uniformly mapped to a high-dimensional space, and learnable modality embedding positions or codes are set for different modalities to distinguish different signal sources and their temporal features. Subsequently, the feature vectors of each modality are concatenated and input into a multi-layer Transformer encoder module. This module includes a multi-head self-attention mechanism and a feedforward neural network layer. The self-attention mechanism dynamically captures the spatiotemporal dependencies between and within each modality, realizing deep feature fusion across modalities and across time slices. The network training stage adopts a supervised learning method, and end-to-end optimization is performed according to the labels of disease state or treatment needs. The optimization objectives include classification loss and regression loss. The model output layer outputs preset parameter combinations or recommended values ​​of treatment parameters using a fully connected layer according to task requirements.

[0024] Preferably, in the signal processing and analysis module, the neural network is combined with the Transformer architecture.

[0025] Preferably, in the selection and feedback module, for epilepsy treatment, ultrasound stimulation is used, set to a low-frequency pulse mode, and the stimulation site is via the vagus nerve, transcranial, or a combination of multiple sites; for depression treatment, magnetic stimulation is used, set to a high-frequency pulse mode, and the site is transcranial, using ultrasound combined with vagus nerve stimulation; for health care treatment to improve sleep disorders and enhance learning ability, direct current stimulation is used, set to a low-frequency mode, and the stimulation site is transcranial, using ultrasound combined with vagus nerve stimulation.

[0026] Compared with the prior art, the present invention has the following advantages: 1. Multimodal biosignal precise synchronous acquisition technology: Design a unique biosignal acquisition system for EEG, ECG and micro-expression, etc., to achieve high-precision synchronous acquisition of multiple signals, optimize the layout and parameters of acquisition equipment, and improve signal quality and synchronization.

[0027] 2. Multimodal signal deep fusion analysis algorithm: Based on a deep neural network, the multimodal signal fusion model effectively mines the spatiotemporal correlation between different modal signals, enabling accurate judgment of disease status.

[0028] 3. Personalized multi-physical treatment plan generation and control: Based on the results of multimodal signal analysis, a personalized ultrasound treatment parameter adjustment model is established to achieve precise treatment of nervous system diseases.

[0029] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A brain-computer interface method based on multimodal biological signals, characterized in that: It includes the following steps: (1) Acquisition of multimodal biological signals: EEG signals are acquired through disc electrodes or by using an integrated EEG cap; Electrode pads are attached to the skin of the chest to collect electrocardiogram signals; A high frame rate camera is used to capture micro-expression signals, and a facial key point detection algorithm is used to capture subtle changes in facial muscle movements; each acquisition device performs synchronous acquisition of multimodal signals through a synchronization clock and communication module; (2) Signal processing and analysis: The acquired multimodal signals are preprocessed. The EEG signal is filtered by bandpass to remove power frequency interference and combined with wavelet transform to remove motion artifacts. The ECG signal is filtered by adaptive filtering algorithm to eliminate baseline drift and noise. The micro-expression signal is processed by image enhancement and normalization. Then, the signal features are extracted. The energy features and entropy features of different frequency bands are extracted from the EEG signal. Features of R-R interval and heart rate variability were extracted from electrocardiogram signals; features of facial key point displacement and muscle contraction intensity were extracted from micro-expression signals; a deep neural network fusion model was used to learn the spatiotemporal correlation between different modal signals and perform deep fusion analysis. (3) Selection and feedback of multiple treatment methods: Based on the results of the fusion analysis, the patient's disease status and treatment needs are judged by the machine learning model, and the parameters of multiple physical therapy devices are adjusted; based on the treatment process and the patient's multimodal biosignal feedback, the parameters of stimulation intensity and frequency are adjusted in real time.

2. The brain-computer interface method based on multimodal biological signals according to claim 1, characterized in that: In step (1), the EEG signal acquisition device, ECG signal acquisition device and high frame rate camera are all connected to the same synchronous clock source through a physical synchronization interface, so that each device uses the same time reference during the acquisition process; before acquisition, clock calibration is completed. The main control terminal issues acquisition commands simultaneously through a unified acquisition control program. Each device starts the acquisition task at the same time. While acquiring data, each device timestamps the signal stream and transmits it to the data central server in real time via wired or wireless local area network. The server performs real-time splicing and caching of multimodal data based on timestamp information, thereby ensuring the timing alignment of multi-source signals.

3. The brain-computer interface method based on multimodal biological signals according to claim 2, characterized in that: In step (2), the deep neural network fusion model is a Transformer architecture. The method is as follows: for the EEG signal, ECG signal and micro-expression signal after preprocessing and feature extraction, construct independent embedding layers, map the feature vector of each modality to a high-dimensional space, and set learnable modality embedding positions or codes for different modalities to distinguish different signal sources and their temporal features. Subsequently, the feature vectors of each modality are concatenated and input into the multi-layer Transformer encoder module, which includes a multi-head self-attention mechanism and a feedforward neural network layer. The self-attention mechanism dynamically captures the spatiotemporal dependencies between and within each modality, thereby achieving deep feature fusion across modalities and across time slices. The network training phase employs a supervised learning method, performing end-to-end optimization based on labels of disease status or treatment needs. The optimization objectives include classification loss and regression loss. The model output layer uses a fully connected layer to output preset parameter combinations or recommended values ​​for treatment parameters, depending on the task requirements.

4. The brain-computer interface method based on multimodal biological signals according to claim 3, characterized in that: In step (3), for epilepsy treatment, ultrasound stimulation is used, set to low-frequency pulse mode, and the stimulation site is via the vagus nerve, transcranial, or combined stimulation of multiple sites; for depression treatment, magnetic stimulation is used, set to high-frequency pulse mode, and the site is transcranial, using ultrasound combined with vagus nerve; for health care treatment to improve sleep disorders and enhance learning ability, direct current stimulation is used, set to low-frequency mode, and the stimulation site is transcranial, using ultrasound combined with vagus nerve.

5. A brain-computer interface system based on multimodal biological signals, characterized in that: It includes: A multimodal biosignal acquisition module, configured to acquire electroencephalogram (EEG) signals via disc electrodes or an integrated EEG cap; Electrode pads are attached to the skin of the chest to collect electrocardiogram signals; A high frame rate camera is used to capture micro-expression signals, and a facial key point detection algorithm is used to capture subtle changes in facial muscle movements; each acquisition device performs synchronous acquisition of multimodal signals through a synchronization clock and communication module; The signal processing and analysis module is configured to preprocess the acquired multimodal signals. For EEG signals, bandpass filtering is used to remove power frequency interference, and wavelet transform is used to remove motion artifacts. For ECG signals, an adaptive filtering algorithm is used to eliminate baseline drift and noise. For micro-expression signals, image enhancement and normalization processing are performed. Then, signal features are extracted, and energy features and entropy features of different frequency bands are extracted from EEG signals. Features of R-R interval and heart rate variability were extracted from electrocardiogram signals; features of facial key point displacement and muscle contraction intensity were extracted from micro-expression signals; a deep neural network fusion model was used to learn the spatiotemporal correlation between different modal signals and perform deep fusion analysis. The selection and feedback module is configured to use machine learning models to determine the patient's disease status and treatment needs based on the fusion analysis results, and adjust the parameters of the multi-physical therapy devices; based on the treatment progress and the patient's multimodal biosignal feedback, it adjusts the parameters of stimulation intensity and frequency in real time.

6. The brain-computer interface system based on multimodal biological signals according to claim 5, characterized in that: In the multimodal biosignal acquisition module, the EEG signal acquisition device, ECG signal acquisition device, and high frame rate camera device are all connected to the same synchronization clock source through a physical synchronization interface, so that each device uses the same time reference during the acquisition process; clock calibration is completed before acquisition. The main control terminal issues acquisition commands simultaneously through a unified acquisition control program. Each device starts the acquisition task at the same time. While acquiring data, each device timestamps the signal stream and transmits it to the data central server in real time via wired or wireless local area network. The server performs real-time splicing and caching of multimodal data based on timestamp information, thereby ensuring the timing alignment of multi-source signals.

7. The brain-computer interface system based on multimodal biological signals according to claim 6, characterized in that: In the multimodal biosignal acquisition module, flexible and stretchable electrodes are used for EEG and ECG acquisition; and infrared micro-expression recognition method is used to acquire micro-expression signals.

8. The brain-computer interface system based on multimodal biological signals according to claim 7, characterized in that: In the signal processing and analysis module, the deep neural network fusion model is based on the Transformer architecture. The method is as follows: for the EEG signal, ECG signal and micro-expression signal after preprocessing and feature extraction, independent embedding layers are constructed respectively. The feature vectors of each modality are uniformly mapped to a high-dimensional space, and learnable modality embedding positions or codes are set for different modalities to distinguish different signal sources and their temporal features. Subsequently, the feature vectors of each modality are concatenated and input into the multi-layer Transformer encoder module, which includes a multi-head self-attention mechanism and a feedforward neural network layer. The self-attention mechanism dynamically captures the spatiotemporal dependencies between and within each modality, thereby achieving deep feature fusion across modalities and across time slices. The network training phase employs a supervised learning method, performing end-to-end optimization based on labels of disease status or treatment needs. The optimization objectives include classification loss and regression loss. The model output layer uses a fully connected layer to output preset parameter combinations or recommended values ​​for treatment parameters, depending on the task requirements.

9. The brain-computer interface system based on multimodal biological signals according to claim 8, characterized in that: In the signal processing and analysis module, neural networks are combined with the Transformer architecture.

10. The brain-computer interface system based on multimodal biological signals according to claim 9, characterized in that: In the selection and feedback module, for epilepsy treatment, ultrasound stimulation is used, set to a low-frequency pulse mode, and the stimulation site is via the vagus nerve, transcranial, or a combination of multiple sites; for depression treatment, magnetic stimulation is used, set to a high-frequency pulse mode, and the site is transcranial, using ultrasound combined with vagus nerve stimulation; for health care treatment to improve sleep disorders and enhance learning ability, direct current stimulation is used, set to a low-frequency mode, and the stimulation site is transcranial, using ultrasound combined with vagus nerve stimulation.

Images