An immersive brain-computer interface dysphagia training device and method

Abstract

The present application relates to a kind of immersive brain-computer interface swallowing disorder training equipment and method, belong to brain-computer interface technical field.The present application is constructed virtual eating scene by VR module, provides visual stimulation for patient, brain electric signal in the swallowing motor imagination process of patient is gathered and decoded by brain electric signal acquisition and decoding module, determine whether it exists swallowing motor imagination and swallowing motor imagination intensity, according to decoding result control nerve muscle electric stimulation module to patient swallowing related muscle output accurate electric stimulation signal, and control VR module to play corresponding swallowing sound.The present application is integrated brain-computer interface technology, nerve muscle stimulation technology and VR multi-sensory immersive experience technology, constructs " active imagination-signal decoding-precise stimulation-multi-sensory feedback " closed-loop rehabilitation training process, realize the personalized, efficient rehabilitation intervention of swallowing disorder patient, improve the rehabilitation effect and training experience of swallowing disorder patient.

Description

Technical Field

[0001] This invention relates to an immersive brain-computer interface training device and method for swallowing disorders, belonging to the field of brain-computer interface technology. Background Technology

[0002] Dysphagia, a common clinical condition, is prevalent among stroke patients, Parkinson's disease patients, Alzheimer's disease patients, and the elderly. It not only directly affects patients' nutrient intake and hydration but also easily leads to serious complications such as aspiration pneumonia and suffocation, significantly reducing patients' quality of life and threatening their lives. Currently, clinical training methods for dysphagia are mainly divided into three categories: traditional rehabilitation training, neuromuscular electrical stimulation training, and cognitive intervention training.

[0003] Traditional rehabilitation training, such as the Mendelssohn maneuver and empty swallowing training, relies heavily on the therapist's professional experience and subjective judgment. The training effect is significantly influenced by human factors and cannot achieve precise monitoring and control of the patient's swallowing-related neural activity. Furthermore, such training is often conducted in open treatment environments, where patients are easily distracted by surrounding people and equipment noise, making it difficult to concentrate on completing the swallowing imagery and training, resulting in low training efficiency and a lengthy rehabilitation period. While neuromuscular electrical stimulation (NMS) can stimulate swallowing-related muscles (such as the infrahyoid muscles and pharyngeal constrictors) with low-frequency current to promote muscle function recovery, existing equipment often uses fixed stimulation parameters. This makes it impossible to dynamically adjust the stimulation intensity, frequency, and duration according to the patient's real-time swallowing status. This can easily lead to insufficient stimulation resulting in poor training effects, or excessive stimulation causing muscle fatigue, pain, and other adverse reactions. In addition, this technology only focuses on passive activation at the muscle level and lacks training for the patient's active swallowing imagery ability, making it difficult to improve swallowing function from the root of neural regulation.

[0004] With the development of brain-computer interface (BCI) technology, some have proposed applying BCI to swallowing disorder training. For example, Chinese patent application CN120361419A discloses a brain-computer hybrid intelligent swallowing rehabilitation system for treating post-stroke dysphagia. The intelligent swallowing BCI module accurately identifies swallowing intentions through multimodal physiological signal acquisition and processing. The intelligent swallowing functional electrical stimulation module finely adjusts stimulation parameters according to the patient's state. The AI-assisted online integration module realizes non-invasive brain-controlled swallowing and high-precision AI control. The closed-loop intelligent feedback control module monitors, warns, and adaptively optimizes electrical stimulation in real time. However, this method only considers electrical stimulation and does not provide sufficient guidance to the patient, resulting in poor training effects.

[0005] With the application of virtual reality (VR) technology in the field of rehabilitation medicine, some studies have attempted to simulate the eating process through VR scenes in order to enhance patients' initiative and interest in training. However, existing VR swallowing training systems only provide a single visual stimulus, making it difficult to build a highly realistic immersive training environment. This results in patients having a weak sense of immersion during training, and training compliance is difficult to maintain in the long term. Summary of the Invention

[0006] The purpose of this invention is to provide an immersive brain-computer interface training device and method for swallowing disorders, in order to solve the problems of weak patient immersion and difficulty in maintaining long-term training compliance during current training.

[0007] To address the aforementioned technical problems, this invention provides an immersive brain-computer interface swallowing disorder training device. The device includes a central control module and connected to it an electroencephalogram (EEG) signal acquisition and decoding module, a neuromuscular electrical stimulation module, and a VR module. The EEG signal acquisition and decoding module acquires EEG signals during the patient's swallowing motor imagery process and decodes these signals to determine the presence and intensity of swallowing motor imagery. The VR module constructs a virtual eating scene to provide visual stimulation for the patient. The central control module controls the neuromuscular electrical stimulation module to output precise electrical stimulation signals to the patient's swallowing-related muscles based on the decoding results, and controls the VR module to play corresponding swallowing sounds when the patient's EEG signal decoding result indicates the presence of swallowing motor imagery.

[0008] Furthermore, when the central control module controls the neuromuscular electrical stimulation module based on the decoding results, if the decoding results show that the intensity of the patient's swallowing motor imagination has not reached the preset standard, the central control module increases the frequency and / or intensity of the output signal of the neuromuscular electrical stimulation module; if the decoding results show that the intensity of the patient's swallowing motor imagination has reached the preset standard, the central control module controls the neuromuscular electrical stimulation module to maintain the current output.

[0009] Furthermore, the electrodes used in the neuromuscular electrical stimulation module are neck electrode pads targeting the infrahyoid muscles or pharyngeal minimally invasive electrodes targeting the pharyngeal constrictor muscles.

[0010] Furthermore, the neuromuscular electrical stimulation module uses electrodes equipped with pressure sensors to monitor the contact pressure between the electrodes and the skin / mucous membrane, and to issue a prompt when the pressure is lower than a set threshold.

[0011] Furthermore, the EEG signal acquisition and decoding module includes an EEG signal acquisition unit, an EEG signal preprocessing unit, and a swallowing motor imagery decoding unit. The EEG signal acquisition unit uses a non-invasive EEG electrode cap. The EEG signal preprocessing unit is used to denoise and normalize the EEG signals acquired by the EEG signal acquisition unit. The swallowing motor imagery decoding unit is used to decode the EEG signals preprocessed by the EEG signal preprocessing unit to determine the patient's swallowing motor imagery.

[0012] Furthermore, the swallowing operation imagination decoding unit uses a deep learning algorithm to construct a decoding model, which is trained and validated using a multi-center swallowing patient EEG dataset.

[0013] Furthermore, the VR module is also used to interact with the patient under the control of the central control module.

[0014] This invention also provides an immersive brain-computer interface method for training swallowing disorders, the method comprising: A virtual eating scenario was constructed using VR models to provide visual stimulation of swallowing for patients and to detect the electroencephalogram (EEG) signals during the patient's imagined swallowing movements under visual stimulation. The EEG signal is decoded to determine whether the patient has swallowing motor imagination and the intensity of swallowing motor imagination, and the electrical stimulation signal output to the relevant muscles of the patient is controlled according to the intensity of swallowing motor imagination; when the decoding result of the patient's EEG signal indicates that swallowing motor imagination exists, the VR module is controlled to play the corresponding swallowing sound.

[0015] Furthermore, the method also includes detecting the quality of the patient's swallowing action when the patient completes a swallowing motion imagery, and providing corresponding feedback to the patient based on the quality of completion.

[0016] Furthermore, the control of the electrical stimulation signal output to the patient's relevant muscles based on the intensity of swallowing motor imagery includes: if the decoding result shows that the intensity of the patient's swallowing motor imagery has not reached the preset standard, the central control module increases the frequency and / or intensity of the output signal of the neuromuscular electrical stimulation module; if the decoding result shows that the intensity of the patient's swallowing motor imagery has reached the preset standard, the central control module controls the neuromuscular electrical stimulation module to maintain the current output.

[0017] The beneficial effects of this invention are as follows: The immersive brain-computer interface swallowing disorder training device of this invention constructs a virtual eating scene through a VR module, providing visual stimulation to the patient. The EEG signal acquisition and decoding module acquires and decodes the patient's EEG signals during swallowing motor imagery to determine the presence and intensity of swallowing motor imagery. Based on the decoding results, the central control module controls the neuromuscular electrical stimulation module to output precise electrical stimulation signals to the patient's swallowing-related muscles. When the decoding result of the patient's EEG signal indicates the presence of swallowing motor imagery, the VR module plays corresponding swallowing sounds. This invention integrates brain-computer interface technology, neuromuscular stimulation technology, and VR multi-sensory immersive experience technology to construct a closed-loop rehabilitation training process of "active imagery - signal decoding - precise stimulation - multi-sensory feedback," achieving personalized and efficient rehabilitation intervention for patients with swallowing disorders, and improving the rehabilitation effect and training experience of patients with swallowing disorders. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the principle of the immersive brain-computer interface swallowing disorder training device of the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0020] This invention integrates brain-computer interface technology, neuromuscular stimulation technology, and VR multi-sensory immersive experience technology to construct a closed-loop rehabilitation training process of "active imagination - signal decoding - precise stimulation - multi-sensory feedback" for swallowing disorder training.

[0021] Implementation of Immersive Brain-Computer Interface Swallowing Disorder Training Device like Figure 1 As shown, the immersive brain-computer interface swallowing disorder training device of the present invention includes a central control module and connected to it an EEG signal acquisition and decoding module, a neuromuscular electrical stimulation module, and a VR module. The EEG signal acquisition and decoding module is used to acquire EEG signals during the patient's swallowing motor imagination process and decode the EEG signals to determine whether the patient is in a swallowing motor imagination. The central control module is used to control the neuromuscular electrical stimulation module to output precise electrical stimulation signals to the patient's swallowing-related muscles according to the decoding results. The VR module constructs a virtual eating scene to provide visual stimulation for the patient, and the central control module controls the VR module to play corresponding swallowing sounds according to the patient's EEG signal decoding results.

[0022] Specifically, the central control module, as the core control unit of the equipment, adopts a high-performance embedded processor (such as the ARM Cortex-A72 architecture, with a main frequency of no less than 1.5GHz) and is equipped with a real-time operating system (RTOS) to achieve unified coordination and control of various modules. Its core functions include: Data interaction and timing control: Data interaction is conducted with the EEG decoding module and VR module via Ethernet (transmission rate 1000Mbps), and wireless data transmission is conducted with the electrical stimulation module via Bluetooth 5.0 to ensure that the data transmission delay between modules does not exceed 100ms; at the same time, a strict working sequence is established, such as "EEG signal acquisition (100ms) - signal decoding (50ms) - electrical stimulation parameter adjustment (30ms) - electrical stimulation output and VR feedback (20ms)" to ensure the real-time and continuity of the training process. Personalized Training Programs: The system includes multiple standardized training programs (targeting different causes such as post-stroke dysphagia and age-related degenerative dysphagia). It also allows doctors to customize training duration (15-60 minutes / session), frequency (1-3 times / day), food texture difficulty gradient, and initial electrical stimulation parameters based on parameters such as the severity of the patient's condition (e.g., VFSS score), age, and cognitive ability). The system automatically records key data from each training session (e.g., EEG decoding accuracy, changes in electrical stimulation parameters, and training completion rate), generating personalized training reports to provide doctors with a basis for adjusting treatment plans. Remote Monitoring and Maintenance: The module supports 4G / 5G network connectivity, allowing doctors to view the patient's training process and data in real time via a remote monitoring platform. When abnormal training occurs (e.g., electrical stimulation alarms, abnormal EEG signals), the platform automatically sends a warning to the doctor. The system also features remote firmware updates, allowing for continuous optimization and fault repair via network-push software updates.

[0023] The EEG signal acquisition and decoding module includes an EEG signal acquisition unit, an EEG signal preprocessing unit, and a swallowing motor imagery decoding unit. The EEG signal acquisition unit employs a non-invasive, high-density EEG electrode cap. The electrodes on the cap are distributed across key brain regions such as the motor cortex and parietal cortex. The motor cortex is used for complex motor imagery, while the parietal cortex is responsible for swallowing-related sensory integration. In this embodiment, the EEG electrode cap contains at least 32 channels to completely cover the aforementioned brain regions. The sampling rate is not lower than a set threshold (e.g., 250Hz) to ensure the capture of weak and complex swallowing motor imagery EEG signals. Simultaneously, to maintain stable conductivity between the electrodes and the scalp and avoid signal distortion due to poor contact, the acquisition duration can be set according to the training plan, for example, 5-30 minutes per session.

[0024] The EEG signal preprocessing unit is used to denoise and normalize the EEG signals acquired by the EEG signal acquisition unit. The denoising process includes: 1) removing power frequency interference, electromyographic interference, and electrooculography artifacts using digital filtering techniques (including 50Hz power frequency notch filtering and 0.5-30Hz bandpass filtering); 2) separating noise components from the effective signal in the EEG signal using independent component analysis (ICA) to further improve the signal-to-noise ratio. Swallowing disorders are divided into oral preparation, oral delivery, pharyngeal, and esophageal phases. The neural regulation and swallowing organ muscles involved differ at each of these different time periods. Therefore, during normalization, EEG signals from different patients and different training periods need to be converted into feature data of the same dimension to provide standardized input for subsequent decoding.

[0025] The swallowing motor imagery decoding unit is used to decode the preprocessed EEG signals from the preprocessing unit to determine the patient's swallowing motor imagery status. This implementation uses a decoding model built on a deep learning algorithm to decode the EEG signals. This model takes the preprocessed EEG signal features as input and outputs three states: "swallowing motor imagery," "non-swallowing motor imagery," and "resting state." "Non-swallowing motor imagery" refers to states outside of VR (Virtual Reality) and includes electrical stimulation during and outside of actual eating. The resting state is a non-treatment state without any intervention. Non-swallowing motor imagery requires specific processing, and the resting state is merely a mode representation under different scenarios. Different intensities of non-swallowing motor imagery correspond to different levels of brain cell activation; currently, it is impossible to classify the intensity based solely on the intensity of motor imagery.

[0026] The EEG signal features include Event-Related Desynchronization / Resynchronization (ERD / ERS) features, temporal features, and frequency domain features. These features represent the determination of brain region activation and excitation levels. The decoding model is trained and validated using a multi-center EEG dataset of patients with swallowing disorders, with a sample size of no less than 500 cases to ensure a decoding accuracy of no less than 85%. "Multi-center" here refers to multiple centers including an information acquisition unit, an information processing unit, and an imagery EEG signal decoding unit. During training, the decoding unit outputs the recognition result and confidence level of the patient's current swallowing motor imagery in real time. If the confidence level is lower than a preset threshold (e.g., 70%, indicating the current recognition result is unreliable), a warning signal is sent to the central control module, prompting adjustments to the training difficulty or patient-guided instruction. This implementation may employ machine learning algorithms such as Support Vector Machines (SVM) and deep learning algorithms such as Convolutional Neural Networks (CNNs).

[0027] The neuromuscular electrical stimulation module is used to output precise electrical stimulation signals to the patient's swallowing-related muscles based on the decoding results of electroencephalogram (EEG) signals, thereby achieving a collaborative training mode of "active imagery triggering passive stimulation". This neuromuscular electrical stimulation module includes stimulation electrodes, an adaptive stimulation parameter control unit, and a safety protection unit. The stimulation electrodes are flexible biocompatible electrode pads, designed with two electrode attachment schemes based on human anatomy: a neck electrode pad for the infrahyoid muscles (including the sternohyoid and omohyoid muscles); and a pharyngeal minimally invasive electrode for the pharyngeal constrictor muscles. This pharyngeal minimally invasive electrode can be inserted through the nasal cavity, with a spherical conductive contact at the end to avoid damage to the mucosa. To ensure the effectiveness of electrical stimulation of the relevant muscles, this invention also incorporates pressure sensors within the electrode pads to monitor the contact pressure between the electrode and the skin / mucosa in real time. When the pressure falls below a set threshold (which is the critical value for effective electrical stimulation of the relevant muscles), a prompt is issued to facilitate personnel examination.

[0028] The adaptive stimulation parameter adjustment unit, controlled by the central control module, adjusts stimulation parameters under its control. These parameters include stimulation frequency (adjustable from 20-80Hz), stimulation intensity (adjustable from 0-30mA), stimulation pulse width (adjustable from 100-500μs), and stimulation duration (synchronized with the patient's swallowing motor imagery duration, ranging from 1-3s). The central control module dynamically adjusts the stimulation parameters based on the "swallowing motor imagery intensity" output by the EEG decoding unit. For example, when the decoding result shows a weak swallowing motor imagery intensity, the central control module automatically controls the output frequency and intensity of the electrodes through the adaptive stimulation parameter adjustment unit to enhance muscle activation. When the imagery intensity reaches a preset standard, the central control module electrodes maintain the current stimulation parameters to avoid muscle over-fatigue.

[0029] Specifically, the central control module in this embodiment uses a step-by-step fine-tuning method for control. During this control process, it can be controlled according to a set frequency, where the set frequency is adjusted in steps of 2-5Hz each time, and the intensity is adjusted in steps of 0.5-1mA each time, to avoid discomfort caused by sudden parameter changes. The specific set frequency and adjustment step size can be adjusted according to the actual situation.

[0030] Upon initial use, the patient's stimulation tolerance threshold and effective activation threshold are determined using electromyography (sEMG) combined with subjective feedback, forming the initial values ​​for the base frequency / intensity. Electrode control strategies without swallowing motor imagery or in a resting state are a crucial safety and energy-saving aspect of the closed-loop system. The core principle is "no intention, no stimulation," specifically divided into two scenarios: (1) Clear rest period (rest period preset in the training plan) Complete cessation of stimulation: The electrode output is directly reduced to zero, and no current is applied, allowing the swallowing muscles to fully relax. Weak sensory stimulation can be added (optional): Some protocols will apply subthreshold sensory stimulation (intensity < 30% of the muscle activation threshold, frequency 5-10Hz) during the rest period to maintain sensory input without triggering muscle contraction, thus helping to maintain the sensitivity of neural pathways.

[0031] (2) No swallowing motor imagery intention was detected (no effective EEG signal during the training period) 1. If no valid signal is detected for the first time, the system keeps the current parameters unchanged and enters a "waiting recognition window" for a certain period of time (e.g., 3 seconds); 2. If no effective decoding result is found within a certain period of time (e.g., 3 seconds), the system will automatically reduce the stimulation parameters: frequency will be reduced by 10-20%, intensity will be reduced by 15-25%, and the system will enter the "low intensity maintenance mode". 3. If no swallowing intention is detected for a continuous period of time (e.g., 10-15 seconds), the system will automatically stop stimulation and prompt the patient to adjust their state through audio and visual cues, while recording the data for subsequent analysis.

[0032] Safety protection: All parameter adjustments are set with hard upper / lower limits (e.g., frequency upper limit 50Hz, intensity upper limit 20mA, lower limits 5Hz and 0.5mA respectively) to prevent parameters from getting out of control.

[0033] The safety protection unit, through its built-in overcurrent protection, overvoltage protection, and temperature monitoring functions, will immediately cut off the stimulation output and issue an audible and visual alarm when the stimulation current / voltage exceeds the safety threshold or the electrode temperature exceeds a certain temperature (generally human body temperature, such as 37°C). Simultaneously, a stimulation interval time (no less than 10 seconds) is set to prevent continuous stimulation in a short period from causing damage to muscles and nerves.

[0034] The VR module provides patients with multi-sensory stimulation, including visual and auditory senses, by constructing highly realistic virtual eating scenarios, enhancing the immersion and initiative of training. Specifically, the VR module of this invention, based on the Unity3D engine, develops various virtual eating scenarios, such as family dining table scenes and restaurant scenes. The food models in these scenarios cover various textures, including liquids (such as milk and porridge), semi-liquids (such as rice porridge and egg custard), and soft foods (such as noodles and cakes). Each simulated food model has realistic appearance and texture (such as the viscosity of porridge and the fluffiness of cake), color, and dynamic effects (such as the shape changes of food when chewed and the simulation of esophageal intubation during swallowing). Patients observe the virtual scenes through a VR headset, which uses a resolution of no less than 2560×1440 and a field of view of no less than 110 degrees. The VR model can dynamically switch the type and difficulty of food according to the patient's training stage through the control system of the central control module. For example, in the early stages of training, the VR model can be controlled to display liquid food scenes, gradually transitioning to soft food scenes.

[0035] The VR module has a built-in swallowing sound database containing characteristic sounds of swallowing different textures of food (such as the "gulp" sound of liquid food, and the chewing and swallowing sounds of soft food). When the central control module detects that the patient has completed a swallowing motion visualization, the VR module simultaneously plays the corresponding swallowing sound at the same time as the stimulation electrodes are triggered, thus achieving "visual-auditory" sensory coordination. At the same time, the VR module provides auditory feedback based on the quality of the patient's swallowing action. For example, it plays a "ding" sound when the quality is good, and a gentle "Please try again" voice prompt when the quality is poor, thereby enhancing the patient's sense of feedback during training.

[0036] To enable interaction with patients, the VR module of this invention also features interactive control functionality. Specifically, patients use a hand motion capture controller (such as infrared optical positioning technology with a positioning accuracy of no less than 1mm) to perform actions such as "grabbing utensils," "putting food into their mouths," and "chewing" in a virtual environment. The VR module responds to the patient's action commands in real time, ensuring synchronization between virtual and real actions. In addition, the VR headset has a built-in eye-tracking module that monitors the patient's gaze focus during training. If the patient's gaze deviates from the virtual eating scene for more than a preset time (e.g., 10 seconds), the system automatically displays guidance prompts to help the patient refocus their attention, thereby further improving the training effect.

[0037] Based on the above introduction of the immersive brain-computer interface swallowing disorder training device of the present invention, the working process of the immersive brain-computer interface swallowing disorder training device of the present invention will be described below. The swallowing disorder training process of the immersive brain-computer interface swallowing disorder training device of the present invention follows a closed-loop process of "preparation-training-assessment-feedback", which specifically includes the following stages.

[0038] Training Preparation Phase: The patient enters the training chamber, and medical staff assist the patient in putting on the EEG electrode cap (ensuring good electrode contact during installation) and attaching the electrical stimulation electrode pads. The patient is then fitted with a VR headset and motion capture controller. Medical staff select or customize training programs via the touchscreen of the central control module, such as setting training duration and initial electrical stimulation parameters. After preparation, they exit the training chamber, and the patient confirms the start of training via the in-chamber touchscreen. In this embodiment, the training chamber's internal dimensions are 2.5m × 2m × 2.2m (length × width × height). The chamber is constructed with sound-insulating materials (sound insulation of no less than 40dB) to effectively isolate external noise interference. The chamber is equipped with a ventilation system (air exchange rate of no less than 10 times / hour) to maintain air circulation, while the temperature is controlled at 22-26℃ and the humidity at 40%-60%, providing a comfortable training environment for the patient. As another implementation method, an emergency call button can be installed inside the cabin. If the patient experiences discomfort during training, they can call medical staff with one click. It is equipped with an adjustable training seat (backrest angle adjustable from 0-90°, seat height adjustable from 40-55cm) to meet the needs of patients of different body shapes. Armrests and a storage table are provided next to the seat. The storage table is used to place charging and storage devices for EEG electrode caps, VR headsets and other equipment to ensure that the cabin environment is clean and orderly.

[0039] Training execution phase: The central control module sends instructions to the VR module to activate the virtual eating scenario. The patient observes the virtual scenario and performs swallowing motion visualization according to the prompts (e.g., "imagine swallowing a mouthful of milk"). The EEG signal acquisition unit collects the patient's EEG signals in real time and transmits them to the EEG signal preprocessing unit for noise reduction. Subsequently, the decoding unit analyzes the signals to identify whether the patient is in a swallowing motion visualization state and the intensity of the visualization. If the decoding result is "effective swallowing motion visualization," the central control module immediately sends instructions to the neuromuscular electrical stimulation module to adjust the stimulation parameters according to the visualization intensity and output electrical stimulation signals. At the same time, it sends a synchronization instruction to the VR module to play the corresponding swallowing sounds and food dynamic effects. If the decoding result is "ineffective swallowing motion visualization" or "resting state," the central control module controls the VR module to prompt the patient to try again through VR voice prompts without triggering electrical stimulation output.

[0040] Training Assessment and Feedback Phase: After each training session, the central control module automatically compiles training data, including the number of effective swallowing motor imagery attempts, EEG decoding accuracy, electrical stimulation parameter adjustment records, and patient gaze concentration time. Based on this data, it generates a training effectiveness assessment report (e.g., "Effective imagery rate in this training was 82%, an improvement of 5% compared to the previous session; it is recommended to appropriately increase the difficulty of food texture in the next training session"). The generated assessment report can be displayed to the patient via a touchscreen and can also be automatically uploaded to the hospital information system (HIS) for doctors to review and analyze. Doctors can adjust subsequent training plans based on the assessment report to achieve personalized iteration of training.

[0041] Implementation of an Immersive Brain-Computer Interface Method for Swallowing Disorder Training The immersive brain-computer interface swallowing disorder training method of the present invention first uses a VR model to construct a virtual eating scene to provide visual stimulation for the patient to swallow, detects the electroencephalogram (EEG) signals of the patient during the swallowing motor imagination process under visual stimulation, then decodes the detected EEG signals to determine whether the patient has swallowing motor imagination and the intensity of swallowing motor imagination, and controls the output of electrical stimulation signals to the relevant muscles of the patient according to the intensity of swallowing motor imagination; if the patient completes swallowing motor imagination, the corresponding swallowing sound is played using the VR model.

[0042] The training method of this invention uses a VR model developed with the Unity 3D engine to construct virtual eating scenarios. These virtual eating scenarios include family dining table scenes, restaurant scenes, etc. The food models in these scenarios cover various textures, including liquids (such as milk, porridge, etc.), semi-liquids (such as rice cereal, egg custard, etc.), and soft foods (such as noodles, cakes, etc.). Each simulated food model has realistic appearance and texture (such as the viscosity of porridge, the fluffiness of cake, etc.), color, and dynamic effects (such as the shape changes of food when chewed, the simulation of esophageal intubation during swallowing, etc.). Patients observe the virtual scenarios through a VR headset with a resolution of at least 2560×1440 and a field of view of at least 110 degrees. The VR model can dynamically switch the type and difficulty of food according to the patient's training stage via the central control module's control system. For example, in the early stages of training, the VR model can be controlled to display liquid food scenarios, gradually transitioning to soft food scenarios.

[0043] The detection of EEG signals during the patient's swallowing motor imagery process under visual stimulation is achieved by an EEG signal acquisition unit. In this embodiment, the EEG signal acquisition unit uses a non-invasive high-density EEG electrode cap. The electrodes of the electrode cap are distributed in key brain regions such as the motor cortex and parietal cortex. The motor cortex is used for complex motor imagery, and the parietal cortex is responsible for the integration of swallowing-related sensations. In this embodiment, the EEG electrode cap has no fewer than 32 channels to completely cover the aforementioned brain regions; the sampling rate is no less than a set threshold (e.g., 250Hz) to ensure that weak and complex swallowing motor imagery EEG signals can be captured. Simultaneously, to maintain stable conductivity between the electrodes and the scalp and avoid signal distortion due to poor contact, the acquisition duration can be set according to the training plan, for example, 5-30 minutes per session.

[0044] The EEG signals acquired by the EEG signal acquisition unit are first processed by the EEG signal preprocessing unit, and then decoded by the swallowing motor imagery decoding unit to determine the patient's swallowing motor imagery. The preprocessing performed by the EEG signal preprocessing unit includes denoising and normalization. The swallowing motor imagery decoding unit uses a decoding model built based on a deep learning algorithm to decode the EEG signals. This decoding model takes the preprocessed EEG signal features as input and outputs three states: "swallowing motor imagery," "non-swallowing motor imagery," and "resting state." The EEG signal features include event-related desynchronization / synchronization (ERD / ERS) features, temporal features, and frequency domain features. The decoding model is trained and validated using a multi-center EEG dataset of patients with swallowing disorders, with a sample size of no less than 500 cases to ensure a decoding accuracy of no less than 85%.

[0045] This invention utilizes a neuromuscular electrical stimulation module to output electrical stimulation signals to the relevant muscles of a patient. In this embodiment, the neuromuscular electrical stimulation module includes stimulation electrodes, an adaptive stimulation parameter control unit, and a safety protection unit. The stimulation electrodes are flexible, biocompatible electrode pads, and two electrode attachment schemes are designed based on human anatomical structure: a neck electrode pad for the infrahyoid muscles (including the sternohyoid and omohyoid muscles); and a pharyngeal minimally invasive electrode for the pharyngeal constrictor muscles. This pharyngeal minimally invasive electrode can be inserted through the nasal cavity, with a spherical conductive contact at the end to avoid damage to the mucosa. To ensure the effectiveness of electrical stimulation to the relevant muscles, this invention also incorporates a pressure sensor within the electrode pads to monitor the contact pressure between the electrode and the skin / mucosa in real time. When the pressure falls below a set threshold (a critical value for effective electrical stimulation of the relevant muscles), a prompt is issued to facilitate personnel examination.

[0046] The adaptive stimulation parameter control unit, controlled by the central control module, adjusts stimulation parameters under its guidance. These parameters include stimulation frequency (adjustable from 20-80Hz), stimulation intensity (adjustable from 0-30mA), stimulation pulse width (adjustable from 100-500μs), and stimulation duration (synchronized with the patient's swallowing motor imagery duration, ranging from 1-3s). The central control module dynamically adjusts these parameters based on the "swallowing motor imagery intensity" output from the EEG decoding unit. For example, when decoding results indicate weak swallowing motor imagery intensity, the central control module automatically controls the electrode output frequency and intensity through the adaptive stimulation parameter control unit to enhance muscle activation. When the imagery intensity reaches a preset standard, the central control module maintains the current stimulation parameters to avoid muscle over-fatigue. The safety protection unit, with built-in overcurrent protection, overvoltage protection, and temperature monitoring, immediately cuts off the stimulation output and issues an audible and visual alarm when the stimulation current / voltage exceeds a safety threshold or the electrode temperature exceeds a certain temperature (generally body temperature, such as 37°C). At the same time, set a stimulation interval (no less than 10 seconds) to prevent continuous stimulation in a short period of time from causing damage to muscles and nerves.

[0047] Based on the above introduction, the immersive brain-computer interface swallowing disorder training method of the present invention mainly includes a training preparation stage, a training execution stage, and a training evaluation and feedback stage. Each stage is described below.

[0048] Training preparation phase: The patient enters the training cabin, where medical staff assist the patient in putting on the EEG electrode cap (ensuring good electrode contact during wearing) and attaching the electrical stimulation electrode pads. The patient is then fitted with a VR headset and motion capture controller. Medical staff select or customize training programs via the touchscreen of the central control module, such as setting the training duration and initial electrical stimulation parameters. After completing the preparation, the medical staff exits the training cabin, and the patient confirms the start of training via the touchscreen inside the cabin.

[0049] Training execution phase: The central control module sends instructions to the VR module to activate the virtual eating scenario. The patient observes the virtual scenario and performs swallowing motion visualization according to the prompts (e.g., "imagine swallowing a mouthful of milk"). The EEG signal acquisition unit collects the patient's EEG signals in real time and transmits them to the EEG signal preprocessing unit for noise reduction. Subsequently, the decoding unit analyzes the signals to identify whether the patient is in a swallowing motion visualization state and the intensity of the visualization. If the decoding result is "effective swallowing motion visualization," the central control module immediately sends instructions to the neuromuscular electrical stimulation module to adjust the stimulation parameters according to the visualization intensity and output electrical stimulation signals. At the same time, it sends a synchronization instruction to the VR module to play the corresponding swallowing sounds and food dynamic effects. If the decoding result is "ineffective swallowing motion visualization" or "resting state," the central control module controls the VR module to prompt the patient to try again through VR voice prompts without triggering electrical stimulation output.

[0050] Training Assessment and Feedback Phase: After each training session, the central control module automatically compiles training data, including the number of effective swallowing motor imagery attempts, EEG decoding accuracy, electrical stimulation parameter adjustment records, and patient gaze concentration time. Based on this data, it generates a training effectiveness assessment report (e.g., "Effective imagery rate in this training was 82%, an improvement of 5% compared to the previous session; it is recommended to appropriately increase the difficulty of food texture in the next training session"). The generated assessment report can be displayed to the patient via a touchscreen and can also be automatically uploaded to the hospital information system (HIS) for doctors to review and analyze. Doctors can adjust subsequent training plans based on the assessment report to achieve personalized iteration of training.

[0051] Therefore, the immersive brain-computer interface swallowing disorder training program of the present invention can integrate brain-computer interface technology, neuromuscular stimulation technology and VR multi-sensory immersive experience technology, thereby constructing a closed-loop rehabilitation training process of "active imagination-signal decoding-precise stimulation-multi-sensory feedback", realizing personalized and efficient rehabilitation intervention for patients with swallowing disorders, and improving the rehabilitation effect and training experience of patients with swallowing disorders.

Claims

1. An immersive brain-computer interface swallowing disorder training device, characterized in that, The device includes a central control module and connected to it an EEG signal acquisition and decoding module, a neuromuscular electrical stimulation module, and a VR module. The EEG signal acquisition and decoding module is used to acquire EEG signals during the patient's swallowing motor imagination process and decode the EEG signals to determine whether swallowing motor imagination exists and its intensity. The VR module constructs a virtual eating scene to provide visual stimulation for the patient. The central control module is used to control the neuromuscular electrical stimulation module to output precise electrical stimulation signals to the patient's swallowing-related muscles based on the decoding results, and to control the VR module to play corresponding swallowing sounds when the patient's EEG signal decoding result indicates the presence of swallowing motor imagination.

2. The immersive brain-computer interface swallowing disorder training device according to claim 1, characterized in that, When the central control module controls the neuromuscular electrical stimulation module based on the decoding results, if the decoding results show that the intensity of the patient's swallowing motor imagination has not reached the preset standard, the central control module increases the frequency and / or intensity of the output signal of the neuromuscular electrical stimulation module; if the decoding results show that the intensity of the patient's swallowing motor imagination has reached the preset standard, the central control module controls the neuromuscular electrical stimulation module to maintain the current output.

3. The immersive brain-computer interface swallowing disorder training device according to claim 1 or 2, characterized in that, The neuromuscular electrical stimulation module uses neck electrode pads targeting the infrahyoid muscles or minimally invasive pharyngeal electrodes targeting the pharyngeal constrictor muscles.

4. The immersive brain-computer interface swallowing disorder training device according to claim 3, characterized in that, The neuromuscular electrical stimulation module uses electrodes equipped with pressure sensors to monitor the contact pressure between the electrodes and the skin / mucous membrane, and issues a prompt when the pressure falls below a set threshold.

5. The immersive brain-computer interface swallowing disorder training device according to claim 1, characterized in that, The EEG signal acquisition and decoding module includes an EEG signal acquisition unit, an EEG signal preprocessing unit, and a swallowing motor imagery decoding unit. The EEG signal acquisition unit uses a non-invasive EEG electrode cap. The EEG signal preprocessing unit is used to denoise and normalize the EEG signals acquired by the EEG signal acquisition unit. The swallowing motor imagery decoding unit is used to decode the EEG signals preprocessed by the EEG signal preprocessing unit to determine the patient's swallowing motor imagery.

6. The immersive brain-computer interface swallowing disorder training device according to claim 5, characterized in that, The swallowing operation imagination decoding unit uses a deep learning algorithm to build a decoding model, which is trained and validated using a multi-center swallowing patient EEG dataset.

7. The immersive brain-computer interface swallowing disorder training device according to claim 1, characterized in that, The VR module is also used to interact with patients under the control of the central control module.

8. An immersive brain-computer interface method for training swallowing disorders, characterized in that, The method includes: A virtual eating scenario was constructed using VR models to provide visual stimulation of swallowing for patients and to detect the electroencephalogram (EEG) signals during the patient's imagined swallowing movements under visual stimulation. The EEG signal is decoded to determine whether the patient has swallowing motor imagination and the intensity of swallowing motor imagination, and the electrical stimulation signal output to the relevant muscles of the patient is controlled according to the intensity of swallowing motor imagination; when the decoding result of the patient's EEG signal indicates that swallowing motor imagination exists, the VR module is controlled to play the corresponding swallowing sound.

9. The immersive brain-computer interface swallowing disorder training method according to claim 8, characterized in that, The method also includes detecting the quality of the patient's swallowing action when the patient completes a swallowing motor imagery, and providing corresponding feedback to the patient based on the quality of completion.

10. The immersive brain-computer interface swallowing disorder training method according to claim 8, characterized in that, The electrical stimulation signals output to the relevant muscles of the patient are controlled according to the intensity of the swallowing motor imagery. If the decoding result shows that the intensity of the patient's swallowing motor imagery has not reached the preset standard, the central control module increases the frequency and / or intensity of the output signal of the neuromuscular electrical stimulation module; if the decoding result shows that the intensity of the patient's swallowing motor imagery has reached the preset standard, the central control module controls the neuromuscular electrical stimulation module to maintain the current output.

Images