A human-computer interaction system and method for extracting electroencephalogram signals assisted by spontaneous breathing

Abstract

The present invention discloses a human-computer interaction system and method for extracting electroencephalogram signals assisted by spontaneous breathing, which relates to the field of brain-computer interfaces. The system includes: a breathing signal acquisition unit, configured to acquire the breathing movement information of a patient and then generate a waveform signal according to the breathing movement information; a control center, connected to the breathing signal acquisition unit, configured to convert at least one waveform signal generated by the breathing signal acquisition unit into a control signal for the cerebral cortex position of the corresponding limb part; an electroencephalogram signal acquisition unit, connected to the control center, configured to extract electroencephalogram signals at the cerebral cortex position of the specified corresponding limb part according to the control signal of the control center and then send them to the corresponding limb control unit. The human-computer interaction system for extracting electroencephalogram signals assisted by spontaneous breathing provided by the present invention completes human-computer interaction through two stages of primary selection by spontaneous breathing and fine selection of electroencephalogram, thereby ensuring a high intention recognition accuracy rate.

Description

Technical Field

[0001] The present invention relates to the field of brain - computer interfaces, and particularly to a human - machine interaction system and method for extracting electroencephalogram (EEG) signals assisted by spontaneous breathing. Background Art

[0002] Patients with severe motor disabilities, such as stroke, amyotrophic lateral sclerosis, and Guillain - Barré syndrome, have severely affected quality of life due to their inconvenient movement. Brain - computer interface (BCI) provides a way of human - machine communication for these patients and external movement - assistance devices; however, due to the complexity of EEG signals, the low recognition rate of EEG intentions is one of the main obstacles hindering the wide application of BCI technology. To improve the recognition rate of EEG intentions, most devices extract EEG signals assisted by visual evoked potentials. These systems require users to focus their vision on a certain part for spatial selection, and eye movement / gaze tracking technology becomes an essential component of such devices. For example, a method and system for combining three - dimensional gaze tracking and BCI to control a robotic arm to grasp an object (publication number: CN 108646915A), and a visual evoked BCI method combining an asynchronous eye movement switch (publication number: CN109508094B).

[0003] Deficiencies of eye movement / gaze tracking technology assisting BCI: 1) For patients with severe motor disabilities, such as patients with amyotrophic lateral sclerosis, the motor abilities of the trunk, head, and neck muscles decline, and they often cannot lower their heads, raise their heads, turn their heads, or move their eyes to fix their vision on a certain part. Therefore, eye movement / gaze tracking cannot assist in extracting EEG signals from these patients. 2) For patients with brain neuron damage, there are not only motor disabilities but also difficulties in extracting EEG signals caused by brain neuron damage. For example, for a patient with a left - brain stroke, due to the damage of neurons in the left brain, there is right - limb motor disability, often accompanied by difficulties in turning the head, lowering the head, and eye movement; in addition, due to the damage of neurons in the left brain, EEG information needs to be extracted from the right brain, but there is a dilemma in unclear intention recognition as to whether the EEG information extracted from the right brain controls the components of the left limb or the right limb. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a human - machine interaction system and method for extracting EEG signals assisted by spontaneous breathing, which completes human - machine interaction through two stages: primary selection by spontaneous breathing and refined selection by EEG, so as to ensure a high intention recognition accuracy rate.

[0005] The present invention is implemented as follows:

[0006] A human - machine interaction system for extracting EEG signals assisted by spontaneous breathing, comprising:

[0007] A respiration signal acquisition unit, configured to acquire a patient's respiratory movement information and then generate a waveform signal according to the respiratory movement information;

[0008] A control center, connected to the respiration signal acquisition unit, configured to convert at least one waveform signal generated by the respiration signal acquisition unit into a control signal for the cerebral cortex position of the corresponding limb part;

[0009] An electroencephalogram (EEG) signal acquisition unit, connected to the control center, configured to extract an EEG signal at the cerebral cortex position corresponding to the specified limb part according to the control signal of the control center, and then convert the extracted EEG signal into an action instruction and send it to the corresponding limb control unit.

[0010] Further, the respiration signal acquisition unit includes a thoracic electrode and a respiration monitoring module. The thoracic electrode is connected to the respiration monitoring module through a wire, and the respiration monitoring module converts the signal acquired by the thoracic electrode into a waveform signal.

[0011] Further, when the control center receives the first waveform signal generated by the respiration signal acquisition unit for controlling the movement of the right upper limb for the first odd number of times, it is converted into a control signal for starting to extract the EEG from the middle part of the left frontal lobe;

[0012] When the control center receives the first waveform signal generated by the respiration signal acquisition unit for controlling the movement of the right upper limb for the first even number of times, it is converted into a control signal for stopping the extraction of the EEG from the middle part of the left frontal lobe.

[0013] Further, the first waveform signal is a waveform signal generated by taking two deep breaths.

[0014] Further, when the control center receives the second waveform signal generated by the respiration signal acquisition unit for controlling the movement of the left upper limb for the first odd number of times, it is converted into a control signal for starting to extract the EEG from the middle part of the right frontal lobe;

[0015] When the control center receives the second waveform signal generated by the respiration signal acquisition unit for controlling the movement of the left upper limb for the first even number of times, it is converted into a control signal for stopping the extraction of the EEG from the middle part of the right frontal lobe.

[0016] Further, the second waveform signal is a waveform signal generated by holding the breath for 5 seconds.

[0017] Further, the limb control unit includes a left upper limb control module and a right upper limb control module.

[0018] Further, the EEG signal acquisition unit is an EEG helmet.

[0019] Further, the respiratory signal acquisition unit is connected to the control center via Bluetooth or wired connection; the control center is connected to the electroencephalogram signal acquisition unit via Bluetooth or wired connection; the electroencephalogram signal acquisition unit is connected to the limb control unit via Bluetooth or wired connection.

[0020] The method of the present invention specifically includes the following steps:

[0021] Step 1: Place the respiratory signal acquisition unit at a specified position on the chest of the user. The respiratory signal acquisition unit collects the respiratory movement information of the patient and then generates a waveform signal according to the respiratory movement information.

[0022] Step 2: The control center converts at least one waveform signal generated by the respiratory signal acquisition unit into cerebral cortex position information of the corresponding limb part, and then transmits the cerebral cortex position information to the electroencephalogram signal acquisition unit; different combinations of respiratory frequencies and / or breath-holding times will generate different sine wave signals, and these signals correspond to different limb movement parts one by one.

[0023] Step 3: Extract or select electroencephalogram signals at the cerebral cortex position of the specified corresponding limb part according to the control signal of the control center.

[0024] Step 4: Convert the extracted electroencephalogram signal into an action instruction and send it to the corresponding limb control unit.

[0025] The present invention has the following advantages:

[0026] By adopting two stages of autonomous breathing primary selection and electroencephalogram fine selection to complete human-computer interaction. In the breathing primary selection stage, the present invention makes full use of the controllability of autonomous breathing. By adjusting the frequency of autonomous breathing and the breath-holding time, various combinations of respiratory signals can be generated, and different combinations can correspond to different limb movement parts. In the electroencephalogram fine selection stage, the electroencephalogram information of the cerebral cortex of the required part is selected according to the autonomous respiratory signal. In this process, the electroencephalogram can ensure a high intention recognition accuracy rate. Description of the Drawings

[0027] The present invention will be further described below with reference to the accompanying drawings in conjunction with embodiments.

[0028] Figure 1 It is a schematic structural diagram of the system according to the embodiment of the present invention;

[0029] Figure 2 It is a schematic flow diagram of the method according to the embodiment of the present invention

[0030] Figure 3 It is the respiratory wave in the resting state according to the embodiment of the present invention;

[0031] Figure 4It is the sine wave generated when taking two deep breaths in the embodiment of the present invention;

[0032] Figure 5 It is the waveform generated when inhaling / exhaling / pausing breathing in the embodiment of the present invention;

[0033] Figure 6 It is the electrode schematic diagram of the electroencephalogram helmet in the embodiment of the present invention. Detailed implementation manners

[0034] In view of the above deficiencies, the present invention identifies electroencephalogram signals through assisted voluntary breathing for the following reasons: 1) For patients with movement disorders, compared with the movement disorders of trunk muscles, the respiratory function is affected later. 2) Various ventilators, electrocardiogram monitors, and sleep apnea monitors widely used clinically can identify the breathing frequency, inhalation / exhalation duration, respiratory movement, etc. of users, and the breathing recognition technology is relatively mature. 3) Different combinations of the breathing frequency and inhalation time of users can be monitored by using nasal and oral airflow or baroreceptors, or thoracic volume / thoracic movement receptors, and used as the input signal of the brain-computer interface. This input method needs to exclude the normal breathing frequency first, which is similar to the current eye movement / gaze tracking technology as the electroencephalogram input signal. The eye movement / gaze tracking technology also needs to exclude inadvertent blinking, eye movement, etc.

[0035] The implementation process of the present invention is as follows: The respiratory signal is collected through the thoracic electrodes, and the respiratory monitoring unit converts the respiratory signal into a sine wave signal. By changing the frequency of voluntary breathing and / or the breath-holding time, different sine wave signals can be generated, and these signals match different limb movement parts. The respiratory monitoring unit transmits the sine wave signal to the control center, and the control center interprets the limb movement part corresponding to the signal and transmits the cerebral cortex position information controlling the limb part to the electroencephalogram signal collector; then, the electroencephalogram helmet collects the electroencephalogram information of the corresponding cortex position, thereby triggering the corresponding limb mechanical movement element. Most of the previous devices for assisting electroencephalogram signal recognition used visual tracking; the present invention overcomes the limitation that patients with movement disorders cannot fix their eyes / vision on a specific part by controlling the frequency of voluntary breathing and / or the breath-holding time.

[0036] Specifically, it can be realized through the following steps:

[0037] 1) Respiratory signal collection: Place the thoracic electrodes on the right midaxillary line and under the left rib of the user. Since the inhalation and exhalation cause changes in the thoracic cavity volume, which can further cause changes in the thoracic cavity resistance, the regular changes in the thoracic cavity resistance can be monitored by the chest electrodes; the chest electrodes transmit the signals to the respiratory monitoring unit through wires, and the respiratory monitoring unit converts the changes in the thoracic cavity resistance into sine wave signals.

[0038] 2) The sine wave signal is converted into the cerebral cortex position information for controlling a certain limb part in the control center: Combinations of different breathing frequencies and / or breath-holding times will generate different sine wave signals, and these signals correspond to different limb movement parts one by one. After receiving the sine wave information, the control center will interpret the cerebral cortex position information for controlling the corresponding limb part and transmit this position information to the electroencephalogram (EEG) signal acquisition unit, which extracts the EEG signal of this cerebral cortex position.

[0039] 3) The EEG signal outputs to control the operation of an external component (robotic arm) corresponding to the limb part.

[0040] Improve the EEG intention recognition rate. For patients with motor disorders, due to limb, neck, and eye movement disorders, they often cannot gaze at a specific limb part; and the respiratory function of this part of patients is affected relatively late. The present invention controls the voluntary breathing frequency and / or breath-holding time to guide the EEG signal acquisition unit to extract the EEG signals of the corresponding parts, with strong controllability.

[0041] The technical advantage of the present invention is mainly to complete human-computer interaction in two stages: primary selection by voluntary breathing and fine selection by EEG. In the primary selection stage of breathing, the present invention makes full use of the controllability of voluntary breathing. By adjusting the frequency of voluntary breathing and the breath-holding time, various breathing signal combinations can be generated, and different combinations can correspond to different limb movement parts. In the fine selection stage of EEG, the cerebral cortex EEG information of the required part is selected according to the voluntary breathing signal. In this process, the EEG can ensure a relatively high intention recognition accuracy rate.

[0042] An embodiment of the present invention provides a human-computer interaction system for assisting EEG signal extraction with voluntary breathing, as Figure 1 shown, including:

[0043] A breathing signal acquisition unit, configured to acquire the patient's breathing movement information and then generate a waveform signal according to the breathing movement information;

[0044] A control center, connected to the breathing signal acquisition unit, configured to convert at least one waveform signal generated by the breathing signal acquisition unit into a control signal for the cerebral cortex position of the corresponding limb part;

[0045] An EEG signal acquisition unit (for example, an EEG helmet), connected to the control center, configured to extract an EEG signal at the cerebral cortex position of the designated corresponding limb part according to the control signal of the control center, and then convert the extracted EEG signal into an action instruction and send it to the corresponding limb control unit (for example, including a left upper limb control module and a right upper limb control module).

[0046] In a possible implementation, the breathing signal acquisition unit includes a thoracic electrode and a breathing monitoring module. The thoracic electrode is connected to the breathing monitoring module through a wire, and the breathing monitoring module converts the signal collected by the thoracic electrode into a waveform signal.

[0047] In a possible implementation, when the control center receives the first waveform signal generated by the breathing signal acquisition unit for controlling the movement of the right upper limb for the first odd number of times, it is converted into a control signal for starting to extract electroencephalogram from the middle part of the left frontal lobe.

[0048] When the control center receives the first waveform signal generated by the breathing signal acquisition unit for controlling the movement of the right upper limb for the first even number of times (for example, the waveform signal generated by taking two deep breaths, or it can be set to other waveform signals according to needs), it is converted into a control signal for stopping the extraction of electroencephalogram from the middle part of the left frontal lobe.

[0049] In a possible implementation, when the control center receives the second waveform signal generated by the breathing signal acquisition unit for controlling the movement of the left upper limb for the first odd number of times (for example, the waveform signal generated by holding the breath for 5 seconds, or it can be set to other waveform signals according to needs, but it needs to be distinguished from the first waveform signal), it is converted into a control signal for starting to extract electroencephalogram from the middle part of the right frontal lobe.

[0050] When the control center receives the second waveform signal generated by the breathing signal acquisition unit for controlling the movement of the left upper limb for the first even number of times, it is converted into a control signal for stopping the extraction of electroencephalogram from the middle part of the right frontal lobe.

[0051] Other combinations of waveform signals can be further set to achieve the purpose of controlling more than two limb control units.

[0052] In a possible implementation, the breathing signal acquisition unit is connected to the control center through Bluetooth or wired connection; the control center is connected to the electroencephalogram signal acquisition unit through Bluetooth or wired connection; the electroencephalogram signal acquisition unit is connected to the limb control unit through Bluetooth or wired connection.

[0053] The embodiment of the present invention also provides a human-computer interaction method for extracting electroencephalogram signals assisted by autonomous breathing, as Figure 2 shown. The above system needs to be provided, including the following steps:

[0054] Step 1: Place the respiratory signal acquisition unit at the designated position on the chest of the user. The respiratory signal acquisition unit collects the patient's respiratory movement information and then generates a waveform signal based on the respiratory movement information. For example, place the thoracic electrodes on the right midaxillary line and under the left rib of the user. Since the change in thoracic cavity volume caused by inhalation and exhalation can further cause a change in thoracic resistance, the regular change in thoracic resistance can be monitored by the chest electrodes. The thoracic electrodes transmit the signal to the respiratory monitoring module through wires.

[0055] Step 2: The control center converts at least one waveform signal generated by the respiratory signal acquisition unit into the cerebral cortex position information for controlling the corresponding limb part, and then transmits the cerebral cortex position information to the electroencephalogram (EEG) signal acquisition unit. Different combinations of respiratory frequencies and / or breath-holding times will generate different sine wave signals, and these signals correspond to different limb movement parts one by one.

[0056] Step 3: Extract or select EEG signals at the cerebral cortex position corresponding to the designated limb part according to the control signal of the control center.

[0057] Step 4: Convert the extracted EEG signals into action instructions and send them to the corresponding limb control unit (such as a robotic arm).

[0058] Example 1

[0059] A patient with amyotrophic lateral sclerosis (ALS) who is unable to move wants to lift the right upper limb. When operating using the system of the present invention, the following steps are required:

[0060] Step 1: The thoracic electrodes collect the patient's respiratory movement information in the resting state. This signal is input into the ventilator monitoring unit to generate a regular sine wave, as Figure 3 shown. When the control center receives the regular sine wave signal, it does not issue an EEG signal extraction instruction.

[0061] Step 2: The patient takes two deep breaths, generating a new sine wave, as Figure 4 shown. The process above the horizontal line is the inhalation process, and the process below the horizontal line is the exhalation process; the amplitude of the deep breath sine wave is equal to 2 times the amplitude of the calm breathing waveform; the interval between the vertical lines is two deep breaths. This waveform corresponds to the right upper limb. When the control center receives this sine wave signal for the first time (odd number), the cerebral cortex position information for controlling the movement of the right upper limb interpreted is the middle part of the left frontal lobe (for example, Figure 6 electrode C3 in). The control center instructs the EEG signal acquisition unit to extract / select the EEG information of the middle part of the left frontal lobe.

[0062] Step 3: The EEG signal acquisition unit selects the EEG information of the middle part of the left frontal lobe and instructs the external component of the right upper limb (such as a robotic arm, etc.) to lift up.

[0063] Step 4: After the movement of the right upper limb ends, the patient takes two deep breaths again, and a sine wave as shown in Figure 4 appears again; when the same waveform appears for the second time (even number), the control center terminates the extraction of EEG information by the EEG signal acquisition unit.

[0064] Embodiment 2

[0065] A patient with amyotrophic lateral sclerosis (ALS) who is unable to move wants to pick up a cup with the left hand. To operate using the system of the present invention, the following steps are required:

[0066] Step 1: The thoracic electrodes collect the patient's respiratory movement information in the resting state, and this signal is input into the ventilator monitoring unit, generating a regular sine wave as shown in Figure 3 . When the control center only receives the regular sine wave signal, it does not issue an EEG signal extraction instruction.

[0067] Step 2: The patient stops breathing for 5 seconds, and the sine wave disappears, as shown in Figure 5 . Above the horizontal line is the inhalation process, below the horizontal line is the exhalation process, and between the vertical lines is a 5-second breath hold. The limb part corresponding to this waveform is the left upper limb. When the control center receives this signal for the first time (odd number), it interprets that the position information of the cerebral cortex controlling the movement of the left upper limb is the middle part of the right frontal lobe (for example, Figure 6 electrode C4 in ). The control center instructs the EEG signal acquisition unit to extract / select the EEG information of the middle part of the right frontal lobe.

[0068] Step 3: The EEG signal acquisition unit selects the EEG information of the middle part of the right frontal lobe and guides the movement of the external component of the left upper limb (such as a robotic arm, etc.).

[0069] Step 4: After the movement ends, the patient stops breathing for 5 seconds again, and a waveform as shown in Figure 5 appears again; when the same waveform appears for the second time (even number), the control center terminates the extraction of EEG information by the EEG signal acquisition unit.

[0070] The technical advantage of the present invention is mainly to complete human-computer interaction through two stages: primary selection by spontaneous breathing and fine selection by EEG. In the primary selection stage of breathing, the present invention makes full use of the controllability of spontaneous breathing. By adjusting the frequency of spontaneous breathing and the breath-holding time, various combinations of breathing signals can be generated, and different combinations can correspond to different limb movement parts. In the fine selection stage of EEG, the EEG information of the cerebral cortex of the required part is selected according to the spontaneous breathing signal. In this process, the EEG can ensure a high intention recognition accuracy.

[0071] Although the specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments we described are illustrative rather than used to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be covered by the scope protected by the claims of the present invention.

Claims

1. A human-computer interaction system for extracting electroencephalogram signals with assisted autonomous breathing, characterized in that Comprising: A respiratory signal acquisition unit, configured to acquire the respiratory movement information of a patient, and then generate a waveform signal according to the respiratory movement information; A control center, connected to the respiratory signal acquisition unit, configured to convert at least one waveform signal generated by the respiratory signal acquisition unit into a control signal for the cerebral cortex position of the corresponding limb part; An electroencephalogram (EEG) signal acquisition unit, connected to the control center, configured to extract EEG signals at the cerebral cortex position of the specified corresponding limb part according to the control signal of the control center, and then convert the extracted EEG signals into action instructions and send them to the corresponding limb control unit.

2. The system according to claim 1, wherein: The respiratory signal acquisition unit includes a thoracic electrode and a respiratory monitoring module. The thoracic electrode is connected to the respiratory monitoring module through a wire, and the respiratory monitoring module converts the signal acquired by the thoracic electrode into a waveform signal.

3. The system according to claim 1, wherein: When the control center receives the first waveform signal generated by the respiratory signal acquisition unit for controlling the movement of the right upper limb for the first odd number of times, it is converted into a control signal for starting to extract EEG signals from the middle part of the left frontal lobe; When the control center receives the first waveform signal generated by the respiratory signal acquisition unit for controlling the movement of the right upper limb for the first even number of times, it is converted into a control signal for stopping to extract EEG signals from the middle part of the left frontal lobe.

4. The system according to claim 3, wherein: The first waveform signal is a waveform signal generated by taking two deep breaths.

5. The system according to claim 1, characterized in that: When the control center receives the second waveform signal generated by the respiratory signal acquisition unit for controlling the movement of the left upper limb for the first odd number of times, it is converted into a control signal for starting to extract EEG signals from the middle part of the right frontal lobe; When the control center receives the second waveform signal generated by the respiratory signal acquisition unit for controlling the movement of the left upper limb for the first even number of times, it is converted into a control signal for stopping to extract EEG signals from the middle part of the right frontal lobe.

6. The system according to claim 5, wherein: The second waveform signal is a waveform signal generated by holding the breath for 5 seconds.

7. The system according to claim 1, wherein: The limb control unit includes a left upper limb control module and a right upper limb control module.

8. The system according to claim 1, characterized in that: The EEG signal acquisition unit is an EEG helmet.

9. The system according to claim 1, wherein: The respiratory signal acquisition unit is connected to the control center via Bluetooth or wired connection; the control center is connected to the EEG signal acquisition unit via Bluetooth or wired connection; the EEG signal acquisition unit is connected to the limb control unit via Bluetooth or wired connection.

10. A human-computer interaction method for extracting electroencephalogram signals with assisted spontaneous breathing, characterized in that, It is necessary to provide any one of the systems described in claims 1-9. The method includes: Step 1: Place the respiratory signal acquisition unit at a specified position on the chest of the user. The respiratory signal acquisition unit acquires the respiratory movement information of the patient, and then generates a waveform signal according to the respiratory movement information; Step 2: The control center converts at least one waveform signal generated by the respiratory signal acquisition unit into cerebral cortex position information of the corresponding limb part, and then transmits the cerebral cortex position information to the EEG signal acquisition unit; different combinations of respiratory frequencies and / or breath-holding times will generate different sine wave signals, and these signals correspond to different limb movement parts one by one; Step 3: Extract or select EEG signals at the cerebral cortex position of the specified corresponding limb part according to the control signal of the control center; Step 4: Convert the extracted EEG signals into action instructions and send them to the corresponding limb control unit.

Images