Brain-computer interface systems and methods

Abstract

The application discloses a brain-computer interface system and method, which comprises a visual stimulation module, a collection module and an analysis module. The visual stimulation module provides a plurality of visual stimulation targets, each of which comprises a background gray adjustable display area and a visual stimulation area formed in the display area. The visual stimulation area does not completely fill the display area, and the display area and the visual stimulation area jointly constitute a visual stimulation code. The collection module is used for collecting brain electrical signals generated by a user to the visual stimulation code. The analysis module extracts features in the brain electrical signals and identifies a visual stimulation target currently gazed by the user. The application adopts a contrast coding method and can be used on a common refresh rate display to realize a high-frequency multi-target steady-state visual evoked potential brain-computer interface system.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, and in particular to a brain-computer interface system and method. Background Technology

[0002] Interaction methods using devices such as keyboards and mice are among the most common forms of human-computer interaction (HCI). Brain-computer interfaces (BCIs), as another direct communication method between the brain and the external world, have attracted increasing attention. In practical scenarios, BCIs can detect brain signals, decode their meaning, and transmit them to externally controlled robotic arms, drones, computers, etc., replacing parts of a disabled patient's body with machines so that the patient can interact with their surroundings.

[0003] It is worth noting that among numerous brain-computer interface (BCI) systems, those based on steady-state visual evoked potentials (SSVEPs) have been widely used in the field of assistive brain-computer interfaces for people with disabilities due to their advantages such as high information transmission rate, short training time, and non-invasiveness. Specifically, steady-state visual evoked potential signals are periodic responses generated by the visual cortex of the human brain when subjected to visual stimuli of a certain frequency, exhibiting the same frequency as the stimulus (the fundamental frequency of the stimulus) and its harmonic components.

[0004] Existing encoding methods for steady-state visual evoked potential brain-computer interfaces mainly include frequency encoding (different targets flash at different frequencies), phase encoding (different targets flash at the same frequency, but with different initial phases), and joint frequency-phase encoding (combining frequency encoding and phase encoding methods). The problems with these methods include at least the following: difficulty in achieving high-frequency multi-target encoding on ordinary displays (common displays on the market have a refresh rate of 60Hz); and reliance on using screen refresh rate division to obtain stimulus frequencies for encoding.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a brain-computer interface system and method that can overcome the technical problem of difficulty in implementing high-frequency multi-target coding on ordinary displays in the prior art.

[0007] To achieve the above objectives, embodiments of the present invention provide a brain-computer interface system, comprising:

[0008] The visual stimulation module provides multiple visual stimulation targets. Each visual stimulation target includes a display area with adjustable background grayscale and a visual stimulation area formed within the display area. The visual stimulation area does not completely fill the display area. The display area and the visual stimulation area together constitute the visual stimulation code.

[0009] The acquisition module is used to acquire the electroencephalogram (EEG) signals generated by the user's encoding of the visual stimuli;

[0010] The analysis module extracts features from the EEG signals and identifies the visual stimulus target that the user is currently looking at.

[0011] In one or more embodiments of the present invention, the visual stimulation area undergoes brightness changes at a certain frequency.

[0012] In one or more embodiments of the present invention, the visual stimulation region includes a plurality of display windows spaced apart within the display region.

[0013] To achieve the above objectives, embodiments of the present invention also provide a brain-computer interface method, comprising:

[0014] Multiple visual stimulus targets are provided. Each visual stimulus target includes a display area with adjustable background grayscale and a visual stimulus area formed in the display area. The visual stimulus area does not completely fill the display area. The display area and the visual stimulus area together constitute the visual stimulus code.

[0015] Collect the electroencephalogram (EEG) signals generated by the user's encoding of the visual stimuli;

[0016] Features are extracted from EEG signals to identify the visual stimulus target that the user is currently looking at.

[0017] In one or more embodiments of the present invention, different display areas have different background grayscale values;

[0018] Visual stimulation areas within different display areas have the same phase and flicker frequency.

[0019] In one or more embodiments of the present invention, the visual stimulation area undergoes brightness changes at a certain frequency.

[0020] In one or more embodiments of the present invention, the visual stimulation region includes a plurality of display windows spaced apart within the display region.

[0021] In one or more embodiments of the present invention, the display window is a square area.

[0022] The spacing between adjacent display windows is the same, and this spacing is equal to the side length of the display window.

[0023] In one or more embodiments of the present invention, a method for extracting features from electroencephalogram (EEG) signals and identifying the visual stimulus target currently being gazed at by the user includes:

[0024] The collected EEG signals were downsampled;

[0025] The downsampled signal is then filtered.

[0026] In one or more embodiments of the present invention, when the flicker frequency of the visual stimulation region is 15Hz, the filtering adopts low-frequency bandpass filtering of 9-90Hz and 20-90Hz.

[0027] When the flicker frequency of the visual stimulation area is 30Hz, high-frequency bandpass filters of 25-90Hz and 55-90Hz are used.

[0028] Compared with existing technologies, this invention employs a contrast encoding method, which can be used to implement a high-frequency, multi-target steady-state visual evoked potential brain-computer interface system on displays with common refresh rates (e.g., typical screens with a refresh rate of 60Hz today). Furthermore, by combining it with stimulus frequency modulation, a joint frequency contrast encoding method can also be implemented, which is of great significance for the research, application, and promotion of steady-state visual evoked potential brain-computer interface systems. Attached Figure Description

[0029] Figure 1 This is a schematic block diagram of a brain-computer interface system according to an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of a single visual stimulus target 10 according to an embodiment of the present invention;

[0031] Figure 3 This is a comparative schematic diagram of different background grayscale values ​​according to an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of a single visual stimulus target switching according to an embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of 11 different checkerboard stimulus patterns based on background contrast according to an embodiment of the present invention.

[0034] Figure 6 This is a schematic diagram of a display interface according to an embodiment of the present invention;

[0035] Figure 7This is a schematic diagram showing the placement of a 9-conducting electrode in the occipital region of a user's brain according to an embodiment of the present invention.

[0036] Figure 8 This is a flowchart illustrating a brain-computer interface method according to an embodiment of the present invention. Detailed Implementation

[0037] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0038] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0039] like Figure 1 As shown, a brain-computer interface system according to a preferred embodiment of the present invention includes a visual stimulation module, a data acquisition module, and an analysis module. This system is particularly applicable in the medical field, for example, for patients whose mobility is impaired or who have lost the ability to move due to certain diseases (such as disability, paralysis, epilepsy, ALS, etc.).

[0040] Combination Figure 2 As shown, the visual stimulation module is used to provide multiple visual stimulation targets 10. Each visual stimulation target 10 includes a display area 11 with adjustable background grayscale and a visual stimulation area 12 formed in the display area 11. The visual stimulation area 12 does not completely fill the display area 11. The display area 11 and the visual stimulation area 12 together constitute the visual stimulation code.

[0041] In some embodiments, in each visual stimulus target 10, the display area 11 is a square region, and the grayscale remains constant when the visual stimulus area 12 flashes. The visual stimulus area 12 includes a plurality of square windows arrayed within the display area 11, representing the switching area from the highest to the lowest brightness of the actual flashing stimulus. Preferably, the spacing between the square windows is equal to the side length of the square window, thus the visual stimulus target has a checkerboard shape.

[0042] It should be noted that the shape of the display area 11 can also be set to a rectangle, a circle or other irregular shape, and the small windows in the visual stimulation area 12 can also be set to a rectangle, a circle, a triangle or other irregular shape. The side length of the small windows in the visual stimulation area 12, the spacing between the small windows, the number of small windows and the array method are not limited in this embodiment.

[0043] The visual stimulus area 12 does not completely fill the display area 11 in order to expose part of the background, so that the display area 11 and the visual stimulus area 12 together constitute the visual stimulus encoding. Therefore, in some embodiments, the visual stimulus area 12 may also be a window generated in a specific area of ​​the display area 11. In order to facilitate the user's gaze, the window may be centered in the center of the display area. The shape of the window may be a regular shape such as a rectangle, circle, or triangle, or it may be other irregular shapes.

[0044] Display area 11 is set using background grayscale values. For ease of engineering application, different background grayscale values ​​are used to indicate different contrast levels. Combined with... Figure 3 As shown, in one embodiment, 11 different background grayscale values ​​were designed, ranging from 0 to 1, with a grayscale value interval of 0.1 between each pair of adjacent stimuli.

[0045] Visual stimulus area 12 is encoded with flickering at a specific frequency and phase. Taking the checkerboard-shaped visual stimulus target 10 as an example, combined with... Figure 4 The diagram shows a checkerboard stimulus with RGB values ​​of 0:25.5:255, and a spatial frequency of 2.13c / o The left image shows a contrast-reversed checkerboard stimulus against a pure black background (background grayscale value of 0), while the right image shows a checkerboard stimulus against a pure white background (background grayscale value of 1).

[0046] Combination Figure 5 As shown, all 11 different checkerboard stimulus patterns based on background contrast are displayed, with corresponding background grayscale values ​​of 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, and 1, respectively. The small black window area in the image (the corresponding black area in the last image) represents the stimulus flashing area, which is visual stimulus area 12. This represents the flip (black-and-white flip) area from the lowest brightness to the highest brightness in actual use scenarios, and the brightness is at the lowest level in all 11 images. The other areas serve as the background area, which is display area 11.

[0047] In practical application, the Michelson contrast ratio is used as the definition of contrast: Cm = (Lc - Lb) / (Lc + Lb), where Lc represents the brightness of the minimum or maximum stimulus area (grayscale value of 0 or 1), and Lb represents the background brightness. The brightness of the stimulus is measured using a screen luminance meter (e.g., Xinbao Science Instrument SM208). The parameters of the 11 different background contrast checkerboard stimuli used in the experiments of this invention are shown in Table 1.

[0048]

[0049] Table 1 Stimulation parameters

[0050] In a steady-state visual evoked potential brain-computer interface system, different stimuli (i.e., flashing novel checkerboard-like stimulus blocks) are presented on a display, with the center of each stimulus block serving as the user's focal point of gaze. During a single recognition process, multiple visual stimulus targets with different background grayscale can be displayed simultaneously on the same screen.

[0051] In one embodiment, combined with Figure 6 As shown, the same interface provides four visual stimulus targets 10. In actual use of the system, the user needs to select one of the four targets for focusing. In each visual stimulus target 10, the dark square window serves as the display area 11, and the white part serves as the visual stimulus area 12. Each visual stimulus target 10 is 300×300 pixels in size and is placed at the top, bottom, left, and right positions on the screen. The top target is 424 pixels vertically from the bottom target, and the left target is 424 pixels horizontally from the right target. The small windows in each visual stimulus area 12 flicker at the same frequency and initial phase.

[0052] In other embodiments, two, three, or more than five visual stimuli can be displayed simultaneously on the same interface.

[0053] Different visual stimulus targets 10 have display areas 11 with different background gray levels, so as to... Figure 6 In the illustrated embodiment, the background grayscale values ​​of the four visual stimulus targets 10 are different, namely 0.1, 0.6, 0.8, and 1. Within the same display interface, the flickering frequency and phase of all visual stimulus areas 12 are the same. Figure 6 In the illustrated embodiment, the phase of all visual stimulation regions 12 is 0.

[0054] The acquisition module is used to acquire the electroencephalogram (EEG) signals generated by the user's encoding of the visual stimuli. In one embodiment, the acquisition module may use a 9-lead electrode, combined with... Figure 7 As shown, the 9-lead electrodes are placed in the occipital region of the user's brain, and their distribution conforms to the international 10-20 system. Specifically, the 9 leads are: Pz, PO5, PO3, POz, PO4, PO6, O1, Oz, and O2. The ground electrode is placed at the AFz electrode point, and the reference electrode is placed at the CPz electrode point. Throughout the entire EEG data acquisition process, the electrode impedance of all electrodes remained below 10kΩ.

[0055] The analysis module extracts features from the EEG signals to identify the visual stimulus target that the user is currently looking at.

[0056] The synchronized trigger signals generated by the stimulation program are recorded on the event channel of an amplifier synchronized with the EEG database. During online program use, EEG data and trigger signals are recorded and analyzed in real time through an online data analysis program.

[0057] To accurately detect steady-state visual evoked potential signals, a filter bank task-related component analysis algorithm was employed. This algorithm combines the detection of the signal's fundamental frequency and harmonic frequency components. Specifically, it includes the following steps:

[0058] First, offline template data acquisition is performed. For offline data, data segments are extracted from continuous EEG recordings based on the event labels at the time of stimulus occurrence. To reduce storage and computation costs, the data is first downsampled (from 1000Hz to 250Hz) and subjected to 50Hz notch filtering to remove power frequency interference. Then, filter bank analysis is performed using an infinite impulse response filter (two sub-bands are used for both low-frequency and high-frequency steady-state visual evoked potential-brain-computer interface systems: 9–90Hz and 20–90Hz bandpass filtering for low-to-mid-frequency systems (e.g., 15Hz); and 25–90Hz and 55–90Hz bandpass filtering for higher-frequency systems (e.g., 30Hz).

[0059] Finally, training templates for the user's response signals to each stimulus are obtained and used in the online program. In the actual use of the online brain-computer interface system, filter bank task-related component analysis is also used for target recognition. During target recognition in the online system, the maximum correlation coefficient between each processed data point and the previously acquired template data is calculated. The target corresponding to the maximum correlation coefficient is the target selected by the user. After receiving feedback after one selection, the user can make the next selection.

[0060] Combination Figure 8 As shown in the figure, this embodiment also provides a brain-computer interface method, which includes the following steps.

[0061] Step s1: Provide multiple visual stimulus targets. Each visual stimulus target includes a display area with adjustable background grayscale and a visual stimulus area formed within the display area. The visual stimulus area does not completely fill the display area. The display area and the visual stimulus area together constitute the visual stimulus code.

[0062] Display area 11 is encoded using background grayscale values. For ease of engineering application, different background grayscale values ​​are used to indicate different contrast levels. Combined with... Figure 3 As shown, in one embodiment, 11 different background grayscale values ​​were designed, ranging from 0 to 1, with a grayscale value interval of 0.1 between each pair of adjacent stimuli.

[0063] Visual stimulus area 12 is encoded with flickering at a specific frequency and phase. Taking the checkerboard-shaped visual stimulus target 10 as an example, combined with... Figure 4The diagram shows a checkerboard stimulus with RGB values ​​of 0:25.5:255, and a spatial frequency of 2.13c / o The left image shows a contrast-reversed checkerboard stimulus against a pure black background (background grayscale value of 0), while the right image shows a checkerboard stimulus against a pure white background (background grayscale value of 1).

[0064] In the steady-state visual evoked potential brain-computer interface method, different stimuli (i.e., flashing novel checkerboard-like stimulus blocks) are presented on the display, with the center of each stimulus block serving as the user's focal point of gaze. During a single recognition process, multiple visual stimulus targets with different background grayscale can be displayed simultaneously on the same screen.

[0065] In one embodiment, combined with Figure 6 As shown, the same interface provides four visual stimulus targets 10. In actual use of the system, the user needs to select one of the four targets for focusing. In each visual stimulus target 10, the dark square window serves as the display area 11, and the white part serves as the visual stimulus area 12. Each visual stimulus target 10 is 300×300 pixels in size and is placed at the top, bottom, left, and right positions on the screen. The top target is 424 pixels vertically from the bottom target, and the left target is 424 pixels horizontally from the right target. The small windows in each visual stimulus area 12 flicker at the same frequency and initial phase.

[0066] Step s2: Collect the electroencephalogram (EEG) signals generated by the user's encoding of the visual stimuli.

[0067] In one embodiment, the acquisition module can use 9 conductive electrodes, combined with Figure 7 As shown, the 9-lead electrodes are placed in the occipital region of the user's brain, and their distribution conforms to the international 10-20 system. Specifically, the 9 leads are: Pz, PO5, PO3, POz, PO4, PO6, O1, Oz, and O2. The ground electrode is placed at the AFz electrode point, and the reference electrode is placed at the CPz electrode point. Throughout the entire EEG data acquisition process, the electrode impedance of all electrodes remained below 10kΩ.

[0068] Step s3: Extract features from the EEG signal and identify the visual stimulus target that the user is currently looking at.

[0069] To accurately detect steady-state visual evoked potential signals, a filter bank task-related component analysis algorithm was employed. This algorithm combines the detection of the signal's fundamental frequency and harmonic frequency components. Specifically, it includes the following steps:

[0070] First, offline template data acquisition is performed. For offline data, data segments are extracted from continuous EEG recordings based on the event labels at the time of stimulus occurrence. To reduce storage and computation costs, the data is first downsampled (from 1000Hz to 250Hz) and subjected to 50Hz notch filtering to remove power frequency interference. Then, filter bank analysis is performed using an infinite impulse response filter (two sub-bands are used for both low-frequency and high-frequency steady-state visual evoked potential-brain-computer interface systems: 9–90Hz and 20–90Hz bandpass filtering for low-to-mid-frequency systems (e.g., 15Hz); and 25–90Hz and 55–90Hz bandpass filtering for higher-frequency systems (e.g., 30Hz).

[0071] Finally, training templates for the user's response signals to each stimulus are obtained and used in the online program. In the actual use of the online brain-computer interface system, filter bank task-related component analysis is also used for target recognition. During target recognition in the online system, the maximum correlation coefficient between each processed data point and the previously acquired template data is calculated. The target corresponding to the maximum correlation coefficient is the target selected by the user. After receiving feedback after one selection, the user can make the next selection.

[0072] In summary, this invention fully utilizes the spatial physical properties of visual stimuli. By altering the background contrast of the stimulus, it induces electroencephalogram (EEG) signals with separable characteristics. Then, using filter bank task-related component analysis (CRB) methods, it extracts corresponding features from the EEG signals to identify the user-selected target, ultimately enabling the encoding of multiple targets using a single frequency and the same initial phase. Compared to traditional frequency and phase encoding methods, the advantage of this invention is that the contrast encoding method can be used on displays with ordinary refresh rates (e.g., a typical 60Hz screen) to implement a high-frequency, multi-target steady-state visual evoked potential (SVP) brain-computer interface system. Furthermore, by combining it with stimulus frequency modulation, a joint frequency-contrast encoding method can be implemented, which is of great significance for the research, application, and promotion of steady-state visual evoked potential (SVP) brain-computer interface systems.

[0073] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A brain-computer interface system, characterized in that, include: The visual stimulation module provides multiple visual stimulation targets. Each visual stimulation target includes a display area with adjustable background grayscale and a visual stimulation area formed within the display area. The visual stimulation area does not completely fill the display area. The display area and the visual stimulation area together constitute the visual stimulation code. The visual stimulation area changes brightness at a certain frequency. The display areas of different visual stimulation targets have different background grayscale values. The visual stimulation areas within different display areas have the same phase and flicker frequency. The grayscale of the display area in each visual stimulation target remains constant when the visual stimulation area flickers. The acquisition module is used to acquire the electroencephalogram (EEG) signals generated by the user's encoding of the visual stimuli; The analysis module extracts features from the EEG signals and identifies the visual stimulus target that the user is currently looking at.

2. The brain-computer interface system as described in claim 1, characterized in that, The visual stimulation area includes multiple display windows spaced apart within the display area.

3. A brain-computer interface method for a brain-computer interface system according to claim 1, characterized in that, include: Multiple visual stimulus targets are provided. Each visual stimulus target includes a display area with adjustable background grayscale and a visual stimulus area formed in the display area. The visual stimulus area does not completely fill the display area. The display area and the visual stimulus area together constitute the visual stimulus code. Collect the electroencephalogram (EEG) signals generated by the user's encoding of the visual stimuli; Features are extracted from EEG signals to identify the visual stimulus target that the user is currently looking at.

4. The brain-computer interface method as described in claim 3, characterized in that, The visual stimulation area includes multiple display windows spaced apart within the display area.

5. The brain-computer interface method as described in claim 4, characterized in that, The display window is a square area, and the spacing between adjacent display windows is the same, and the spacing is equal to the side length of the display window.

6. The brain-computer interface method as described in claim 3, characterized in that, Methods for extracting features from electroencephalogram (EEG) signals to identify the visual stimulus target currently being gazed at by the user include: The collected EEG signals were downsampled; The downsampled signal is then filtered.

7. The brain-computer interface method as described in claim 6, characterized in that, When the flicker frequency of the visual stimulation area is 15Hz, the filtering adopts a low-frequency bandpass filter of 9-90Hz. When the flicker frequency of the visual stimulation area is 30Hz, a high-frequency bandpass filter of 25-90Hz is used.

Images