A spatial positioning system based on steady-state visual evoked potentials

Abstract

The application discloses a kind of space positioning systems based on steady-state visual evoked potential.The stimulation module in the system utilizes the spatial coding strategy of four flicker arrays, the SSVEP signal induced is amplified and collected after signal pretreatment is carried out by electroencephalogram signal collection module, then the amplitude, phase, correlation coefficient of each stimulation frequency component and the ratio of the correlation coefficient corresponding to each coding stimulation frequency or phase are used as characteristic information to decode spatial information to determine the coordinates of visual fixation, finally the coordinate information is converted into instruction and given to controlled operation module, to realize visual feedback.The application can effectively solve the problem that the area of divided region and the number of target in each partition are limited in existing spatial coding, and the edge information of each partition cannot be distinguished, realizes the BCI of low visual load and accurate positioning in screen, and provides the possibility for realizing high-integration embedded BCI system.

Description

Technical Field

[0001] This invention relates to a spatial localization technology based on SSVEP (Steady-State Visual Evoked Potentials), belonging to the field of brain-computer interface technology. Background Technology

[0002] Brain-computer interface (BCI) technology establishes an alternative, direct connection between the brain and external devices, bypassing the brain's normal peripheral neural and muscle output pathways. SSVEP (Sequential Visual Evoked Potential), a type of evoked potential that responds to continuous visual stimuli, is a major type of BCI method. Compared to other types of EEG signals, such as motor imagery and event-related potentials (ERPs), it boasts a shorter reaction time (less than 100ms), closely approximating the communication speed humans are accustomed to. Current BCI research highlights SSVEP's advantages: high signal-to-noise ratio, high information transmission rate, good repeatability, high recognition and classification accuracy, and the low barrier to entry where users only need to passively receive stimuli without training. With continuous breakthroughs in system performance, SSVEP BCI has demonstrated excellent positive feedback effects at the application level.

[0003] Temporal encoding and frequency encoding methods are widely used encoding strategies in visual BCI (Brain-Computer Interface). Temporal encoding uses orthogonal or near-orthogonal time series, evoking a corresponding response within a specific time window when a user gazes at a particular target. Frequency encoding uses different frequency sequences as visual stimuli, and feature extraction and frequency identification can be achieved by analyzing the spectrum of the obtained stimulus frequencies and their harmonics. Both methods have certain limitations in practical human-computer interaction scenarios, mainly including a limited target range, significant challenges in portability development, and high visual load on users, easily causing visual fatigue.

[0004] Spatial coding, another common coding method in communication system theory, is currently less used in the field of brain-computer interfaces (BCIs) but still has room for development. Spatial coding directly establishes a mapping relationship between neural responses and spatial location information, decoding spatial information from the activity topography distribution of EEG on the scalp, with low dependence on temporal or spectral patterns. Its main characteristics are target-stimulus separation, scalable spatial range, and greater flexibility in practical applications. Chen Jingjing et al., in their paper "A Spatially-coded Visual Brain-Computer Interface for Flexible Visual Spatial Information Decoding" published in IEEE Transactions on Neural Systems and Rehabilitation Engineering, implemented the expansion of the spatial coding BCI range and flexible spatial information decoding under various stimulus combinations. However, this method cannot distinguish the edges of each target region and cannot determine the specific location of the gaze within the target region, i.e., it cannot achieve precise or continuous coordinate positioning of the gaze within a small area on the screen. Summary of the Invention

[0005] Technical Problem: The purpose of this invention is to provide a spatial positioning system based on steady-state visual evoked potentials. This system can be used to expand the spatial range and enable the user's gaze point to be located at any position on the interface, thus solving the problem of inaccurate and continuous coordinate positioning in the prior art.

[0006] Technical solution: In order to achieve the above objectives, the present invention provides a spatial localization system based on steady-state visual evoked potentials, comprising a stimulation module, an EEG signal acquisition module, an EEG signal processing module, and a controlled operation module.

[0007] The stimulation module consists of a scintillation block array designed in a specific coding method at the edge of a computer display screen or a controlled area;

[0008] The EEG signal acquisition module acquires steady-state visual evoked potential (SSVEP) response signals generated in the visual cortex by scintillation block array stimulation, which are related to the stimulation frequency or phase, through non-invasive dry electrodes, and transmits these signals to the EEG signal processing module.

[0009] The EEG signal processing module performs signal downsampling, filtering and other preprocessing on the acquired SSVEP signal, and extracts features based on the rhythm assimilation characteristics of SSVEP. It extracts the fundamental frequency and higher harmonic frequency components related to the stimulation frequency or phase in the collected EEG signal, and obtains the correlation coefficient, amplitude and phase corresponding to each stimulation frequency in the user's EEG signal, as well as the ratio of the correlation coefficients corresponding to each coded stimulation frequency or phase. Based on these feature information, it classifies the signal, and then calculates the user's gaze position through interpolation or fitting methods, and transmits the position information to the controlled operation module.

[0010] The controlled operation module is a computer or mobile terminal, which controls the computer or mobile terminal by obtaining the position information of the gaze point.

[0011] The signal transmission between the stimulation module, the EEG signal acquisition module, the EEG signal processing module, and the controlled operation module is wired or wireless, or wireless transmission via Bluetooth, WIFI, radio frequency, or infrared.

[0012] The stimulation module has a scintillation block array, with a user interface in the middle, referred to as the operation interface or feedback interface; when the subject fixates on any fixation point in the user interface, the visual cortex generates an SSVEP signal, which is a neural response corresponding to the spatial location information.

[0013] The position display of the stimulation module is located at the top, bottom, left, and right edges of the screen, or at the top, bottom, left, and right edges of the controlled operation area; its shape is rectangular or elliptical; its encoding method is based on frequency or phase or a combination of both; and it is a sine wave, square wave, or triangular wave.

[0014] The EEG signal acquisition module uses a dry electrode type EEG acquisition system, which mainly includes an electrode headband, an amplifier, and EEG recording software. The electrode headband selects several channels in the occipital region of the brain related to vision. The SSVEP signal is amplified and converted into a digital signal for acquisition. The EEG recording software can view the SSVEP signal in real time and manage the data.

[0015] The EEG signal acquisition module has multiple channel selection combinations, including channels covering the entire cerebral cortex, eight channels in the occipital region of the brain, or even a more streamlined number of channels.

[0016] The EEG signal processing module, as the core of the system, first downsamples the acquired data to reduce the amount of computation and improve the processing speed. Then, it removes external high-frequency noise and power frequency interference, and performs bandpass filtering on the SSVEP signal. By calculating the amplitude, phase, correlation coefficient, and the ratio of the correlation coefficients corresponding to each frequency component and each coded stimulus frequency or phase, it analyzes the differences and connections of spatial feature information to locate the fixation point.

[0017] The EEG signal processing module obtains its feature information by simultaneously extracting the frequency components corresponding to each stimulus in the EEG signal, calculating the amplitude, phase, correlation coefficient, and pairwise ratios of the frequency components, analyzing the differences and connections in neural representations, and using interpolation or fitting methods to calculate the two-dimensional coordinates of the gaze point, forming different commands and position information that are transmitted to the controlled module.

[0018] The working steps of the aforementioned EEG signal processing module are as follows:

[0019] S1, EEG signal downsampling, power frequency filtering, baseline drift removal, bandpass filtering and adaptive filtering;

[0020] S2, extract the ratio of the amplitude, phase and correlation coefficient of each frequency component in the EEG signal to the frequency or phase of each coded stimulus;

[0021] S3, determine the coordinates of the line of sight based on the feature information.

[0022] The controlled operation module obtains the information after decoding the spatial features of the gaze point in the EEG signal processing module, that is, the two-dimensional coordinates of the gaze point determined by interpolation or fitting. Then, it calls the system application programming interface (API) to control the mouse or keyboard and provide visual feedback to the user with the cursor.

[0023] The controlled module can be integrated with the stimulation module in a single display screen, or it can function as a separate external device; that is, the position information of the gaze point can be displayed in real time on the screen of a computer or mobile terminal to provide visual feedback, or it can be converted into control commands for the control of external devices.

[0024] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows:

[0025] This invention utilizes the correlation coefficients, amplitudes, and phases corresponding to various stimulation frequencies in the user's electroencephalogram (EEG) signal, as well as the ratio of the correlation coefficients corresponding to various coded stimulation frequencies or phases, to calculate the two-dimensional coordinates of the user's gaze point through interpolation or fitting methods, and presents them in the form of a cursor.

[0026] This invention decodes EEG signals spatially, incorporating the ratios of frequencies or phases of coded stimuli into feature information. It monitors the user's gaze coordinates in real time, achieving positional recognition based on the continuity of gaze coordinates. This breaks away from classification-based recognition methods that rely on screen area division, providing users with a novel screen interaction mode. It achieves a visually low-load, highly accurate screen-based BCI, better aligning with users' daily screen browsing habits. The reduced number of stimuli also makes the BCI system more portable and universally applicable, resulting in a more natural and comfortable interactive experience and enhancing the user experience. This fully demonstrates its value in practical applications. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a system structure block diagram of the present invention;

[0029] Figure 2 This is an electrode distribution diagram of a 10-20 system that can be selected according to the present invention;

[0030] Figure 3 This is a schematic diagram of the stimulation interface and feature extraction method of the present invention;

[0031] Figure 4 The diagram shows the overall structure of the system described in this invention, which includes: 1. a flashing stimulus for the interactive interface; 2. a user interface; 3. an EEG headband; 4. a user; 5. a wireless EEG signal amplifier; 6. an eight-channel dry electrode; and 7. a reference electrode. Detailed Implementation

[0032] Preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings:

[0033] Example 1:

[0034] See Figure 1This invention relates to a spatial positioning system based on SSVEP, comprising a stimulation module, an EEG signal acquisition module, an EEG signal processing module, and a controlled operation module. The stimulation module consists of a flashing block array designed with a specific encoding method at the edge of a computer display screen or a controlled area. The EEG signal acquisition module non-invasively acquires SSVEP response signals generated in the visual cortex by the flashing array, which are related to the stimulation frequency or phase, and transmits these signals to the EEG signal processing module. The EEG signal processing module preprocesses and extracts features from the acquired SSVEP signals, extracting the fundamental frequency and higher harmonic frequency components related to the stimulation frequency or phase from the collected EEG signals. Based on this feature information, it classifies the signals and calculates the two-dimensional coordinates of the user's gaze using interpolation or fitting methods, transmitting this positional information to the controlled operation module. The controlled operation module is a computer or mobile terminal, which controls the computer or mobile terminal by obtaining the positional information of the gaze point.

[0035] Example 2 is basically the same as Example 1, with the following differences:

[0036] When the optic nerve is stimulated by a fixed frequency of visual signals, the visual cortex of the human brain generates a SSVEP response, which is related to the fundamental frequency and its odd / even harmonic components, in relation to the stimulation frequency or phase. This response is processed by the EEG signal acquisition module and captured by SSVEP-related electrodes in the occipital region. The electrode distribution can include channels covering the entire cerebral cortex, or selectable channels from the occipital region, or even a more streamlined number of channels. The electrode distribution diagram is shown below. Figure 2 As shown, the collected data is transmitted to the EEG signal processing module after passing through an EEG amplifier.

[0037] The EEG signal processing module includes:

[0038] S1: The acquired SSVEP signal is downsampled to reduce computational costs. Since visual stimulation is used during use, the user's gaze will constantly move, and the induced EEG signals are easily interfered with by physiological signals such as electrooculography and electromyography. Therefore, preprocessing operations such as power frequency filtering, baseline drift removal, and bandpass filtering are performed.

[0039] S2: Extract the amplitude, phase, correlation coefficient, and the pairwise ratio of the correlation coefficients corresponding to each coded stimulus frequency or phase in the SSVEP signal as feature information.

[0040] S3: After decoding the spatial information, determine the coordinates of the location of the line of sight based on the neural representation.

[0041] The controlled operation module is a PC or a smart mobile device. By calling the API, when the user gazes at any position on the interface, the cursor will appear accordingly, achieving spatial positioning and providing visual feedback to the user. When the user's gaze point moves continuously on the user interface, the cursor will move continuously following the gaze point based on the coordinate information transmitted in real time, achieving smooth and continuous cursor displacement.

[0042] Example 3: This example is based on Examples 1 and 2, with the following special features:

[0043] See Figure 3 The stimulation module consists of four stimulation blocks located at the top, bottom, left, and right edges of the display screen or at the top, bottom, left, and right edges of the controlled operation area. The flashing frequencies are 15Hz, 16Hz, 17Hz, and 18Hz, commonly used frequencies. Its shape can be rectangular or elliptical; its encoding method can be based on frequency, phase, or a combination of both; it can be a sine wave, square wave, or triangular wave. The user interface within the stimulation module is the area of ​​the user's gaze. When the user gazes at any point within it, due to the different spatial positions of the gaze point relative to the stimulation module, such as... Figure 3 As shown in the bar chart, the four frequency components of the acquired EEG signals differ. Additionally, as... Figure 3 As shown by curves f1 and f2, when the fixation point moves continuously horizontally from the left end of the user interface to the right, the correlation coefficients and amplitudes of the frequency components corresponding to the two stimuli in the horizontal direction monotonically increase and decrease as the horizontal coordinate moves. Furthermore, as the fixation point moves continuously back and forth in the user interface along the horizontal or vertical direction, the correlation coefficients or amplitudes of each frequency component also exhibit a continuous alternating increase or decrease pattern. This regular change reflects the positional information of the fixation point on the horizontal and vertical coordinate axes. Therefore, the multi-channel EEG signals acquired by the EEG acquisition module, after spatial information decoding, yield the amplitude, phase, and correlation coefficient values ​​corresponding to each frequency component. Through experimental data from multiple continuous fixation point movements, the horizontal or vertical coordinates of the fixation point and the correlation coefficient ratios corresponding to the coded stimulus frequencies or phases in that direction can be fitted. The mapping relationship between the two is used to determine the two-dimensional coordinates of the gaze point based on this relationship and feature information. Finally, the API is called to present the gaze point as a cursor in this area as visual feedback information.

[0044] Example 4:

[0045] See Figure 4This SSVEP-based spatial positioning system includes the following steps: When using the system, user 4 needs to wear the EEG headband 3 on their head, place electrodes 6 in the occipital region of the brain, and place two reference electrodes 7 on the mastoid processes on both sides behind the ears, and perform impedance checks, ensuring that the impedance values ​​of all lead channels are below 5kΩ. The blinking block array stimulation 1 of the interactive interface is designed with a specific encoding method. The user only needs to look at the position to be operated on in the user interface 2. The collected EEG signals are processed by the EEG signal amplifier 5 and then connected to the SSVEP spatial decoding unit via wired or wireless means. The wireless transmission method can be Bluetooth, WIFI, radio frequency, or infrared. The spatial coordinates of the user's gaze are converted into corresponding control commands. The control commands can display a cursor on the user interface 2 to achieve visual feedback; they can also control independent external devices.

[0046] This invention utilizes a spatial encoding strategy with four stimulus blocks to effectively address the limitations of time-encoding and frequency-encoding, which result in a limited number of targets and high visual load for users. Furthermore, it addresses the issues in existing spatial encoding methods where the area of ​​the partitioned region and the number of targets in each partition are limited, making it impossible to distinguish the edges and details of each partition. This invention fuses the ratio of correlation coefficients with features such as amplitude, phase, and correlation coefficients corresponding to each stimulus frequency component commonly used in classification and recognition algorithms. By using the amplitude, phase, and correlation coefficients of each stimulus frequency component, as well as the ratio of the correlation coefficients corresponding to each encoded stimulus frequency or phase, as feature information, the coordinates of the fixation point can be determined and fed back in a certain visual form, enabling continuous control of the mouse cursor position.

[0047] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A spatial positioning system based on steady-state visual evoked potentials, characterized in that: The system includes a stimulation module, an EEG signal acquisition module, an EEG signal processing module, and a controlled operation module; The stimulation module consists of a scintillation block array designed in a specific coding method at the edge of a computer display screen or a controlled area; The EEG signal acquisition module acquires steady-state visual evoked potential (SSVEP) response signals generated in the visual cortex by scintillation block array stimulation, which are related to the stimulation frequency or phase, through non-invasive dry electrodes, and transmits these signals to the EEG signal processing module. The EEG signal processing module downsamples and filters the acquired SSVEP signals, and extracts features based on the rhythm assimilation characteristics of SSVEP. It extracts the fundamental frequency and higher harmonic frequency components related to the stimulation frequency or phase in the collected EEG signals, obtains the correlation coefficient, amplitude and phase corresponding to each stimulation frequency in the user's EEG signal, and the ratio of the correlation coefficients corresponding to each coded stimulation frequency or phase. Based on these feature information, it classifies the EEG signals and calculates the user's gaze position using interpolation or fitting methods, and transmits the position information to the controlled operation module. The controlled operation module is a computer or mobile terminal, which controls the computer or mobile terminal by obtaining the position information of the gaze point; The stimulation module has a scintillation block array, with a user interface in the middle, referred to as the operation interface or feedback interface; when the subject fixates on any fixation point in the user interface, the visual cortex will generate an SSVEP signal, which is a neural response corresponding to the spatial location information. The position of the stimulation module is at the top, bottom, left, and right edges of the display screen, or at the top, bottom, left, and right edges of the controlled operation area. Its shape is rectangular or oval; Its encoding method is based on frequency, phase, or a combination of both; it can be a sine wave, square wave, or triangle wave. The EEG signal processing module, as the core of the system, first downsamples the acquired data to reduce the amount of computation and improve the processing speed. Then, it removes external high-frequency noise and power frequency interference, and performs bandpass filtering on the SSVEP signal. By calculating the amplitude, phase, correlation coefficient, and the ratio of the correlation coefficients corresponding to each frequency component and each coded stimulus frequency or phase, it analyzes the differences and connections of spatial feature information to locate the fixation point.

2. The spatial positioning system based on steady-state visual evoked potentials according to claim 1, characterized in that, The signal transmission between the stimulation module, the EEG signal acquisition module, the EEG signal processing module, and the controlled operation module is wired or wireless, or wireless transmission via Bluetooth, WIFI, radio frequency, or infrared.

3. A spatial positioning system based on steady-state visual evoked potentials according to claim 2, characterized in that, The EEG signal acquisition module uses a dry electrode type EEG acquisition system, which mainly includes an electrode headband, an amplifier, and EEG recording software. The electrode headband selects several channels in the occipital region of the brain related to vision. The SSVEP signal is amplified and converted into a digital signal for acquisition. The EEG recording software views the SSVEP signal in real time and manages the data.

4. A spatial positioning system based on steady-state visual evoked potentials according to claim 3, characterized in that, The EEG signal acquisition module has multiple channel selection combinations, including channels covering the entire cerebral cortex, eight channels in the occipital region of the brain, or even a more streamlined number of channels.

5. A spatial positioning system based on steady-state visual evoked potentials according to claim 4, characterized in that, The EEG signal processing module obtains its feature information by simultaneously extracting the frequency components corresponding to each stimulus in the EEG signal, calculating the amplitude, phase, correlation coefficient, and pairwise ratios of the frequency components, analyzing the differences and connections in neural representations, and using interpolation or fitting methods to calculate the two-dimensional coordinates of the gaze point, forming different commands and position information that are transmitted to the controlled module.

6. A spatial positioning system based on steady-state visual evoked potentials according to claim 5, characterized in that, The working steps of the EEG signal processing module are as follows: S1, EEG signal downsampling, power frequency filtering, baseline drift removal, bandpass filtering and adaptive filtering; S2, extract the ratio of the amplitude, phase and correlation coefficient of each frequency component in the EEG signal to the frequency or phase of each coded stimulus; S3, determine the coordinates of the line of sight based on the feature information.

7. A spatial positioning system based on steady-state visual evoked potentials according to claim 6, characterized in that, The controlled operation module obtains the information after decoding the spatial features of the gaze point in the EEG signal processing module, that is, the two-dimensional coordinates of the gaze point determined by interpolation or fitting. Then, it calls the system application programming interface (API) to control the mouse or keyboard and provide visual feedback to the user with the cursor. The controlled operation module can be integrated with the stimulation module in a single display screen, or it can function as a separate external device; that is, the position information of the gaze point can be displayed in real time on the screen of a computer or mobile terminal to provide visual feedback, or it can be converted into control commands for the control of external devices.

Images