A method, system, device and medium for SSVEP-based brain-computer interface stimulation modulation

Abstract

The application provides a SSVEP-based brain-computer interface stimulation modulation method, system, device and medium, comprising the following steps: acquiring a preset code sequence set, a stimulation frequency set and a stimulation paradigm; the code sequence set comprises a plurality of code sequences; the code sequence comprises a plurality of code elements, and the code element corresponds to a brightness amplitude depth coefficient; the stimulation frequency set comprises a plurality of stimulation frequencies; the stimulation paradigm is used for modulating the brightness amplitude depth and the flicker frequency of visual stimulation based on the code sequence and the stimulation frequency; based on the code sequence and the stimulation frequency, a code stimulation combination of the code sequence and the stimulation frequency is generated; and based on the code stimulation combination, the visual stimulation corresponding to each stimulation block is modulated by using the stimulation paradigm. The SSVEP-based brain-computer interface stimulation modulation method, system, device and medium provided by the application not only expand the number of stimulation targets, but also improve the comfort of users when gazing at the visual stimulation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain-computer interface, and particularly relates to a brain-computer interface stimulation modulation method, system and device based on SSVEP and a medium. BACKGROUND

[0002] Steady-State Visual Evoked Potentials (SSVEP) is a neural response of the brain to periodic visual stimulation. When the visual system receives regular visual stimulation, certain areas of the brain will produce electrical activity corresponding to the visual stimulation (usually in the form of stimulation blocks).

[0003] At present, SSVEP is widely used in the field of Brain Computer Interface (BCI), that is, based on the SSVEP electroencephalogram response signal, the brain instructions of paralyzed patients or patients with motor dysfunction are extracted, and based on the brain instructions, the control of external devices or communication with the outside world is realized, such as: the user gazes at different stimulation blocks on the screen to form electroencephalogram response signals corresponding to each stimulation block, thereby generating corresponding control instructions, and based on different control instructions, external devices such as wheelchairs or prostheses are controlled. With the increasing demand for high communication rate and complex control tasks, the existing BCI technology uses a stimulation paradigm constructed by combining the frequency and phase of the stimulation modulation method, which expands the display form of the stimulation block and improves the coding efficiency; and the existing display form of the stimulation block is usually according to the preset brightness amplitude depth, according to the combination of different stimulation frequencies and phases, that is, the corresponding visual stimulation is presented with different flashing frequencies and times; however, since the brightness amplitude depth of the visual stimulation modulated by the existing stimulation modulation method is a fixed value (usually a high brightness amplitude depth), long-term gazing at such visual stimulation can easily cause visual and mental fatigue of the user, affect the amplitude and signal-to-noise ratio of the electroencephalogram response signal, and reduce the classification accuracy of the electroencephalogram response signal.

[0004] Therefore, how to provide a stimulation modulation method with variable brightness amplitude depth of visual stimulation on the basis of expanding the display form of the visual stimulation, so as to reduce the visual and mental fatigue of the user is an important problem to be solved at present. SUMMARY

[0005] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a brain-computer interface stimulation modulation method based on SSVEP, which is used to solve the problem that the visual stimulation modulated by the existing brain-computer interface stimulation modulation method based on SSVEP maintains a fixed brightness amplitude depth, which easily causes visual and mental fatigue of the user.

[0006] To achieve the above object and other related objects, the present application provides a SSVEP-based brain-computer interface stimulation modulation method, comprising the following steps:

[0007] acquiring a preset code sequence set, a preset stimulation frequency set and a preset stimulation paradigm; the code sequence set comprises a plurality of code sequences; each code sequence comprises a plurality of code elements, each code element corresponding to a luminance amplitude depth coefficient; the stimulation frequency set comprises a plurality of stimulation frequencies; the stimulation paradigm is used to modulate the luminance amplitude depth and the flicker frequency of visual stimulation based on the code sequence and the stimulation frequency; wherein the luminance amplitude depth of the visual stimulation varies with time in a sinusoidal wave;

[0008] generating code stimulation combinations of the code sequence and the stimulation frequency based on the code sequence and the stimulation frequency; each code stimulation combination is a combination of a single code sequence and a single stimulation frequency;

[0009] modulating the visual stimulation corresponding to each stimulation block based on the code stimulation combination and the stimulation paradigm.

[0010] In an embodiment of the present application, the stimulation paradigm is constructed in the following manner:

[0011] The amplitude shift keying method is used to control the amplitude of the sinusoidal wave by using the luminance amplitude depth, and the stimulation frequency is used as the number of periods of the sinusoidal wave in a unit of time, so as to construct the stimulation paradigm and load signals onto the sinusoidal wave.

[0012] In an embodiment of the present application, the code stimulation combinations of the code sequence and the stimulation frequency are generated based on the code sequence and the stimulation frequency, and the method comprises:

[0013] all possible code stimulation combinations between the code sequence and the stimulation frequency are generated according to the code sequence and the stimulation frequency.

[0014] In an embodiment of the present application, the visual stimulation corresponding to each stimulation block is modulated based on the code stimulation combination and the stimulation paradigm, and the method comprises:

[0015] each stimulation block corresponding to each code stimulation combination is generated respectively, and the visual stimulation corresponding to each stimulation block is modulated according to the stimulation paradigm based on the code stimulation combination.

[0016] In an embodiment of the present application, when the code sequence is encoded by using the binary encoding method and the total number of code elements in the code sequence is even, the number of code elements of two types of code elements in the code sequence is equal.

[0017] In an embodiment of the present application, the stimulation blocks with the same stimulation frequency and / or the same encoding sequence are distributed non-adjacently.

[0018] In an embodiment of the present application, after modulating the visual stimulation corresponding to each of the stimulation blocks, the method further comprises:

[0019] presenting each of the stimulation blocks as a corresponding visual stimulation to cause the subject to generate a brain electrical response signal corresponding to a target stimulation block based on the target stimulation block.

[0020] Correspondingly, the present application provides a SSVEP-based brain-computer interface stimulation modulation system, comprising:

[0021] an acquisition module configured to acquire a preset encoding sequence set, a stimulation frequency set, and a stimulation paradigm; the encoding sequence set comprises a plurality of encoding sequences; the encoding sequence comprises a plurality of symbols corresponding to a luminance amplitude depth coefficient; the stimulation frequency set comprises a plurality of stimulation frequencies; the stimulation paradigm is used to modulate the luminance amplitude depth and the flicker frequency of the visual stimulation based on the encoding sequence and the stimulation frequency; wherein the luminance amplitude depth of the visual stimulation varies as a sine wave over time;

[0022] an encoding stimulation combination generation module configured to generate an encoding stimulation combination of the encoding sequence and the stimulation frequency based on the encoding sequence and the stimulation frequency; a single encoding stimulation combination is a combination of a single encoding sequence and a single stimulation frequency;

[0023] a stimulation modulation module configured to modulate the visual stimulation corresponding to each stimulation block based on the encoding stimulation combination and using the stimulation paradigm.

[0024] Correspondingly, the present application provides a computer device, comprising:

[0025] a memory configured to store a computer program;

[0026] a processor configured to execute the computer program stored in the memory, so that the device performs the SSVEP-based brain-computer interface stimulation modulation method as described above.

[0027] Correspondingly, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the SSVEP-based brain-computer interface stimulation modulation method as described above.

[0028] As described above, the present application provides a SSVEP-based brain-computer interface stimulation modulation method, system, device, and medium, which at least has the following beneficial effects:

[0029] By acquiring a preset stimulation paradigm, an encoding sequence set and a stimulation frequency set, determining the number of stimulation blocks based on the number of encoding sequences and the number of stimulation frequencies, and modulating the visual stimulation corresponding to the stimulation block according to the stimulation paradigm based on the encoding sequence and the stimulation frequency, the visual stimulation with the sinusoidal variation of the brightness amplitude depth of the stimulation block is displayed, which not only expands the display form of the stimulation block, but also improves the comfort of the user when gazing at the visual stimulation presented by the stimulation block, and reduces the visual and mental fatigue of the user. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A flowchart of a SSVEP-based brain-computer interface stimulation modulation method provided by the application is shown in an embodiment.

[0031] Figure 2 A stimulation modulation process diagram of a SSVEP-based brain-computer interface stimulation modulation method provided by the application is shown in an embodiment.

[0032] Figure 3 An encoding stimulation combination generation process diagram of a SSVEP-based brain-computer interface stimulation modulation method provided by the application is shown in an embodiment.

[0033] Figure 4 A flowchart of a SSVEP-based brain-computer interface stimulation modulation method provided by the application is shown in another embodiment.

[0034] Figure 5 A stimulation layout diagram of a SSVEP-based brain-computer interface stimulation modulation method provided by the application is shown in an embodiment.

[0035] Figure 6 An experimental effect comparison diagram of a SSVEP-based brain-computer interface stimulation modulation method provided by the application is shown in an embodiment.

[0036] Figure 7 A module structure diagram of a SSVEP-based brain-computer interface stimulation modulation system provided by the application is shown in an embodiment.

[0037] Figure 8 A structural diagram of a computer device provided by the application is shown.

[0038] REFERENCE SIGNS

[0039] S1-S4, step; 200, brain-computer interface stimulation modulation system; 201, acquisition module; 202, encoding stimulation combination generation module; 203, stimulation modulation module; 300, computer device; 301, memory; 302, processor. DETAILED DESCRIPTION

[0040] The above and other advantages and effects of the present application will become readily apparent to those of ordinary skill in the art from the description set forth below and the accompanying drawings. The present application can be applied or implemented in other different embodiments, and the details of the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0041] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the diagrams, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change, and the component layout may be more complex.

[0042] In order to facilitate the understanding of the technical solutions provided by the present application, the related terms in the present application are explained before the specific embodiments, as follows:

[0043] Amplitude keying mode: a modulation mode that controls the amplitude change of the carrier by baseband digital signal, which represents the digital signal by changing the amplitude of the carrier signal.

[0044] The embodiments of the present application will be described in detail below with reference to the accompanying drawings. The features in the following embodiments and implementation modes can be combined with each other without conflict.

[0045] The following embodiments of the present application provide a SSVEP-based brain-computer interface stimulation modulation method, which acquires a stimulation paradigm, an encoding sequence set and a stimulation frequency set, determines the number of stimulation blocks based on the number of encoding sequences and the number of stimulation frequencies, and modulates the visual stimulation with sinusoidal variation of brightness amplitude depth corresponding to the stimulation block according to the stimulation paradigm based on the encoding sequence and the stimulation frequency. The method not only expands the display form of the stimulation block, but also improves the comfort of the user when gazing at the visual stimulation presented by the stimulation block, and reduces the visual and mental fatigue of the user.

[0046] Please refer to Figure 1 , which shows a flowchart of a SSVEP-based brain-computer interface stimulation modulation method provided by the present application in an embodiment.

[0047] As shown in Figure 1 , in the present embodiment, the SSVEP-based brain-computer interface stimulation modulation method provided by the present application includes the following steps:

[0048] Step S1, acquiring a preset encoding sequence set, a stimulation frequency set and a stimulation paradigm;

[0049] The encoding sequence set comprises a plurality of encoding sequences; the encoding sequence comprises a plurality of symbols corresponding to luminance amplitude depth coefficients, used to regulate the luminance amplitude depth of the visual stimulus; the luminance amplitude depth represents the degree of luminance variation of the visual stimulus.

[0050] The stimulation frequency set comprises a plurality of stimulation frequencies; the stimulation frequencies are used to regulate the flicker frequency of the visual stimulus;

[0051] The stimulation paradigm is used to modulate the luminance amplitude depth and flicker frequency of the visual stimulus based on the encoding sequence and the stimulation frequency; wherein the luminance amplitude depth of the visual stimulus varies as a sinusoidal wave over time.

[0052] Optionally, the stimulation paradigm is constructed in the following manner:

[0053] An amplitude shift keying method is used to control the amplitude of the sinusoidal wave using the luminance amplitude depth, and the stimulation frequency is used as the number of cycles of the sinusoidal wave per unit time to construct the stimulation paradigm to load signals onto the sinusoidal wave.

[0054] Exemplarily, the stimulation paradigm is as follows:

[0055]

[0056] In the above formula, S represents the current gray value of the stimulation block; the sin() function is used to generate a sinusoidal wave sequence; n represents the frame sequence of the stimulation block; f represents the stimulation frequency; k represents the luminance amplitude depth coefficient; r represents the refresh rate of the stimulation display device; wherein, represents the ratio between the frame sequence and the refresh rate of the stimulation display device, i.e., the time for which the frame is displayed on the stimulation display device.

[0057] Optionally, the luminance amplitude depth coefficient takes a value of 1 or 0.5.

[0058] Optionally, the refresh rate of the stimulation display device is 60 Hz.

[0059] Optionally, the symbol width of each symbol in the encoding sequence is 1 second; the symbol width is the corresponding continuous flickering time of the symbol.

[0060] Optionally, each symbol in the encoding sequence is encoded based on a preset luminance amplitude depth coefficient sequence using a binary encoding method; the symbol corresponds to the luminance amplitude depth coefficient in the stimulation paradigm, so as to regulate the luminance amplitude depth of the visual stimulus corresponding to the stimulation block based on the luminance amplitude depth coefficient corresponding to each symbol; the stimulation block is used to present the visual stimulus.

[0061] In a specific embodiment, when the encoding sequence is encoded by using a binary encoding method, and the total number of symbols of the encoding sequence is even, the number of symbols of two types of binary encoding in the encoding sequence is set to be equal, that is, the number of symbols corresponding to 0 in the binary encoding is set to be equal to the number of symbols corresponding to 1, so as to balance the overall brightness level of the visual stimulus presented by the stimulation block, to improve the visual comfort of the subject, reduce the pressure on the eyes of the subject caused by the visual stimulus, and reduce visual fatigue; and the brightness balance helps to reduce visual interference, improve the distinguishability of the visual stimulus, make the response of the brain to the visual stimulus clear, thereby improving the signal-to-noise ratio of the electroencephalogram response signal and the accuracy of classification of the electroencephalogram response signal.

[0062] Optionally, the method for obtaining the stimulation frequency comprises:

[0063] obtaining a preset lower limit of frequency, an upper limit of frequency, and a total number of frequencies; constructing a frequency sequence based on the lower limit of frequency, the upper limit of frequency, and the total number of frequencies; the frequency sequence contains the same number of stimulation frequencies as the total number of frequencies.

[0064] Optionally, the frequency sequence is an arithmetic sequence, so that the frequency intervals between different visual stimuli are uniform, the interference between frequencies is reduced, the distinguishability of each stimulation frequency in the frequency domain is improved, and the classification accuracy of the electroencephalogram response signals corresponding to different visual stimuli is improved.

[0065] Illustratively, the method for constructing the frequency sequence comprises:

[0066] obtaining a frequency difference value, which is the difference between the upper limit of frequency and the lower limit of frequency, based on the lower limit of frequency and the upper limit of frequency; obtaining a subtraction times by subtracting one from the total number of frequencies; obtaining a tolerance, which is the ratio of the frequency difference value to the subtraction times, based on the frequency difference value and the subtraction times; obtaining each stimulation frequency based on the lower limit of frequency, the upper limit of frequency, and the tolerance; wherein the stimulation frequency is greater than or equal to the lower limit of frequency and less than or equal to the upper limit of frequency.

[0067] Step S2, generating encoding stimulus combinations of the encoding sequence and the stimulation frequency based on the encoding sequence and the stimulation frequency;

[0068] wherein each encoding stimulus combination is a combination of a single encoding sequence and a single stimulation frequency.

[0069] Specifically, all possible encoding stimulus combinations between the encoding sequence and the stimulation frequency are generated according to the encoding sequence and the stimulation frequency.

[0070] Step S3, modulating the visual stimulus corresponding to each of the stimulation blocks based on the coded stimulation combination and using the stimulation paradigm;

[0071] The stimulation block is used to present a visual stimulus. The stimulation block and the coded stimulation combination have a one-to-one correspondence.

[0072] Specifically, based on each of the coded stimulation combinations, a corresponding stimulation block is generated. Based on the coded stimulation combination, the visual stimulus corresponding to each of the stimulation blocks is modulated according to the stimulation paradigm.

[0073] In an embodiment, based on the coded stimulation combination and according to the stimulation paradigm, the visual stimulus corresponding to each of the stimulation blocks is modulated. When modulating the visual stimulus corresponding to a single stimulation block, it includes:

[0074] According to the stimulation frequency in the coded stimulation combination, the flicker frequency of the visual stimulus corresponding to the stimulation block is determined. Based on the coded sequence in the coded stimulation combination, the luminance amplitude depth of the visual stimulus presented by the stimulation block under the corresponding symbol is determined according to the correspondence between the symbol and the luminance amplitude depth coefficient.

[0075] For example, when the stimulation block presents the corresponding visual stimulus, the luminance amplitude depth of the visual stimulus corresponding to the stimulation block is controlled according to each symbol in the coded sequence in sequence according to the time sequence and the preset symbol width, and the visual stimulus flickers at the stimulation frequency during the presentation.

[0076] According to the stimulation frequency in the coded stimulation combination, the preset symbol width, and the visual stimulus flickering at the stimulation frequency,

[0077] Each of the coded stimulation combinations is respectively assigned to the corresponding stimulation block, so that the stimulation block respectively corresponds to one of the coded stimulation combinations. The stimulation paradigm is obtained. Based on the coded stimulation combination and according to the stimulation paradigm, the visual stimulus corresponding to each of the stimulation blocks is modulated.

[0078] Optionally, the stimulation blocks with the same stimulation frequency and / or the same coded sequence are distributed non-adjacent to each other. In this embodiment, by setting the non-direct adjacent distribution of the stimulation blocks with the same stimulation frequency and / or the same coded sequence, the interference between adjacent visual stimuli is reduced, and the discrimination degree of the electroencephalogram response signal is improved.

[0079] For example, as Figure 2As shown, the encoding sequence is encoded by using a binary encoding method, and the symbol width of each symbol is 1 second, when the luminance amplitude depth coefficient is 1, it corresponds to binary symbol 1, at this time, the gray value range of the stimulation block is [0, 1]; when the luminance amplitude depth coefficient is 0.5, it corresponds to binary symbol 0, at this time, the gray value range of the stimulation block is [0.25, 0.75].

[0080] As shown in the example, Figure 3 As shown, when the encoding sequence is encoded by using a binary encoding method, and the total number of symbols in the encoding sequence is 4, the total number of symbol 0 and symbol 1 in each encoding sequence is set to 2, the encoding sequence includes: '1010', '0101', '1001', '0110', '1100' and '0011', in which Figure 3 The encoding sequence is sequentially identified by a, b, c, d, e, and f; the stimulation frequency includes: 10.0Hz, 10.8Hz, 11.6Hz, 12.4Hz, 13.2Hz and 14.0Hz, which are sequentially identified by 1, 2, 3, 4, 5, and 6; based on the encoding sequence and the stimulation frequency, 36 encoding stimulation combinations are generated as shown in the table on the right side, and based on the encoding stimulation combination, the visual stimulation of 36 stimulation blocks is modulated. Figure 3 The encoding sequence is sequentially identified by a, b, c, d, e, and f; the stimulation frequency includes: 10.0Hz, 10.8Hz, 11.6Hz, 12.4Hz, 13.2Hz and 14.0Hz, which are sequentially identified by 1, 2, 3, 4, 5, and 6; based on the encoding sequence and the stimulation frequency, 36 encoding stimulation combinations are generated as shown in the table on the right side, and based on the encoding stimulation combination, the visual stimulation of 36 stimulation blocks is modulated.

[0081] It should be noted that the change of the luminance amplitude depth of the visual stimulation will affect the change of the SSVEP brain electrical response signal amplitude, that is, the luminance amplitude depth of the visual stimulation can be used to modulate the amplitude of the SSVEP brain electrical response signal, and the visual stimulation with changing luminance can effectively reduce the visual and mental fatigue of the user, therefore, the dynamic changing luminance amplitude depth stimulation mode is adopted in the present application, which not only can expand the number of stimulation targets, but also can improve the user experience.

[0082] Please refer to Figure 4 , which shows a flowchart of a SSVEP-based brain-computer interface stimulation modulation method provided by the present application in another embodiment;

[0083] As shown in the example, Figure 4 As shown in the example,

[0084] Step S4, presenting each of the stimulation blocks as corresponding visual stimulation, so that the subject generates a brain electrical response signal corresponding to the target stimulation block based on the target stimulation block.

[0085] Specifically, based on the coding stimulus combination, the target stimulus block corresponding to the coding stimulus combination is controlled according to the stimulation paradigm, and the corresponding visual stimulus is presented for the subject to fixate the target stimulus block, so as to generate the electroencephalogram response signal corresponding to the target stimulus block.

[0086] To verify the performance of the SSVEP-based brain-computer interface stimulation modulation method provided in the present application in the target stimulus identification problem and the comfort of the visual stimulus modulated thereby, the following experiment is performed:

[0087] In the experiment, the subject is 60 cm away from the stimulus display device, the electroencephalogram response signal acquisition device is a Neuroscan SynAmps2 64-256 channel electroencephalogram amplifier, which is used to acquire electroencephalogram data of the parietal lobe and occipital lobe of the subject, the sampling rate is 1000 Hz, the reference electrode is located in the central region, the grounding electrode is located in the frontal lobe region, and the impedance of each electrode is less than 10 kΩ; the coding sequence used in the experiment includes '1010', '0101', '1001', '0110', '1100' and '0011'; the stimulation frequency includes 10.0 Hz, 10.8 Hz, 11.6 Hz, 12.4 Hz, 13.2 Hz and 14.0 Hz; the above coding sequence and the above stimulation frequency form 36 coding stimulus combinations, as shown in Figure 5 The 36 coding stimulus combinations correspond one-to-one to the 36 stimulus blocks.

[0088] In the experiment, 5 groups of experimental data are collected, each group of experiment including a target prompt stage and a target fixation stage. In the target prompt stage (with a duration of 1 second), the display screen displays a target position prompt point, and the subject moves the fixation angle to the target position prompt point in the target prompt stage. After the target prompt stage ends, the target fixation stage (with a duration of 4 seconds) starts automatically, and the display screen simultaneously displays 36 stimulus blocks, each stimulus block presenting its corresponding visual stimulus. In the target fixation stage, the subject fixates the target stimulus block corresponding to the target position prompt point, and the acquisition device collects the electroencephalogram response signal generated by the subject when fixating the visual stimulus presented by the target stimulus block. After the target fixation stage ends, the subject rests for 5 minutes, and after the rest ends, the next group of experiment is started.

[0089] In the experiment, the presentation time of the stimulus block corresponding to each encoding stimulus combination is 3s, a total of 6 repetitions, a total presentation time of 18s, 8 subjects are collected for measurement scale and electroencephalogram response signal data, and the comfort and decoding performance of different stimulation paradigms are discussed through offline analysis. The comfort and the degree of induced visual fatigue of the two visual stimulation paradigms are evaluated through comparative analysis of subjective comfort and visual fatigue scale, and the performance of the stimulation paradigm is evaluated based on the electroencephalogram response signal data.

[0090] After collecting the electroencephalogram response signal of the subject, the electroencephalogram response signal is preprocessed; the preprocessing includes: downsampling the electroencephalogram response signal from 1000Hz to 250Hz; using a 50Hz notch filter to process the downsampled electroencephalogram response signal to exclude power line interference, and removing the baseline of the electroencephalogram response signal to eliminate signal fluctuations; using a digital bandpass filter of 0.1Hz to 90Hz to filter the electroencephalogram response signal after the above processing; using a task-related component analysis algorithm to classify the preprocessed electroencephalogram response signal data, including:

[0091] Assuming that the electroencephalogram response signal of the lth test is Where N c represents the number of channels, N s represents the number of sampling points, and N t represents the number of tests for each visual stimulus, then the linear combination of the lth test is as follows:

[0092] Y l =w T X l

[0093] In the above formula, represents a spatial filter, which is the optimization target of the related task analysis algorithm.

[0094] The optimal spatial filter is obtained by maximizing the covariance of all possible test combinations, and the calculation method of the covariance of each test combination is as follows:

[0095]

[0096] In the above formula, represents the signal of the ith channel in the l1th test; represents the signal of the jth channel in the l2th test; S represents the covariance matrix between different tests.

[0097] Let represent the connection matrix of all tests, and normalize the variance of signal Y l to 1, as follows:

[0098]

[0099] In the above formula, j1 and j2 represent the index corresponding to the acquisition of the brain response signal channel; Q represents the normalized constraint covariance matrix between different experiments; the above-mentioned restricted optimization problem is represented by the Rayleigh-Ritz method as follows:

[0100]

[0101] As can be seen from the above, the spatial filter can be obtained by Q -1 The eigenvector corresponding to the maximum eigenvalue of the S matrix is obtained.

[0102] The brain response signal data is divided into training data and test data; the brain response signal in the training data is averaged to obtain an individual template And by giving the optimal spatial filter, the correlation coefficient between the brain response signal and the individual template of the nth visual stimulus is calculated, as shown below:

[0103]

[0104] In the above formula, p represents the correlation coefficient; X represents the brain response signal in the test data; w n represents the nth spatial filter.

[0105] Using filter bank analysis, the signal is decomposed into sub-band components to effectively extract harmonics, and the recognition formula of the target visual stimulus is as follows:

[0106]

[0107] In the above formula, N k represents the total number of target visual stimuli; N m represents the number of sub-bands; a(m) represents the weight function.

[0108] In this experiment, the brain response signal data of ten subjects was collected, and the brain response signal formed by the visual stimulus modulated based on the brain-computer interface stimulus modulation method provided in the application was obtained on the brain response signal data of the ten subjects. The average classification accuracy of 4s is 81.7±12.6%; and

[0109] In this experiment, the same stimulus frequency and stimulus layout were used to evaluate the user experience of the stimulus paradigm and the standard stimulus paradigm provided in the application, and the evaluation results are as follows:

[0110] As Figure 6As shown, compared with the standard stimulus paradigm of fixed brightness amplitude depth, the stimulus paradigm provided by the application significantly improves the comfort of the user (p<0.01); and significantly reduces the flicker perception (p<0.05).

[0111] The SSVEP-based brain-computer interface stimulation modulation method provided in the above embodiment determines the number of stimulation blocks based on the number of encoding sequences and the number of stimulation frequencies, and modulates the visual stimulation corresponding to the stimulation blocks according to the stimulation paradigm based on the encoding sequences and the stimulation frequencies, so that the brightness amplitude depth of the visual stimulation changes in a sinusoidal wave. The method not only expands the display form of the stimulation blocks, but also improves the comfort of the user when gazing at the visual stimulation presented by the stimulation blocks, and reduces the visual and mental fatigue of the user.

[0112] As Figure 7 shown, in the present embodiment, the application provides an SSVEP-based brain-computer interface stimulation modulation system, comprising:

[0113] The stimulation block generation module 201 is configured to obtain a preset encoding sequence set, a stimulation frequency set and a stimulation paradigm; the encoding sequence set comprises a plurality of encoding sequences; the encoding sequence comprises a plurality of symbols, the symbols correspond to brightness amplitude depth coefficients; the stimulation frequency set comprises a plurality of stimulation frequencies; the stimulation paradigm is used to modulate the brightness amplitude depth and the flicker frequency of the visual stimulation based on the encoding sequence and the stimulation frequency; wherein the brightness amplitude depth of the visual stimulation changes in a sinusoidal wave over time;

[0114] The encoding stimulation combination generation module 202 is configured to generate an encoding stimulation combination of the encoding sequence and the stimulation frequency based on the encoding sequence and the stimulation frequency; a single encoding stimulation combination is a combination of a single encoding sequence and a single stimulation frequency;

[0115] The stimulation modulation module 203 is configured to modulate the visual stimulation corresponding to each stimulation block based on the encoding stimulation combination and the stimulation paradigm.

[0116] Please refer to Figure 8 , which shows the structural schematic diagram of a computer device provided by the application in an embodiment;

[0117] As Figure 8As shown, in the embodiment, the computer device 300 provided by the present application comprises a memory 301 and a processor 302, the memory 301 is used to store a computer program; the processor 302 is used to execute the computer program stored in the memory 301, so that the computer device 300 executes the SSVEP-based brain-computer interface stimulation modulation method of any of the above-mentioned embodiments. Since the specific implementation process of the steps of the SSVEP-based brain-computer interface stimulation modulation method has been described in detail in the above-mentioned embodiments, it will not be repeated here.

[0118] The memory 301 comprises a ROM (Read Only Memory image), a RAM (Random Access Memory), a magnetic disk, a U disk, a memory card or an optical disk and the like various media which can store program codes.

[0119] The processor 302 is connected with the memory 301, and is used to execute the computer program stored in the memory 301, so that the computer device 300 executes the above-mentioned SSVEP-based brain-computer interface stimulation modulation method.

[0120] The embodiment of the present application also provides a computer readable storage medium. Those skilled in the art can understand that all or part of the steps of the method of the above-mentioned embodiments can be completed by a program instructing a processor, and the program can be stored in a computer readable storage medium. The storage medium is a non-transitory medium, for example, a random access memory, a read only memory, a flash memory, a hard disk, a solid state disk, a magnetic tape, a floppy disk, an optical disc and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center and the like integrated with one or more available medium sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (digital video disc, DVD)) or a semiconductor medium (for example, a solid state disk (solid state disk, SSD)) and the like.

[0121] The embodiments of the present application can also provide a computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer or data center to another website, computer or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.).

[0122] The computer program product is executed by a computer, and the computer executes the method described in the foregoing method embodiments. The computer program product can be a software installation package, and when the foregoing method needs to be used, the computer program product can be downloaded and executed on the computer.

[0123] The description of the processes or structures corresponding to the above respective figures has different focuses, and the parts not described in detail in a certain process or structure can be referred to the related description of other processes or structures.

[0124] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A SSVEP-based brain-computer interface stimulation modulation method, characterized in that, The method comprises: acquiring a preset code sequence set, a preset stimulation frequency set and a preset stimulation mode; the code sequence set comprises a plurality of code sequences; the code sequence comprises a plurality of code elements corresponding to luminance amplitude depth coefficients; the stimulation frequency set comprises a plurality of stimulation frequencies; the stimulation mode is used to modulate the luminance amplitude depth and the flicker frequency of visual stimulation based on the code sequence and the stimulation frequency; wherein the luminance amplitude depth of the visual stimulation varies in a sinusoidal wave over time, and the stimulation mode is constructed in a manner comprising: using an amplitude shift keying mode to control the amplitude of the sinusoidal wave by using the luminance amplitude depth, and taking the stimulation frequency as the number of cycles of the sinusoidal wave in a unit time, to construct the stimulation mode to load a signal onto the sinusoidal wave, wherein the amplitude shift keying mode is a modulation mode for controlling the amplitude variation of a carrier wave by a baseband digital signal, and the amplitude of the carrier signal is changed to represent the digital signal; generating code stimulation combinations of the code sequence and the stimulation frequency based on the code sequence and the stimulation frequency; each code stimulation combination is a combination of a single code sequence and a single stimulation frequency; modulating the visual stimulation corresponding to each stimulation block based on the code stimulation combination and the stimulation mode.

2. The method of claim 1, wherein, The generating of the code stimulation combinations of the code sequence and the stimulation frequency based on the code sequence and the stimulation frequency comprises: generating all possible code stimulation combinations between the code sequence and the stimulation frequency according to the code sequence and the stimulation frequency.

3. The method of claim 1, wherein, The modulating of the visual stimulation corresponding to each stimulation block based on the code stimulation combination and the stimulation mode comprises: generating the corresponding stimulation block based on each code stimulation combination; and modulating the visual stimulation corresponding to each stimulation block according to the stimulation mode based on the code stimulation combination.

4. The method of claim 1, wherein, When the code sequence is encoded by using a binary encoding method, and the total number of code elements in the code sequence is even, the number of code elements of two types of code elements in the code sequence is equal.

5. The method of claim 1, wherein, The stimulation blocks with the same stimulation frequency and / or the same code sequence are distributed non-adjacently.

6. The method of claim 1, wherein, After the modulating of the visual stimulation corresponding to each stimulation block, the method further comprises: presenting each stimulation block as the corresponding visual stimulation to make the subject generate an electroencephalogram response signal corresponding to the target stimulation block based on a target stimulation block.

7. A SSVEP-based brain-computer interface stimulation modulation system, characterized in that, comprises: an acquisition module, configured to acquire a preset code sequence set, a preset stimulation frequency set and a preset stimulation mode; the code sequence set comprises a plurality of code sequences; the code sequence comprises a plurality of code elements corresponding to luminance amplitude depth coefficients; The stimulation frequency set comprises a plurality of stimulation frequencies; the stimulation paradigm is used to modulate the luminance amplitude depth and the flicker frequency of the visual stimulus based on the coding sequence and the stimulation frequency; wherein the luminance amplitude depth of the visual stimulus varies with time in a sinusoidal wave, and the stimulation paradigm is constructed in the following way: using an amplitude shift keying method to control the amplitude of the sinusoidal wave by using the luminance amplitude depth, and using the stimulation frequency as the number of cycles of the sinusoidal wave in a unit of time, to construct the stimulation paradigm to load the signal onto the sinusoidal wave, wherein the amplitude shift keying method is a modulation method that controls the amplitude variation of the carrier by using the baseband digital signal, and the amplitude of the carrier signal is changed to represent the digital signal. The coding stimulation combination generation module is configured to generate a coding stimulation combination of the coding sequence and the stimulation frequency based on the coding sequence and the stimulation frequency; and each coding stimulation combination is a combination of a single coding sequence and a single stimulation frequency. The stimulation modulation module is configured to modulate the visual stimulus corresponding to each stimulation block based on the coding stimulation combination and using the stimulation paradigm.

8. A computer apparatus, comprising: The device comprises: a memory for storing a computer program; a processor for executing the computer program stored in the memory, so that the device executes the method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed to implement the method in any one of claims 1 to 6.

Images