SSVEP (Steady-State Visual Evoked Potential)-based brain-computer interface stimulation normal form generation and detection method, system, medium and equipment

Abstract

The invention provides a brain-computer interface stimulation normal form generation and detection method and system based on SSVEP (Steady-State Visual Evoked Potential), a medium and equipment. The method comprises the following steps: setting a plurality of stimulation targets; using parameters of the stimulation target are obtained, different using parameters correspond to different stimulation sizes, and the using parameters comprise one or more combinations of the using frequency, the confusion degree and the personal preference degree; and displaying the stimulation target according to the stimulation size corresponding to the use parameter. According to the SSVEP-based brain-computer interface stimulation normal form generation method and system, the SSVEP-based brain-computer interface stimulation normal form detection method and system, the medium and the equipment, the size of the stimulation normal form can be distributed based on the use frequency, the easy confusion degree and the like, and visual fatigue is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-computer interface (BCI), and in particular to a brain-computer interface stimulation paradigm generation, detection method, system, medium and equipment based on steady-state visually evoked potential (SSVEP). Background Art

[0002] BCI technology can build a pathway between the brain and peripherals. It uses specific EEG signal acquisition equipment to detect neural activity in the brain, analyzes the signal and converts it into control commands to achieve the function of controlling the device. After the device is running, the result is fed back to the human brain to generate new EEG signals, thus achieving interaction between the human brain and the machine.

[0003] BCI technology has applications in various fields. For example, in the medical field, a BCI-based typing system can enable people who are paralyzed or have impaired speech function to communicate with the outside world without relying on their own muscle system. In addition, BCI has broad application prospects in the fields of games and human-computer interaction.

[0004] In terms of detection, electroencephalogram (EEG) is an electrophysiological monitoring method that records brain electrical activity. It usually uses a non-invasive method to place electrodes on the scalp and measure the voltage fluctuations caused by ionic currents in brain neurons through signal acquisition equipment.

[0005] The acquisition of EEG signals by acquisition equipment can be divided into wet electrodes and dry electrodes according to the type of electrodes. For wet electrodes, this method requires applying a layer of conductive paste on the scalp surface of the subject to effectively reduce the scalp resistance and facilitate the acquisition of EEG signals; for dry electrodes, this method places a tentacle-like conductor with good conductivity on the electrode, which can penetrate the hair and increase the contact area between the electrode and the scalp, thereby reducing the resistance at the contact between the scalp and the electrode. Because wet electrodes can provide EEG signals with higher accuracy and higher signal-to-noise ratio, this method is more popular among researchers. However, due to the advantages of dry electrodes in terms of ease of use, many studies are working on optimizing the design of dry electrodes. It can be foreseen that with the development of materials science, the EEG signals of dry electrodes will also be further improved in terms of accuracy and signal-to-noise ratio.

[0006] In general, using EEG signals as detection content has many advantages: first, the operation required to obtain EEG signals is relatively simple, and because most detection equipment uses a non-invasive method to detect EEG signals, it will not cause side effects to the human body; in addition, the equipment required to collect EEG signals is relatively inexpensive, and the process of collecting signals is short and efficient. In addition, the spatial resolution of the collected signals is easy to identify and contains sufficient information, so there is no need to collect additional information.

[0007] SSVEP is a periodic evoked potential caused by rapid repetitive visual stimulation and is a special EEG signal. When visual stimulation is repeated at a certain frequency, such as a flashing light source or a vibrating pattern, this frequency-specific stimulation can produce a potential response in the visual area of ​​the brain that matches the stimulation frequency.

[0008] Regarding the principle of SSVEP, the mainstream view in academia is that the neural networks distributed in the human brain each have their own inherent resonant frequency. Under normal conditions, these neural networks are asynchronous and chaotic, and the EEG signals at this time are spontaneous EEG. Under the influence of external constant-frequency visual stimulation, the neural network consistent with the stimulation frequency or harmonic frequency will resonate, causing the brain's potential activity to change significantly at the stimulation frequency or harmonic frequency, thereby generating SSVEP signals.

[0009] From the perspective of physiological characteristics, different functional areas of the brain have their own division of labor, and the sensory, motor, and cognitive modules of different cortical areas are independent of each other. Figure 1 As shown in the figure, the frontal lobe 1 is used for movement and understanding, the parietal lobe 2 is used for touch and spatial perception, the occipital lobe 3 is used for vision, and the temporal lobe 4 is used for hearing and language. The above functional modules cooperate with each other to form an organic whole. In the process of the brain processing perceptual information, multiple modules work in parallel. The BCI system based on SSVEP signals determines the brain's thinking activities by detecting the EEG signals in the occipital visual area.

[0010] SSVEP consists of many discrete frequency components, usually covering the stimulation frequency range of 3.5Hz to 75Hz. These frequency components include the fundamental wave of visual stimulation and its harmonics. When different stimuli are presented to the subject, the brain will produce different EEG signals. By analyzing these signals, the specific stimulation frequency range can be determined, thereby judging the subject's instructions and using this information to control the device.

[0011] Compared with several other common BCI systems: P300, motor imagery, the advantages of the BCI system based on SSVEP are: the BCI system based on SSVEP has a higher signal-to-noise ratio, recognition accuracy, has a larger number of classifications, is easy to implement a high information transfer rate (ITR) system, and can be used without prior training. Due to the accuracy, speed and reliability of SSVEP, it is widely used in the BCI field to identify and decode the subject's intention and attention, so that humans can manipulate and control external devices by observing visual stimuli, and interact with computers and other intelligent systems.

[0012] However, objectively speaking, the SSVEP-based paradigm still has the following shortcomings:

[0013] (1) Long-term stimulation will cause visual fatigue to the subjects; at the same time, the stimulus sizes in the mainstream stimulation paradigm are the same and arranged regularly, such as Figure 2 As shown, there is no personalized design based on actual application conditions. In actual application scenarios, since not all functions in the system require the same response amplitude, it is impossible to achieve a balance between user experience and system functions.

[0014] (2) SSVEP experiments usually require specialized experimental equipment, such as high-refresh-rate monitors and precise stimulus control systems, which increases the cost and complexity of the experiment. Summary of the invention

[0015] In view of the shortcomings of the prior art mentioned above, the purpose of the present invention is to provide a brain-computer interface stimulation paradigm generation, detection method, system, medium, and equipment based on SSVEP, which can allocate the size of the stimulation paradigm based on frequency of use, degree of confusion, etc., and effectively reduce visual fatigue.

[0016] In a first aspect, the present invention provides a method for generating a brain-computer interface stimulation paradigm based on SSVEP, the method comprising the following steps: setting multiple stimulation targets; obtaining usage parameters of the stimulation targets, different usage parameters corresponding to different stimulation sizes, the usage parameters including one or more combinations of usage frequency, degree of confusion, and personal preference; and displaying the stimulation targets according to the stimulation sizes corresponding to the usage parameters.

[0017] In an implementation manner of the first aspect, the frequencies of the multiple stimulation targets are increased at equal intervals in a sequence from left to right and from top to bottom.

[0018] In an implementation manner of the first aspect, the phases of the multiple stimulation targets are increased at equal intervals in a sequence from left to right and from top to bottom.

[0019] In a second aspect, the present invention provides a method for detecting a brain-computer interface stimulation paradigm based on SSVEP, the method comprising the following steps:

[0020] Generate a SSVEP-based brain-computer interface stimulation paradigm according to the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generation method;

[0021] Collecting SSVEP electroencephalogram signals when the user is looking at the stimulation target;

[0022] Based on the filter bank canonical correlation analysis algorithm, the correlation coefficient of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target is obtained;

[0023] For each stimulus target, the weighted square sum of the correlation coefficients corresponding to each sub-band component is calculated;

[0024] The stimulus target with the largest weighted square sum is selected as the stimulus target that the user is fixated on.

[0025] In an implementation of the second aspect, obtaining the correlation coefficient of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target includes:

[0026] Decomposing the SSVEP electroencephalogram signal into a plurality of sub-bands by using a filter bank based on a filter bank canonical correlation analysis algorithm;

[0027] For each sub-band, a canonical correlation analysis is performed for each stimulus target to obtain the corresponding correlation coefficient.

[0028] In an implementation of the second aspect, according to Calculate the weighted square sum of the correlation coefficients corresponding to each sub-band component, where N represents the number of sub-band components. represents the correlation coefficient between the kth stimulus target and the nth sub-band component, and w(n) represents the weight of the nth sub-band component.

[0029] In an implementation of the second aspect, w(n)=n -a +b,n∈[1,N], where a and b are constants.

[0030] In a third aspect, the present invention provides a brain-computer interface stimulation paradigm detection system based on SSVEP, the system comprising a generation module, a collection module, an acquisition module, a calculation module and a detection module;

[0031] The generating module is used to generate a SSVEP-based brain-computer interface stimulation paradigm according to the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generating method;

[0032] The acquisition module is used to collect the SSVEP electroencephalogram signal when the user looks at the stimulation target;

[0033] The acquisition module is used to acquire the correlation coefficient of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target based on a filter bank canonical correlation analysis algorithm;

[0034] The calculation module is used to calculate the weighted square sum of the correlation coefficients corresponding to each sub-band component for each stimulation target;

[0035] The detection module is used to select the stimulus target with the largest weighted square sum as the stimulus target that the user is fixated on.

[0036] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generation method.

[0037] In a fifth aspect, the present invention provides an electronic device, comprising: a processor and a memory;

[0038] The memory is used to store computer programs;

[0039] The processor is used to execute the computer program stored in the memory so that the electronic device executes the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generation method.

[0040] As described above, the SSVEP-based brain-computer interface stimulation paradigm generation, detection method, system, medium, and device of the present invention have the following beneficial effects:

[0041] (1) The size of the stimulus can be assigned based on a number of factors such as frequency of use, degree of confusion, and personal preference, so that different sized stimuli can be used in the same task;

[0042] (2) Based on the stimulation paradigm, the usability and flexibility of the system are improved, and a balance is achieved between user experience and system functionality. This can reduce the visual fatigue caused by long-term gaze at stimuli while achieving the desired functional effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram showing the functional distribution of brain layers in one embodiment;

[0044] Figure 2 A schematic diagram showing a conventional stimulation paradigm in the prior art in one embodiment;

[0045] Figure 3 Shown is a flow chart of a method for generating a brain-computer interface stimulation paradigm based on SSVEP in one embodiment of the present invention;

[0046] FIG. 4( a ) is a schematic diagram showing a stimulation paradigm of the present invention in one embodiment;

[0047] FIG4( b ) is a schematic diagram showing the frequency phase of the stimulation paradigm of FIG4( a ) in one embodiment;

[0048] FIG4( c ) is a graph showing the amplitude-frequency characteristics of four sizes of the stimulation paradigm of FIG4( a ) in one embodiment;

[0049] FIG5( a ) is a schematic diagram showing a stimulation paradigm of the present invention in another embodiment;

[0050] FIG5( b ) is a schematic diagram showing the frequency phase of the stimulation paradigm of FIG5( a ) in one embodiment;

[0051] Figure 6 FIG5( a ) is a schematic diagram showing the stimulus presentation of a stimulus paradigm in one embodiment;

[0052] Figure 7 Shown is a schematic diagram of a SSVEP-based brain-computer interface stimulation paradigm detection method in one embodiment of the present invention;

[0053] FIG8( a ) is a schematic diagram showing a 64-channel extended international 10-20 system in one embodiment;

[0054] FIG8( b ) is a schematic diagram showing an arrangement of electrodes in an embodiment;

[0055] Fig. 9 Shown is a schematic diagram of SSVEP signal recognition in one embodiment;

[0056] Fig.10 Shown is a schematic structural diagram of a SSVEP-based brain-computer interface stimulation paradigm generation system in one embodiment of the present invention;

[0057] Fig.11 It is a schematic structural diagram of an electronic device of the present invention in one embodiment. DETAILED DESCRIPTION

[0058] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0059] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0060] The following embodiments of the present invention provide a method for generating a brain-computer interface stimulation paradigm based on SSVEP, which can be applied to electronic devices. The electronic devices described in the present invention may include mobile phones, tablet computers, laptop computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc., which have wireless charging functions. The embodiments of the present invention do not impose any restrictions on the specific types of electronic devices.

[0061] For example, the electronic device may be a station (STAION, ST) in a WLAN with a wireless charging function, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with a wireless charging function, a computing device or other processing device, a computer, a laptop computer, a handheld communication device, a handheld computing device, and / or other devices for communicating on a wireless system and a next generation communication system, such as a mobile terminal in a 5G network, a mobile terminal in a future evolved Public Land Mobile Network (PLMN), or a mobile terminal in a future evolved Non-terrestrial Network (NTN).

[0062] For example, the electronic device can communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobilecommunication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS) and / or Satellite Based Augmentation Systems (SBAS).

[0063] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0064] like Figure 3 As shown, in one embodiment, the SSVEP-based brain-computer interface stimulation paradigm generation method of the present invention includes steps S31 to S33.

[0065] Step S31, setting multiple stimulation targets.

[0066] Specifically, in the present invention, multiple stimulation targets are set according to the needs of practical applications. As shown in Figures 4(a) and 5(a), two stimulation paradigms of the typing system based on SSVEP are provided, in which the stimulation targets represent numbers and letters, respectively. It should be noted that in practical applications, the order of letters and symbols can be set according to the needs of the user or the user's usage habits and EEG characteristics. In addition, the operations corresponding to the stimulation targets are not limited to typing, but can also be control instructions for other devices, such as drones, robot cars, robot arms, etc.

[0067] Step S32: obtaining usage parameters of the stimulation target, where different usage parameters correspond to different stimulation sizes, and the usage parameters include one or more combinations of usage frequency, confusion degree, and personal preference.

[0068] Specifically, in the study of stimulation paradigms, researchers' studies on vision have shown that the larger the size of the visual stimulus, the larger the response amplitude generated in the brain; conversely, the smaller the stimulus size, the smaller the response amplitude. Therefore, in order to achieve a balance between user experience and system function, the present invention selects different sizes of stimulation paradigms for different functions. Among them, the size of the stimulation paradigm is associated with the usage parameters of the stimulation target, and a corresponding relationship is established between the two. Preferably, the usage parameters include one or more combinations of frequency of use, degree of confusion, and personal preference.

[0069] For example, for larger stimulus targets, it is suitable for functions that require strong EEG signal response or focus on user perception, making it easier for users to notice and identify target keys. Larger stimulus targets can attract more attention and help users accurately select target keys. Therefore, by assigning larger stimulus targets to commonly used or important function keys, users can locate and select these keys faster, reducing the user's visual search time, thereby improving operational efficiency and reducing the degree of visual fatigue to a certain extent. In addition, some functional instructions are prone to misoperation. For example, taking the typing system as an example: m and n, i and l, the shapes of the letters are similar, which can easily cause recognition errors. By assigning these easily confused keys to stimulus targets of different sizes, the risk of misoperation can be reduced and the reliability of the system can be improved.

[0070] Take the stimulation paradigm in Figure 4(a) as an example. This stimulation paradigm can be used for an EEG typing system with 12 keys. 12 stimulation targets are presented on the display (the actual system can have other numbers). These 12 different stimulation targets are represented in the order from left to right and from top to bottom: 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, a total of 10 numbers, a decimal point "." and the delete key "delete". This stimulation paradigm arranges 4 stimulation sizes. When setting, the largest size is arranged for the three numbers in the first column of "1, 5, 9", so that the user can more easily focus on this column when looking for the keys they need, and can also issue instructions faster, and the accuracy of the instructions will be the highest and most stable; the second column has the second highest priority, and so on. For the two function keys, since they are used less frequently, two small-sized stimulation targets are arranged for them, so that the user's attention will not be too focused on them. When looking at a small stimulus target, the EEG response is relatively weak, which is not easy to cause false touches and is suitable for high-precision control. Because the frequency of use and the demand for precision of "." are greater than those of "delete", the stimulus size of the former is set to be larger than that of the latter.

[0071] Similarly, after changing the size, layout and key settings of the stimulus target, an EEG typing system with 12 keys can be realized. For example, in the stimulus paradigm shown in Figure 5(a), the 12 keys are: a, e, i, o, u, s, h, m, n, g, l, v. This stimulus paradigm sets three stimulus sizes: large, medium and small. The allocation of letters is mainly set according to their frequency of use. For example: considering the high frequency of vowels, the four vowel letters "a, e, i, o" are set to a "large" size, and "u" is a medium size; at the same time, the stimulus paradigm also arranges different sizes for "m" and "n", two shapes that are easily confused.

[0072] When setting the frequency and phase of the stimulation target, such as Figure 4(b) and 5(b) As shown, the setting is increased in equal intervals from left to right and from top to bottom. For example, the stimulation frequency range is: 8Hz-13.5Hz, and the frequency interval between two adjacent stimulation blocks on the left and right is Δf=0.5Hz; 12 characters are marked with equal intervals of frequency, and their increments are proportional to the target index. The frequency value of each character in the matrix is ​​set as follows:

[0073] f(k x ,k y ) = f 0 +Δf×[(k y -1)×3+(k x -1)],

[0074] k x ∈[1,3],ky ∈[1,4]

[0075] Among them, f 0 Indicates the starting frequency, i.e. 8Hz, k x With k y Respectively represent the row index and column index of the stimulus block. In addition, the stimulus phase range is: 0-3.85π, and the phase interval between two adjacent stimulus blocks is 0.35π.

[0076] Step S33, displaying the stimulation target according to the stimulation size corresponding to the usage frequency.

[0077] Specifically, the corresponding stimulation size is set for the usage frequency of each stimulation frequency, so as to achieve stimulation targets of different sizes in the same task, effectively reduce visual fatigue, and improve the recognition accuracy of SSVEP electroencephalogram signals.

[0078] It should be noted that the present invention does not limit the specific distribution, specific size, and size type of the stimulation paradigm. In practical applications, the selection of the size of the stimulation block needs to comprehensively consider the application scenario, user needs, and technical requirements.

[0079] like Figure 7 As shown, in one embodiment, the SSVEP-based brain-computer interface stimulation paradigm detection method of the present invention includes steps S71 to S75.

[0080] Step S71: Generate a SSVEP-based brain-computer interface stimulation paradigm according to the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generation method.

[0081] Step S72: collecting the SSVEP electroencephalogram signal when the user is gazing at the stimulation target.

[0082] Specifically, in the stimulation mode of FIG. 4(a) or FIG. 5(a), all stimulation targets begin to flash, and the user can stare at the control instructions or characters he wants to operate, and the duration of the stimulation can be set by himself. For example, in the typing system shown in FIG. 5(a), the duration of the stimulation is set to 4 seconds. Then, considering that staring at the stimulation for a long time will cause visual fatigue to the user, there will be a 2-second rest period after the stimulation is presented. During this rest period, the typing system will display the following on the screen: Figure 6 The typing box shown displays in real time the typing system's identification results based on the user's SSVEP signal.

[0083] In the present invention, the SSVEP EEG signal is collected using the "EEG Wireless Digital EEG Acquisition System" under the "NeuroHub Wearable Multimodal Research Platform" of "neuracle". The user needs to wear an EEG signal collection cap to collect SSVEP EEG signals in a non-invasive way. At the same time, in order to prevent the problem of high impedance caused by hair, scalp and other debris, the user needs to wash the scalp before use, and inject conductive paste into the electrode channel used after wearing the EEG cap.

[0084] The placement of electrodes follows the 64-channel extended international 10-20 system as shown in Figure 8(a), with a sampling rate of 1000Hz, which is down-sampled to 256Hz after acquisition. The channel selection of the present invention is set in the occipital region based on the research on the distribution of SSVEP on the scalp, and the number of channels is set to 9 based on technical indicators (the number of electrode channels is not more than 12), namely: Pz, POz, Oz, O1, O2, PO3, PO4, PO5, PO6. The schematic diagram of electrode placement is shown in Figure 8(b). After acquisition, the EEG signal is transmitted to the computer via Wi-Fi. This process is achieved through the "synchronization box" in the "EEG wireless digital EEG acquisition system".

[0085] Step S73: Based on the filter bank canonical correlation analysis algorithm, the correlation coefficients of the sub-band components of the SSVEP electroencephalogram signal for each stimulation target are obtained.

[0086] Specifically, Filter Bank Canonical Correlation Analysis (FBCCA) is a feature recognition algorithm that can decompose SSVEP EEG signals into multiple sub-band components, maximize the use and combination of the harmonics of SSVEP EEG signals for analysis, and thus make more effective judgments. This method does not require prior training of users.

[0087] In one embodiment, obtaining the correlation coefficient of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target includes:

[0088] 731) The filter bank based on the filter bank canonical correlation analysis algorithm decomposes the SSVEP electroencephalogram signal into multiple sub-bands.

[0089] like Fig. 9 As shown, the filter bank typical correlation analysis includes filter bank analysis.

[0090] First, multiple filters with different passbands are used to filter the original EEG signal. Perform subband decomposition and obtain SB 1 ,SB 2 SB3 ,…SB N There are N subbands in total, and N=5 in this example.

[0091] 732) For each sub-band, a canonical correlation analysis is performed for each stimulus target to obtain the corresponding correlation coefficient.

[0092] Among them, canonical correlation analysis is a statistical analysis method that studies the relationship between two groups of variables and is used to measure the basic correlation between two multidimensional variables.

[0093] The core idea of ​​CCA is to find the linear combination of two sets of variables and use the correlation between the linear combinations of the two variables to reflect the original relationship between the two. A simple explanation of CCA is: this method first finds a pair of linear combinations with the highest correlation, called typical variables; then, continue to find the second pair of typical variables so that their correlation is the second highest, and so on, until the correlation is extracted, that is, the number of typical variable pairs equal to the number of variables in the shorter data set is extracted.

[0094] In the frequency detection of SSVEP EEG signals, for two sets of variables X and Y, where X represents the multi-channel SSVEP signal, that is, The subscript of channel indicates the number of different transmission channels. Y refers to the sinusoidal signal Y related to the stimulus frequency f. f ,Right now Where f is the stimulation frequency, N h is the number of harmonics, in this example N h =5.

[0095] In order to identify the frequency of SSVEP EEG signals, CCA calculates the typical correlation between SSVEP EEG signals of multiple channels and the reference signal corresponding to each stimulation frequency. The CCA algorithm finds a pair of weight vectors W for the two multidimensional variables X and Y mentioned above. X and W Y , through W X and W Y To make their linear combination x = X T W X and y = Y T W Y Maximize the correlation between Thus, the maximum value of the correlation coefficient ρ between X and Y is obtained, and ρ is calculated by selecting different f. Finally, the stimulation frequency corresponding to the maximum correlation coefficient ρ is determined as the frequency of the SSVEP EEG signal.

[0096] In the present invention, after the filter bank analysis, each sub-band component is processed by standard CCA to obtain the sub-band component and all stimulus frequencies. For the kth reference signal, the correlation vector ρ consisting of N correlation values ​​is k as follows:

[0097]

[0098] Here, ρ(x,y) represents the correlation coefficient between x and y.

[0099] Step S74: For each stimulation target, calculate the weighted square sum of the correlation coefficients corresponding to each sub-band component.

[0100] Specifically, according to Calculate the weighted square sum of the correlation coefficients corresponding to each sub-band component, where N represents the number of sub-band components. represents the correlation coefficient between the kth stimulus target and the nth sub-band component, and w(n) represents the weight of the nth sub-band component. Calculate the features for target recognition.

[0101] In one embodiment, w(n)=n -a +b,n∈[1,N], where a and b are constants. Preferably, w(n)=n -1.25 +0.5.

[0102] Step S75: Select the stimulation target with the largest weighted square sum as the stimulation target that the user is fixated on.

[0103] Specifically, using ρ corresponding to all stimulation frequencies k To determine the frequency of the SSVEP EEG signal, by comparison, the one with the largest ρ k The corresponding stimulation frequency is identified as the response frequency of the SSVEP EEG signal, which will have the largest ρ k The corresponding stimulus target is used as the stimulus target that the user is fixated on.

[0104] For the stimulation paradigm shown in FIG4(a), under the online typing system based on SSVEP, the average recognition accuracy of 6 subjects can achieve an accuracy of 76.39% when the signal sampling time is 4 seconds and the FBCCA algorithm is used for feature recognition. The effect of the scheme is analyzed from the perspective of EEG response as shown in FIG4(c). For the function keys with 4 stimulation sizes set in the same row of the stimulation paradigm shown in FIG4(a), size 1> size 2> size 3> size 4, the amplitude of the EEG signal stimulated by the stimulation of larger size is significantly higher than that of the stimulation of smaller size. The above results show that the stimulation paradigm design of the present invention can be used in actual scenarios, and through the optimization of the algorithm, the recognition accuracy and signal sampling time of the present invention are further improved.

[0105] Smaller stimulus blocks are suitable for functions that require high-precision control. Using these buttons requires higher attention to avoid misoperation. In addition, small-sized stimuli can be assigned to buttons with relatively low frequency of use, making them easier for users to ignore, so that more attention can be allocated to those function buttons with higher priority.

[0106] The protection scope of the SSVEP-based brain-computer interface stimulation paradigm generation method described in the embodiment of the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0107] An embodiment of the present invention also provides a brain-computer interface stimulation paradigm generation system based on SSVEP, which can implement the brain-computer interface stimulation paradigm generation method based on SSVEP described in the present invention. However, the implementation device of the brain-computer interface stimulation paradigm generation system based on SSVEP described in the present invention includes but is not limited to the structure of the brain-computer interface stimulation paradigm generation system based on SSVEP listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0108] like Fig.10 As shown, in one embodiment, the SSVEP-based brain-computer interface stimulation paradigm detection system of the present invention includes a generation module 101, a collection module 102, an acquisition module 103, a calculation module 104 and a detection module 105.

[0109] The generating module 101 is used to generate a SSVEP-based brain-computer interface stimulation paradigm according to the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generating method.

[0110] The acquisition module 102 is connected to the generation module 101 and is used to acquire the SSVEP electroencephalogram signal when the user is gazing at the stimulation target.

[0111] The acquisition module 103 is connected to the acquisition module 102 and is used to obtain the correlation coefficients of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target based on a filter bank canonical correlation analysis algorithm.

[0112] The calculation module 104 is connected to the acquisition module 103 and is used to calculate the weighted square sum of the correlation coefficients corresponding to each sub-band component for each stimulation target.

[0113] The detection module 105 is connected to the calculation module 104 and is used to select a stimulus target with the largest weighted square sum as the stimulus target that the user is fixated on.

[0114] Among them, the structures and principles of the generation module 101, the collection module 102, the acquisition module 103, the calculation module 104 and the detection module 105 correspond one by one to the steps in the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generation method, so they will not be repeated here.

[0115] In the several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0116] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0117] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0118] The embodiment of the present invention also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for generating a brain-computer interface stimulation paradigm based on SSVEP in the above embodiment can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0119] An embodiment of the present invention further provides an electronic device, which includes a processor and a memory.

[0120] The memory is used to store computer programs.

[0121] The memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0122] The processor is connected to the memory and is used to execute the computer program stored in the memory so that the electronic device executes the above-mentioned SSVEP-based brain-computer interface stimulation paradigm generation method.

[0123] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0124] like Fig.11As shown, the electronic device of the present invention is in the form of a general computing device. The components of the electronic device may include but are not limited to: one or more processors or processing units 111, a memory 112, and a bus 113 connecting different system components (including the memory 112 and the processing unit 111).

[0125] Bus 113 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0126] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0127] The memory 112 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1121 and / or cache memory 1122. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 1123 may be used to read and write non-removable, non-volatile magnetic media ( Fig.11 not shown, usually called a "hard drive"). Although Fig.11 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 113 via one or more data medium interfaces. The memory 112 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0128] A program / utility 1124 having a set (at least one) of program modules 11241 may be stored, for example, in the memory 112, such program modules 11241 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 11241 generally perform the functions and / or methods of the embodiments described herein.

[0129] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via input / output (I / O) interface 114. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 115. Fig.11 As shown, the network adapter 115 communicates with other modules of the electronic device via the bus 113. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0130] In summary, the SSVEP-based brain-computer interface stimulation paradigm generation, detection method, system, medium, and device of the present invention can allocate the size of the stimulation paradigm based on a series of factors such as frequency of use, accuracy of use, degree of confusion, personal preference, etc., so that different sizes of stimulation can be used in the same task; based on the stimulation paradigm, the usability and flexibility of the system are improved, and a balance is achieved between user experience and system function, which can reduce the visual fatigue of users caused by long-term gaze at stimulation, and obtain the desired functional effect. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0131] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for generating a brain-computer interface stimulation paradigm based on SSVEP, characterized in that: The method comprises the following steps: Set multiple stimulus goals; Acquiring usage parameters of the stimulation target, where different usage parameters correspond to different stimulation sizes, and the usage parameters include one or more combinations of usage frequency, confusion degree, and personal preference; The stimulation target is displayed according to the stimulation size corresponding to the frequency of use.

2. The method for generating a brain-computer interface stimulation paradigm based on SSVEP according to claim 1, characterized in that: The frequencies of the multiple stimulation targets are increased at equal intervals in a sequence from left to right and from top to bottom.

3. The method for generating a brain-computer interface stimulation paradigm based on SSVEP according to claim 1, characterized in that: The phases of the multiple stimulation targets are increased at equal intervals in a sequence from left to right and from top to bottom.

4. A brain-computer interface stimulation paradigm detection method based on SSVEP, characterized in that: The method The following steps are involved: Generate a SSVEP-based brain-computer interface stimulation paradigm according to the SSVEP-based brain-computer interface stimulation paradigm generation method according to any one of claims 1-3; Collecting SSVEP electroencephalogram signals when the user is looking at the stimulation target; Based on the filter bank canonical correlation analysis algorithm, the correlation coefficient of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target is obtained; For each stimulus target, the weighted square sum of the correlation coefficients corresponding to each sub-band component is calculated; The stimulus target with the largest weighted square sum is selected as the stimulus target that the user is fixated on.

5. The SSVEP-based brain-computer interface stimulation paradigm detection method according to claim 4, characterized in that: Obtaining the correlation coefficients of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target includes: The filter bank based on the filter bank canonical correlation analysis algorithm decomposes the SSVEP electroencephalogram signal into multiple sub-bands; for each sub-band, a canonical correlation analysis is performed for each stimulation target to obtain a corresponding correlation coefficient.

6. The SSVEP-based brain-computer interface stimulation paradigm detection method according to claim 4, characterized in that: according to Calculate the weighted square sum of the correlation coefficients corresponding to each sub-band component, where N represents the number of sub-band components. represents the correlation coefficient between the kth stimulus target and the nth sub-band component, and w(n) represents the weight of the nth sub-band component.

7. The SSVEP-based brain-computer interface stimulation paradigm detection method according to claim 6, characterized in that: w(n)=n -a +b,n∈[1,N], where a and b are constants.

8. A brain-computer interface stimulation paradigm detection system based on SSVEP, characterized in that: The system includes a generation module, a collection module, an acquisition module, a calculation module and a detection module; The generation module is used to generate a SSVEP-based brain-computer interface stimulation paradigm according to the SSVEP-based brain-computer interface stimulation paradigm generation method according to any one of claims 1 to 3; The acquisition module is used to collect the SSVEP electroencephalogram signal when the user looks at the stimulation target; The acquisition module is used to acquire the correlation coefficient of each sub-band component of the SSVEP electroencephalogram signal for each stimulation target based on a filter bank canonical correlation analysis algorithm; The calculation module is used to calculate the weighted square sum of the correlation coefficients corresponding to each sub-band component for each stimulation target; The detection module is used to select the stimulus target with the largest weighted square sum as the stimulus target that the user is fixated on.

9. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the SSVEP-based brain-computer interface stimulation paradigm generation method described in any one of claims 4 to 7 is implemented.

10. An electronic device, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory so that the electronic device executes the SSVEP-based brain-computer interface stimulation paradigm generation method described in any one of claims 4 to 7.

Images