Large instruction set visual evoked potential brain-computer interface coding and decoding method, device and system

Abstract

The invention discloses a large instruction set visual evoked potential brain-computer interface coding and decoding method, device and system. The large instruction set visual evoked potential brain-computer interface coding method comprises the steps that a plurality of instruction blocks distributed in rows and columns are displayed on a visual stimulation interface, each instruction block is jointly coded by two frequencies, and the row and column positions of each instruction block correspond to the two coding frequencies of the instruction block. The large instruction set visual evoked potential brain-computer interface decoding method comprises the following steps: modeling each row instruction block and each column instruction block to obtain a row identification model and a column identification model; and respectively inputting the visual evoked potential Xtest to be identified into the row identification model and the column identification model to obtain a row number and a column number of the visual evoked potential to be identified. According to the technical scheme provided by the embodiment of the invention, the identification model capable of accurately identifying the user instruction is established by fully utilizing a small amount of visual evoked potential for calibration of the user, so that more convenient and high-performance brain-computer interaction of a large instruction set is realized.

Description

Technical Field

[0001] The embodiments of the present invention relate to brain-computer interface signal acquisition and encoding and decoding technology, and in particular to a large instruction set visual evoked potential brain-computer interface encoding and decoding method, device and system. Background Art

[0002] With the in-depth study of electroencephalogram (EEG) by scientific researchers, non-invasive brain-computer interface (BCI) based on brain waves has developed rapidly in recent years, establishing a communication channel between the brain and the computer in a non-invasive way. In particular, brain-computer interface based on visually evoked potential (VEP) has attracted more attention due to its high information transmission rate and decoding reliability. Steady-State Visual Evoked Potentials (SSVEP) BCI paradigm has become one of the mainstream paradigms of BCI because of its advantages such as stable induced characteristics, high signal-to-noise ratio and high information transmission rate. In recent years, the information transmission rate of high-speed SSVEP-BCI can reach more than 300 bits / min, which is one of the preferred solutions for achieving efficient brain-computer interaction. The principle of SSVEP-BCI is that when a user looks at visual stimulation with a certain frequency and duration, components consistent with the stimulation frequency or its higher harmonic frequency appear in the EEG signal. The user's intention is judged by detecting and classifying the SSVEP components in the EEG signal.

[0003] In the prior art, the SSVEP-BCI stimulation interface consists of multiple instruction blocks arranged in a matrix, each of which represents an instruction. Each instruction block flashes at a specific frequency and phase to present visual stimulation to the user. The computer decodes the EEG after the user looks at the visual stimulation and completes the operation corresponding to the instruction represented by the visual stimulation. In the case of a large number of instruction blocks, the SSVEP-BCI encoding and decoding method in the prior art has the problems of long calibration time, low decoding accuracy and low information transmission rate. Summary of the invention

[0004] The present invention provides a large instruction set visual evoked potential brain-computer interface encoding and decoding method, device and system to realize a technical solution in which multiple instruction blocks distributed in rows and columns are displayed on a visual stimulation interface, each instruction block is jointly encoded by two frequencies, and the row and column positions of each instruction block correspond to the two encoding frequencies of the instruction block, thereby achieving the technical effects of shortening the modeling calibration time, improving the decoding accuracy, and increasing the information transmission rate.

[0005] In a first aspect, an embodiment of the present invention provides a large instruction set visual evoked potential brain-computer interface encoding method, comprising:

[0006] A plurality of instruction blocks distributed in rows and columns are displayed on a visual stimulation interface, each instruction block is encoded by two frequencies, and the row and column positions of each instruction block correspond to the two encoding frequencies of the instruction block.

[0007] In a second aspect, an embodiment of the present invention further provides a large instruction set visual evoked potential brain-computer interface decoding method, corresponding to the large instruction set visual evoked potential brain-computer interface encoding method described in the first aspect, the large instruction set visual evoked potential brain-computer interface decoding method comprising:

[0008] Model each row instruction block and each column instruction block respectively to obtain a row identification model and a column identification model;

[0009] The visual evoked potential X test The row identification model and the column identification model are input respectively to obtain the row number and column number of the visual evoked potential to be identified.

[0010] In a third aspect, an embodiment of the present invention further provides a large instruction set visual evoked potential brain-computer interface encoding device, comprising:

[0011] The display module is used to display multiple instruction blocks distributed in rows and columns on a visual stimulation interface, each instruction block is jointly encoded by two frequencies, and the row and column position of each instruction block corresponds to the two encoding frequencies and phases of the instruction block.

[0012] In a fourth aspect, an embodiment of the present invention further provides a large instruction set visual evoked potential brain-computer interface decoding device, corresponding to the large instruction set visual evoked potential brain-computer interface encoding device described in the third aspect, comprising:

[0013] The recognition model acquisition module is used to model each row instruction block and each column instruction block respectively to obtain a row recognition model and a column recognition model;

[0014] The number acquisition module 902 is used to obtain the visual evoked potential X to be identified. test The row identification model and the column identification model are input respectively to obtain the row number and column number of the visual evoked potential to be identified.

[0015] In the fifth aspect, an embodiment of the present invention further provides a large instruction set visual evoked potential brain-computer interface system, comprising the large instruction set visual evoked potential brain-computer interface encoding device described in the third aspect and the large instruction set visual evoked potential brain-computer interface decoding device described in the fourth aspect.

[0016] The technical solution of the embodiment of the present invention is that each instruction block is designed as dual-frequency encoding, a larger number of instruction blocks are obtained by using fewer stimulation frequencies, and a large instruction set visual evoked potential brain-computer interface encoding method and a corresponding decoding method are proposed. By making full use of the user's relatively small amount of calibration visual evoked potential, a recognition model that can accurately recognize user instructions is established, thereby achieving a more convenient and high-performance large instruction set brain-computer interaction. The encoding and decoding method proposed by the present invention can significantly reduce the time required for calibration while ensuring high recognition performance, thereby improving the practicality of the large instruction set visual evoked potential brain-computer interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the principle of signal acquisition of visual evoked potential brain-computer interface (VEP-BCI);

[0018] Figure 2 The coding diagram of SSVEP-BCI is a spectrum dense coding technology;

[0019] Figure 3 A schematic diagram of a visual stimulation interface corresponding to the large instruction set visual evoked potential brain-computer interface encoding method according to Embodiment 1 of the present invention;

[0020] Figure 4 Schematic diagram of the instruction block of the checkerboard structure and the left and right field of view structure;

[0021] Figure 5 A flowchart of a large instruction set visual evoked potential brain-computer interface decoding method provided in Embodiment 2 of the present invention;

[0022] Figure 6 A flowchart of the "decode by target" decoding strategy;

[0023] Figure 7 It is a flowchart of the "decoding by row and column" decoding strategy;

[0024] Figure 8 This is a comparison chart of the accuracy of "decoding by target" and "decoding by rows and columns";

[0025] Fig. 9 A schematic diagram of the structure of a large instruction set visual evoked potential brain-computer interface decoding device provided in Embodiment 4 of the present invention;

[0026] Fig.10 This is a schematic diagram of the structure of a large instruction set visual evoked potential brain-computer interface system provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0028] The main purpose of brain-computer interface (BCI) research is to create non-muscle communication channels so that people with severe motor impairments or other healthy people in need can use this technology to communicate with and control computers. Visual evoked potential-based brain-computer interface (VEP-BCI) modulates EEG activity through visual channels, and then decodes the user's intentions by decoding specific modulation information in the EEG.

[0029] Figure 1 This is a schematic diagram of the principle of signal acquisition for visual evoked potential brain-computer interface (VEP-BCI). As shown in the figure, VEP-BCI signal acquisition and processing mainly includes three stages: stimulus presentation, data acquisition, and data processing and analysis. In the stimulus presentation stage, in order to make the visual stimulus well controllable, a traditional computer screen or a customized light-emitting diode (LED) and other visual stimulation interfaces are usually used. Figure 1 The eight circular icons displayed on the computer screen are eight different visual coding units, i.e., instruction blocks. When the user looks at different instruction blocks, different EEG signals can be induced at the scalp. During the data collection phase, the non-invasive brain-computer interface on the user's head collects scalp EEG. Figure 1 As shown in the frequency domain EEG signals in , different instruction blocks generate different frequency domain EEG signals. For example, instruction blocks flashing at different frequencies induce different base frequencies and multiple frequency peaks of frequency domain EEG signals. The data processing and analysis stage analyzes the instruction block that the user is looking at and outputs the user's intention corresponding to the instruction block.

[0030] VEP-BCI modulates the user's EEG signals through the visual channel, and then decodes the EEG signals to achieve intention decoding. In order to achieve the encoding and decoding of multiple instruction block information and avoid mutual interference between different instruction block information, it is necessary to make the EEG signals induced by different instruction blocks orthogonal or nearly orthogonal to each other.

[0031] In the prior art, the number of instruction blocks of the visual evoked potential brain-computer interface (VEP-BCI) is usually less than 40, and less than 40 instruction blocks are displayed on the same visual stimulation interface for the user to select by fixation, that is, the user can issue up to 40 control instructions through VEP-BCI. The more instruction blocks on the same visual stimulation interface, the greater the range of choices for the user, thereby achieving a more convenient brain-computer interaction experience. However, the existing large instruction set (large number of instruction blocks) visual evoked potential brain-computer interface encoding and decoding methods are limited by the limited stimulation frequency bandwidth, the limited number of instruction blocks, and have the disadvantages of low decoding accuracy and long calibration time. It is difficult to balance system performance and user convenience, resulting in poor practicality of large instruction set visual evoked potential brain-computer interface.

[0032] In the prior art, there are few technical solutions for visual evoked potential brain-computer interface with a large instruction set having hundreds of instruction blocks. A solution that is closer to the present invention is SSVEP-BCI using spectral dense coding technology. Figure 2 Figure 1 is a schematic diagram of the coding of SSVEP-BCI, a spectrum-dense coding technology. Figure 2 As shown, in this scheme, a single instruction block is encoded by a single frequency with a minimum frequency interval of 0.1 Hz and decoded using the Task-Related Component Analysis (TRCA) algorithm. Six rounds of calibration data need to be collected and the calibration time is about 23.8 minutes.

[0033] The shortcomings of the prior art mainly include the following three points:

[0034] 1. The calibration time is too long: The calibration time refers to the time required to collect calibration visual evoked potentials to train the brain-computer interface system before using the brain-computer interface system to identify the visual evoked potentials to be identified. Due to the large individual differences and cross-time variability of EEG signals, in order to ensure the accuracy of EEG signal decoding, it is necessary to collect enough calibration visual evoked potentials before each use of the brain-computer interface to identify the visual evoked potentials to be identified, so as to train a brain-computer interface system for a specific user to ensure that the brain-computer interface system can efficiently and accurately decode the user's intentions when it is officially used. Since the brain-computer interface system requires sufficient training data when there are more instruction blocks, the length of the required calibration time increases accordingly. During the calibration process of SSVEP-BCI, the user is required to stare at the flickering visual stimulation. Long-term calibration will cause visual fatigue in the user, resulting in damage to the quality of the calibration visual evoked potential signal.

[0035] 2. Low decoding accuracy: BCI systems with a large number of instruction blocks are more difficult to decode, so the recognition accuracy of large instruction set visual evoked potential BCI systems is generally lower than that of 40 instruction blocks. Some existing large instruction set visual evoked potential BCI systems have low decoding accuracy while shortening the calibration time, which is not enough to reach a practical level.

[0036] 3. Low information transmission rate: Information Translation Rate (ITR) is an important performance indicator of BCI, which is related to the number of instruction blocks, decoding accuracy and instruction output time. Instruction output time refers to the time required to identify a single instruction (intention) of the user. Compared with Motor Imagery (MI) BCI and P300-BCI, SSVEP-BCI has a higher information transmission rate. The problem with the prior art is that due to insufficient decoding accuracy or long instruction output time, as the number of instruction blocks of the large instruction set visual evoked potential BCI increases, its information transmission rate decreases significantly.

[0037] In summary, the three indicators of calibration time, decoding accuracy and information transmission rate restrict each other. If the calibration time is reduced, the decoding accuracy will decrease. If the decoding accuracy is ensured by increasing the instruction output time, the information transmission rate will decrease.

[0038] The present invention designs each instruction block as dual-frequency encoding, uses fewer stimulation frequencies to obtain more instruction blocks, proposes a large instruction set visual evoked potential brain-computer interface encoding method and a corresponding decoding method, and establishes a recognition model that can accurately recognize user instructions by making full use of the user's relatively small amount of calibration visual evoked potential, thereby achieving a more convenient and high-performance large instruction set brain-computer interaction. The encoding and decoding method proposed by the present invention can significantly reduce the time required for calibration while ensuring high recognition performance, and improve the practicality of the large instruction set visual evoked potential brain-computer interface.

[0039] Embodiment 1

[0040] A large instruction set visual evoked potential brain-computer interface coding method, this embodiment can be applied to the situation where the instruction block needs to be reasonably displayed on the visual stimulation interface. The method can be executed by a large instruction set visual evoked potential brain-computer interface coding device, and specifically includes the following steps:

[0041] A plurality of instruction blocks distributed in rows and columns are displayed on a visual stimulation interface, each instruction block is encoded by two frequencies, and the row and column positions of each instruction block correspond to the two encoding frequencies of the instruction block.

[0042] Figure 3This is a schematic diagram of the visual stimulation interface corresponding to the large instruction set visual evoked potential brain-computer interface encoding method described in Example 1 of the present invention. Figure 3 The visual stimulation interface corresponding to the SSVEP-BCI encoding method of 120 instruction blocks has 8 rows × 15 columns of instruction blocks, each of which represents an optional instruction and is encoded by two frequencies. Figure 3 The coding method shown in FIG. 1 adopts a frequency-phase joint coding method, so that each coding frequency has a specific initial phase, which is used to increase the difference between adjacent frequency stimulation signals. Figure 3 As shown, the row and column where the instruction block is located can be characterized by its encoding frequency and phase. All instruction blocks in the same row contain the same frequency components, and all instruction blocks in the same column also contain the same frequency components. For example, 14.4Hz, 14.6Hz, 14.8Hz, 15.0Hz, 15.2Hz, 15.4Hz, 15.6Hz, and 15.8Hz encode the 1st to 8th rows respectively, and 8.0Hz, 8.3Hz, 8.6Hz, 8.9Hz, 9.2Hz, 9.5Hz, 9.8Hz, 10.1Hz, 10.4Hz, 10.7Hz, 11.0Hz, 11.3Hz, 11.6Hz, 11.9Hz, and 12.2Hz encode the 1st to 15th columns respectively. If the EEG signal of a certain trial is decoded and it is found that it contains two frequency components of 14.4Hz and 12.2Hz, it can be determined that the user is looking at the instruction block in the 1st row and the 15th column at this time.

[0043] Optionally, the instruction block may adopt a checkerboard structure or a left-right view structure. Figure 4 Schematic diagram of the instruction block of the chessboard structure and the left and right field of view structure. Figure 4 As shown in the figure, the two frequencies encoding the same instruction block are represented by black and gray respectively. The black and gray parts flicker at a specific frequency and initial phase respectively. The flickering brightness s(t) is related to time t, frequency f, initial phase The relationship is as follows:

[0044]

[0045] in, Figure 4 The left side is a checkerboard structure, and the right side is a left-right field of view structure. The command block of the checkerboard structure is divided into multiple small squares, one group of small squares flashes at the f1 frequency, and the other group flashes at the f2 frequency. The small squares in the same group are not adjacent, so the two groups of small squares form a checkerboard structure as a whole. The command block of the left-right field of view structure is divided into the left field of view part and the right field of view part. The left field of view part flashes at the f1 frequency, and the right field of view part flashes at the f2 frequency.

[0046] The technical solution of this embodiment takes an 8-row × 15-column instruction block as an example. 120 (8×15) instructions can be implemented through 23 (8+15) frequencies, that is, a large instruction set is encoded through a small number of frequencies. Compared with spectrum-intensive encoding, the intervals between different frequencies are larger, which is more conducive to frequency recognition.

[0047] The technical solution of this embodiment, by displaying multiple instruction blocks distributed in rows and columns on a visual stimulation interface, each instruction block is jointly encoded by two frequencies, and the row and column positions of each instruction block correspond to the two encoding frequencies of the instruction block; solves the technical problem of low decoding accuracy when the number of instruction blocks is large. This technical problem is solved because the intervals between different frequencies of the technical solution of this embodiment are larger, which makes it easier to identify and decode; and achieves the technical effect of improving decoding accuracy.

[0048] In this embodiment, the instruction block can adopt a checkerboard structure or a left-right field of view structure. The advantage of this arrangement is that two instruction block design schemes are provided, and a suitable instruction block structure can be selected according to actual conditions and needs.

[0049] Embodiment 2

[0050] Figure 5 A flowchart of a large instruction set visual evoked potential brain-computer interface decoding method provided in Example 2 of the present invention, which can be implemented by a large instruction set visual evoked potential brain-computer interface decoding device. Example 2 corresponds to the large instruction set visual evoked potential brain-computer interface encoding method described in Example 1. After encoding the instruction block displayed on the visual stimulation interface by the encoding method described in Example 1, the EEG signal generated by the user looking at the visual stimulation interface is collected, and the EEG signal is decoded. The large instruction set visual evoked potential brain-computer interface decoding method specifically includes:

[0051] S510 , model each row instruction block and each column instruction block respectively to obtain a row identification model and a column identification model.

[0052] S520, the visual evoked potential to be identified X test The row identification model and the column identification model are input respectively to obtain the row number and column number of the visual evoked potential to be identified.

[0053] Different from the decoding strategy of "decoding by target" in the prior art, the "decoding by rows and columns" corresponding to the encoding method of the first embodiment is adopted in the second embodiment of the present application. "Decoding by target" in the prior art means that the decoding algorithm needs to model each instruction block. As the number of instruction blocks increases, the workload of modeling increases significantly. As the number of instruction blocks increases, spectrally dense coding will also increase the difficulty of modeling and reduce the decoding accuracy. And for each instruction block modeling, it is necessary to collect 5 to 10 calibration visual evoked potentials, so "decoding by target" requires a long calibration time. "Decoding by rows and columns" is a decoding strategy, not a decoding algorithm. This decoding strategy can be combined with any decoding algorithm, such as Task Related Component Analysis (TRCA), Canonical Correlation Aaronnalysis (CCA) and other commonly used algorithms for SSVEP decoding.

[0054] Figure 6 The figure is a flowchart of the "decode by target" decoding strategy. Figure 6 As shown in the figure, the decoding strategy of "target decoding" includes the modeling process and the recognition process. Taking the TRCA algorithm as an example, TRCA is an algorithm based on spatial filtering and template matching. It calculates the correlation coefficient between the data to be recognized after spatial filtering and the template data, and outputs the instruction block with the largest correlation coefficient with the data to be recognized as the recognition result. Collect all calibration data from a user to Indicates, where m represents the number of optional instructions, n represents the number of trials, and N s Represents the number of sampling points, N c represents the number of EEG channels, u is the instruction block number, c is the EEG channel number, and h is the trial number.

[0055] Figure 6 Instruction block template Take the average of multiple trials of the same instruction block, such as the template data of instruction block u The calculation method is as follows:

[0056]

[0057] The purpose of spatial filtering in the TRCA algorithm is to extract task-related components. The spatial filter W corresponding to the u-th instruction block u Calculated from the calibration data, k is the channel number, as follows:

[0058]

[0059]

[0060] Where the operator Cov(a, b) means to calculate the covariance between vectors a and b, S is the inter-trial covariance matrix, Q is the intra-trial covariance matrix, and solving w maximizes the inter-trial repeatability and minimizes the intra-trial variability, so that the spatial filter w can extract the task-related components, w T Qw=1 is the constraint condition for solving w.

[0061] In the recognition process, the correlation coefficient between the data to be recognized and each instruction block template is calculated, that is, the test data of a single trial is and template data After processing, multiply it by the spatial filter w u , and then calculate the correlation coefficient between them:

[0062]

[0063] Figure 6 The correlation coefficient r is used directly in u The recognition result is calculated. In order to improve the recognition accuracy, the r u Calculated ρ u , according to ρ u Further calculations are performed to obtain the recognition result. That is, if the above spatial filtering process uses a filter bank, the above calculations are performed on each sub-band of the test data, and then the obtained correlation coefficients are weighted and summed according to specific weights.

[0064]

[0065] Where f is the sub-band number, N f is the total number of sub-bands, and the sub-band weight is set to a(f)=f -1.25 +0.25. The recognition result τ can be obtained by the following formula:

[0066]

[0067] The following focuses on the decoding strategy of "decoding by row and column". Optionally, S510 models each row instruction block and each column instruction block separately, including: modeling each row instruction block and each column instruction block separately through a task-related component analysis TRCA algorithm.

[0068] Figure 7 FIG. 2 is a flow chart of the "decoding by row and column" decoding strategy, which is the key content of the second embodiment of the present invention. Figure 7 As shown in Figure 1, the decoding strategy of "decoding by row and column" includes the modeling process and the recognition process. Taking the TRCA algorithm as an example, all calibration data from a user are collected. Where n represents the number of trials, m represents the number of optional instructions, and N sRepresents the number of sampling points, N c represents the number of EEG channels, i represents the row number of the instruction block, j represents the column number of the instruction block, h is the trial number, (i, j) is equivalent to the instruction block number u, and N i Represents the total number of instruction blocks, N j Represents the total number of columns in the instruction block. For convenience, Figure 7 The example of n=1 is used for explanation.

[0069] Optionally, the step of modeling each row instruction block and each column instruction block to obtain a row recognition model and a column recognition model includes: modeling each row instruction block and each column instruction block by a task-related component analysis (TRCA) algorithm to obtain an i-th row template. and the jth column template and the i-th row of spatial filters and the jth column spatial filter

[0070] Template for row i Template for column j The calculation method is:

[0071]

[0072] The i-th row of spatial filters and the jth column spatial filter The calculation method of is similar to the spatial filter calculation method of "decoding by target", but all calibration data of the row / column are used when calculating the row / column covariance matrix, such as the covariance matrix of the i-th row The calculation method is as follows:

[0073]

[0074] Referring to formula (4), we can get the i-th row spatial filter The jth spatial filter

[0075] Optionally, the visual evoked potential X to be identified test Input the row recognition model and column recognition model respectively to obtain the visual evoked potential X to be recognized test The row and column numbers of test , the i-th row template and the i-th row of spatial filters Calculate the visual evoked potential X to be identified test With the i-th row template The row correlation coefficient between According to the visual evoked potential X to be identified test , j-th column template and the jth spatial filter Calculate the visual evoked potential X to be identified test With the jth column template The column correlation coefficient between According to N i Row correlation coefficient and N j Column correlation coefficient Get the visual evoked potential X to be identified test The row and column numbers of i is the total number of instruction blocks, N j is the total number of columns in the instruction block.

[0076] Calculate the visual evoked potential to be identified after filtering by spatial filter Correlation coefficient with each row / column template:

[0077]

[0078] Figure 7 The correlation coefficient r is used directly in i The recognition result is calculated. In order to improve the recognition accuracy, the r i Calculated according to Further calculations are performed to obtain the recognition result. That is, if the above spatial filtering process uses a filter bank, the above calculations are performed on each sub-band of the test data, and then the obtained correlation coefficients are weighted and summed according to specific weights. Referring to the idea of ​​formula (6), weighted summation is performed on r to obtain the row correlation coefficient and column correlation coefficient

[0079] "Decoding by row and column" is equivalent to performing spatial filtering and template matching on the row and column respectively, and outputting two recognition results, namely, the visual evoked potential to be recognized X test The row and column numbers:

[0080]

[0081] According to equations (14) and (15), the visual evoked potential X to be identified can be located test The row and column number.

[0082] The technical solution of this embodiment is to obtain a row recognition model and a column recognition model by modeling each row instruction block and each column instruction block respectively; testThe row recognition model and the column recognition model are input respectively to obtain the row number and column number of the visual evoked potential to be identified; an operation scheme of "decoding by row and column" is provided, which corresponds to the large instruction set visual evoked potential brain-computer interface encoding method described in Example 1; and the technical problems of long modeling and calibration time, low recognition and decoding accuracy, and low information transmission rate in the prior art are solved.

[0083] This embodiment can significantly reduce the calibration time by making full use of the calibration visual evoked potential through a specific decoding method and encoding method. Relatively accurate decoding can be achieved by collecting only one calibration visual evoked potential for each instruction block, and the modeling calibration time can be shortened to less than 5 minutes. In view of the problem of a large number of instruction blocks, this embodiment proposes a "decoding by row and column" method. The number of recognition models required is only the number of rows / columns. The increase in the number of instruction blocks will not cause a significant increase in the number of recognition models, and all calibration recognition evoked potentials in the same row / column can be used for modeling of the row / column. At least only one calibration recognition evoked potential needs to be collected for use, which greatly reduces the demand for calibration visual evoked potentials, reduces the difficulty of decoding, and can achieve more accurate decoding when there are fewer calibration visual evoked potentials. The method of "decoding by row and column" based on SSVEP proposed in this embodiment can achieve an information transmission rate of more than 300 bits / min when the decoding is accurate, thereby meeting the needs of practical applications.

[0084] Figure 8 This is a comparison chart of the accuracy of "decoding by target" and "decoding by row and column". Figure 8 As shown in the figure, when using the ensemble task-related component analysis (ensemble TRCA, eTRCA) algorithm or the ensemble canonical correlated component analysis (ensemble CCA, eCCA) algorithm, the decoding accuracy of the "decoding by rows and columns" method is higher than that of the "decoding by targets" method. The advantage of the "decoding by rows and columns" algorithm is more obvious when the number of trials is small.

[0085] Embodiment 3

[0086] Figure 1 The display on the left is a schematic diagram of the structure of a large instruction set visual evoked potential brain-computer interface encoding device provided in Embodiment 3 of the present invention. The large instruction set visual evoked potential brain-computer interface encoding device can execute the large instruction set visual evoked potential brain-computer interface encoding method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0087] A large instruction set visual evoked potential brain-computer interface encoding device, characterized by comprising:

[0088] The display module is used to display multiple instruction blocks distributed in rows and columns on a visual stimulation interface, each instruction block is jointly encoded by two frequencies, and the row and column position of each instruction block corresponds to the two encoding frequencies and phases of the instruction block.

[0089] Optionally, the instruction block adopts a checkerboard structure.

[0090] Optionally, the instruction block adopts a left and right field of view structure.

[0091] The technical solution of this embodiment, by displaying multiple instruction blocks distributed in rows and columns on a visual stimulation interface, each instruction block is jointly encoded by two frequencies, and the row and column positions of each instruction block correspond to the two encoding frequencies of the instruction block; solves the technical problem of low decoding accuracy when the number of instruction blocks is large. This technical problem is solved because the intervals between different frequencies of the technical solution of this embodiment are larger, which makes it easier to identify and decode; and achieves the technical effect of improving decoding accuracy.

[0092] Embodiment 4

[0093] Fig. 9 This is a schematic diagram of the structure of a large instruction set visual evoked potential brain-computer interface decoding device provided in Embodiment 4 of the present invention. The large instruction set visual evoked potential brain-computer interface decoding device can execute the large instruction set visual evoked potential brain-computer interface decoding method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0094] A large instruction set visual evoked potential brain-computer interface decoding device, corresponding to the large instruction set visual evoked potential brain-computer interface encoding device described in Example 3, comprising:

[0095] The recognition model acquisition module 901 is used to model each row instruction block and each column instruction block respectively to obtain a row recognition model and a column recognition model;

[0096] The number acquisition module 902 is used to obtain the visual evoked potential X to be identified. test The row identification model and the column identification model are input respectively to obtain the row number and column number of the visual evoked potential to be identified.

[0097] Optionally, an identification model acquisition module (901) is used to model each row instruction block and each column instruction block respectively through a task-related component analysis TRCA algorithm to obtain the i-th row template and the jth column template and the i-th row of spatial filters and the jth column spatial filter

[0098] Optionally, a number acquisition module (902) is used to obtain a number according to the visual evoked potential X to be identified. test , the i-th row template and the i-th row of spatial filters Calculate the visual evoked potential X to be identified test With the i-th row template The row correlation coefficient between According to the visual evoked potential X to be identified test , j-th column template and the jth spatial filter Calculate the visual evoked potential X to be identified test With the jth column template The column correlation coefficient between According to N i Row correlation coefficient and N j Column correlation coefficient Get the visual evoked potential X to be identified test The row and column numbers of i is the total number of instruction blocks, N j is the total number of columns in the instruction block.

[0099] Embodiment 5

[0100] Fig.10 This is a structural schematic diagram of a large instruction set visual evoked potential brain-computer interface system provided in Example 4 of the present invention. The large instruction set visual evoked potential brain-computer interface system can execute the large instruction set visual evoked potential brain-computer interface encoding method or the large instruction set visual evoked potential brain-computer interface decoding method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0101] The system includes the large instruction set visual evoked potential brain-computer interface encoding device described in Example 3 and the large instruction set visual evoked potential brain-computer interface decoding device described in Example 4. Fig.10 As shown, the large instruction set visual evoked potential brain-computer interface system may specifically include a processor 1010, a memory 1020, an input device 1030, and an output device 1040; the number of processors 1010 in the device may be one or more, Fig.10 A processor 1010 is taken as an example; the processor 1010, the memory 1020, the input device 1030 and the output device 1040 in the device can be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0102] The memory 1020, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the large instruction set visual evoked potential brain-computer interface encoding method or the large instruction set visual evoked potential brain-computer interface decoding method in the embodiments of the present invention (for example, the recognition model acquisition module 901 and the number acquisition module 902). The processor 1010 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 1020, that is, realizing the above-mentioned large instruction set visual evoked potential brain-computer interface encoding method or large instruction set visual evoked potential brain-computer interface decoding method.

[0103] The memory 1020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 1020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 1020 may further include a memory remotely arranged relative to the processor 1010, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0104] The input device 1030 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 1040 may include a display device such as a display screen.

[0105] Embodiment 6

[0106] Embodiment 6 of the present invention also provides a computer-readable storage medium containing a computer program, which, when executed by a processor, is used to execute a large instruction set visual evoked potential brain-computer interface encoding method or a large instruction set visual evoked potential brain-computer interface decoding method.

[0107] Of course, the computer-readable storage medium containing a computer program provided by an embodiment of the present invention is not limited to the method operations described above, and can also execute related operations in the large instruction set visual evoked potential brain-computer interface encoding method or the large instruction set visual evoked potential brain-computer interface decoding method provided by any embodiment of the present invention.

[0108] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0109] It is worth noting that in the above-mentioned embodiment of the head-mounted display, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0110] Although the various embodiments of the present invention are mostly described using SSVEP-BCI as an example, the corresponding design concepts are also applicable to various VEP-BCIs.

[0111] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A large instruction set visual evoked potential brain-computer interface coding method, characterized in that: include: A plurality of instruction blocks distributed in rows and columns are displayed on a visual stimulation interface, each instruction block is encoded by two frequencies, and the row and column positions of each instruction block correspond to the two encoding frequencies of the instruction block.

2. The large instruction set visual evoked potential brain-computer interface coding method according to claim 1, characterized in that: The instruction block adopts a checkerboard structure.

3. The large instruction set visual evoked potential brain-computer interface coding method according to claim 1, characterized in that: The instruction block adopts a left and right field of view structure.

4. A large instruction set visual evoked potential brain-computer interface decoding method, corresponding to the large instruction set visual evoked potential brain-computer interface encoding method according to claims 1-3, the large instruction set visual evoked potential brain-computer interface decoding method comprising: Model each row instruction block and each column instruction block respectively to obtain a row identification model and a column identification model; The visual evoked potential X test The row identification model and the column identification model are input respectively to obtain the row number and column number of the visual evoked potential to be identified.

5. The large instruction set visual evoked potential brain-computer interface decoding method according to claim 4, characterized in that: The modeling of each row instruction block and each column instruction block to obtain a row identification model and a column identification model comprises: The task-related component analysis (TRCA) algorithm is used to model each row instruction block and each column instruction block, and the template for the i-th row is obtained. and the jth column template and the i-th row of spatial filters and the jth column spatial filter 6. The large instruction set visual evoked potential brain-computer interface decoding method according to claim 5, characterized in that: The visual evoked potential X to be identified test Input the row recognition model and the column recognition model respectively to obtain the visual evoked potential X to be recognized test The row and column numbers of , including: According to the visual evoked potential X to be identified test , the i-th row template and the i-th row of spatial filters Calculate the visual evoked potential X to be identified test With the template of row i The row correlation coefficient between According to the visual evoked potential X to be identified test , j-th column template and the jth column of spatial filters Calculate the visual evoked potential X to be identified test With the jth column template The column correlation coefficient between According to N i Row correlation coefficient and N j Column correlation coefficient Get the visual evoked potential X to be identified test The row and column numbers of ; Among them, N i is the total number of instruction blocks, N j is the total number of columns in the instruction block.

7. A large instruction set visual evoked potential brain-computer interface encoding device, characterized in that: include: The display module is used to display multiple instruction blocks distributed in rows and columns on a visual stimulation interface, each instruction block is jointly encoded by two frequencies, and the row and column position of each instruction block corresponds to the two encoding frequencies and phases of the instruction block.

8. A large instruction set visual evoked potential brain-computer interface decoding device, corresponding to the large instruction set visual evoked potential brain-computer interface encoding device described in claim 8, characterized in that: include: The recognition model acquisition module (901) is used to model each row instruction block and each column instruction block respectively to obtain a row recognition model and a column recognition model; The number acquisition module (902) is used to obtain the visual evoked potential X to be identified. test The row identification model and the column identification model are input respectively to obtain the row number and column number of the visual evoked potential to be identified.

9. A large instruction set visual evoked potential brain-computer interface system, characterized in that: It includes the large instruction set visual evoked potential brain-computer interface encoding device as described in claim 7 and the large instruction set visual evoked potential brain-computer interface decoding device as described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the large instruction set visual evoked potential brain-computer interface encoding method as described in any one of claims 1-3 and / or the large instruction set visual evoked potential brain-computer interface decoding method as described in any one of claims 4-6.

Images