A high-interactive naturalness brain-computer interface system based on common stimulus coding

Abstract

The application discloses a high-interaction naturalness brain-computer interface system based on common stimulation coding, which comprises a common stimulation setting module, a basic code element construction module and a time domain coding / decoding module; the common stimulation setting module is used for setting common stimulations in a space division multiple access manner within a user's gaze visual range; the basic code element construction module is used for constructing basic code elements according to the control conditions of each stimulation, including each stimulation and the relative position of characters relative to each stimulation; and the time domain coding / decoding module is used for time domain coding of each operable object according to the time sequence of common stimulation presentation after the basic code elements of each stimulation are determined, and the exact characters gazed by the user are obtained by analyzing the code elements corresponding to the stimulations at different moments through the decoding result of the time domain coding and the space-time code sequence. The application realizes a high-interaction brain-computer interface system for consumer-level users.

Description

Technical Field

[0001] This invention relates to brain-computer interface technology, and more specifically, to a brain-computer system with high degree of natural interaction based on retinal mapping. Background Technology

[0002] Brain-computer interface (BCI) is a novel human-computer interaction method based on neuroscience and engineering technology. It can establish a direct communication and control channel between the human brain and computers or other electronic devices by acquiring and analyzing brain signals in real time, and directly convert the activity of the central nervous system into information output. Users no longer need to express their thoughts or operate external devices through language or body movements.

[0003] Electroencephalogram (EEG) signals are a comprehensive reflection of the electrophysiological activity of brain nerve cells in the cerebral cortex. They can be recorded by scalp electrodes. Compared with other brain information acquisition methods such as functional near-infrared spectroscopy (fNIRS), magnetoencephalography (MEG), and invasive electrocorticography (ECoG), scalp EEG signals have advantages such as high temporal resolution, easy signal acquisition, and low acquisition cost, and therefore have received more widespread attention.

[0004] In recent years, the principle of retinal mapping has been well applied in brain-computer interface (BCI) research. This principle states that different spatial orientations of a user's gaze at a visual stimulus elicit different spatial patterns of EEG responses. In BCI systems, this translates to dividing the physical space into multiple subspaces, each corresponding to a different command, stimulating different locations within the user's visual field to induce transient visual evoked stimuli (VEP) or spontaneous visual evoked stimuli (SSVEP) signals with different spatial patterns. Compared to traditional visual BCI systems, users do not need to directly view flickering stimuli, resulting in lower visual load and a better experience. Furthermore, it is more economical in terms of computer screen usage and can be better integrated into more natural or complex application contexts, showing promising practical applications.

[0005] The asymmetric visual evoked potential (aVEP-BCI) brain-computer interface system has attracted widespread attention due to its high degree of natural interaction. The retinal mapping principle states that optical stimulation on either side of the visual field center activates different brain regions in the visual cortex. Based on this, aVEP-BCI applies blinking stimuli to both sides of the encoded command. These stimuli are extremely small and do not require explicit fixation from the user, thus reducing visual load to some extent. Furthermore, the EEG signals evoked by the stimuli on both sides contain specific spatial features. This BCI system encodes each command using a space code division multiple access (SCR) encoding method. The BCI system processes the acquired EEG signals, identifies different spatial features of the EEG, and then determines the target the user is gazing at. Overall, aVEP-BCI exhibits high natural interaction and is a suitable brain-computer interface system for consumer applications. However, further departure from the laboratory environment still presents the following challenges:

[0006] 1. Low system performance. Currently, the information transmission rate (ITR) of aVEP-BCI is low, and a high recognition accuracy is based on a single stimulus over a relatively long period of time.

[0007] 2. Numerous stimuli and low encoding efficiency. Although the aVEP-BCI stimulus size is small, one coded instruction corresponds to multiple flashing stimulus codes. On the one hand, this increases the visual burden on the user; on the other hand, a large number of flashing stimuli also increases the burden on the device, which is not conducive to the use of portable devices.

[0008] Traditional visual brain-computer interfaces (BCIs) often improve system performance by sacrificing the naturalness of brain-computer interaction, such as by increasing the size or number of stimuli. This significantly reduces user comfort and experience, severely limiting the application of BCI technology in daily life. From the perspective of naturalness and comfort of interaction, traditional visual BCI systems have the following problems:

[0009] A. Regarding the number of stimuli: Traditional visual brain-computer interface systems typically use one or more stimuli to correspond to one system command. On the one hand, this results in a large number of stimuli relative to the number of commands, thus increasing user fatigue. On the other hand, encoding a single command with multiple stimuli also limits the efficiency of the system's encoding.

[0010] B. From the perspective of stimulus location: The flashing stimuli of traditional visual brain-computer interfaces are usually superimposed on the instructions and require the user to focus on a concentrated gaze target. On the one hand, direct eye contact with the flashing stimulus can aggravate visual fatigue and is not conducive to the long-term use of the system. On the other hand, this is not friendly to patients with certain neurological diseases, because they often cannot accurately control their eye movements.

[0011] C. Stimulus size: Traditional visual brain-computer interfaces have relatively large stimulation sizes, which usually provide users with stronger stimulation and increase the user's visual load.

[0012] Among EEG-based brainstem induction (BCI) systems, the paradigm utilizing visual stimulation is the most widely applied, with SSVEP-BCI based on steady-state visual evoked potentials (SSVEP) and P300-BCI based on event-related potentials (ERPs) being the most typical. The shortcomings of existing technologies are described below:

[0013] I. The P300-BCI system has been developed to a relatively stable and mature stage. The P300 signal is a positive shift in the EEG signal approximately 300 milliseconds after the stimulus occurs. For the subject, the stimulus itself is uncommon and unpredictable (e.g., a sudden appearance of the stimulus), yet it is closely related to the subject. The P300 amplitude directly depends on the relevance of the stimulus and is inversely proportional to the probability of the stimulus's occurrence.

[0014] (1) The classic P300 paradigm is typically displayed on a computer screen in a 6x6 matrix format, with rows and columns flashing repeatedly in a random order. All rows and columns are displayed once per round. A noticeable P300 signal is generated in the subject's brain only when a row or column contains the letter or instruction selected by the subject. This signal can be detected by a classifier. The P300-BCI system has become relatively stable and mature, achieving high information transmission rates and possessing a solid technical foundation. However, the following problems still exist:

[0015] (2) Low coding efficiency. The visual BCI system based on P300 requires all instructions to flash once to complete one instruction recognition. However, due to the weak signal characteristics and low amplitude of the P300 signal itself, the signal often needs to be superimposed multiple times to obtain a highly dissimilar P300 signal. Therefore, the high-quality output of an instruction often requires multiple rounds of flashing of all instructions to complete. This makes the P300 paradigm difficult to apply to multi-instruction and complex application scenarios.

[0016] The P300 brain-computer interface suffers from low interactivity. It superimposes flashing stimuli onto system commands and requires users to maintain prolonged focus on them. On one hand, prolonged direct viewing of the flashing stimuli increases visual burden, reducing the interactive experience and impacting system performance. On the other hand, the stimuli superimposed on the system commands are identical in size and number; when the number of commands is large or the stimuli are numerous, it negatively affects the interactive experience.

[0017] II. SSVEP-BCI uses stable, periodically repetitive visual stimuli (such as a graphic flashing at a fixed frequency or a pattern flipping repeatedly on the screen) to elicit responses. Based on the frequency characteristics of SSVEP-BCI, different frequency stimuli can be used to encode different operational commands. The user simply needs to focus on a specific target stimulus in their visual field according to their intention. The BCI system processes the acquired EEG signals, identifies EEG characteristics, and determines the target the user is focusing on through spectral analysis.

[0018] To make brain-computer interface (BCI) interactions more comfortable and natural, current SSVEP-BCI systems primarily employ mid-to-high frequency flickering stimulation. SSVEP-based BCIs have achieved relatively mature development, exhibiting high encoding efficiency and system stability. However, the following issues remain when transitioning from laboratory-scale applications to consumer-grade practical applications:

[0019] (1) The interaction of low and medium frequency signals is low. The flickering stimulation of low and medium frequency signals is strong, which causes high user fatigue and is not suitable for long-term use.

[0020] (2) High-frequency signals require expensive equipment and have poor system performance. High-frequency SSVEP signals overcome the shortcomings of low-frequency signals, such as strong stimulation and low interactivity. However, their signal-to-noise ratio is significantly lower than that of low-frequency signals, resulting in significantly reduced system performance. In addition, as the selected frequency of SSVEP-BCI continues to increase, the refresh rate requirements of the equipment are also increasing. High refresh rate equipment is expensive and not easy to carry.

[0021] Therefore, SSVEP-based BCI systems are not suitable for being moved out of the laboratory environment and directly into the consumer market.

[0022] III. Brain-computer interfaces (BCIs) based on spatial division multiple access (SDMA) have garnered widespread attention due to their simplicity and naturalness of interaction. They divide the physical space into multiple subspaces, each corresponding to a different command. The retinal principle states that light stimulation of different regions lateral to the fovea elicits different responses in the visual cortex, exhibiting lateralization. Based on the retinal mapping principle, BCI systems using SDMA encoding stimulate different locations within the user's visual field to induce transient visual evoked stimuli (VEPs) or spatially modulated spontaneous evoked stimuli (SSVEPs). SDMA-based BCIs typically place the stimulus centrally and then encode different surrounding regions to recognize different commands, which can be induced through SSVEPs, mVEPs, etc. Users do not need to directly view strong flashing stimuli. While SDMA-based BCIs have been widely used due to their naturalness of interaction and simplicity of construction, the following problems still exist in practical applications.

[0023] (1) Limited number of instructions, unsuitable for complex situations. Due to the spatial resolution limitations of EEG, the basic code elements available for spatial coding are limited, thus limiting the number of instructions that can be encoded and making it unsuitable for complex systems.

[0024] (2) The stimulus size is generally large and the signal-to-noise ratio is low. Most brain-computer interfaces based on space division multiple access induce ERP signals through a single stimulus. However, since the signal-to-noise ratio of ERP signals is low, large-size stimuli are usually used to ensure system performance, which affects system comfort.

[0025] IV. The asymmetric visual evoked potential (aVEP-BCI) system has attracted widespread attention due to its high degree of natural interaction. The retinal mapping principle indicates that optical stimulation on both sides of the visual field center activates different brain regions in the visual cortex. Based on this, aVEP-BCI sets flashing stimuli on both sides of the encoded command. These stimuli are extremely small and do not require explicit fixation from the user, thus reducing visual load to some extent. Furthermore, the EEG signals evoked by the stimuli on both sides contain specific spatial features. This BCI system encodes each command using a space code division multiple access (SCR) encoding method. The BCI system processes the acquired EEG signals, identifies different spatial features of the EEG, and then determines the target the user is gazing at. Overall, aVEP-BCI has a high degree of natural interaction and is a suitable brain-computer interface system for consumer use. However, further departure from the laboratory environment still presents the following challenges:

[0026] (1) The system performance is low. Currently, the information transmission rate (ITR) of aVEP-BCI is low, and the high recognition accuracy is based on a single stimulus over a long period of time.

[0027] (2) The number of stimuli is large and the encoding efficiency is low. Although the aVEP-BCI stimulus size is small, one coded instruction corresponds to multiple flashing stimulus codes. On the one hand, this increases the visual burden on the user. On the other hand, a large number of flashing stimuli will also increase the burden on the device, which is not conducive to the use of portable devices. Summary of the Invention

[0028] To address the limitations of the existing technologies, this invention proposes a highly interactive and natural brain-computer interface system and method based on shared stimulus encoding. Through a novel paradigm combining the retinal mapping principle and the P300 paradigm, shared stimuli are set at specific locations around the operable object. These shared stimuli encode the operable object in the form of space-division multiple access, allowing users to complete target recognition without directly looking at the flashing stimuli, thereby improving the interactivity of the system.

[0029] This invention is achieved using the following technical solution:

[0030] A highly interactive and natural brain-computer interface system based on shared stimulus coding, comprising a shared stimulus setting module, a basic code construction module, and a temporal encoding / decoding module; wherein:

[0031] The shared stimulus setting module is used to set shared stimuli within the user's visual gaze range using a spatial division multiple access method, satisfying the following setting principles:

[0032] Principle 1: Each operable object is governed by one or more shared stimuli, and multiple shared stimuli can govern the same operable object;

[0033] Principle 2: An operable object controlled by the same shared stimulus must include at least the parameters of stimulus magnitude, stimulus distance, and stimulus presentation method;

[0034] Principle 3: For a shared stimulus controlling m (m = 1, ..., k) operable objects, the total number of operable objects controlled should be greater than or equal to the total number of operable objects, i.e.:

[0035] 1*n1+2*n2+…+m*n m ≥k

[0036] Principle 4: The number of shared stimuli for controlling m (m = 1, ..., k) operable objects is n. m It should be less than the total number of operable objects k, that is:

[0037] n1+n2+…+n m ≤k

[0038] Where k is the total number of operable objects in the system, and n is the total number of visual stimuli;

[0039] The basic code element construction module is used to construct basic code elements based on the control status of each stimulus, including each stimulus and the relative position of characters relative to each stimulus.

[0040] The time-domain encoding / decoding module is used to encode each operable object in the time domain according to the time sequence of the common stimulus presentation after the basic code of each stimulus is determined. After time-domain encoding, a spatiotemporal code sequence is generated. The time-domain encoding is decoded, and the code corresponding to the stimulus at different times is parsed from the decoding result of the time-domain encoding and the spatiotemporal code sequence to obtain the exact target of the user's gaze.

[0041] The shared stimulus consists of multiple groups, each group including stimuli that control a different number of operable objects.

[0042] Multiple stimuli can share a set of basic code elements or construct their own basic code elements according to their respective control capabilities.

[0043] The stimulus controls adjacent characters, and its control capability is related to the relative position of the character and the stimulus.

[0044] The basic code elements include valid stimulus code elements and invalid stimulus code elements.

[0045] When the stimulus is a flashing stimulus, the flashing stimulus is presented sequentially in the order of flashing, or presented simultaneously in a few or all of them.

[0046] The design elements of the shared stimulus include at least the stimulus location, the number of stimuli, the stimulus evoked form, or the stimulus shape parameter.

[0047] The shape of the stimulus can be any different shape.

[0048] Compared with existing technologies, the high-interaction naturalness brain-computer interface system based on shared stimulus coding of the present invention can achieve the following positive technical effects:

[0049] 1) Aiming to improve the naturalness of system interaction and coding efficiency, the problems to be solved or the advantages that should be possessed are as follows:

[0050] 2) The human-computer interaction is highly natural, with small size and few number of stimuli, and the method is user-friendly; however, multiple large-sized, strongly flashing stimuli superimposed on the operable object, and looking directly at these stimuli, will significantly increase the user's fatigue, which greatly limits the practical application of brain-computer interface systems.

[0051] 3) High encoding efficiency, a large number of encodeable instructions, and good system performance. How to support complex systems (multi-instruction systems) while maintaining high interactivity and good performance (achieving high recognition accuracy in a short time)?

[0052] 4) Low cost and easy setup. The designed system should have minimal hardware requirements and be as easy to set up as possible.

[0053] 5) A brain-computer interface system for consumer users has been implemented. Attached Figure Description

[0054] Figure 1 This is a block diagram of the high-interaction naturalness brain-computer interface system based on shared stimulus coding of the present invention;

[0055] Figure 2 The diagram shows the stimulation paradigms; (2a) shows the flashing stimulus of the prior art superimposed on the gaze character, and (2b) shows the common stimulus of the present invention located around the gaze character.

[0056] Figure 3 (3a)(3b)(3c) are design elements of a single stimulus paradigm; (3d) is the shared stimulus paradigm design element of this invention.

[0057] Figure 4 To construct a basic symbol diagram;

[0058] Figure 5 This is a schematic diagram of time-domain coding;

[0059] Figure 6 The diagram below is a schematic diagram of a specific embodiment. (6a) and (6b) are brain-computer interface embodiments for mobile phone dialing. Detailed Implementation

[0060] The technical solution will now be clearly described in conjunction with the accompanying drawings and embodiments. All other embodiments and technical substitutions or modifications of the embodiments obtained by those skilled in the art based on the embodiments of this invention without departing from the spirit of the invention and without creative effort, fall within the protection scope of this invention.

[0061] The core of this invention is the design of a novel visual stimulus paradigm, which specifically includes a shared stimulus paradigm and an encoding strategy.

[0062] like Figure 1 The diagram shows a block diagram of a high-interaction naturalness brain-computer interface system based on shared stimulus coding according to the present invention. The system includes a shared stimulus paradigm setting module 100, a basic code construction module 200, and a temporal encoding / decoding module 300.

[0063] like Figure 2 The diagram shows a stimulation paradigm. (2a) In the prior art, flashing stimuli are superimposed on the gazed character. (2b) In this invention, the common stimuli (dots) are located around the gazed character. (2a) is the flashing stimulation interface of a traditional brain-computer spelling device (only 16 characters are shown). The stimuli are superimposed on each character. When using it, the user needs to look directly at the character flashing stimulation for several seconds. After the system recognizes it, it can output the recognition command. (2b) is the common stimulation paradigm designed in this invention (only one stimulation arrangement is shown). The dots are stimuli. These stimuli are distributed around the character. Each character is encoded by one or more stimuli. Each stimulus is presented in a specified encoding method to induce the separability characteristics of the EEG signal. After the system recognizes it, it can output the character that the user is looking at.

[0064] like Figure 3 The diagram shown illustrates the design elements of a stimulus paradigm. Common stimulus design elements include parameters such as stimulus location, number of stimuli, stimulus evoked form, or stimulus shape.

[0065] The retinal mapping principle states that the spatial patterns of brain electrical responses induced by a user's gaze at different areas around a visual stimulus differ. (3a), (3b), and (3c) are design elements of the single-stimulus paradigm. For example, as shown in (3a), when the stimulus is located in the center, gazing at different areas around it leads to differences in brain signal generation characteristics. These differences depend on the relative position, relative distance, and stimulus size of the gazing area and the stimulus. For example, as shown in (3b) and (3c), operable objects at different locations can be encoded by one or more stimuli using space-division multiple access. That is, a single stimulus can control multiple operable objects based on factors such as position, size, and distance. Let the number of operable objects that a stimulus can control be called the stimulus control capability m. In (3b), the stimulus control capability m = 4, and in (3c), the stimulus control capability m = 2. Depending on the parameters such as the stimulus position, size, and distance from the operable object, the number, position, and distance of the operable objects that the stimulus can control are different. This should be considered a variant of the present invention and protected. For example, in Figures b and c, although both contain only one stimulus and have the same stimulus control capability m=2, the relative positions of the stimulus and the operable object are different. Therefore, these two stimulus settings should be considered different cases. Furthermore, any differences in stimulus control capability caused by variations in parameters such as stimulus position, size, and distance from the operable object should be considered variations of this invention and protected.

[0066] As shown in (3d), the shared stimulus paradigm design element of this invention is specifically set up in a space-division multiple access manner in a brain-computer system containing an arbitrary number of operable objects. Due to the influence of the number and location of operable objects, there will be multiple sets of shared stimuli. Each set includes stimuli that control a different number of operable objects. The setting of each set of shared stimuli should meet four basic principles:

[0067] Principle 1: Each operable object is controlled by one or more shared stimuli, and multiple shared stimuli can control the same operable object;

[0068] Principle 2: Operable objects controlled by the same shared stimulus should be separable according to the principle of retinal mapping. The strength of their separability is affected by parameters including at least stimulus size, stimulus distance, and stimulus presentation method. Other parameters are considered variations of this invention and should be protected.

[0069] Principle 3: Let the total number of operable objects in the system be k, the total number of visual stimuli be n, and the number of shared stimuli for controlling m (m = 1, ..., k) operable objects be n. m The total number of operable targets for shared stimulus control should be greater than or equal to the total number of operable targets, that is:

[0070] 1*n1+2*n2+…+m*n m ≥k

[0071] Principle 4: The number of shared stimuli for controlling m (m = 1, ..., k) operable objects is n. m It should be less than the total number of operable objects k, that is:

[0072] n1+n2+…+n m ≤k.

[0073] Taking (3d) as an example, the left figure shows four common stimuli, each of which controls four adjacent characters. Therefore, including the four stimuli with a control capacity of m=4, it is represented as n4=4; this example is described as follows:

[0074] Description 1: All stimuli are controlled;

[0075] Description 2: Characters controlled by the same stimulus are separable;

[0076] Description 3: The total number of operable objects is k = 16, and the fourth shared stimulus is n4 = 4. Therefore, in this example, the number of operable objects controlled by the four shared stimuli is 4*n4 = 16, which is equal to the total number of operable objects being k = 16. That is, 4*n4 = 16 = k.

[0077] Description 4: The number of shared stimuli n for controlling m (m = 1, ..., k) operable objects. m The number of operable objects is less than the total number of objects k, i.e., n = 4 < k.

[0078] Similarly, the diagram on the right shows 5 shared stimuli. Three stimuli control the four adjacent characters, and two stimuli control the two adjacent characters. Therefore, the three stimuli with a control capacity of m = 4 are represented as n4 = 3; and the two stimuli with a control capacity of m = 2 are represented as n2 = 2. This example is described as follows:

[0079] 1. All stimuli are controlled;

[0080] 2. Characters controlled by the same stimulus are separable;

[0081] 3. The total number of operable objects is k=16. The three stimuli with controllability m=4 are represented as n4=3; and the two stimuli with controllability m=2 are n2=2. Then 4*n4+2*n2=16=k.

[0082] 4. The number of shared stimuli n for controlling m (m=1,…,k) operable objects. m The total number of operable objects is less than t, and n = 5 < k.

[0083] In summary, the 16-instruction scheme can be represented by a shared stimulus code of 4 control 4-instructions, or by 3 control 4-instructions and 2 control 2-instructions. These two schemes of different shared stimulus arrangements do not change the core design concept of this invention and should be considered as variations of this design for protection.

[0084] To maximize the naturalness of the system's interaction and minimize the device load, the final selection should be made from multiple arrangement schemes, choosing one or more schemes with the fewest total stimuli for subsequent encoding. For example, suppose a system has x possible stimulus arrangements that meet certain conditions, numbered y1, y2, y3, ..., y x The total number of stimuli for each option is n1, n2, n3, ..., n x Then the final stimulus arrangement y used for encoding is one or more schemes with the fewest stimuli among all the schemes that meet the conditions:

[0085]

[0086] The visual stimuli are graphic representations of various shapes, such as circles, squares, or triangles. Figure 3 The example is a circle.

[0087] Regarding the quantity and location of visual stimuli, as long as the controlled stimuli can control all operable objects and meet the four principles, the combination of their quantity and location should be considered an effective arrangement.

[0088] like Figure 4 The diagram shown illustrates the construction of basic code elements. The total number of operable objects in the system is n. After determining the shared stimulus paradigm, the stimuli are numbered s1, s2, s3, ..., s... n Based on the control status of each stimulus, including the individual stimuli and their relative positions to the characters, basic code units are constructed. These basic code units include valid stimulus code units and invalid stimulus code units, i.e., empty code P. The empty code P is used to enhance the separability of the temporal coding and represents an invalid stimulus (a stimulus that cannot control a certain character). The brain-computer interface system shown in (4a) consists of 16 instruction characters and 4 stimuli, which are encoded as s1, s2, s3, and s4 in sequence. Then, basic code units are constructed for each stimulus.

[0089] Each stimulus can only control the adjacent characters, and its control ability is only related to the relative position of the character and the stimulus. Taking s1 as an example, according to its relative position, it can be divided into four cases: stimulus in the upper left of the character, stimulus in the lower left of the character, stimulus in the upper right of the character, and stimulus in the lower right of the character.

[0090] Since the relative positions of the characters controlled by the four stimuli in this example are similar, these four stimuli share a set of basic code elements. Table 1 shows the specific encoding of the basic code element composition table in this example.

[0091] Table 1

[0092] Stimulus at the top left of the character C1 Stimulus at the lower left of the character C2 Stimulus in the upper right corner of the character C3 Stimulus in the lower right corner of the character C4 Ineffective stimuli P

[0093] The construction of basic code elements depends on the separability between each stimulus control character. The strength of separability is controlled by factors such as the distance between the stimulus and the operable object, the size of the stimulus, and the location of the stimulus. Multiple stimuli may share a set of basic code elements or construct their own basic code elements, depending on their respective control capabilities. Essentially, it depends on the control capability of the stimulus. As long as there are differential spatial features that can generate EEG signals in the visual cortex of the brain, and the code elements are separable, they should be considered as variations of this design and protected.

[0094] like Figure 5 The diagram shows a temporal coding scheme. The stimuli are presented in the order of time t1, t2, t3, t4, and the specific coding of each stimulus's basic code is shown in Table 1.

[0095] Once the basic codewords for each stimulus are determined, each operable object is temporally encoded according to the chronological order of presentation of shared stimuli. Flashing stimuli are presented sequentially in the order of flashing, or in small numbers or all at the same time. This increases the separability of different operable objects, and the presentation method is not limited to SSVEP, ERP, or fade-out mVEP.

[0096] For each manipulated object, different visual stimuli correspond to different code elements. After temporal encoding, a specific spatiotemporal code sequence is generated. Taking character A as an example, the stimulus s1 presented at time t1 has the basic code element c1. The stimulus s2 presented at time t2 cannot control character A, so its corresponding code element is the empty code P. Similarly, the stimuli s3 presented at time t3 and s4 presented at time t4 both correspond to the empty code element P. Therefore, the spatiotemporal code sequence of character A is s1(C4). Figure 5 All characters A through P in the example are time-domain encoded. As shown in Table 2, ... Figure 5 The example is a time-domain encoded sequence of all characters.

[0097] Table 2

[0098] A S1(C4) P P P B S1(C2) P P P C P S2(C4) P P D P S2(C2) P P E S1(C3) P P P F S1(C1) P P P G P S2(C3) P P H P S2(C1) P S4 I P P S3(C4) P J P P S3(C2) P K P P P S4(C4) L P P P S4(C2) M P P S3(C3) P N P P S3(C1) P O P P P S4(C3) P P P P S4(C1)

[0099] In terms of the timing of visual stimulus presentation, stimuli can be presented sequentially or simultaneously, with the aim of maximizing the spatiotemporal code differences corresponding to each character. By using temporal coding, the differences between operation commands are maximized, thereby obtaining the largest possible number of operation commands while minimizing decoding difficulty and improving decoding speed and accuracy.

[0100] The temporal code is decoded, and the code elements corresponding to stimuli at different times are parsed from the decoding result of the temporal code and the spatiotemporal code sequence to obtain the exact character that the user is looking at.

[0101] As brain-computer interface systems move from the laboratory to consumer applications, the naturalness of interaction, system performance, and device cost are the three core indicators.

[0102] Specific Implementation Example 1: This example demonstrates the application of brain-controlled mobile phone dialing keys, as described in detail below:

[0103] like Figure 6 The diagram shown is a schematic diagram of a specific embodiment.

[0104] As shown in (6a), the left side is the dialing interface of the mobile phone, and the right side is the stimulus arrangement designed according to the present invention, where the dots represent the positions of visual stimuli and the internal numbers represent the stimuli induction order. The system uses 4 stimuli to control 12 characters.

[0105] As shown in (6b), for each character, the eight positions surrounding it are considered as valid code elements, and stimuli outside the surrounding area are considered as invalid code elements, i.e., empty code. Taking the '#' key as an example, its valid code elements are numbered 1-8, and the empty code is numbered P. In this case, the brain will only generate obvious separability features when there are visual stimuli around the letter or instruction character selected by the user.

[0106] Table 3 shows the time-space code sequence corresponding to each key character on the dialer interface.

[0107] Table 3

[0108]

[0109]

[0110] By decoding the timing and type of the effective stimulus, the button commands to be executed can be obtained.

[0111] The technical effects achieved by the specific embodiment of the present invention can be summarized as follows:

[0112] 1-1. The ERP-BCI system designed according to this invention brings the following beneficial effects during user interaction. 1-2. Under this system design, the number of stimuli is small and located in the surrounding field of vision, resulting in low visual load on the user and making it suitable for long-term use scenarios. 1-3. Under this system design, the system is simple to build, has low requirements for computer screens, and can be well integrated into more natural or complex application backgrounds, showing good practical prospects. 1-4. Under this system design, the coding efficiency is improved compared to the traditional P300 paradigm, overcoming the problem of significantly increased time costs due to amplified instructions in the P300 paradigm.

[0113] A comparison of a specific embodiment of this invention with the P300 paradigm: The P300 paradigm, with its 12 (4*3) instruction method, uses 12 stimuli superimposed on the system instruction. The operator directly observes the flashing stimuli to complete the instruction output. A single instruction recognition involves 7 flashes, with each flash containing at least 3 stimuli. The paradigm designed according to the encoding method of this invention uses 4 stimuli located around the system instruction. The operator does not need to directly observe the flashing stimuli. A single instruction recognition involves 4 flashes, with each flash containing 1 stimulus. It can be seen that, with the same number of instructions, the BCI system designed using the encoding method of this invention is superior to the traditional P300 paradigm in both encoding efficiency and stimulus-friendliness.

[0114] Specific Embodiment Two of the Invention: An Application Case of Intelligent Games Based on a Small Wearable Head Sensor. The user wears a data acquisition device with an electroencephalogram (EEG) sensor on their head. This device can be integrated into wearable devices such as glasses or headphones, or it can be a standalone piece of hardware worn on the user's head.

[0115] The data acquisition module is deployed on a data acquisition unit equipped with an EEG sensor;

[0116] The data processing module is deployed on the user's mobile terminal. The data collector communicates with the mobile terminal via a wireless network.

[0117] The visual presentation module is deployed to present corresponding feedback in games, such as action feedback in fighting games and platformers.

[0118] Users can directly control in-game functions by looking at the character. For example, jumping, running left or right in a parkour game.

[0119] 1. Present the game interface, collect EEG signals, and complete preprocessing and A / D conversion, etc.

[0120] Users wear microcomputer devices or VR / AR devices, and the data acquisition module built on them collects the brain signals generated according to the user's operating intentions, and completes analog signal processing such as noise reduction and amplification, as well as A / D conversion.

[0121] 2. Complete feature extraction and pattern recognition of EEG signals.

[0122] The data processing module deployed in wearable devices can use a variety of pattern recognition algorithms to perform feature processing and pattern recognition of EEG signals for user-device interaction.

[0123] 3. Provide feedback to the user on the execution results of the operation commands.

[0124] The visual presentation module deployed in smart home devices provides feedback to the user on the execution results of operation commands based on the judgment results.

[0125] The technical effects achieved by the second specific embodiment of the present invention can be summarized as follows:

[0126] 2-1. Game interaction is currently a major research direction in BCI (Brain-Induced Interaction). Since games are primarily for entertainment and rehabilitation, BCI systems designed for games should possess high interactivity to provide players with a better experience. 2-2. In the user interaction process, the paradigm designed in this invention is used for visual stimulus presentation, achieving the same beneficial technical effects as in Embodiment 1. Furthermore, due to the significant differences in EEG characteristics generated by spatial division multiple access (SDMA) and time division multiple access (TDMA), classification and recognition time can be reduced under low signal-to-noise ratio conditions, resulting in a smoother operating experience.

Claims

1. A highly interactive and natural brain-computer interface system based on shared stimulus encoding, characterized in that, The system includes a shared stimulus setting module, a basic symbol construction module, and a time-domain encoding / decoding module; wherein: The shared stimulus setting module is used to set shared stimuli within the user's visual gaze range using a spatial division multiple access method, satisfying the following setting principles: Principle 1: Each operable object is governed by one or more shared stimuli, and multiple shared stimuli can govern the same operable object; Principle 2: An operable object controlled by the same shared stimulus must include at least the parameters of stimulus magnitude, stimulus distance, and stimulus presentation method; Principle 3: For a shared stimulus controlling m (m = 1, ..., k) operable objects, the total number of operable objects controlled should be greater than or equal to the total number of operable objects, i.e.: 1*n1+2*n2+…+m*n m ≥k Principle 4: The number of shared stimuli for controlling m (m = 1, ..., k) operable objects is n. m It should be less than the total number of operable objects k, that is: n1+n2+…+n m ≤k Where k is the total number of operable objects in the system, and n is the total number of visual stimuli; The basic code element construction module is used to construct basic code elements based on the control status of each stimulus, including each stimulus and the relative position of characters relative to each stimulus. The time-domain encoding / decoding module is used to encode each operable object in the time domain according to the time sequence of the common stimulus presentation after the basic code of each stimulus is determined. After time-domain encoding, a spatiotemporal code sequence is generated. The time-domain encoding is decoded, and the code corresponding to the stimulus at different times is parsed from the decoding result of the time-domain encoding and the spatiotemporal code sequence to obtain the exact target of the user's gaze.

2. The high-interaction naturalness brain-computer interface system based on shared stimulus coding as described in claim 1, characterized in that, The shared stimulus consists of multiple groups, each group including stimuli that control a different number of operable objects.

3. The high-interaction naturalness brain-computer interface system based on shared stimulus encoding as described in claim 1, characterized in that, Multiple stimuli can share a set of basic code elements or construct their own basic code elements according to their respective control capabilities.

4. The high-interaction naturalness brain-computer interface system based on shared stimulus encoding as described in claim 1, characterized in that, The stimulus controls adjacent characters, and its control capability is related to the relative position of the character and the stimulus.

5. The high-interaction naturalness brain-computer interface system based on shared stimulus coding as described in claim 1, wherein the basic code elements include effective stimulus code elements and ineffective stimulus code elements.

6. The high-interaction naturalness brain-computer interface system based on shared stimulus coding as described in claim 1, wherein when the stimulus is a flashing stimulus, the flashing stimulus is presented sequentially in the order of flashing, or presented simultaneously in a few or all of them.

7. The high-interaction naturalness brain-computer interface system based on shared stimulus coding as described in claim 1, wherein the design elements of the shared stimulus include at least stimulus location, stimulus quantity, stimulus evoked form, or stimulus shape parameter.

8. The high-interaction naturalness brain-computer interface system based on shared stimulus encoding as described in claim 1, wherein the shape of the stimulus is any different shape.

Images