Brain computer interface

Abstract

Methods and machines herein describe methods and machines for control of real-world objects by a brain computer interface. The method includes displaying a visual stimulus associated with a controllable object to a user; capturing a neural signal from the user using a neural signal capture device; determining, based on the neural signal, that a visual stimulus of the visual stimulus is a subject of interest; visually displaying feedback in the vicinity of the controllable object associated with the visual stimulus, where the feedback highlights the object of interest; and creating a positive feedback loop using the feedback to play a brain response through the extended magnified attention to confirm the subject of interest as an intended target subject.

Description

[0001] This application is a divisional application of invention patent application No. 202080080282.4, with the application date of November 20, 2020, international application number PCT / EP2020 / 082836, and the invention name “Brain-Computer Interface” which entered the Chinese national phase on May 19, 2022.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 62 / 938,112, filed on November 20, 2019, entitled “BRAIN-COMPUTER INTERFACE,” the entire contents of which are incorporated herein by reference. Technical Field

[0004] The present invention relates to the control of real-world objects through a brain-computer interface involving visual sensing. Background Art

[0005] In visual brain-computer interfaces (BCIs), neural responses to target stimuli are typically used to infer (or "decode") which stimulus is the object of attention at any given time among multiple generated visual stimuli presented to the user. The object of attention can then be associated with a user-selectable or user-controllable action.

[0006] A variety of known techniques can be used to obtain neural responses. A convenient method relies on surface electroencephalography (EEG), which is non-invasive, has fine-grained temporal resolution, and is based on well-known empirical foundations. Surface EEG makes it possible to measure changes in the diffusion potential on the surface of the subject's skull (i.e., the scalp) in real time. These changes in potential are commonly referred to as electroencephalographic signals or EEG signals.

[0007] In a typical BCI, visual stimuli are presented in a display generated by a display device. Suitable display devices (some of which are Figure 3Examples of visual stimuli (shown in ) include television screens and computer monitors 302, projectors 310, virtual reality headsets 306, interactive whiteboards, and display screens of tablet computers 304, smartphones, smart glasses 308, etc. The visual stimuli 311, 311', 312, 312', 314, 314', 316 may form part of a generated graphical user interface (GUI), or they may be presented as augmented reality (AR) or mixed reality graphical objects 316 overlaying a base image: the base image may simply be the user's actual field of view (as in the case of a mixed reality display function projected onto an otherwise transparent display of a set of smart glasses) or a digital image corresponding to the user's field of view but captured in real time by an optical capture device (which in turn may capture images corresponding to the user's field of view among other possible views).

[0008] Inferring which of multiple visual stimuli (if any) is the object of attention at any given time is fraught with difficulty. For example, when a user is presented with multiple stimuli (e.g., numbers displayed on an on-screen keyboard), it has proven almost impossible to infer directly from brain activity which stimulus is being attended to at a given time. The user perceives the number being attended to (let's say the number 5), so the brain must contain information to distinguish this number from other numbers, but current methods are unable to extract this information. That is, current methods can infer that a stimulus has been perceived, but these methods cannot determine which specific stimulus is being attended to using brain activity alone.

[0009] In order to overcome this problem and to provide sufficient contrast between the stimulus and the background (and between the stimuli), it is known to configure the stimuli used by visual BCIs to flicker or pulse (e.g., a large surface of pixels switches from black to white and vice versa) so that each stimulus has a distinguishable characteristic distribution that varies over time. The flickering stimulus causes a measurable electrical response. Specific techniques monitor different electrical responses, such as steady-state visual evoked potentials (SSVEPs) and P-300 event-related potentials. In typical implementations, the stimulus flickers at a rate of more than 6 Hz. As a result, such visual BCIs rely on an approach that involves displaying various stimuli in a display device discretely rather than continuously, and typically at different points in time. Brain activity associated with attention paid to a given stimulus is found to correspond to (i.e., correlate with) one or more aspects of the temporal distribution of the stimulus, such as the frequency with which the stimulus flickers and / or the duty cycle with which the stimulus alternates between a flickering state and a resting state.

[0010] Therefore, decoding of neural signals relies on the fact that when the stimulus is turned on, it will trigger a characteristic pattern of neural responses in the brain, which can be determined from the electrical signals, namely the SSVEP or P-300 potential, which is picked up by the electrodes of the EEG device (such as the electrodes of an EEG helmet). This neural data pattern can be very similar or even identical for different digits, but it is time-locked to the digit being perceived: only one digit can pulse at any one time, so that the correlation between the timing of the digit pulse and the pulsing neural response can be determined as an indication that the digit is the object of attention. By displaying each digit at a different time point, turning the digit on and off at different rates, applying different duty cycles, and / or simply applying the stimulus at different time points, the BCI algorithm can establish which stimulus is most likely to trigger a given neural response when turned on, thereby enabling the system to determine the object of attention.

[0011] In recent years, visual BCIs have improved significantly, making it increasingly practical to decode a user's attention in real time and accurately. However, the constant flickering of stimuli, which sometimes flickers across the entire screen when there are many stimuli, is an inherent limitation of large-scale use of this technology. In fact, this can cause discomfort and mental fatigue, and if it persists, can also cause physiological responses such as headaches. In addition, the flickering effect can hinder the user's ability to focus on a specific target, as well as the system's ability to quickly and accurately determine the object of attention. For example, when a user tries to focus on the number 5, other (i.e., peripheral) numbers act as distractors, which temporarily attract the user's attention and interfere with the user's visual system. This interference, in turn, can hinder the performance of the BCI. Therefore, there is a need for improved methods for distinguishing between screen targets and their displayed stimuli to determine which one the user is focusing on.

[0012] The requirement for a certain display device to present visual stimuli limits the application of the aforementioned technologies. In particular, a suitable display may not be available or desired. In some applications, interacting with an object through a screen may be inconvenient or impractical. In addition, user acceptability of EEG devices (and their electrodes) brings aesthetic constraints, as well as constraints in terms of comfort and ease of use. In many cases, these constraints are effectively major obstacles to the adoption of EEG technology. Examples of applications where long-term use comfort and the need for technology to help hinder adoption include applications such as video games, training (e.g., for health and safety or flight simulation), sleep aids, etc.

[0013] Therefore, it is desirable to provide a brain-computer interface that addresses the above challenges. Summary of the Invention

[0014] The present disclosure relates to brain-computer interfaces in which visual stimuli are presented in direct association with real-world objects so that a user's intent can be extended to objects in the real world without the need for an intervening screen or other display device, thereby providing an improved and intuitive user experience.

[0015] The present disclosure relates to techniques for applying visual stimulation to otherwise conventional real-world objects that appear to be objects of potential interest when within a user's field of view.

[0016] In some embodiments, the applied visual stimulation may include projecting an overlay image with temporal modulation onto one or more real-world objects. The modulation causes the object to flicker or otherwise change visually, such that the modulation acts as a stimulus for a relevant neural response in the user's brain. The neural response, in turn, can be measured and decoded to determine which object of interest is the focus of the user's attention.

[0017] In other embodiments, the object itself may include one or more light sources capable of emitting light with temporal modulation. Here again, the modulation causes the object to flicker or otherwise visually change, such that the modulation acts as a stimulus for a relevant neural response in the user's brain. The neural response can in turn be measured and decoded to determine which object of interest is the focus of the user's attention.

[0018] In other embodiments, an electronic badge can be provided that is separate from the real-world controllable objects but logically associated with at least one of them. The electronic badge includes one or more light sources capable of emitting temporally modulated light. Unlike traditional screens and displays, electronic badges are typically small in size and can be dedicated to outputting visual stimuli. The modulation in the emitted light causes the displayed portion of the electronic badge to flicker or otherwise visually change, causing the modulation to stimulate a relevant neural response in the user's brain. The neural response can then be measured and decoded to determine which electronic badge is the focus of the user's attention, and because the badge is logically associated with the real-world object, it can be determined that the real-world object is the object of interest.

[0019] In each of the above embodiments, modulation may be applied preferentially or exclusively to high spatial frequency components of the projected overlay image.

[0020] According to a first aspect, the present disclosure relates to a brain-computer interface system, comprising: at least one light-emitting unit that outputs a corresponding visual stimulus generated by a stimulus generator, the visual stimulus having a characteristic modulation; at least one controllable object that is configured to receive user instructions, each controllable object being associated with at least one visual stimulus; a neural signal capture device that is configured to capture neural signals associated with the user; an interface device that is operatively coupled to the neural signal capture device and the controllable object, the interface device comprising: a memory; and a processor that is operatively coupled to the memory and configured to: receive neural signals from the neural signal capture device; determine which of the at least one visual stimulus is the user's object of interest based on the neural signal, the object of interest being inferred from the presence in the neural signal of a component having a property associated with the characteristic modulation of the visual stimulus; and send a command to the controllable object determined to be associated with the object of interest, wherein the controllable object implements an action based on the command.

[0021] In some embodiments, performing the action includes controlling the controllable object to change state from a standby state.

[0022] In some embodiments, at least one controllable object includes a stimulus generator and a lighting unit for outputting visual stimuli generated by the stimulus generator.

[0023] In some embodiments, the light emitting unit and the stimulus generator are provided in an electronic badge that is separate from one or more controllable objects and logically associated with at least one of the controllable objects. Likewise, the light emitting unit outputs visual stimuli generated by the stimulus generator.

[0024] In certain embodiments, the lighting unit is a projector or a laser display device that is operatively coupled to the stimulus generator and projects corresponding visual stimuli onto the controllable object; and wherein the controllable object reflects the projected stimuli.

[0025] In certain embodiments, the or each light emitting unit comprises at least one of: a light emitting diode (LED); an LED array; a liquid crystal display (LCD) device; an organic light emitting diode (OLED) display; an active matrix organic light emitting diode (AMOLED) display; or an electric arc.

[0026] In certain embodiments, the system further includes a processing device comprising the stimulus generator, wherein the processing device is communicatively coupled to the interface device, the processing device being configured to transmit information indicative of the generated visual stimulus to the interface device.

[0027] In certain embodiments, modulation is selectively applied to high spatial frequency (HSF) components of the display data.

[0028] According to a second aspect, the present disclosure relates to a method for operating a brain-computer interface system, the brain-computer interface system comprising a neural signal capture device and at least one light-emitting unit for outputting a visual stimulus generated by a stimulus generator, the visual stimulus having a characteristic modulation, wherein the method comprises, in a hardware interface device operatively coupled to the neural signal capture device and a controllable real-world object: forming an association between the controllable real-world object and at least one visual stimulus; receiving a neural signal associated with a user captured by the neural signal capture device; determining, based on the neural signal, which of the at least one visual stimulus is an object of interest of the user, the object of interest being inferred from the presence in the neural signal of a component having a property associated with the characteristic modulation of the visual stimulus; and sending a command to the controllable object determined to be associated with the object of interest, thereby controlling the controllable object to perform an action based on the command.

[0029] In some embodiments, the method further comprises implementing an action, the implementing action comprising controlling the controllable object to change state from the standby state.

[0030] In some embodiments, at least one controllable object includes a stimulus generator and a lighting unit, and forming an association between the controllable real-world object and the at least one visual stimulus includes controlling the lighting unit to output the visual stimulus generated by the stimulus generator.

[0031] In some embodiments, the light-emitting unit and the stimulus generator are provided in an electronic badge that is separate from one or more controllable objects, and forming an association between the controllable real-world object and at least one visual stimulus includes logically associating the electronic badge with the at least one controllable object, and controlling the light-emitting unit of the electronic badge to output the visual stimulus generated by the stimulus generator.

[0032] In some embodiments, the lighting unit is a projector that is operatively coupled to the stimulus generator, and forming an association between the controllable real-world object and at least one visual stimulus includes controlling the projector to project the corresponding visual stimulus onto the controllable object such that the controllable object reflects the projected stimulus.

[0033] In some embodiments, the brain-computer interface system further comprises a processing device comprising a stimulus generator; the processing device is communicatively coupled to the hardware interface device, and forming an association between the controllable real-world object and at least one visual stimulus further comprises causing the processing device to transmit information indicative of the generated visual stimulus to the interface device.

[0034] In certain embodiments of the method, modulation is selectively applied to high spatial frequency (HSF) components of the display data.

[0035] According to a third aspect, the present disclosure relates to a computer-readable storage medium comprising instructions which, when executed by a machine, cause the machine to perform the above method.

[0036] According to a fourth aspect, the present disclosure relates to a method comprising: displaying visual stimuli associated with a controllable object to a user; capturing neural signals from the user using a neural signal capture device; determining that a visual stimulus in the visual stimuli is an object of attention based on the neural signals; visually displaying feedback near the controllable object associated with the visual stimuli, wherein the feedback highlights the object of attention; and creating a positive feedback loop, using the feedback to amplify the brain response through prolonged and amplified attention, thereby confirming the object of attention as the intended target object.

[0037] According to a fifth aspect, the present disclosure relates to a machine comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising: displaying visual stimuli associated with a controllable object to a user; capturing neural signals from the user using a neural signal capture device; determining that a visual stimulus in the visual stimuli is an object of attention based on the neural signals; visually displaying feedback near the controllable object associated with the visual stimuli, wherein the feedback highlights the object of attention; and creating a positive feedback loop, using the feedback to amplify a brain response through prolonged and amplified attention, thereby confirming the object of attention as an intended target object.

[0038] According to a fifth aspect, the present disclosure relates to a machine-readable medium comprising instructions, which, when executed by a machine, causes the machine to perform operations, the operations comprising: displaying visual stimuli associated with a controllable object to a user; capturing neural signals from the user using a neural signal capture device; determining that a visual stimulus in the visual stimulus is an object of attention based on the neural signals; visually displaying feedback near the controllable object associated with the visual stimulus, wherein the feedback highlights the object of attention; and creating a positive feedback loop, using the feedback to amplify the brain response through prolonged and amplified attention, thereby confirming the object of attention as the intended target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] To easily identify the discussion of any particular element or act, the highest digit or digits in a reference number refer to the figure number in which the element is first introduced.

[0040] Figure 1 shows an electronic architecture for receiving and processing EEG signals according to the present disclosure;

[0041] Figure 2 A system incorporating a brain-computer interface (BCI) according to the present disclosure is shown;

[0042] Figure 3 Various examples of display devices suitable for use with the BCI system of the present disclosure are shown;

[0043] Figure 4 A first exemplary embodiment of a BCI system of the present disclosure is shown;

[0044] Figure 5 A second exemplary embodiment of a BCI system of the present disclosure is shown;

[0045] Figure 6A and Figure 6B Another exemplary embodiment of the BCI system of the present disclosure is shown;

[0046] Figure 7 The main functional blocks in the operation method of the BCI according to the present disclosure are shown.

[0047] Figure 8 is a block diagram illustrating a software architecture in which the present disclosure may be implemented according to some example embodiments; and

[0048] Figure 9 is a diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed, causing the machine to perform any one or more of the methodologies discussed herein, according to some example embodiments. DETAILED DESCRIPTION

[0049] The following description includes systems, methods, techniques, instruction sequences, and computer program products that implement the illustrative embodiments of the present disclosure. In the following description, for illustrative purposes, many specific details are set forth to provide an understanding of the various embodiments of the subject matter of the invention. However, it is apparent to those skilled in the art that the embodiments of the subject matter of the invention can be practiced without these specific details. Generally, known instruction instances, protocols, structures, and techniques do not need to be shown in detail.

[0050] Figure 1 An example of an electronic architecture for receiving and processing EEG signals by means of an EEG device 100 according to the present disclosure is shown.

[0051] To measure diffuse potentials on the surface of a subject's 110 skull, EEG device 100 includes a portable device 102 (ie, a cap or headgear), analog-to-digital conversion (ADC) circuitry 104 , and a microcontroller 106 . Figure 1 The portable device 102 comprises one or more electrodes 108 , typically between 1 and 128 electrodes, advantageously between 2 and 64 electrodes, advantageously between 4 and 16 electrodes.

[0052] Each electrode 108 may include a sensor for detecting electrical signals generated by the subject's neuronal activity and electronic circuitry for pre-processing (e.g., filtering and / or amplifying) the detected signals prior to analog-to-digital conversion: such electrodes are referred to as "active." Figure 1 , wherein the sensor is in physical proximity to the subject's scalp. The electrodes may be suitable for use with a conductive gel or other conductive liquid (referred to as "wet" electrodes) or without such a liquid (i.e., a "dry" electrode).

[0053] Each ADC circuit 104 is configured to convert signals from a given number (eg, between 1 and 128) of active electrodes 108 .

[0054] The ADC circuit 104 is controlled by a microcontroller 106 and communicates with the microcontroller 106, for example, via the SPI (“Serial Peripheral Interface”) protocol. The microcontroller 106 packages the received data for transmission, for example, via Bluetooth, Wi-Fi (“Wireless Fidelity”), or Li-Fi (“Light Fidelity”) to an external processing unit (not shown), such as a computer, a mobile phone, a virtual reality headset, a car or an aircraft computer system, for example, a car computer or computer system.

[0055] In certain embodiments, each active electrode 108 is powered by a battery ( Figure 1 The battery is conveniently provided in the housing of the portable device 102.

[0056] In certain embodiments, each active electrode 108 measures a corresponding potential value, from which the potential measured by the reference electrode (Ei=Vi−Vref) is subtracted, and the difference is digitized by means of the ADC circuit 104 and then sent by the microcontroller 106 .

[0057] In certain embodiments, the method of the present disclosure introduces a target object for display in a graphical user interface of a display device. The target object includes a control item, which in turn is associated with a user-selectable action.

[0058] Figure 2A system incorporating a brain computer interface (BCI) according to the present disclosure is shown. The system incorporates, for example, Figure 1 2. The neural response device 206 of the EEG device 100 is shown in FIG. In the system, an image is displayed on a display of the display device 202. The subject 204 watches the image on the display and focuses on the target object 210.

[0059] In an embodiment, the display device 202 displays at least the target object 210 as a graphical object having a varying temporal signature that is different from the temporal signature of the background and / or other displayed objects in the display. The varying temporal signature can be, for example, a continuous or time-locked flickering effect that changes the appearance of the target object at a rate greater than 6 Hz. In the case where more than one graphical object is a potential target object (i.e., in the case where a viewing subject is provided with a choice of target objects to focus attention on), each object is associated with a discrete spatial and / or temporal code.

[0060] The neural response device 206 detects neural responses associated with attention focused on the target object (i.e., tiny electrical potentials indicative of brain activity in the visual cortex); thus, the visual perception of the changing temporal features of the target object acts as a stimulus in the subject's brain, generating a specific brain response consistent with the code associated with the target object of attention. The detected neural response (e.g., electrical potential) is then converted into a digital signal and transmitted to the processing device 208 for decoding. Examples of neural responses include visual evoked potentials (VEPs), which are commonly used in neuroscience research. The term VEP encompasses: traditional SSVEPs, as mentioned above, in which the stimulus oscillates at a specific frequency; and other methods such as code-modulated VEPs, in which the stimulus is subjected to variable or pseudo-random temporal coding.

[0061] The processing device 208 executes instructions to interpret the received neural signals to determine, in real time, feedback indicating the target object with the current (visual) focus of attention. Decoding the information in the neural response signal relies on the correspondence between that information and one or more aspects of the temporal distribution of the target object (i.e., stimulus). In some embodiments, the processing device 208 and the neural response device 206 can be provided in a single device so that the decoding algorithm is performed directly on the detected neural response. Thus, a BCI that utilizes visually associated neural signals can be used to determine which objects on the screen the user is focusing on.

[0062] In some embodiments, the processing device may conveniently generate image data for presentation on the display device 202 that includes the target object changing over time.

[0063] Feedback can be conveniently visually presented on a display screen. For example, a display device can display an icon, cursor, crosshairs, or other graphical objects or effects very close to the target object, highlighting the object that appears to be the focus of current visual attention. Obviously, such a visual display of feedback has a reflexive cognitive effect on the perception of the target object, amplifying the brain response. This positive feedback (wherein the seemingly target object is confirmed as the expected target object due to the extended amplified attention) is referred to as "neural synchronization" in this article.

[0064] Studies of the way human visual sensing operates have shown that when staring at a screen with multiple objects and focusing on one of these objects, the human visual system will be good at accepting both high spatial frequencies (HSF) and low spatial frequencies (LSF). Evidence shows that the human visual system is primarily sensitive to the HSF component of a specific display area of ​​interest (e.g., the object the user is looking at). In contrast, for peripheral objects, the human visual system is primarily sensitive to their LSF components. In other words, the picked-up neural signal will be substantially affected by both the HSF component from the target of interest and the LSF component from the peripheral targets. However, since all objects cause a certain proportion of both HSF and LSF, processing the neural signal to determine the object of interest may be hindered by the LSF noise contributed by the peripheral objects. This tends to make the identification of the object of interest less accurate and timely.

[0065] As the human visual system is tuned to process multiple stimuli at different locations in the visual field in parallel (usually unconsciously), peripheral object stimuli will continue to trigger neural responses in the user's brain even if they appear at the periphery of the visual field. As a result, this causes competition between multiple stimuli and makes specific neural decoding of the object of attention (target) more difficult.

[0066] Co-pending International Patent Application No. PCT / EP2020 / 081348 (Docket No. 5380.002WO1), filed on November 6, 2020 (the entire specification of which is incorporated herein by reference), describes a method for the following challenge: determining an object of interest (target) from objects in the periphery of the target (distractors) with a certain speed and accuracy. The method described in the international patent application relies on the characteristics of the human visual system discussed above. Multiple objects are displayed so that each object is divided into a version consisting only of the LSF component of the object and a version consisting only of the HSF component. In one example, a flickering visual stimulus used to elicit a decodable neural response (e.g., SSVEP) is transmitted only through the HSF version of the object. The flickering HSF version is superimposed on the LSF version (which does not flicker).

[0067] Known systems in the medical or related research fields generally include a head-mounted device with attachment locations for receiving various sensors / electrodes. The electronic circuit is then connected to the housing of the electrodes and the acquisition chain (i.e., an assembly of connected components for acquiring EEG signals). Therefore, EEG devices are usually formed of three different components that the operator / presenter must assemble each time they are used. Similarly, the nature of EEG devices makes technical assistance desirable, if not required.

[0068] The BCI described above can be used in conjunction with real-world objects to enable the objects to be controlled or otherwise interacted with. In some embodiments, stimulus generation is handled by one or more light sources (e.g., light emitting diodes, LEDs, or small display units) provided in association with (or on the surface of) the controllable object.

[0069] In certain embodiments, the generation of the stimuli is handled by a projector or a scanning laser device, such that the visual stimuli are projected onto the controllable object, and the controllable object outputs the visual stimuli by reflecting the projected stimuli.

[0070] As in the case of a BCI using a display screen through which a user interacts with on-screen objects, the controllable objects of the present disclosure can themselves be made to present visual stimuli with characteristic modulations (e.g., flickering stimuli) so that the neural responses to the presence of these stimuli become apparent and can be decoded from neural signals captured by a neural signal capture device (e.g., an EEG device).

[0071] In some embodiments, determining the focus of attention on a visual display of a controllable device is used to send a command to the controllable object. The controllable object can then perform an action based on the command: for example, the controllable object can emit an audible sound, unlock a door, turn on or off, change operating state, etc. The action can also provide the user with visual or other feedback associated with the controllable object: this can be used in the positive feedback loop discussed above, but it can also provide real-time feedback indicating the degree of certainty that the object has been identified as the focus of attention, to give the user an idea of ​​his or her level of concentration.

[0072] Figure 4A first exemplary embodiment is shown, in which at least one surface of a remote-controlled RC car 404 incorporates an LED array 410 capable of generating a visual display. In one embodiment, the RC car includes: a receiving circuit that picks up commands sent from a stimulus generator (which can conveniently be the decoding computer 104, as shown) via wireless, one-way communication means (e.g., radio frequency (RF); infrared; or near-field communication (NFC); ​​a processing unit (i.e., a microcontroller) that interprets the received commands and controls the operation of the RC car and / or the visual display accordingly; and a display driver circuit that applies light modulation to a display device (here, an LED panel 410) of the type used to generate the visual display. RC car 404 can also be configured to receive commands from a conventional radio controller 414 via a radio communication link 412. Thus, the operation of the RC car can be controlled by commands originating from the decoding computer 104 (as a result of decoded brain activity in the user 402) and commands from the radio controller 414. In some embodiments, the stimulus generator, along with the visual display, is embedded in the RC car 404. The commands from the decoding computer 104 may include a signal for synchronizing or configuring the modulation generated by the stimulus generator with the decoding computer. If it is assumed that brain activity is monitored using SSVEP at a predetermined set frequency, such a link between the decoding computer and the subject can be omitted. The microcontroller can be programmed to modulate light of the same frequency in an asynchronous manner, so that the modulation does not need to be transmitted or even synchronized with the decoding computer 104.

[0073] The visual display (i.e., LED array 410) emits visual stimuli that include temporal modulation (i.e., a flickering effect) unique to a given RC car 404. By visually attending to that car (and thus the stimuli output by that car), the user generates a neural response that can, in turn, be captured by a BCI (e.g., the BCI described above). The neural signals captured from the user's brain encode the temporal modulation. The BCI then decodes the neural signals to determine which car (if any) is the object of the user's attention (or focus). In certain embodiments, the BCI includes a communication unit through which commands can be sent to the remote-controlled car. This command can, for example, be an instruction to enter an active state, whereupon control signals sent by a conventional remote control device can be transmitted to the car. The command can conveniently be a "keep on" command, whereby continued attention results in repeated transmission of the command, but shifted attention results in the car being deactivated. In the presence of more than one such remote-controlled car (each generating different stimuli), the user can change attention (resume the unattended car to a standby or inactive state) and apply the remote control to the car that forms the new focus of attention.

[0074] Figure 5A second exemplary embodiment is shown in which a stimulus comprising a temporal modulation (i.e., a flickering effect) is projected onto at least one surface of a real-world object 504 associated with a controllable object 514 using a projector 510, the stimulus being generated by a stimulus generator 512. When a user 502 equipped with a BCI views the light reflected from the object 504, the reflected light produces a decodable neural response in the user's brain (which can be captured by the BCI). Figure 5 A plurality of real-world objects 504, 506, etc. are shown - a projector 510 projects a corresponding stimulus onto each object. The BCI then decodes the neural signals to determine which of the objects (if any) is the user's object of attention. Each real-world object 504, 506, etc. is associated with a corresponding command or interaction with an associated controllable object 514. In some embodiments, the BCI includes a communication unit by which a command can be sent to a controllable object identified as an object of attention. The command can be, for example, an instruction to enter an active state. In one embodiment, as shown, the controllable object includes a speaker 514, and the command can be a command to control the speaker to reproduce an audio signal such as an alarm or a musical tone. In some embodiments, the real-world object onto which the stimulus is projected or each real-world object itself can be a controllable object.

[0075] Figure 6A Yet another exemplary embodiment is shown in which three visual displays 610, 610', 610" are provided remotely from the controllable object 604. Each of the visual displays 610, 610', 610" (e.g., respective LED arrays) generates a corresponding different visual stimulus including a temporal modulation (i.e., a flickering effect). The visual stimulus used here utilizes the high decodability of HSF elements, such as a plurality of tiny line drawings (by using an opaque physical mask over the LED arrays, which mask can be considered a printed Gabor filter). In some embodiments, the mask itself is controllable: a controllable motor can switch a shutter device between open and closed states, alternatively, a transparent liquid crystal device can be used to controllably mask or transmit incident light from one or more light sources, providing a backlit LCD display with a controllable temporally and / or spatially variable mask.

[0076] In the example shown, each visual display 610, 610', 610" incorporates a logo or icon (here, colors: red R; green G; blue B) indicating a selectable option. The logo or icon may be formed by providing an engraved pattern in an otherwise opaque physical mask or by controlling the pattern of light transmitted by a shutter device or LCD. By focusing attention on one of the visual displays 610, 610', 610", the user 602 equipped with the BCI generates a neural response corresponding to the visual stimulus unique to that visual display 610. This is then interpreted as an intention to select the corresponding option (assuming red R). The intended option is then used by the processor 104 of the BCI to generate a command that is sent to the controllable object 604. In the example shown, the controllable object is a color tunable lamp 604, and the command may be an instruction to control the lamp to emit red light. As Figure 4 The RC car example in Figure 6A The circuitry used at each of the visual displays 610, 610', 610" in the example includes: a receiving circuit that picks up commands sent from the stimulus generator 612; a processing unit; and a display driver circuit. Although the communication link between the processor 104 (i.e., the decoding computer) of the BCI and the subject 604 is Figure 6A It is shown as a wireless link, but it can also be wired.

[0077] In some embodiments, each visual display 610, 610', 610" can display a visual stimulus associated with a respective command to be sent to a corresponding different controllable object. Thus, there may be multiple different lights, each obeying a command associated with a respective one of the displayed visual stimuli. Rather than simply controlling the color of light emitted by one tunable lamp, note that one visual stimulus can activate one or more of the lights; note that another visual stimulus (displayed on another display) can, for example, dim one or more of the lights; and note that a third visual stimulus (displayed on yet another display) can cause multiple different lights to enter an economy mode in which the lights only illuminate when there is not enough natural light.

[0078] exist Figure 6BIn another exemplary embodiment shown in , a visual display referred to as an "electronic badge" takes the form of a physical, movable object, separate from the real-world controllable objects, but logically associated with at least one of them. In the present disclosure, these electronic badges provide a small, movable, programmable screen (e.g., 3 cm x 3 cm) that displays one or more visual stimuli associated with other (controllable) objects or specific functions of such objects. The screen may be, for example, a backlit LCD, OLED, or AMOLED display. The screen may be programmable to emit light with a temporal modulation similar to the light sources described above, thereby generating one or more visual stimuli. Unlike Figure 6A Visual display, Figure 6B The electronic badges 620, 620', 620" each include their own stimulus generator 612' (see inset "exploded" view of badge 620") in addition to a processing unit 614, display driver circuitry 616, and a screen 618 for outputting visual stimuli.

[0079] The electronic badge may be embedded within a physical object (which may be a controllable object associated with the or each stimulus generated by the electronic badge): badge 620 is embedded in controllable light 604'. Alternatively, the badge may be placed on or affixed (temporarily or permanently) to the surface of a physical object, or indeed any surface from which one can conveniently interact with or control a controllable physical object, such as a nearby wall, control panel, or item of furniture. Figure 6B In the embodiment of the present invention, the controllable light 604' can also be controlled by interacting with the electronic badge 620' in the control panel 606. In other embodiments, the electronic badge 620" can be portable or even wearable. In some embodiments, the electronic badge includes a battery and a wireless communication module. The electronic badge can be attached to the surface in any conventional manner, such as a clip, adhesive, magnet, screw, bolt, or any other such fixing means.

[0080] In some cases, such as Figure 6B As in , more than one electronic badge can be used to control different types of interactions with a single physical object: each electronic badge displays a different visual stimulus and a different action is associated with each stimulus. Controllable light 604' can be turned on by badge 620 and dimmed by badge 620'. In further illustration, consider a case where instead of Figure 6AWith the three mask visual displays, three electronic badges can be arranged (presumably on a nearby table) to display visual stimuli that differ in their respective modulations, each badge having a different associated command for the color tunable light 604. The respective badges can also be controlled to have different visual appearances, such as emitting light in a color that represents the desired effect of the color tunable light.

[0081] In some cases, the visual stimulus displayed on the screen of the electronic badge can be applied to the entire screen or to corresponding portions of the screen according to the programming of the badge. As previously described, the visual stimulus is generated to induce a decodable neural response in the brain of the BCI user who pays attention to the stimulus. The visual stimulus can be presented as a plurality of tiny lines or "beans" with modulation applied (reducing the flicker effect without compromising the ability to evoke a neural response when the given stimulus is the focus of the BCI user's attention). The neural response can be measured and decoded to determine which electronic badge is the focus of the user's attention, and because the badge is logically associated with a real-world object and / or a control action with a real-world object, it is determined that the real-world object is the object of interest and / or control.

[0082] In some cases, each badge, or each of the multiple portions of a badge, may display a respective symbol corresponding to an associated action / command (e.g., open / close, play / pause, forward, etc.) that controls the operation of one or more controllable real-world objects. Different visual stimuli may then be applied to the respective symbols. In some embodiments, the displayed symbols may be changed based on the programming of the badge. For example, a badge 620″ showing a “play” symbol may have the play symbol replaced with a “pause” symbol when the controlled audio system 604″ is in playback. Similarly, in a badge affixed to a controllable door, note that the stimulus displayed on the badge may be interpreted as an “open” command or a “close” command based on the current state of the door.

[0083] The controllable real-world objects to which the light-emitting unit applies visual stimulation are not limited to the objects mentioned in the examples above. In addition to controllable doors, lights, remote-controlled toys, audio systems / speakers, more generally, the same contactless control can be permitted on the operation of switches and valves. Users who cannot operate controllable devices with their hands can still control their operations: from operating a faucet to provide water in a sink to selecting items to be retrieved by a robot from a shelf in a warehouse. Various aspects of the present disclosure can be applied in many different situations, including home, healthcare, commerce (retail, wholesale and logistics), agriculture and maritime.

[0084] Figure 7 A BCI system according to the present disclosure (e.g., Figure 2) is a main functional block in a method of operation of a BCI system (shown in FIG). In block 702, the interface device 208 forms an association between a controllable real-world object and at least one visual stimulus. In block 704, the interface device 208 receives a neural signal associated with the user captured by the neural signal capture device 206. In block 706, the interface device 208 determines which of the at least one visual stimulus is the user's object of interest based on the neural signal, the object of interest being inferred from the presence in the neural signal of a component having a property associated with a characteristic modulation of the visual stimulus. In block 708, the interface device 208 sends a command to the controllable object determined to be associated with the object of interest, thereby controlling the controllable object to implement an action based on the command.

[0085] You can use about Figure 2 The BCI described in

[15] is used to confirm the user's intention, such as to initiate an action such as a request for information, a switch in the control state of a real-world object, or the activation / selection of an object in a mixed reality setting (e.g., for control).

[0086] Figure 8 is a block diagram illustrating an example software architecture 806 that can be used with various hardware architectures described herein (e.g., Figure 2 The processing device 208 is used in combination. Figure 8 is a non-limiting example of a software architecture, and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. The software architecture 806 may be implemented in a manner such as Figure 2 The processing device 208 or Figure 9 900, which includes, among other things, a processor 904, a memory 906, and input / output (I / O) components 918. A representative hardware layer 852 is shown and may represent, for example, Figure 9 900. A representative hardware layer 852 includes a processing unit 854 having associated executable instructions 804. Executable instructions 804 represent executable instructions of a software architecture 806, including implementations of the methods, modules, etc. described herein. The hardware layer 852 also includes a memory and / or storage module, shown as a memory / storage device 856, which also has executable instructions 804. The hardware layer 852 may also include other hardware 858, such as specialized hardware for interfacing with EEG electrodes and / or for interfacing with a display device.

[0087] exist Figure 8In the example architecture of , software architecture 806 can be conceptualized as a stack of layers in which each layer provides specific functionality. For example, software architecture 806 may include layers such as operating system 802, library 820, framework or middleware 818, application 816, and presentation layer 814. Operationally, application 816 and / or other components within a layer may call application programming interface (API) call 808 through the software stack and receive a response as message 810. The layers shown are representative in nature, and not all software architectures have all layers. For example, some mobile operating systems or dedicated operating systems may not provide framework / middleware 818, while other operating systems may provide such a layer. Other software architectures may include additional or different layers.

[0088] The operating system 802 can manage hardware resources and provide common services. The operating system 802 can include, for example, a kernel 822, services 824, and drivers 826. The kernel 822 can act as an abstraction layer between the hardware and other software layers. For example, the kernel 822 can be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. The services 824 can provide other common services to other software layers. The drivers 826 can be responsible for controlling or interfacing with the underlying hardware. For example, depending on the hardware configuration, the drivers 826 can include a display driver, an EEG device driver, a camera driver, drives, flash drives, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Drivers, audio drivers, power management drivers, etc.

[0089] The libraries 820 can provide common infrastructure that can be used by applications 816 and / or other components and / or layers. The libraries 820 generally provide functionality that enables other software modules to perform tasks more easily than by directly interfacing with underlying operating system 802 functionality (e.g., kernel 822, services 824, or drivers 826). The libraries 820 can include system libraries 844 (e.g., the C standard library), which can provide functionality such as memory allocation, string manipulation, mathematical functions, and the like. Additionally, the libraries 820 can include API libraries 846, such as media libraries (e.g., libraries that support rendering and manipulation of various media formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG), graphics libraries (e.g., the OpenGL framework that can be used to render 2D and 3D graphics content on a display), database libraries (e.g., SQLite that can provide various relational database functions), web libraries (e.g., WebKit that can provide web browsing functionality), and the like. The libraries 820 can also include a variety of other libraries 848 to provide numerous other APIs to the applications 816 and other software components / modules.

[0090] The framework 818 (sometimes also referred to as middleware) provides a higher-level common infrastructure that can be used by applications 816 and / or other software components / modules. For example, the framework / middleware 818 can provide various graphical user interface (GUI) functions, advanced resource management, advanced location services, etc. The framework / middleware 818 can provide a wide range of other APIs that can be used by applications 816 and / or other software components / modules, some of which may be specific to a particular operating system or platform.

[0091] Applications 816 include built-in applications 838 and / or third-party applications 840 .

[0092] Applications 816 may create user interfaces to interact with users of the system using built-in operating system functionality (e.g., kernel 822, services 824, and / or drivers 826), libraries 820, or framework / middleware 818. Alternatively or additionally, in some systems, interaction with the user may occur through a presentation layer, such as presentation layer 814. In these systems, application / module "logic" may be separate from aspects of the application / module that interact with the user.

[0093] Figure 9 900 according to some example embodiments (e.g. Figure 2 The block diagram of the processing device 208 of the machine 900 is capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any one or more of the methods discussed herein. Specifically, Figure 9A graphical representation of a machine 900 is shown in the form of an example computer system in which instructions 911 (e.g., software, programs, applications, applet, app, or other executable code) for causing the machine 900 to perform any one or more of the methodologies discussed herein may be executed. Similarly, the instructions 911 may be used to implement the modules or components described herein. The instructions 911 transform the general, unprogrammed machine 900 into a specific machine that is programmed to perform the functions described and illustrated in the manner described. In alternative embodiments, the machine 900 operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 900 may operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 900 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch or head-mounted display), a smart home device (e.g., a smart appliance), other smart devices, a network service tool, a network router, a network switch, a network bridge, or any machine capable of executing, sequentially or otherwise, the instructions 911 specifying actions to be taken by the machine 900. Furthermore, while only a single machine 900 is shown, the term "machine" shall also be taken to include an aggregate of machines that individually or jointly execute the instructions 911 to perform any one or more of the methodologies discussed herein.

[0094] The machine 900 may include a processor 904, a memory 906, and input / output (I / O) components 918 that may be configured to communicate with each other, for example, via a bus 902. In an example embodiment, the processor 904 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), other processors, or any suitable combination thereof) may include, for example, a processor 908 and a processor 912 that may execute instructions 911. The term "processor" is intended to include a multi-core processor that may include two or more independent processors (sometimes referred to as "cores") that may execute instructions concurrently. Although Figure 9 Multiple processors are shown, but the machine 900 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0095] The memory 906 may include a memory 914, such as a main memory, static memory, or other memory storage device, and a storage unit 916, both of which are accessible by the processor 904, for example, via the bus 902. The storage unit 916 and the memory 914 store instructions 911 that embody any one or more of the methods or functions described herein. The instructions 911 may also reside, completely or partially, within the memory 914, within the storage unit 916, within at least one of the processors 904 (e.g., within a cache memory of the processor), or any suitable combination thereof during execution thereof by the machine 900. Thus, the memory 914, the storage unit 916, and the memory of the processor 904 are examples of machine-readable media.

[0096] As used herein, a "machine-readable medium" refers to a device capable of temporarily or permanently storing instructions and data, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage devices (e.g., erasable programmable read-only memory (EEPROM)), and / or any suitable combination thereof. The term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database or associated caches and servers) that can store instructions 911. The term "machine-readable medium" should also be taken to include any medium or combination of multiple media that can store instructions (e.g., instructions 911) for execution by a machine (e.g., machine 900), such that the instructions, when executed by one or more processors (e.g., processor 904) of machine 900, cause machine 900 to perform any one or more of the methodologies described herein. Thus, a "machine-readable medium" refers to a single storage device or device, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or devices. The term "machine-readable medium" does not include the signal itself.

[0097] The input / output (I / O) components 918 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific input / output (I / O) components 918 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone or a user interface machine will likely include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It will be appreciated that the input / output (I / O) components 918 may include Figure 9 Many other components not shown.

[0098] The input / output (I / O) components 918 are grouped by function for the purpose of simplifying the following discussion only and are by no means limiting. In various example embodiments, the input / output (I / O) components 918 may include output components 926 and input components 928. The output components 926 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), auditory components (e.g., speakers), tactile components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. The input components 928 may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric input, optical keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or other pointing instruments), touch-sensitive input components (e.g., physical buttons, touch screens that provide touch gestures or the location and / or force of touch, or other touch-sensitive input components), audio input components (e.g., microphones), etc.

[0099] In yet another example embodiment, the input / output (I / O) component 918 may include various other components such as a biometric component 930, a motion component 934, an environmental component 936, or a positioning component 938. For example, the biometric component 930 may include components for detecting expressions (e.g., hand expressions, facial expressions, vocal expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves such as output from an EEG device), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 934 may include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The environment component 936 may include, for example, an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects the concentration of hazardous gases for safety or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The positioning component 938 may include a position sensor component (e.g., a global positioning system (GPS) receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc.

[0100] Various technologies may be used to implement communications. Input / output (I / O) components 918 may include communications components 940 operable to couple machine 900 to network 932 or device 920 via coupling 924 and coupling 922, respectively. For example, communications components 940 may include a network interface component or other suitable device to interface with network 932. In yet another example, communications components 940 may include wired communications components, wireless communications components, cellular communications components, near field communications (NFC) components, Components (e.g. Low power consumption), Device 920 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via a universal serial bus (USB)). In the case where the EEG device or display device is not integrated with machine 900, device 920 may be an EEG device (e.g., neural response device 206) and / or a display device (e.g., Figure 2 and Figure 3 display device).

[0101] Although described through a number of detailed exemplary embodiments, the portable device for acquiring electroencephalographic signals according to the present disclosure includes various variations, modifications and improvements that are obvious to those skilled in the art, and it should be understood that these various variations, modifications and improvements fall within the scope of the subject matter of the present disclosure as defined by the appended claims.

[0102] Although an overview of the inventive subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of the embodiments of the present disclosure. Such embodiments of the inventive subject matter may be referred to herein individually or collectively by the term "invention," which is merely a matter of convenience and is not intended to autonomously limit the scope of this application to any single disclosure or inventive concept in the event that more than one disclosure or inventive concept is in fact disclosed.

[0103] The embodiments shown herein are described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, the specific embodiments should not be considered in a limiting sense, and the scope of the various embodiments is limited only by the appended claims and the full scope of equivalents to which such claims are entitled.

[0104] As used herein, the term "or" may be interpreted as inclusive or exclusive. In addition, multiple instances may be provided for resources, operations, or structures described herein as single instances. In addition, the boundaries between various resources, operations, modules, engines, and data stores are arbitrary to some extent, and specific operations are shown in the context of specific illustrative configurations. Other allocations of functions are conceivable, and other allocations of functions may fall within the scope of various embodiments of the present disclosure. Typically, structures and functions presented as separate resources in the example configurations may be implemented as combined structures or resources. Similarly, structures and functions presented as single resources may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of the embodiments of the present disclosure as represented by the appended claims. Therefore, the specification and drawings are to be considered in an illustrative rather than a restrictive sense.

[0105] Thus, this disclosure describes systems and methods for improving the accuracy, speed performance, and visual comfort of BCIs.

[0106] Example

[0107] To better illustrate the systems and methods disclosed herein, a non-limiting list of examples is provided here:

[0108] 1. A brain-computer interface system comprising:

[0109] at least one light emitting unit, the at least one light emitting unit outputting a corresponding visual stimulus generated by a stimulus generator, the visual stimulus having a characteristic modulation;

[0110] at least one controllable object configured to receive user commands, each controllable object being associated with at least one visual stimulus;

[0111] a neural signal capturing device configured to capture neural signals associated with the user;

[0112] an interface device operatively coupled to the neural signal capturing device and the controllable object, the interface device comprising:

[0113] Memory; and

[0114] a processor operatively coupled to the memory and configured to:

[0115] receiving the neural signal from the neural signal capturing device;

[0116] determining, based on the neural signal, which of the at least one visual stimulus is an object of attention of the user, the object of attention being inferred from the presence in the neural signal of a component having a property associated with a characteristic modulation of the visual stimulus; and

[0117] sending a command to the controllable object determined to be associated with the object of interest,

[0118] The controllable object performs an action based on the command.

[0119] 2. The brain-computer interface system of Example 1, wherein implementing the action comprises controlling the controllable object to change state from a standby state.

[0120] 3. The brain-computer interface system according to Example 1 or 2, wherein the at least one controllable object includes the stimulus generator and the light-emitting unit for outputting the visual stimulus generated by the stimulus generator.

[0121] 4. A brain-computer interface system according to Example 1 or Example 2, wherein the stimulation generator and the light-emitting unit are arranged in an electronic badge, which is separate from the at least one controllable object but logically associated with the at least one controllable object.

[0122] 5. The brain-computer interface system according to example 1 or example 2, wherein the light emitting unit is a projector that is operatively coupled to the stimulus generator and projects the corresponding visual stimulus onto the controllable object; and

[0123] Therein, the controllable object reflects the projected stimulus.

[0124] 6. A brain-computer interface system according to any one of Examples 1 to 5, wherein the light-emitting unit or each light-emitting unit includes at least one of the following: a single light-emitting diode (LED); an LED array; a liquid crystal display (LCD) device; an organic light-emitting diode (OLED) display; or an electric arc.

[0125] 7. The brain-computer interface system according to any one of Examples 1 to 6 further includes a processing device, which includes the stimulus generator, wherein the processing device is communicatively coupled to the interface device and is configured to transmit information indicative of the generated visual stimulus to the interface device.

[0126] 8. A brain-computer interface system according to any one of Examples 1 to 7, wherein the modulation is selectively applied to a high spatial frequency (HSF) component of the display data.

[0127] 9. A method of operating a brain-computer interface system, the brain-computer interface system comprising a neural signal capture device and at least one light-emitting unit for outputting a visual stimulus generated by a stimulus generator, the visual stimulus having a characteristic modulation,

[0128] The method includes, in a hardware interface device operatively coupled to the neural signal capture device and a controllable real-world object:

[0129] forming an association between the controllable real-world object and the at least one visual stimulus;

[0130] receiving a neural signal associated with a user captured by the neural signal capturing device;

[0131] determining, based on the neural signal, which of the at least one visual stimulus is an object of attention of the user, the object of attention being inferred from the presence in the neural signal of a component having a property associated with a characteristic modulation of the visual stimulus; and

[0132] A command is sent to the controllable object determined to be associated with the object of interest, thereby controlling the controllable object to perform an action based on the command.

[0133] 10. The method of example 9, wherein implementing the action comprises controlling the controllable object to change state from a standby state.

[0134] 11. A method according to Example 9 or Example 10, wherein the at least one controllable object includes the stimulus generator and the light-emitting unit, and wherein forming an association between the controllable real-world object and the at least one visual stimulus includes controlling the light-emitting unit to output the visual stimulus generated by the stimulus generator.

[0135] 12. A method according to Example 9 or Example 10, wherein the stimulus generator and the light-emitting unit are arranged in an electronic badge, the electronic badge is separated from the at least one controllable object, and wherein forming an association between the controllable real-world object and the at least one visual stimulus includes logically associating the electronic badge with the at least one controllable object, and controlling the light-emitting unit of the electronic badge to output the visual stimulus generated by the stimulus generator.

[0136] 13. A method according to Example 9 or Example 10, wherein the light-emitting unit is a projector, the projector is operatively coupled to the stimulus generator, and wherein forming an association between the controllable real-world object and the at least one visual stimulus includes controlling the projector to project the corresponding visual stimulus onto the controllable object so that the controllable object reflects the projected stimulus.

[0137] 14. A method according to any one of Examples 9 to 13, wherein the brain-computer interface system further includes a processing device, the processing device including the stimulus generator; wherein the processing device is communicatively coupled to the hardware interface device, and wherein forming an association between the controllable real-world object and the at least one visual stimulus further includes causing the processing device to transmit information indicative of the generated visual stimulus to the interface device.

[0138] 15. A method according to any one of Examples 9 to 14, wherein the modulation is selectively applied to a high spatial frequency (HSF) component of the display data.

[0139] 16. A computer-readable storage medium carrying instructions that, when executed by a machine, cause the machine to perform the method of any one of Examples 9 to 15.

[0140] 17. An electronic badge for cooperating with the brain-computer interface system according to any one of Examples 1 to 8, the electronic badge comprising at least one of the light emitting units and a stimulus generator.

Claims

1. A method comprising: displaying visual stimuli associated with the controllable object to the user; capturing neural signals from the user using a neural signal capturing device; determining, based on the neural signal, that a visual stimulus in the visual stimulus is an object of attention; visually displaying feedback in proximity to the controllable object associated with the visual stimulus, wherein the feedback highlights the object of interest; and A positive feedback loop is created, using the feedback to amplify the brain's response through prolonged, amplified attention, thereby confirming the object of attention as the intended target object.

2. The method according to claim 1, wherein The visual stimulus has a characteristic modulation including a temporal modulation such that the visual stimulus flickers at a rate exceeding 6 Hz.

3. The method according to claim 1, wherein The neural signal includes steady-state visual evoked potential (SSVEP).

4. The method according to claim 1, wherein The neural signal capture device includes an electroencephalogram (EEG) device having electrodes positioned on the user's scalp to detect electrical potentials indicative of brain activity.

5. The method according to claim 1, wherein The feedback includes at least one of an icon, a cursor, or a crosshair displayed proximate to the controllable object.

6. The method according to claim 1, wherein The visual stimulation is selectively applied to a high spatial frequency HSF component of the visual stimulation.

7. The method according to claim 1, wherein The controllable object includes at least one of a remote controlled car, a light, an audio system, a door, a switch, and a valve.

8. The method of claim 1, further comprising using the extended, magnified attention to determine the intended target object from peripheral objects.

9. A machine comprising: at least one processor; as well as at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising: displaying visual stimuli associated with the controllable object to the user; capturing neural signals from the user using a neural signal capturing device; determining, based on the neural signal, that a visual stimulus in the visual stimulus is an object of attention; visually displaying feedback in proximity to the controllable object associated with the visual stimulus, wherein the feedback highlights the object of interest; and A positive feedback loop is created, using the feedback to amplify the brain's response through prolonged, amplified attention, thereby confirming the object of attention as the intended target object.

10. A machine-readable medium comprising instructions that, when executed by a machine, cause the machine to perform operations comprising: displaying visual stimuli associated with the controllable object to the user; capturing neural signals from the user using a neural signal capturing device; determining, based on the neural signal, that a visual stimulus in the visual stimulus is an object of attention; visually displaying feedback in proximity to the controllable object associated with the visual stimulus, wherein the feedback highlights the object of interest; and A positive feedback loop is created, using the feedback to amplify the brain's response through prolonged, amplified attention, thereby confirming the object of attention as the intended target object.

Images