Brain-computer interface with vibro-tactile stimulations provided by a transducer

Abstract

The invention relates to a method for the somaesthetic stimulation of at least one user (UT) for the purpose of collecting at least one electroencephalogram signal (EEG) comprising an evoked potential with a view to activating a brain-computer interface (BCI) based on somaesthetic stimuli applied to the user's skin. In particular, with the user positioned in an environment (ENV), the stimuli (STS) are applied to the environment by a transducer (HP), the user's skin being in contact with the environment and thus receiving the stimuli via the environment.

Description

Direct neural interface with vibro-tactile stimulation via a transducer

[0001] This disclosure falls within the domain of the detection of potentials evoked by direct neural interfaces (or BCIs hereafter, for "Brain Computer Interface").

[0002] More specifically, a somatosensory evoked potential (SEP) is a signal that appears in EEG (electroencephalogram) signals when the subject (typically a human) is subjected to somatosensory stimulation, that is, sensory stimulation (pressure, heat, pain, etc.) of one or more regions of the body (skin, tendon, joints, etc.), notably through tactile stimulation. Specific equipment (a BCI headset, for Brain-Computer Interface) collects the EEG signals in order to measure them and identify the frequency of the tactile stimulation within these signals.

[0003] This approach is commonly referred to as "SSSEP" for "Steady-State Somatosensory Evoked Potential".

[0004] When using a BCI interface based on such a SSSEP technique, the stimuli typically consist of a carrier frequency (e.g., 200 Hz) modulated by a signal at a lower frequency, such as a square wave with a frequency between 10 and 40 Hz. This stimulus is generally delivered to the user either electrically (electrodes placed on the skin) or mechanically via a vibro-tactile device also placed on the user's skin. The stimulus thus delivered generates a somatosensory evoked potential in the brain, and the recorded EEG signals have the same frequency as the modulation at the aforementioned lower frequency (between 10 and 40 Hz). This measurement can be performed, among other methods, by a device such as an EEG sensor headset (or any other medical imaging system, such as MRI, MEG, or others) worn by the person exposed to the stimulus.

[0005] One drawback of this approach is that the device that applies the stimulus to the user must be in contact with the user's skin, in addition to the EEG sensor device that collects and measures brain signals. Summary

[0006] This disclosure improves the situation.

[0007] A method is proposed for somatosensory stimulation of at least one user to collect at least one electroencephalogram signal containing an evoked potential in order to activate a direct neural interface based on somatosensory stimuli from the user's skin. In particular, the user being placed in an environment, said stimuli are applied to the environment by a transducer, the user's skin being in contact with the environment and receiving said stimuli via said environment.

[0008] Such a design makes it possible not to apply tactile stimuli directly to the user's skin, but to transmit them into the user's environment, which then acts as a transducer for these stimuli.

[0009] In one implementation, these stimuli can be applied by emitting an acoustic wave into the environment.

[0010] This can typically be a low-frequency acoustic wave (below the hearing threshold of 50 Hz typically), or a high-frequency ultrasonic wave (above the hearing threshold of 20 kHz typically).

[0011] Thus, the acoustic wave may then contain at least one carrier frequency which is inaudible to the user.

[0012] The acoustic wave can be infrasonic, with a frequency between 4 and 40 Hz, or alternatively, ultrasonic and with a frequency greater than 20 kHz.

[0013] In a given implementation, the acoustic wave may contain a plurality of carrier frequencies inaudible to the user, forming a noise inaudible to the user.

[0014] This noise is a mixture of frequencies that are all in ranges inaudible to the user.

[0015] This noise can therefore include a plurality of frequencies ranging from 4 to 40 Hz, for example.

[0016] To form the stimulus signal that can cause an evoked potential, the acoustic wave, at the aforementioned inaudible carrier frequency, is further modulated by a chosen modulation frequency, and thus the electroencephalogram signal, collected to activate the direct neural interface, includes an evoked potential at the aforementioned chosen modulation frequency.

[0017] The aforementioned modulation frequency can be chosen, for example, in a range between 10 and 40 Hz.

[0018] In a typical implementation, the modulation can be applied in the form of beeps of a chosen frequency, between 10 and 40 Hz.

[0019] In an implementation, stimuli are applied to the environment by a directionality source chosen to deliver contactless stimuli to one or more users simultaneously.

[0020] For example, stimuli can be applied to the environment by a chosen directionality source to deliver contactless stimuli to a plurality of users simultaneously. In such an implementation, each of these users can be equipped with a direct neural interface activated to identify whether or not an evoked potential is present. The direct neural interfaces present in the environment can be connected to at least one processing unit to detect at least one user who is present in the environment and perceiving the somatosensory stimuli.

[0021] In one implementation, the processing unit can further determine a state of concentration of each user present in the environment (by detecting or not evoked potentials in response to stimuli, for example).

[0022] It also refers to a computer program containing instructions to build a signal intended to power the transducer for the implementation of a process of the type described above, when executed by a processor.

[0023] In another aspect, a non-transient, computer-readable recording medium is proposed, on which such a program is recorded.

[0024] Also included is a device comprising a processing unit to construct a signal intended to power the transducer for the implementation of a process of the type described above.

[0025] For example, the processing unit may include an output interface to deliver data from one of at least one user attendance report, one user concentration report, and one user voting report.

[0026] Other features, details, and advantages will become apparent upon reading the detailed description below and analyzing the attached drawings, on which: Fig. 1

[0027] illustrates an embodiment of a method for constructing a modulated SIG signal to generate a somatosensory stimulus STS diffused by an HP transducer in an ENV environment. Fig. 2

[0028] illustrates an embodiment for adjusting the frequency and amplitude parameters of the GIS signal in order to maximize the possibility of detecting evoked potentials in a group of users, and thus to identify in particular absent or non-concentrated users in an RP presence report. Fig. 3

[0029] illustrates an implementation in which a user's lack of concentration can be detected during a given time sequence. Fig. 4

[0030] illustrates an example of application to a secret ballot implementation with multiple participants in the same room, implemented by a centralized processing unit and connected to each direct neural interface present in the room. Fig. 5

[0031] illustrates an example of a process according to an alternative embodiment in which each user's direct neural interface is connected to that user's personal device. Fig. 6

[0032] illustrates an example of application of the process to a visit to a place such as a museum with several rooms SA1, SA2, SA3 in each of which a transducer HP1, HP2, HP3 emits a somatosensory stimulus at a modulation frequency specific to the room.

[0033] This description proposes delivering stimuli to a user to activate a direct neural interface (or "DNI"), specifically via a wave propagating through the user's environment. In a detailed embodiment below, this wave can be an acoustic wave, inaudible to the user, such as infrasound and ultrasound. Alternatively, it can be heat radiation propagating through the environment and felt by the user's skin. This radiation is intermittent and has an intermittency frequency corresponding to the frequency of the evoked potential that can be identified in the electroencephalogram (EEG) signals collected and analyzed by the DNI worn by the user.

[0034] Very low frequency sounds (infrasound) are not perceived by the human ear but are directly felt by the body, particularly by the user's skin, through air vibrations. This is known as "vibro-tactile perception." It has been shown that for deaf people, this vibro-tactile perception is possible from 4 Hz, which constitutes an infrasound frequency imperceptible to the human ear, but perceptible notably by the skin, or even the user's chest.

[0035] Thus, from 4 Hz to 20 Hz, infrasound can be perceived by the skin via a contactless vibrotactile phenomenon (the air acting as a vector for the vibrations). Then, from 20 to 40 Hz, it is generally accepted that a transitional zone exists between infrasound and audible sounds. A complete frequency range, from 4 to 40 Hz, is therefore usable for delivering vibrotactile stimuli via acoustic waves, and thus without direct contact between a physical device and the user's skin.

[0036] Similarly, ultrasound (frequencies above 20,000 Hz) can also be perceived by the human body, particularly the skin, via certain mechanoreceptors. Ultrasonic frequencies can therefore also serve as a vector for delivering vibro-tactile stimuli.

[0037] These infrasonic or ultrasonic frequencies mentioned above constitute the carrier frequency of the aforementioned acoustic wave. Furthermore, this carrier frequency can be replaced by a noise band (white, pink, or any other type) containing several frequencies inaudible to the user.

[0038] The signal at this carrier frequency or noise band can be modulated at a modulation frequency remaining within the [10-40 Hz] range identified as generating the best stationary evoked potentials in the human brain. This frequency range is given for guidance only and is not exhaustive.

[0039] To maintain a usual signal architecture where the carrier frequency is higher than the modulation frequency, a modulation can be chosen in a frequency range between 10 and 20 Hz for example, and one or more infrasonic carrier frequency(ies) (in particular to construct a noise signal) for example between 20Hz and 35-40Hz.

[0040] We illustrated on the implementation in which carrier frequencies F p 1 , F p 2 , F p 3The frequencies, …, can be more generally chosen within a frequency range between 4 Hz and 40 Hz, for example, when the inaudible acoustic wave to be generated is infrasound. The signals corresponding to these frequencies each have an amplitude A1, A2, A3, …, these amplitudes being adjustable. Thus, an overall signal can be constructed from a weighted sum (SUM) of these signals, this overall signal then corresponding to a noise signal (NOS). A modulation (MOD) is then applied to this noise signal (NOS), starting from a modulation frequency (F). mod for example, between 10Hz and 40Hz, to construct the SIG signal intended to power the HP transducer. In the example above, the modulation signal is in successive pulses so that the SIG signal powering the HP transducer consists of a succession of "beeps" (inaudible but stimulating to the skin of the user UT), the frequency of these beeps being that of the modulation F mod.

[0041] The aforementioned HP transducer can be a type of loudspeaker adapted, for example, to deliver infrasonic acoustic waves, and thus diffuse a somatosensory stimulus (STS) perceived by the skin of at least one user (UT) into the environmental environment (ENV). The UT is equipped with a direct neural interface (BCI) containing sensors to collect an electroencephalogram (EEG) signal from the user. If the UT has managed to focus correctly on the perception of the STS stimulus, a processing unit (reference CT) is able to detect in this EEG signal a somatosensory evoked potential (SEP) with a frequency corresponding to the modulation frequency F mod .

[0042] Thus, by using a single HP transducer (a specific loudspeaker capable of emitting very low or, alternatively, very high frequencies), it is possible to deliver stimuli without direct contact to one user, or even to several users simultaneously. This improves user acceptance of the SSSEP approach thanks to an HP / CT device that delivers stimuli without direct contact to multiple users simultaneously, notably through the emission of inaudible acoustic waves. This allows users, for example, to continue hearing a speaker or multimedia audio content while being stimulated by an inaudible acoustic wave.

[0043] We illustrated a situation in which the HP transducer has low directivity to allow it to diffuse an acoustic wave intended to be perceived by several users simultaneously (UT1, UT2, etc.). For example, several direct neural interface headsets (BCI1, BCI2, BCI3, etc.) can be made available in a room environment, such as an augmented reality room, a museum room, or similar setting. Users UT1 and UT2 can then wear these BCI interface headsets so that the CT processing unit collects, via its IN input interface, the various EEG signals that may or may not be captured by the BCI1, BCI2, BCI3, etc. interfaces.

[0044] A PROC processor in the CT processing unit can then determine the presence of an evoked potential in each of the received EEG signals. If no such evoked potential is present in any of the EEG signals received from a BCI3 interface, the PROC processor can determine that the signal received from that BCI3 interface is simply noise, and that the BCI3 interface is not in use. From this, the absence of a user of that BCI3 interface can be deduced.

[0045] Thus, the CT processing unit can be programmed to deliver, via its OUT output interface, a presence report (RP) indicating the BCI interfaces in use and, consequently, the users present (UT1, UT2) or absent in the ENV environment of a room, for example. In such an implementation, the PROC processor can cooperate with a MEM memory within the CT processing unit, which contains instruction data from a computer program that can be read and executed by the PROC processor to detect the users present and generate the data for this RP report.

[0046] More generally, the CT processing unit may include local MEM memory, or access remote memory (for example, accessible via a remote server), containing instruction data from a computer program as defined herein. This data can be read and executed by the CT processing unit's PROC processor to construct a SIG signal to feed the HP transducer. For example, the PROC processor, following this computer program, may select the inaudible carrier frequencies F p 1 , F p 2 , F p 3 , …, and adjust their respective amplitudes A1, A2, A3, …, as well as the modulation frequency F mod, in order to construct a modulated SIG noise signal intended to feed the HP transducer. More specifically, these frequency and amplitude parameters can be adjusted to elicit, for example in a group of users present, the evoked potentials most detectable in the EEG signals collected by the BCI interfaces used.

[0047] In addition to the presence report (RP) that the CT processing unit's OUT output interface can provide, the PROC processor can generate a user concentration report for the room environment. Typically, it can be observed that, among the somatosensory evoked potentials (PES1, PES2) of different users, one of the evoked potentials (PES2) exhibits a loss of the modulation frequency F modDuring a time sequence SEQ, which typically corresponds to a state of distraction in user UT2, the output interface OUT can also deliver a concentration (or distraction) report for the users present. This report typically indicates distraction in user UT2 during a time sequence SEQ, which could correspond to the playback of a speech signal from a speaker or multimedia content (while somatosensory evoked potentials were successively measured in the users). For example, in an augmented reality game, representatives from different teams might listen to a speech signal revealing a secret, and only those already aware of the secret can focus on perceiving the inaudible acoustic wave and react with a detectable evoked potential.

[0048] It was further observed that the vast majority of users could perceive (particularly in the low infrasound frequencies) at least two distinct modulation frequencies F mod1 , F mod2 One being "faster" (higher) than the other. Thus, it is possible to envision another application, for example, a secret ballot in which users are not allowed to collude or influence each other. Referring now to the [previous example], different users participating in the vote listen to successive questions and focus (or not) on different vibro-tactile stimuli on their skin. This results in evoked potentials PES1 and PES2, collected respectively from the users, which successively exhibit frequencies that may be a function of their voting preference: for example, the frequency F mod1 for "yes" and the frequency F mod2for "no". Users hearing the questions focus on the perceived "fast" frequency (to vote "yes", for example) or on the "slow" frequency (to vote "no"). In such an implementation, two SIG signals with respective modulations at these two different frequencies F mod1 and F mod2 feed the HP transducer and the two stimuli corresponding to these two signals are diffused into the environment by the transducer.

[0049] Such an implementation allows for the collection of a "binary" signal (yes or no) from users, or even a ternary one, since users can also be asked not to focus on any vibro-tactile stimulus if they wish to abstain from voting. Thus, the detection of a sequence without a modulation frequency (in the evoked potential PES2 in the example) indicates a choice of abstention by the corresponding user, UT2, for a question posed during that sequence.

[0050] The CT processing unit can then deliver, via its OUT output interface, the data of a voting report to the different questions asked during these sequences.

[0051] In such a design, direct neural interfaces are connected to a centralized CT device, which can be controlled to configure the SIG signals to be generated. Alternatively, each direct neural interface can be connected to a personal device that can be individually controlled by each user to transmit instructions to the HP transducer to broadcast a specific signal.

[0052] In such a variant, a controllable personal device includes actuators that receive direct neural commands from the user's direct neural interface: for example, upon entering a museum, each user's neural interface controls the personal device to trigger the playback of multimedia content from a museum guide via headphones. Thus, each user can experience the museum at their own pace, or even according to their visitor profile.

[0053] For example, a high-frequency (HP) transducer is installed in a room containing an artwork and emits a vibro-tactile signal at a specific frequency. The user is equipped with a BCI interface and headphones connected to their personal device (e.g., a smartphone or audio guide). When the user enters the room and is ready to receive information about the artwork, they focus on the vibro-tactile stimulus emitted by the transducer. This state of concentration is detected by their BCI interface, which can identify an evoked potential at the specific frequency corresponding to the room. The BCI interface can then transmit data to the personal device indicating the user's presence in the room and their level of concentration.Upon receiving this data, the personal device can then order the start of playback of multimedia audio content related to the work present in this room, on the headphones available to the user.

[0054] We have illustrated a method for implementing such an application. In step S1, a transducer HPi is installed in a room SAi, for example, in a museum. This transducer emits a somatosensory SIG signal with a modulation frequency of Fmodi (step S2) for one or more users visiting this museum room, each equipped with a direct neural interface (BCI). When the BCI detects a somatosensory evoked potential at the Fmodi modulation frequency in a user's EEG signals (in step S3), this means that the user is present in room SAi and focused on listening to multimedia content, audio or video, related to the room SAi in which the user is located. Thus, in the example above, the BCI can transmit to a user's personal device DP, UT, an identifier IDi that corresponds to the Fmodi modulation frequency and therefore to room SAi (step S4).This IDi identifier can be used to identify specific CMi multimedia content corresponding to the SAi room. Thus, in step S5, the DP personal device, upon receiving this IDi identifier, can access a memory storing various content and query the memory using the IDi identifier to retrieve the data for the corresponding CMi content. In step S6, this data is used by a human-machine interface (such as a CA headset) to play the CMi content on the user's CA interface.

[0055] Thus, as illustrated in the image, multimedia content CM1, CM2, CM3,… adapted to each room, can be broadcast in the CA audio headphones of visitors, and this from a given modulation frequency of the somatosensory signals broadcast by the transducers HP1, HP2, HP3, …, of the respective rooms SA1, SA2, SA3… Such an implementation can also make it possible to determine the attendance rate of each room, individually, if the IDi identifiers are successively transmitted (by personal devices for example) to a centralized processing unit.

[0056] In a variation of the same use case, particularly when the user belongs to a group, a centrally controlled device receives direct neural commands from several direct neural interfaces of the group's users. Content playback is performed by this centrally controlled device and reproduced by each of the group's headphones, or directly by each of the group's headphones. Audio playback might only be triggered, for example, when the centrally controlled device receives commands from all the direct neural interfaces of all the group's users, as they are then all present in the environment where the somatosensory stimulus is being delivered. In this case, all users can experience the same museum visit, for instance, which could potentially be adapted to the group to ensure that users remain focused.In addition, the group can be generated from the user profiles of the group retrieved by means of a second somatosensory stimulus, for example.

[0057] Of course, this disclosure is not limited to the implementation examples described above as examples; it encompasses other variations.

[0058] As shown in the implementation examples in Figures 3 and 4, a given command can be associated with a given somatosensory stimulation frequency, thus generating the same simultaneous neural command for all users stimulated by the somatosensory stimulation. For example, depending on the frequency, the command can be one or more of the following: - a request for presence information (as shown with reference to Figure 3), - a request for identity confirmation (by a binary response "yes" or "no" from the user, for example, as shown with reference to Figure 4), - or a more complex request for the transmission of profile data, for example, - or the triggering of an adaptation of the environment to the user (for example, in augmented reality), - or something else.

[0059] Indeed, typically for profile data transmission, the direct neural interface (DNI) is associated with a user. Activating it can trigger a request for the user's profile (via the DNI to a profile database). The required profile can then be transmitted, upon request from the DNI, to electronic equipment, a smart TV, a home automation system, or other devices to personalize their use based on the current user's profile. This environmental adaptation can, for example, involve adapting a home automation system or a vehicle's system to personalize it for the user, or even for multiple users, based on their profile(s).

[0060] Thus, among the various possible applications, somatosensory vibratory stimulation makes it possible to note the people present for audience calculation, profile data retrieval, or other purposes, or even to control the adaptation of an environment to the person(s) present.

Claims

Method of somatosensory stimulation of at least one user to collect at least one electroencephalogram (EEG) signal including an evoked potential in order to activate a direct neural interface (BCI) based on somatosensory stimuli from the user's skin, wherein, the user being placed in an environment, said stimuli are applied to the environment by a transducer (HP), the user's skin being in contact with the environment and receiving said stimuli via said environment. A method according to claim 1, wherein said stimuli are applied by emission of an acoustic wave into said environment. A method according to claim 2, wherein the acoustic wave comprises at least one carrier frequency which is inaudible to the user. A method according to claim 3, wherein the acoustic wave is infrasonic, and of frequency between 4 and 40 Hz. A method according to claim 3, wherein the acoustic wave is ultrasonic, and of a frequency greater than 20 kHz. A method according to any one of claims 3 to 5, wherein the acoustic wave comprises a plurality of carrier frequencies inaudible to the user, to form a noise inaudible to the user. A method according to claim 6, wherein the noise comprises a plurality of frequencies between 4 and 40 Hz. A method according to any one of claims 3 to 7, wherein the acoustic wave, at said inaudible carrier frequency, is further modulated by a chosen modulation frequency, the electroencephalogram signal, collected to activate the direct neural interface, comprising an evoked potential at said chosen modulation frequency. A method according to claim 8, wherein the modulation frequency is chosen in a range between 10 and 40 Hz. A method according to any one of claims 8 and 9, wherein the modulation is applied in the form of beeps of a chosen frequency, between 10 and 40 Hz. A method according to any one of the preceding claims, wherein said stimuli are applied to the environment by a directionality source chosen to deliver contactless stimuli to one or more users simultaneously. A method according to claim 11, wherein said stimuli are applied to the environment by a source of directionality chosen to deliver contactless stimuli to a plurality of users simultaneously, said users each being equipped with a direct neural interface activated to identify or not an evoked potential, the direct neural interfaces of the environment being connected to at least one processing unit to detect at least users present in the environment and perceiving the somatosensory stimuli. A method according to claim 12, wherein the processing unit further determines a concentration state of each user present in the environment. Computer program comprising instructions for constructing a signal (SIG) intended to power the transducer (HP) for implementing the method according to one of the preceding claims, when executed by a processor. Device comprising a processing unit (CT) for constructing a signal (SIG) intended to power the transducer (HP) for implementing the method according to any one of claims 1 to 13. Device according to claim 15, wherein the processing unit (CT) includes an output interface (OUT) for delivering data from a report among at least one user presence report (RP), one user concentration report, and one user voting report.

Images