Direct neural interface with vibro-tactile stimulation via a transducer

Abstract

A method for somatosensory stimulation of at least one user (UT) is described to acquire at least one electroencephalogram (EEG) signal containing an evoked potential for activating a direct neural interface (BCI) based on somatosensory stimuli from the user's skin. Specifically, with the user placed in an environment (ENV), the stimuli (STS) are applied to the environment by a transducer (HP), the user's skin being in contact with the environment and thus receiving the stimuli via the environment. Abstract figure: Figure 1

Description

Title of the invention: Direct neural interface with vibro-tactile stimulation via a transducer technical field

[0001] This disclosure falls within the domain of the detection of potentials evoked by direct neural interfaces (or BCIs hereafter, for "Brain Computer Interface"). Previous technique

[0002] More specifically, a so-called "somatosensory" evoked potential is a signal appearing in EEG (electroencephalogram) signals when the subject (typically a human being) is subjected to somatosensory stimulation, that is, sensory stimulation (pressure, heat, pain, etc.) of one or more regions of the body (skin, tendon, joints, etc.), particularly through tactile stimulation. Specific equipment (a "BCI" headset for "Brain-Computer Interface") collects the EEG signals in order to measure them and identify the frequency of the tactile stimulation within these signals.

[0003] This approach is commonly referred to as "SSSEP" for "Steady-State Somatosensory Evoked Potential".

[0004] When using a BCI interface based on such an SSSEP technique, the stimuli generally consist of a carrier frequency (e.g., 200 Hz) modulated by a signal at a lower frequency, for example, a square wave with a frequency between 10 and 40 Hz. This stimulus is generally delivered to the user either electrically (electrodes placed on the skin) or mechanically via a vibro-tactile device also placed on the user's skin. The stimulus thus delivered then generates a somatosensory evoked potential in the brain, and the recorded EEG signals have the same frequency as the modulation at the aforementioned lower frequency (between 10 and 40 Hz). This measurement can, among other methods, be performed by a device such as an EEG signal sensor headset (or any other medical imaging system, for example, MRI, MEG, or others) worn by the person subjected to the stimulus.

[0005] One drawback of this approach is that the device used to apply the stimulus to the user must be in contact with the user's skin, in addition to the EEG sensor device used to collect and measure brain signals. Summary

[0006] This disclosure improves the situation.

[0007] A method of somatosensory stimulation of at least one user is proposed to collect at least one electroencephalogram signal including an evoked potential in order to activate a direct neural interface based on somatosensory stimuli from the user's skin. In particular, with the user placed in an environment, said stimuli are applied to the environment by a transducer, the user's skin being in contact with the environment and receiving said stimuli via said environment.

[0008] Such an embodiment makes it possible not to apply tactile stimuli directly to the user's skin, but to transmit them into the user's environment, which then forms a transducer of these stimuli.

[0009] In one embodiment, these stimuli can be applied by emitting an acoustic wave into the environment.

[0010] This can typically be a low frequency acoustic wave (below the hearing threshold of 50 Hz typically), or a high frequency ultrasonic wave (above the hearing threshold of 20 kHz typically).

[0011] Thus, the acoustic wave may then include at least one carrier frequency which is inaudible to the user.

[0012] The acoustic wave can be infrasonic, and of frequency between 4 and 40 Hz, or alternatively, ultrasonic and of frequency greater than 20 kHz.

[0013] In one embodiment, the acoustic wave may include a plurality of carrier frequencies inaudible to the user, to form a noise inaudible to the user.

[0014] This noise is a mixture of frequencies which are all in ranges inaudible to the user.

[0015] This noise can therefore include a plurality of frequencies between 4 and 40 Hz, for example.

[0016] To form the stimulus signal that can cause an evoked potential, the acoustic wave, at the aforementioned inaudible carrier frequency, is further modulated by a chosen modulation frequency, and thus the electroencephalogram signal, collected to activate the direct neural interface, includes an evoked potential at the aforementioned chosen modulation frequency.

[0017] The aforementioned modulation frequency can be chosen, for example, in a range between 10 and 40 Hz.

[0018] In a typical embodiment, the modulation can be applied in the form of beeps of a chosen frequency, between 10 and 40 Hz.

[0019] In one embodiment, the stimuli are applied to the environment by a directionality source chosen to deliver contactless stimuli to one or more users simultaneously.

[0020] For example, stimuli can be applied to the environment by a chosen directionality source to deliver contactless stimuli to a plurality of users simultaneously. In such an embodiment, each of these users can be equipped with a direct neural interface activated to identify whether or not an evoked potential is present. The direct neural interfaces present in the environment can be connected to at least one processing unit to detect at least some users who are present in the environment and perceiving the somatosensory stimuli.

[0021] In one embodiment, the processing unit can further determine a state of concentration of each user present in the environment (by detecting or not evoked potentials in response to stimuli, for example).

[0022] Also referred to is a computer program comprising instructions for constructing a signal intended to power the transducer for the implementation of a process of the type described above, when executed by a processor.

[0023] According to another aspect, a non-transient, computer-readable recording medium is proposed on which such a program is recorded.

[0024] Also referred to is a device comprising a processing unit for constructing a signal intended to supply the transducer for the implementation of a process of the type described above.

[0025] For example, the processing unit may include an output interface for delivering data from a report among at least one user attendance report, one user concentration report, and one user voting report. Brief description of the drawings

[0026] Other features, details and advantages will become apparent from reading the detailed description below and from analyzing the accompanying drawings, in which: Fig. 1

[0027] [Fig.1] illustrates an embodiment of a method for constructing a modulated SIG signal to generate a somatosensory stimulus STS diffused by an HP transducer in an ENV environment. Fig. 2

[0028] [Fig.2] illustrates an embodiment for adjusting the frequency parameters and amplitudes of the SIG signal in order to maximize the possibility of detecting evoked potentials in a group of users, and thus to identify in particular absent or non-concentrated users in an RP presence report. Fig. 3

[0029] [Fig.3] illustrates an embodiment in which the absence of concentration of a The user can be detected during a given time sequence. Fig. 4

[0030] [Fig.4] illustrates an example of application to an implementation of a secret ballot with several participants in the same room, implemented by a centralized processing unit and connected to each direct neural interface present in the room. Fig. 5

[0031] [Fig. 5] illustrates an example of a process according to an alternative embodiment in which each user's direct neural interface is connected to that user's personal device. Fig. 6

[0032] [Fig.6] illustrates an example of applying the process of [Fig.5] to a visit of a a place such as a museum with several rooms SA1, SA2, SA3 in each of which a transducer HP1, HP2, HP3 emits a somatosensory stimulus at a modulation frequency specific to the room. Description of the implementation methods

[0033] This description proposes delivering stimuli to a user to activate a direct neural interface (or "DNI"), in particular via a wave propagating through the user's environment. In a detailed embodiment below, this wave can be an acoustic wave, inaudible to the user, such as infrasound and ultrasound. Alternatively, it can be heat radiation propagating through the environment and felt by the user's skin, this radiation being intermittent and having an intermittency frequency corresponding to the frequency of the evoked potential that can be identified in the electroencephalogram (or "EEG") signals collected and analyzed by the DNI worn by the user.

[0034] Very low frequency sounds (infrasound) are not perceived by the human ear but are directly felt by the body, particularly by the user's skin, through air vibrations. This is referred to as "vibro-tactile perception." It has been shown that for deaf people, this vibro-tactile perception is possible from 4 Hz, which constitutes an infrasound frequency imperceptible to the human ear, but perceptible in particular by the skin, or even by the user's chest.

[0035] Thus, from 4 Hz to 20 Hz, infrasound can be perceived by the skin via a contactless vibrotactile phenomenon (the air acting as a vector for the vibrations). Then, from 20 to 40 Hz, it is generally accepted that a transitional zone exists between infrasound and audible sounds. A complete frequency range, from 4 to 40 Hz, can therefore be used to deliver vibrotactile stimuli via acoustic waves, and thus without applying direct contact between a physical device and the user's skin.

[0036] Similarly, ultrasound (frequencies above 20,000 Hz) can also be perceived by the human body, particularly the skin, via certain mechanoreceptors. Ultrasonic frequencies can therefore also serve as a vector for delivering vibro-tactile stimuli.

[0037] These infrasonic or ultrasonic frequencies above thus constitute the carrier frequency of the aforementioned acoustic wave. Furthermore, this carrier frequency can be replaced by a noise band (white, pink, or of any other nature) comprising several frequencies inaudible to the user.

[0038] The signal at this carrier frequency or noise band can be modulated at a modulation frequency remaining within the range [10-40 Hz] identified as generating the best stationary evoked potentials in the human brain. This frequency range is given for guidance purposes only and is not limiting.

[0039] To maintain a usual signal architecture where the carrier frequency is greater than the modulation frequency, a modulation can be chosen in a frequency range between 10 and 20 Hz for example, and one or more infrasonic carrier frequency(ies) (in particular to construct a noise signal) for example between 20Hz and 35-40Hz.

[0040] Figure 1 illustrates an implementation in which carrier frequencies Fp1, Fp2, Fp3, ... can be chosen more generally within a frequency range between 4 Hz and 40 Hz, for example, when the inaudible acoustic wave to be generated is chosen to be infrasonic. The signals corresponding to these frequencies each have an amplitude Ai, A2, A3, ..., these amplitudes being adjustable. Thus, an overall signal can be constructed from a weighted sum SUM of these signals, this overall signal then corresponding to a NOS noise signal. A MOD modulation is then applied to this NOS noise, using a modulation frequency Fmod, for example, between 10 Hz and 40 Hz, to construct the SIG signal intended to drive the HP transducer. In the example of Figure 1,[l], the modulation signal is in successive squares so that the SIG signal feeding the HP transducer has a succession of "beeps" (inaudible but stimulating the skin of the UT user), the frequency of these beeps being that of the Fmod modulation. .

[0041] The aforementioned HP transducer can be a type of loudspeaker adapted, for example, to deliver infrasonic acoustic waves, and thus diffuse a somatosensory stimulus (STS) perceived by the skin of at least one user (UT) into the environmental environment (ENV). The user (UT) is equipped with a direct neural interface (BCI) comprising sensors to collect an electroencephalogram (EEG) signal from the user. If the user (UT) has managed to concentrate correctly on the perception of the STS stimulus, a processing unit (reference CT in [Fig. 2]) is capable of detecting in this EEG signal a somatosensory evoked potential which has a frequency corresponding to the modulation frequency Fmod.

[0042] Thus, by using a single HP transducer (a specific loudspeaker capable of emitting very low or, alternatively, very high frequencies), it is possible to deliver stimuli without contact to one user, or even to several users simultaneously. It is therefore possible to improve user acceptance of the SSSEP approach thanks to an HP, CT device that allows stimuli to be delivered without direct contact to several users simultaneously, notably by emitting inaudible acoustic waves. This allows users, for example, to continue hearing a speaker or multimedia audio content while being stimulated by an inaudible acoustic wave.

[0043] Figure 2 illustrates such a situation in which the HP transducer has low directivity to allow the diffusion of an acoustic wave intended to be perceived by several users simultaneously (UTi, UT2, etc.). For example, several BCIi, BCI2, BCI3, etc., direct neural interface headsets can be made available in a room environment, such as an augmented reality room, a museum room, or other similar setting. Users (UTi, UT2) can then wear such BCI interface headsets so that the CT processing unit collects, via its IN input interface, the various EEG signals that the BCIi, BCI2, BCI3, etc., interfaces may or may not capture.

[0044] A PROC processor in the CT processing unit can then determine the presence of an evoked potential in each of the received EEG signals. If no such evoked potential is present in one of the EEG signals received from a BCI3 interface, the PROC processor can determine that the signal received from that BCI3 interface is simply noise, and that the BCI3 interface is not in use. It can thus be deduced that there is no user of that BCI3 interface.

[0045] Thus, the CT processing unit can be programmed to deliver, via its OUT output interface, a presence report RP indicating the BCI interfaces in use and, consequently, the users present (UTb UT2) or absent in the ENV environment of a room, for example. In such an embodiment, the PROC processor can cooperate with a MEM memory within the CT processing unit, containing, in particular, instruction data from a computer program that can be read and executed by the PROC processor to detect the users present and generate the data for this RP report.

[0046] More generally, the CT processing unit may include a local MEM memory, or access a remote memory (for example, accessible via a remote server), containing instruction data from a computer program. As described herein, these parameters can be read and executed by the PROC processor of the CT processing unit to construct a SIG signal to feed the HP transducer. For example, the PROC processor, following this computer program, can select the inaudible carrier frequencies Fp*, Fp2, Fp3, ..., and adjust their respective amplitudes Ai, A2, A3, ..., as well as the modulation frequency Fmod, to construct a modulated SIG noise signal to feed the HP transducer. More specifically, these frequency and amplitude parameters can be adjusted to elicit, for example, in a group of users present, the most detectable evoked potentials in the EEG signals collected by the BCI interfaces used.

[0047] In addition to the presence report RP that the OUT output interface of the CT processing unit can provide, the PROC processor can generate a concentration report of users present in the room environment. With reference to [Fig. 3], it can typically be observed that, among the somatosensory evoked potentials PESi, PES2 of different users, one of the evoked potentials PES2 exhibits a loss of the modulation frequency Fmod during a time sequence SEQ, which typically corresponds to a state of deconcentration of a user UT2.The OUT output interface can thus also deliver a concentration (or deconcentration) report for the users present, typically indicating a deconcentration of user UT2 during a time sequence SEQ that could correspond to a passage of a speech signal from a speaker or a multimedia content being played (while somatosensory evoked potentials were successively measured on the users). For example, in the context of an augmented reality game, representatives of different teams can listen to a passage of a speech signal revealing a secret, and only those already knowing this secret can concentrate on perceiving the inaudible acoustic wave to react with a detectable evoked potential.

[0048] It has also been observed that the vast majority of users can perceive (particularly in the low infrasonic frequencies) at least two distinct modulation frequencies Fmod1, Fmod2, one being “faster” (higher) than the other. Thus, it is possible to foresee another application, for example, a secret ballot in which users are not allowed to collude or influence each other. Referring now to [Fig. 4], different users participating in the vote listen to successive questions and focus (or not) on different vibro-tactile stimuli on their skin. This results in evoked potentials PESi, PES2, collected respectively from the users, which successively exhibit frequencies that can be a function of their voting preference: for example, the Fmod1 frequency for “yes” and the The Fmod2 frequency is used for "no". Users hearing the questions focus on the perceived "fast" frequency (to vote "yes", for example) or the "slow" frequency (to vote "no"). In such an implementation, two SIG signals with respective modulations at these two different frequencies, Fmod1 and Fmod2, feed the HP transducer, and the two stimuli corresponding to these two signals are broadcast into the environment by the transducer.

[0049] Such an implementation makes it possible to collect a "binary" (yes or no) signal from users, or even a ternary one, since users can also be asked not to focus on any perception of a vibro-tactile stimulus if they wish to abstain from voting. Thus, the detection of a sequence without a modulation frequency (in the evoked potential PES2 in the example in [Fig. 4]) indicates a choice of abstention by the corresponding user UT2, for a question posed during this sequence.

[0050] The CT processing unit can then deliver via its OUT output interface the data of a voting report to the different questions asked during these sequences.

[0051] In such an embodiment, the direct neural interfaces are connected to a centralized CT device, controllable to parameterize the SIG signals to be generated. Alternatively, each direct neural interface can be connected to a personal device individually controllable by each user to transmit instructions to the HP transducer to broadcast a particular signal.

[0052] In such an example of a variant, a controllable personal device includes actuators receiving direct neural commands from the user's direct neural interface: for example, upon entering a museum, each user's neural interface controls the personal device to trigger the playback, via headphones, of multimedia content from a museum guide for the user. Thus, each user can experience the museum tour tailored to their own pace, or even to their visitor profile.

[0053] For example, an HP transducer is installed in a room containing a work of art and emits a vibro-tactile signal at a given frequency. Furthermore, the user is equipped with a BCI interface and headphones connected to their personal device (e.g., a smartphone or an audio guide). When the user enters the room and is ready to receive explanations about the artwork displayed there, they focus on the vibro-tactile stimulus emitted by the transducer. This state of concentration is detected by their BCI interface, which can identify an evoked potential at the given frequency corresponding to the room. The BCI interface can then transmit to the personal device data indicating the user's presence in the room and their level of concentration. Upon receiving this data, the personal device can then control the beginning of a reading of multimedia audio content relating to the work present in this room, on the audio headphones available to the user.

[0054] Figure 5 illustrates an embodiment of a method for implementing such an application. During step S1, an HPi transducer is installed in a room SAi, for example in a museum. This transducer emits a somatosensory SIG signal with a modulation frequency of Fmodi (step S2) for one or more users visiting this room of the museum, each equipped with a direct neural interface (DNI). When the DNI detects a somatosensory evoked potential at the modulation frequency Fmodi in a user's EEG signals (in step S3), this means that the user is present in the room SAi and focused on listening to multimedia content, audio or video, related to the room SAi in which the user is located. Thus, in the example of Figure 5,[5] The BCI interface can transmit to a user's personal device (DP) an IDi identifier that corresponds to the Fmodi modulation frequency and therefore to the SAi room (step S4). This IDi identifier can be used to identify specific CMi multimedia content corresponding to the SAi room. Thus, in step S5, the DP, upon receiving this IDi identifier, can access a memory storing various content and query the memory with the IDi identifier to retrieve the data for the corresponding CMi content. In step S6, this data is used by a human-machine interface (such as a CA headset) to play the CMi content on the user's CA interface.

[0055] Thus, as illustrated in [Fig.6], multimedia content CMI, CM2, CM3,... adapted to each room, can be broadcast in the CA audio headphones of visitors, and this from a given modulation frequency of the somatosensory signals broadcast by the transducers HPI, HP2, HP3, ..., of the respective rooms SAI, SA2, SA3... Such an implementation can also make it possible to determine the attendance rate of each room, individually, if the IDi identifiers are successively transmitted (by personal devices for example) to a centralized processing unit.

[0056] In a variant of the same use case, particularly when the user belongs to a group, a centrally controlled device receives direct neural commands from several direct neural interfaces of the users belonging to the group. Content playback is performed by a centrally controlled device and reproduced by each of the headphones of the users in the group, or directly by each of the headphones of the users in the group. Audio playback may, for example, only be triggered when the centrally controlled device receives the command from all the direct neural interfaces of all the users in the group, since they are all then present in the environment in which the audio is being played. The somatosensory stimulus. In this case, all users can experience the same museum visit, for example, which can potentially be adapted to the group to ensure users remain focused. Furthermore, the group can be generated from user profiles retrieved using a second somatosensory stimulus, for example.

[0057] Of course, this disclosure is not limited to the embodiment examples described above by way of example; it encompasses other variants.

[0058] As shown in the embodiment examples in Figures 3 and 4, a given command can be associated with a given somatosensory stimulation frequency, thus generating the same simultaneous neural command for all users stimulated by the somatosensory stimulation. For example, depending on the frequency, the command can be one or more of the following: - a request for presence information (as shown with reference to [Fig. 3]), - a request for identity confirmation (by a binary response "yes" or "no" from the user, for example, as shown with reference to [Fig. 4]), - or a more complex request for the transmission of profile data, for example, - or triggering an adaptation of the environment to the user (for example, in augmented reality), - or other.

[0059] Indeed, typically for profile data transmission, since the direct neural interface is associated with a user, its activation can trigger a request for the profile of the user of the direct neural interface (via the direct neural interface to a profile database). The required profile can then be transmitted, upon request from the direct neural interface, to electronic equipment, a connected television, a home automation system, or other devices to personalize the use of this equipment according to the profile of the user present. The aforementioned adaptation of the environment can correspond, for example, to an adaptation of the home automation system or a vehicle system to personalize it for the user, or even for the users present, based on their profile(s).

[0060] Thus, among the various possible applications, somatosensory vibratory stimulation makes it possible to note the people present for audience calculation, profile data retrieval, or other purposes, or even to control the adaptation of an environment to the person(s) present.

Claims

Demands

1. A method for somatosensory stimulation of at least one user to collect at least one electroencephalogram (EEG) signal including an evoked potential for the purpose of activating a direct neural interface (BCI) based on somatosensory stimuli from the user's skin, wherein, the user being placed in an environment, said stimuli are applied to the environment by a transducer (HP), the user's skin being in contact with the environment and receiving said stimuli via said environment.

2. A method according to claim 1, wherein said stimuli are applied by emission of an acoustic wave into said environment.

3. A method according to claim 2, wherein the acoustic wave comprises at least one carrier frequency which is inaudible to the user.

4. A method according to claim 3, wherein the acoustic wave is infrasonic, and of frequency between 4 and 40 Hz.

5. A method according to claim 3, wherein the acoustic wave is ultrasonic, and of a frequency greater than 20 kHz.

6. A method according to any one of claims 3 to 5, wherein the acoustic wave comprises a plurality of carrier frequencies inaudible to the user, to form a noise inaudible to the user.

7. A method according to claim 6, wherein the noise comprises a plurality of frequencies between 4 and 40 Hz.

8. A method according to any one of claims 3 to 7, wherein the acoustic wave, at said inaudible carrier frequency, is further modulated by a chosen modulation frequency, the electroencephalogram signal, collected to activate the direct neural interface, comprising an evoked potential at said chosen modulation frequency.

9. A method according to claim 8, wherein the modulation frequency is chosen in a range between 10 and 40 Hz.

10. A method according to any one of claims 8 and 9, wherein the modulation is applied in the form of beeps of a chosen frequency, between 10 and 40 Hz.

11. A method according to any one of the preceding claims, wherein said stimuli are applied in the environment by a directionality source chosen to deliver contactless stimuli to one or more users simultaneously.

12. A method according to claim 11, wherein said stimuli are applied to the environment by a source of directionality chosen to deliver contactless stimuli to a plurality of users simultaneously, said users each being equipped with a direct neural interface activated to identify or not an evoked potential, the direct neural interfaces of the environment being linked to at least one processing unit to detect at least users present in the environment and perceiving the somatosensory stimuli.

13. A method according to claim 12, wherein the processing unit further determines a concentration state of each user present in the environment.

14. A computer program comprising instructions for constructing a signal (SIG) intended to power the transducer (HP) for implementing the method according to any one of the preceding claims, when executed by a processor.

15. Device comprising a processing unit (CT) for constructing a signal (SIG) intended to power the transducer (HP) for implementing the method according to any one of claims 1 to 13.

16. Device according to claim 15, wherein the processing unit (CT) has an output interface (OUT) for delivering data from a report among at least one user attendance report (RP), one user concentration report, and one user voting report.

Images