Apparatus and method for stimulating the vestibular system of a user, and head-mounted equipment

Abstract

Provided is an apparatus for stimulating the vestibular system of a user. The apparatus includes a first plurality of bone conduction transducers configured to emit vibrations into a first area of the user's skull for stimulating the vestibular system. In addition, the apparatus includes a second plurality of bone conduction transducers configured to emit vibrations into a second area of the user's skull for stimulating the vestibular system. The apparatus further includes control circuitry configured to control, based on a target directional motion to be perceived by the vestibular system of the user, emission of the vibrations by the first plurality of bone conduction transducers and the second plurality of bone conduction transducers.

Description

[0001] APPARATUS AND METHOD FOR STIMULATING THE VESTIBULAR SYSTEM OF A USER, AND HEAD-MOUNTED EQUIPMENT

[0002] Field

[0003] The present disclosure relates to stimulation of the vestibular system. In particular, examples of the present disclosure relate to an apparatus and a method for stimulating the vestibular system of a user, a head-mounted equipment, a non-transitory machine-readable medium and a program.

[0004] Background

[0005] The vestibular system of a human being is a sensory system that creates the sense of balance and spatial orientation for the purpose of coordinating movement with balance. Together with the respective cochlea, a part of the auditory system, the vestibular apparatuses of the vestibular system constitute the labyrinths of the inner ears. The vestibular system serves as an organ that detects acceleration along all three axes on both sides of the head through its semicircular canals. The semicircular canals are filled with fluid. The fluid moves with inertia as the head changes position. The movement of fluid pushes on a structure called the cupula which contains hair cells that transduce the mechanical movement to electrical signals. The electrical signals activate the vestibular nerve. This, in turn, stimulates various regions of the brain, including the vestibular cortex, to recognize and respond to the interpreted motion.

[0006] For example, if the vestibular system does not perceive the same motion as the eyes when viewing motion through a Virtual Reality (VR) headset, a user will experience Visually Induced Motion Sickness (VIMS). Even if this is not the case, the user does not have the feeling of being fully immersed in the content.

[0007] Conventional approaches typically employ techniques such as Galvanic Vestibular Stimulation (GVS) to induce motion perception in users or to selectively inhibit the functioning of the vestibular system in order to alleviate VIMS. However, GVS is characterized by its invasive nature, involving the placement of numerous electrodes on the user's head. Additionally, the application of electrode gel is necessary for optimal contact. GVS tends to be uncomfortable for users and may lead to skin irritation and burns due to the electrical nature of the device.

[0008] Hence, there may be a demand for improved stimulation of the vestibular system

[0009] Summary

[0010] This demand is met by an apparatus and a method for stimulating the vestibular system of a user, a head-mounted equipment, a non-transitory machine-readable medium and a program in accordance with the independent claims. Advantageous embodiments are defined by the dependent claims.

[0011] According to a first aspect, the present disclosure provides an apparatus for stimulating the vestibular system of a user. The apparatus comprises a first plurality of bone conduction transducers configured to emit vibrations into a first area of the user’s skull for stimulating the vestibular system. In addition, the apparatus comprises a second plurality of bone conduction transducers configured to emit vibrations into a second area of the user’s skull for stimulating the vestibular system. The apparatus further comprises control circuitry configured to control, based on a target directional motion to be perceived by the vestibular system of the user, emission of the vibrations by the first plurality of bone conduction transducers and the second plurality of bone conduction transducers.

[0012] According to a second aspect, the present disclosure provides a method for stimulating the vestibular system of a user. The method comprises controlling, based on a target directional motion to be perceived by the vestibular system of the user, emission of vibrations by a first plurality of bone conduction transducers and a second plurality of bone conduction transducers. The first plurality of bone conduction transducers are configured to emit vibrations into a first area of the user’s skull for stimulating the vestibular system. The second plurality of bone conduction transducers are configured to emit vibrations into a second area of the user’s skull for stimulating the vestibular system.

[0013] According to a third aspect, the present disclosure provides a head-mounted equipment. The head-mounted equipment comprises a display configured to graphically output a VR environment. Additionally, the head-mounted equipment comprises an apparatus for stimulating the vestibular system of a user according to the first aspect. The target directional motion is based on motion in the VR environment graphically output to the user.

[0014] According to a fourth aspect, the present disclosure provides a non-transitory machine-readable medium having stored thereon a program having a program code for performing the method according to the second aspect, when the program is executed on a processor or a programmable hardware.

[0015] According to a fifth aspect, the present disclosure provides a program having a program code for performing the method according to the second aspect, when the program is executed on a processor or a programmable hardware.

[0016] Brief description of the Figures

[0017] Some examples of apparatuses and / or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

[0018] Fig. 1 illustrates an exemplary apparatus for stimulating the vestibular system of a user;

[0019] Fig. 2 illustrates an effect of emitted vibrations with phase shift;

[0020] Fig. 3 illustrates an effect of asymmetric amplitudes for emitted vibrations;

[0021] Fig. 4 illustrates an exemplary head-mounted equipment; and

[0022] Fig. 5 illustrates a flowchart of an example of a method for stimulating the vestibular system of a user.

[0023] Detailed Description

[0024] Some examples are now described in more detail with reference to the enclosed figures. However, other possible examples are not limited to the features of these embodiments described in detail. Other examples may include modifications of the features as well as equivalents and alternatives to the features. Furthermore, the terminology used herein to describe certain examples should not be restrictive of further possible examples.

[0025] Throughout the description of the figures same or similar reference numerals refer to same or similar elements and / or features, which may be identical or implemented in a modified form while providing the same or a similar function. The thickness of lines, layers and / or areas in the figures may also be exaggerated for clarification.

[0026] When two elements A and B are combined using an “or”, this is to be understood as disclosing all possible combinations, i.e. only A, only B as well as A and B, unless expressly defined otherwise in the individual case. As an alternative wording for the same combinations, "at least one of A and B" or "A and / or B" may be used. This applies equivalently to combinations of more than two elements.

[0027] If a singular form, such as “a”, “an” and “the” is used and the use of only a single element is not defined as mandatory either explicitly or implicitly, further examples may also use several elements to implement the same function. If a function is described below as implemented using multiple elements, further examples may implement the same function using a single element or a single processing entity. It is further understood that the terms "include", "including", "comprise" and / or "comprising", when used, describe the presence of the specified features, integers, steps, operations, processes, elements, components and / or a group thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components and / or a group thereof.

[0028] Fig- 1 illustrates an apparatus 100 for stimulating the vestibular system of a user. As a reference, the head 150 of the user - including the ears 151 and 152 at the left and the right side of the head 150 - is schematically illustrated in Fig. 1. The vestibular system of the user comprises the central system in the brain and the brainstem as well as the peripheral system in the inner ears (and the pathways to the brainstem). For reasons of simplicity, only the vestibular apparatuses 141 and 142 in the inner ears are schematically illustrated in Fig. 1. The vestibular apparatus 141 is arranged behind the left ear 151 in the inner ear of the left ear 151. Analogously, the vestibular apparatus 142 is arranged behind the right ear 152 in the inner ear of the right ear 152. The apparatus 100 comprises a first plurality of bone conduction transducers 110 arranged in a first array. In the example of Fig. 1, the first plurality of bone conduction transducers 110 comprises twelve bone conduction transducers. However, it is to be noted that the present disclosure is not limited thereto. Any other number N > 2 of bone conduction transducers may be used. The bone conduction transducers of the first plurality of bone conduction transducers 110 are arranged in a two-dimensional array in the example of Fig. 1. In other words, the bone conduction transducers of the first plurality of bone conduction transducers 110 are arranged in rows and columns. However, it is to be noted that the present disclosure is not limited thereto. The bone conduction transducers of the first plurality of bone conduction transducers 110 may be arranged in any other topography as well.

[0029] In addition, the apparatus 100 comprises a second plurality of bone conduction transducers 120 arranged in a second array. In the example of Fig. 1, the second plurality of bone conduction transducers 120 comprises twelve bone conduction transducers. However, it is to be noted that the present disclosure is not limited thereto. Any other number M > 2 of bone conduction transducers may be used. The bone conduction transducers of the second plurality of bone conduction transducers 120 are arranged in a two-dimensional array in the example of Fig. 1. In other words, the bone conduction transducers of the second plurality of bone conduction transducers 120 are arranged in rows and columns. However, it is to be noted that the present disclosure is not limited thereto. The bone conduction transducers of the second plurality of bone conduction transducers 120 may be arranged in any other topography as well.

[0030] As illustrated in Fig. 1, the first plurality of bone conduction transducers 110 may comprise the same number of bone conduction transducers like the second plurality of bone conduction transducers 120. However, it is to be noted that the present disclosure is not limited thereto. In other examples, the first plurality of bone conduction transducers 110 may comprise a different number of bone conduction transducers than the second plurality of bone conduction transducers 120.

[0031] The first plurality of bone conduction transducers 110 are configured to emit vibrations (oscillatory waves, pressure waves) 111 into a first area of the user’s skull for stimulating the vestibular system. The second plurality of bone conduction transducers 120 are configured to emit vibrations (oscillatory waves, pressure waves) 121 into a second area of the user’s skull for stimulating the vestibular system. A bone conduction transducer is a type of device that conveys vibrations by (directly) vibrating the bones of the user's skull, bypassing the outer and middle ear. Bone conduction transducers take advantage of the fact that bones are capable of conducting vibrations. Accordingly, the bone conduction transducer is able to deliver vibrations to the inner ear. For example, the bone conduction transducer may comprise one or more electromagnets attracting and repelling one or more vibration plates for emitting vibrations. In other examples, the bone conduction transducer may comprise one or more piezoelectric vibrators for emitting vibrations. In still other examples, the bone conduction transducer may capacitively attract and repel one or more vibration plates for emitting vibrations. The bone conduction transducer may be a Micro-Elec- troMechanical Systems (MEMS) device.

[0032] A bone conduction transducer generates vibrations that are transmitted through the bones of the user’s skull (e.g., the cheekbones or the temporal bones, in particular the mastoid parts of the temporal bones) to the inner ear. The organs in the inner ear such as the vestibular apparatuses 141 and 142 or the respective cochlea translate these vibrations into electrical signals that the brain interprets. In the proposed technology, the vestibular system of the user is to be stimulated rather than the auditory systems. Accordingly, the frequencies of the vibrations emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 are such that the vestibular apparatuses 141 and 142 can recognize the vibrations. The apparatuses 141 and 142 are particularly sensitive for vibrations at a frequency of approx. 250 Hz. For example, the frequency or frequencies of the vibrations emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may be at least 10 Hz, 20 Hz or 50 Hz. On the other hand, the frequency or frequencies of the vibrations emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may be at maximum 1000 Hz, 750 Hz or 500 Hz. In some examples, the frequency or frequencies of the vibrations emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may be between 50 Hz and 500 Hz, and in particular between 200 Hz and 300 Hz.

[0033] As described above, the first plurality of bone conduction transducers 110 are configured to emit vibrations into a first area of the user’s skull and the second plurality of bone conduction transducers 120 are configured to emit vibrations into a second area of the user’s skull. The second area of the user’s skull is different from the first area of the user’s skull. In the example of Fig. 1, the first plurality of bone conduction transducers 110 emit vibrations 111 into the left part of the user’s skill, whereas the second plurality of bone conduction transducers 120 emit vibrations 121 into the right part of the user’s skill. For example, the first area of the user’s skull may be the mastoid part of one of the temporal bones of the user’s head (such as the mastoid part of the left temporal bone of the user’s head), and the second area of the user’s skull may be the mastoid part of the other one of the temporal bones (such as the mastoid part of the right temporal bone of the user’s head). However, it is to be noted that the present disclosure is not limited thereto. Other parts / areas of the skull may be used as well (e.g., the parietal bones). In general, the first area of the user’s skull and the second area of the user’s skull are opposite to each other. In other words, the first area of the user’s skull is directly contrary to the second area of the user’s skull. In still other words, the first area of the user’s skull and the second area of the user’ s skull are corresponding (the same, matching, congruent) areas on different sides of the user’s skull such as the mastoid part of the left temporal bone and the mastoid part of the right temporal bone of the user’s head.

[0034] The first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may be configured for directly contacting the user’s head (e.g., contacting the user’s scalp). In other examples, a contact material may be provided between the user’s head and the respective one of the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120. The contact material is a material enabling transmission of the vibrations from the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 into the user’s skull. For example, the contact material may be a skin-compatible material such as leather or cloth. The contact material may be part of the apparatus 100 or be part of a device (e.g., a head-mounted equipment) comprising the apparatus 100.

[0035] The apparatus 100 further comprises control circuitry 130 coupled to each of the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120. For example, the control circuitry 130 may be a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which or all of which may be shared, a digital signal processor (DSP) hardware, an application specific integrated circuit (ASIC), a neuromorphic processor or a field programmable gate array (FPGA). The control circuitry 130 may optionally be coupled to, e.g., memory such as read only memory (ROM) for storing software, random access memory (RAM) and / or non-volatile memory. For example, the apparatus 100 may comprise memory configured to store instructions, which when executed by the control circuitry 130, cause the control circuitry 130 to perform the steps and methods described herein.

[0036] The control circuitry 130 is configured to control, based on a target directional motion (movement) to be perceived by the vestibular system of the user, emission of the vibrations 111 and 121 by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120. The target directional motion is motion in a specific direction that the vestibular system of the user is (supposed) to perceive. In other words, the target directional motion is a change in position and / or orientation along a particular path or trajectory that the vestibular system of the user is (supposed) to perceive. For example, target directional motions may be a rotation of the user’s head (e.g., a rotation of the whole body of the user, a motion of the user’s head from left to right, from right to left, from bottom to top, head from top to bottom, a tilt of the user’s head to the left, a tilt of the user’s head to the right, or any combination thereof), a linear movement of the user’ s head without rotation of the head (as the whole body of the user moves; e.g., sideways, frontwards or backwards), or any combination thereof. The rotation of the user’s head may be along any (suitable) axis of rotation. That is, the control circuitry 130 is configured to control the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 such that the emitted vibrations 111 and 121 induce perception of the target directional motion in the vestibular system of the user. For example, the control circuitry 130 may determine parameters for the vibrations 111 and 121 emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 based on the target directional motion and control the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 emit the vibrations 111 and 121 according to the determined parameters. The parameters may, e.g., comprise one or more of respective amplitude, respective phase and respective frequency of the vibrations 111 and 121.

[0037] The vibrations emitted by the individual bone conduction transducers of the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 are subject to constructive and destructive interference. In particular, the vibrations 111 emitted by the individual bone conduction transducers of the first plurality of bone conduction transducers 110 combine to a first resulting combined vibration and the vibrations 121 emitted by the individual bone conduction transducers of the second plurality of bone conduction transducers 120 combine to a second resulting combined vibration. The respective directionality of the resulting combined vibrations is adjusted by the control circuitry 130 by adjusting parameters such as amplitude, phase and / or frequency of the individual vibrations emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120. Accordingly, the directionality of the resulting combined vibrations may be adjusted to align with (correspond to) the directionality of the target directional motion to be perceived by the vestibular system of the user. The resulting combined vibrations are transmitted to the vestibular apparatuses 141 and 142 by bone conduction and causes an according stimulation of the vestibular system of the user. Hence, the user has the impression that the user’s head performs the target directional motion. For example, the semicircular canals of the vestibular organs 141 and 142 may be stimulated to selectively perceive motion about the roll, pitch and / or yaw rotational axes of the user’s head. Similarly, the respective otolithic organs (i.e.., utricle and saccule) of the vestibular organs 141 and 142 may be stimulated to selectively perceive linear accelerations.

[0038] The apparatus 100 allows to create the impression in the user’ s vestibular system that the user’ s head performs a certain directional motion. Compared to conventional approaches such as GVS, neither electrodes nor electrode gel need to be placed on the user’s head. Furthermore, the apparatus 100 improves the comfort for the user as skin irritations or burns may be avoided.

[0039] The apparatus 100 may be used for various applications such as medical applications or VR applications. For example, the apparatus 100 may be used in vertigo treatment to create the impression in the user’s vestibular system that the user’s head performs a certain directional motion. The apparatus 100 may further be used in VR applications to align the motion perceived by the user’s vestibular system with the motion perceived by the user’s eyes to avoid VIMS and / or improve the feeling of immersion.

[0040] As indicated in Fig. 1, the control circuitry 130 may be configured to receive, from an external device such as a server, a computer or a game console, data 101 indicating the target directional motion (movement) to be perceived by the vestibular system of the user. In other examples, the control circuitry 130 may be configured to determine the target directional motion. For example, the control circuitry 130 may compare motion in a VR environment graphically output the user to an actual motion of the user’ s head to determine the target directional motion (e.g., based on a discrepancy or non-alignment between the actual motion of the user’s head and the motion in the VR environment). Further examples how to determine the target directional motion based on motion in a VR environment graphically output the user are described below with respect to Fig. 4. The control circuitry 130 may determine the target directional motion analogously.

[0041] As described above, adjusting the individual phases of the emitted vibrations 111 and 121 allows to adjust the directionality of the resulting combined vibrations. Accordingly, the control circuitry 130 may be configured to determine, based on the target directional motion, a respective phase for the vibrations 111 emitted by each of the first plurality of bone conduction transducers 110 and the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120. The control circuitry 130 is then further configured to control the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 to emit the vibrations 111 and 121 according to the determined respective phase.

[0042] In the following, the effect of adjusting the phases of the vibrations 111 and 121 emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 will be described in more detail with reference to Fig. 2. Fig. 2 exemplarily illustrates in subfigures (a) and (b) two situations with different relative phases between the vibrations 121 emitted by the second plurality of bone conduction transducers 120.

[0043] In the example of subfigure (a), the phases of the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120 are identical. In other words, there is no phase delay between the vibrations 121 emitted by consecutive bone conduction transducers of the second plurality of bone conduction transducers 120. As indicated in subfigure (a) by arrow 210, the user’s vestibular system perceives a motion perpendicular to the second plurality of bone conduction transducers 120.

[0044] In the example of subfigure (b), the phases of the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120 change gradually. In other words, there is a relative phase delay between the vibrations 121 emitted by consecutive bone conduction transducers of the second plurality of bone conduction transducers 120. As indicated in subfigure (a) by arrow 220, the user’s vestibular system perceives a motion diagonal to the second plurality of bone conduction transducers 120.

[0045] In the array of bone conduction transducers 120, the relative phases between the bone conduction transducers may be adjusted such that there is a variable delay between the consecutive bone conduction transducers. The shorter the delay, the more perpendicular is the direction of the perceived vibrations. Vice versa, the longer the relative delay, the more orthogonal the user will feel the direction of motion.

[0046] As is evident from the examples described above with respect to subfigures (a) and (b) of Fig. 2, the directionality of the motion perceived by the user’s vestibular system may be adjusted by adjusting the relative phase delay between the vibrations 111 and 121 emitted by consecutive ones of the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120. In particular, the control circuitry 130 may be configured to determine, based on the target directional motion, a first relative phase delay between the vibrations emitted by two or more (e.g., all) of the first plurality of bone conduction transducers 110 and a second relative phase delay between the vibrations emitted by two or more (e.g., all) of the second plurality of bone conduction transducers 120. The first relative phase delay may be identical to or be different from the second relative phase delay depending on the target directional motion. Accordingly, the control circuitry 130 may determine the respective phase for the vibrations emitted by each of the first plurality of bone conduction transducers 110 and the vibrations emitted by each of the second plurality of bone conduction transducers 120 based on the first relative phase delay and the second relative phase delay.

[0047] Alternatively or additionally, the directionality of the perceived motion may be adjusted by adjusting the individual amplitudes of the emitted vibrations 111 and 121. Accordingly, the control circuitry 130 may be configured to determine, based on the target directional motion, a respective amplitude for the vibrations 111 emitted by each of the first plurality of bone conduction transducers 110 and the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120. The control circuitry 130 is then further configured to control the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 to emit the vibrations 111 and 121 according to the determined respective amplitude. In the following, the effect of adjusting the amplitudes of the vibrations 111 and 121 emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 will be described in more detail with reference to Fig. 3.

[0048] In the example of Fig. 3, the vibrations 111 emitted by the first plurality of bone conduction transducers 110 have a lower amplitude than the vibrations 121 emitted by the second plurality of bone conduction transducers 120. As indicated by arrow 310, the user’s vestibular system perceives a motion to the left, i.e., vestibular system perceives a motion having a directionality from the position of the second plurality of bone conduction transducers 120 to the position of the first plurality of bone conduction transducers 110.

[0049] Similarly, the user’s vestibular system may be stimulated to perceives a motion to the right by controlling the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 such that the vibrations 111 emitted by the first plurality of bone conduction transducers 110 have a higher amplitude than the vibrations 121 emitted by the second plurality of bone conduction transducers 120.

[0050] As is evident from the example described above with respect to Fig. 3, the directionality of the perceived motion may be adjusted by adjusting the amplitudes (amplitude levels) of the vibrations 111 emitted by each of the first plurality of bone conduction transducers 110 and the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120. For example, the control circuitry 130 may be configured to determine, based on the target directional motion, a relative amplitude difference between the vibrations emitted by the first plurality of bone conduction transducers 110 and the vibrations emitted by the second plurality of bone conduction transducers 120 (e.g., that the amplitudes of the vibrations 121 emitted by the second plurality of bone conduction transducers 120 are larger than the amplitudes of the vibrations 111 emitted by the first plurality of bone conduction transducers 111 by a factor X or an absolute value Y - similar to the situation is shown in Fig. 3). The control circuitry 130 is then further configured to determine the respective amplitude for the vibrations 111 emitted by each of the first plurality of bone conduction transducers 110 and the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120 based on the relative amplitude difference. The amplitudes of the vibrations 111 and 121 may be modulated at different timings of the vibration. For example, a driving current or a driving voltage of the bone conduction transducers 110 and 120 may be modulated according to the desired amplitudes of the vibrations 111 and 121. If a higher amplitude (i.e., a stronger vibration or movement speed) is applied in one direction versus the other, the vibrations will not only feel stronger in one direction, but, actually, also physically push / pull the user in one direction, giving an added sense of direction.

[0051] Similar to the amplitude of the vibrations 111 and 121, the frequency of the vibrations 111 and 121 may be used to modulate the intensity of the stimulation of the user’s vestibular system. The ability of the user’s skull to transmit vibrations depends on the frequency of the vibration. Accordingly, varying the frequency of the vibrations 111 and 121 allows to control the amount of vibration and, hence, the amount of motion (in a certain direction) perceived by the user’s vestibular system. The control circuitry 130 may alternatively or additionally be configured to determine, based on the target directional motion, a respective frequency for the vibrations 111 emitted by each of the first plurality of bone conduction transducers 110 and the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120. The control circuitry 130 is then further configured to control the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 to emit the vibrations 111 and 121 according to the determined respective frequency.

[0052] For example, the control circuitry 130 may be configured to adjust, based on the target directional motion, a frequency range for the vibrations emitted by one of the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 different from the frequency range for the vibrations emitted by the other one of the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120. With this configuration, the amount vibration perceived by the user’s vestibular system from each of the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may be adjusted in accordance with the target directional motion to cause an according stimulation of the user’s vestibular system.

[0053] The frequency spectrum emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may further be shaped by means of Head-Related Transfer Functions (HRTFs). A HRTF is a filtering function mimicking the frequency-dependent directional filtering that occurs when waves such as vibrations or sound waves travel from a source to the ears, taking into account the unique characteristics of the user's head, outer ears, and torso.

[0054] The frequency spectrum emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may, e.g., be pre-filtered with the various HRTFs in the various directions from which the vibration should align. The HRTF filtering makes the brain perceive the vibrations as if they had entered the vestibular apparatuses 141 and 142 from a different direction and have been attenuated by different parts of the skull. This principle works for the cochlea in case of spatial audio and works analogously for the vestibular system. The interpretation of directionality is at a higher cortical level and could also potentially be learnt. In other words, the brain could learn to interpret various induced stimuli. For example, a new directional sensation may feel strange in the beginning to a user. However, coupling this feeling with some visual cues a few times may allow the brain to just learn the meaning of the new directional sensation and correctly interpret it in the future for a more effective stimulation of motion perception.

[0055] For example, the control circuitry 130 may additionally be configured to select, based on the target directional motion, a pair of HRTFs from a set of predetermined HRTFs. Accordingly, a set of HRTFs for the vibrations emitted by the first plurality of bone conduction transducers 110 and the second plurality of bone conduction transducers 120 may be selected that aligns with the target directional motion. The selected HRTFs allow to make the brain perceive the vibrations 111 and 121 as if they had entered vestibular apparatuses 141 and 142 from the direction of the target directional motion. The control circuitry 130 is then further configured to filter the determined frequencies for the vibrations 111 emitted by each of the first plurality of bone conduction transducers 110 with one of the selected HRTFs and filter the determined frequencies for the vibrations 121 emitted by each of the second plurality of bone conduction transducers 120 with the other one of the selected HRTFs.

[0056] Combing all of the above described effects by adjusting phase, amplitude and frequency together with HRTF filtering may provide an increased effect, i.e., an improved perception of the target directional motion by the user’s vestibular system. As stated above, the apparatus 100 may be used for VR applications. Fig. 4 illustrates sideview of an exemplary head-mounted equipment 400 using the proposed apparatus 100 for stimulating the vestibular system of a user.

[0057] The head-mounted equipment 400 comprises a display (e.g., a stereoscopic display) 410 configured to graphically output a VR environment (scene). The head-mounted equipment 400 comprises a fastening means (structure) 440 such as one or more straps for mounting the display 410 and other elements of the head-mounted equipment 400 to the user’s head.

[0058] The VR environment (scene) is an environment (scene) in a virtual world, which is simulated to give the user an immersive feel of the virtual world. For example, the virtual reality environment may be an environment in a video game or a metaverse. The user is visually perceiving the VR environment via the display 410. Depending on the user’s viewing direction and the user’ s viewing angle, the user visually perceives a certain part of the VR environment. The viewing direction of the user is the sight direction of the user along which the user is viewing to perceive the VR environment. The viewing angle of the user is the angular range of the VR environment which the user is viewing (that is visible to the user) to perceive the VR environment.

[0059] The VR environment may be generated internally by the head-mounted equipment 400 or be received from an external device such as a server, a computer or a game console. For example, processing circuitry 420 of the head-mounted equipment 400 may be configured to generate the VR environment. In other examples, the head-mounted equipment 400 may comprise a transmitter or transceiver (not illustrated in Fig. 4) configured to receive (e.g., wirelessly or wired) data related the VR environment and the display 410 may be configured to graphically output the VR environment based on the received data.

[0060] The user’s viewing direction and / or viewing angle in the VR environment may be adjusted in various ways. For example, an operator of the VR environment or a VR engine may change the viewing direction and / or viewing angle (e.g., according to a script). In other examples, the viewing direction and / or viewing angle may be changed based on one or more inputs at one or more controllers operable / operated by the user via his / her hands. Additionally or alternatively, the head-mounted equipment 400 may comprise one or more sensors (e.g., one or more accelerometers or one or more gyroscopes; not illustrated in Fig. 1) configured to measure the motion of the user’s head and the viewing direction and / or viewing angle is changed according to the tracked motion of the user’s head.

[0061] Irrespective of how the user’s viewing direction and / or viewing angle in the VR environment is adjusted, there may be a discrepancy between the motion perceived by the user’s eyes from the graphical output of the VR environment and the motion perceived by the user’s vestibular system. For example, if the user’s viewing direction and / or viewing angle in the VR environment is changed based on the tracked motion of the user’s head, the change in the viewing direction and / or viewing angle perceived by the user’s eyes from the graphical output of the VR environment may be more or less than the motion perceived by the user’s vestibular system due to the actual motion of the user’s head in the real world. Similarly, if an avatar of the user moves in the VR environment (e.g., because the avatar is walking / running in the VR environment or because the avatar is located in a moving object such as a vehicle / car or a rollercoaster in the VR environment), the change in position and / or orientation perceived by the user’s eyes from the graphical output of the VR environment may be more or less than the motion perceived by the user’s vestibular system due to the actual motion of the user’s head in the real world. This may cause VIMS or at least reduce the feeling of immersion.

[0062] The apparatus 100 allows align the motion perceived by the user’s vestibular system with the motion perceived by the user’s eyes from the graphical output of the VR environment as the target directional motion input to the control circuitry of the apparatus 100 is based on (derived from, corresponding to) motion in the VR environment graphically output the user. For example, the motion in the VR environment may be a change of the viewing direction in the VR environment, a change of the viewing angle in the VR environment, motion (movement) of the user’s avatar of the user in the VR environment, or any combination thereof.

[0063] For example, the processing circuitry 420 may be configured to determine the target directional motion based on the motion in the VR environment graphically output the user. For example, the processing circuitry 420 may be configured to determine the target directional motion based on the change of the viewing direction in the VR environment, the change of the viewing angle in the VR environment, the motion of the user’s avatar of the user in the VR environment, or any combination thereof. The processing circuitry 420 may compare the motion in the VR environment graphically output the user to an actual motion of the user’s head to determine the target directional motion (e.g., based on a discrepancy or non-alignment between the actual motion of the user’s head and the motion in the VR environment). For example, the processing circuitry 420 may compare a change of the viewing direction in the VR environment, a change of the viewing angle in the VR environment, motion of the user’s avatar of the user in the VR environment, or any combination thereof to actual motion of the user’s head to determine the target directional motion (e.g., based on a discrepancy or non- alignment between the actual motion of the user’s head and the one or more of the change of the viewing direction in the VR environment, the change of the viewing angle in the VR environment and the motion of the user’s avatar of the user in the VR environment). The processing circuitry 420 may, e.g., map the motion in the VR environment graphically output the user to a corresponding target directional motion. For example, the processing circuitry 420 may refer to a Look-Up Table (LUT) for mapping the motion in the VR environment graphically output the user to a corresponding target directional motion. The LUT comprises a plurality of entries denoting a corresponding target directional motion for a given motion in the VR environment. The LUT may be previously learnt. The LUT may be adjusted over time in closed-loop with optimization techniques or machine-learning techniques. However, it is to be noted that the present disclosure is not limited to mapping based on LUTs. In other examples, the processing circuitry 420 may, e.g., analyze the VR environment graphically output the user and determine the motion in the VR environment based on the optical flow of the VR environment graphically output to the user. This may be done on the spot, i.e., online (live, at runtime) during the graphical output of the VR environment to the user.

[0064] The target directional motion is encoded to the data 101 by the processing circuitry 420 and provided to the apparatus 100. However, it is to be noted that the present disclosure is not limited thereto. In other examples, the data 101 indicating the target directional motion may be received from an external device such as a server, a computer or a game console. For example, the external device may provide the data 101 together with the data related the VR environment. The external device may generate the data 101 analogously to what is described above for the processing circuitry 420.

[0065] The bone conduction transducers of the apparatus 100 may, e.g., be integrated into headphones 430 of the head-mounted equipment 400. The headphones 430 are configured to output sounds related to the VR environment, and may be integrated into the same device as the other headmounted equipment. However, in some examples the headphones 430 may be provided separately from other components of the head-mounted equipment 400. The headphones 430 comprise two earpads for the two ears of the user. One of the earpads is exemplarily illustrated in Fig. 4. In the example of Fig. 4, the earpad 435 of the headphones comprises the first plurality of bone conduction transducers 110. Analogously, the other earpad of the headphones 430, which is not depicted in Fig. 4, comprises the first plurality of bone conduction transducers. In other example, the bone conduction transducers of the apparatus 100 may be integrated into another part of the headphones 430 of the head-mounted equipment 400 other than the respective earpads. For example, the bone conduction transducers of the apparatus 100 may be integrated into a support structure (e.g. a headband securing the earpads to the user’s head) of the headphones 430.

[0066] However, it is to be noted that the bone conduction transducers of the apparatus 100 need not be integrated into the headphones 430 of the head-mounted equipment 400. In some examples, the first plurality of bone conduction transducers and the second plurality of bone conduction transducers may be provided separate from the headphones 430.

[0067] In some examples, the head-mounted equipment 400 does not comprise the headphones 430.

[0068] The processing circuitry 420 may be a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which or all of which may be shared, a DSP hardware, an ASIC, a neuromorphic processor or a FPGA. The processing circuitry 420 may optionally be coupled to, e.g., memory such as ROM for storing software, RAM and / or nonvolatile memory. For example, the head-mounted equipment 400 may comprise memory configured to store instructions, which when executed by the processing circuitry 420, cause the processing circuitry 420 to perform the steps and methods described herein.

[0069] The control circuitry 130 of the apparatus 100 and the processing circuitry 420 may be separate elements as illustrated in Fig. 4. In alternative examples, the control circuitry 130 and the processing circuitry 420 may be the same circuitry. In other words, the functionalities of the control circuitry 130 and the processing circuitry 420 may be integrated into a single circuitry.

[0070] The head-mounted equipment 400 may optionally comprise further elements such as one or more sensors (e.g., for eye-tracking) or a haptic feedback device. For further highlighting the stimulation of a user’s vestibular system described above, Fig. 5 illustrates a flowchart of a method for stimulating the vestibular system of a user. The method 500 comprises controlling 502, based on a target directional motion to be perceived by the vestibular system of the user, emission of vibrations by a first plurality of bone conduction transducers and a second plurality of bone conduction transducers. The first plurality of bone conduction transducers are configured to emit vibrations into a first area of the user’s skull for stimulating the vestibular system. The second plurality of bone conduction transducers are configured to emit vibrations into a second area of the user’s skull for stimulating the vestibular system.

[0071] Analogously to what is described above, the method 500 provides improved stimulation of the user’ s vestibular system. In particular, the method 500 allows to induce perception of the target directional motion in the user’s vestibular system.

[0072] More details and aspects of the method 500 are explained in connection with the proposed technique or one or more examples described above (e.g., Fig. 1 to Fig. 4). The method 500 may comprise one or more additional optional features corresponding to one or more aspects of the proposed technique or one or more examples described above.

[0073] The proposed technique provides non-invasive stimulation of directional motion perception.

[0074] The following examples pertain to further embodiments:

[0075] (1) An apparatus for stimulating the vestibular system of a user, comprising: a first plurality of bone conduction transducers configured to emit vibrations into a first area of the user’s skull for stimulating the vestibular system; a second plurality of bone conduction transducers configured to emit vibrations into a second area of the user’s skull for stimulating the vestibular system; and control circuitry configured to control, based on a target directional motion to be perceived by the vestibular system of the user, emission of the vibrations by the first plurality of bone conduction transducers and the second plurality of bone conduction transducers.

[0076] (2) The apparatus of (1), wherein the control circuitry is configured to: determine, based on the target directional motion, a respective amplitude for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers; and control the first plurality of bone conduction transducers and the second plurality of bone conduction transducers to emit the vibrations according to the determined respective amplitude.

[0077] (3) The apparatus of (2), wherein the control circuitry is configured to: determine, based on the target directional motion, a relative amplitude difference between the vibrations emitted by the first plurality of bone conduction transducers and the vibrations emitted by the second plurality of bone conduction transducers; and determine the respective amplitude for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers based on the relative amplitude difference.

[0078] (4) The apparatus of any one of (1) to (3), wherein the control circuitry is configured to: determine, based on the target directional motion, a respective phase for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers; and control the first plurality of bone conduction transducers and the second plurality of bone conduction transducers to emit the vibrations according to the determined respective phase.

[0079] (5) The apparatus of (4), wherein the control circuitry is configured to: determine, based on the target directional motion, a first relative phase delay between the vibrations emitted by two or more of the first plurality of bone conduction transducers and a second relative phase delay between the vibrations emitted by two or more of the second plurality of bone conduction transducers; and determine the respective phase for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers based on the first relative phase delay and the second relative phase delay.

[0080] (6) The apparatus of any one of (1) to (5), wherein the control circuitry is configured to: determine, based on the target directional motion, a respective frequency for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers; and control the first plurality of bone conduction transducers and the second plurality of bone conduction transducers to emit the vibrations according to the determined respective frequency.

[0081] (7) The apparatus of (6), wherein the control circuitry is configured to adjust, based on the target directional motion, a frequency range for the vibrations emitted by one of the first plurality of bone conduction transducers and the second plurality of bone conduction transducers different from the frequency range for the vibrations emitted by the other one of the first plurality of bone conduction transducers and the second plurality of bone conduction transducers.

[0082] (8) The apparatus of (6) or (7), wherein the control circuitry is configured to: select, based on the target directional motion, a pair of head-related transfer functions from a set of predetermined head-related transfer functions; filter the determined frequencies for the vibrations emitted by each of the first plurality of bone conduction transducers with one of the selected head-related transfer functions; and filter the determined frequencies for the vibrations emitted by each of the second plurality of bone conduction transducers with the other one of the selected head-related transfer functions.

[0083] (9) The apparatus of any one of (1) to (8), wherein the first area of the user’s skull and the second area of the user’s skull are opposite to each other.

[0084] (10) The apparatus of any one of (1) to (9), wherein the first area of the user’s skull is the mastoid part of one of the temporal bones of the user’s head, and wherein the second area of the user’s skull is the mastoid part of the other one of the temporal bones.

[0085] (11) A head-mounted equipment, comprising: a display configured to graphically output a virtual reality environment; and an apparatus for stimulating the vestibular system of a user according to any one of (1) to (10), wherein the target directional motion is based on motion in the virtual reality environment graphically output to the user. (12) The head-mounted equipment of (11), further comprising processing circuitry configured to determine the target directional motion based on motion in the virtual reality environment graphically output to the user.

[0086] (13) The head-mounted equipment of (11) or (12), further comprising headphones configured to output sounds related to the virtual reality environment, wherein a first earpad of the headphones comprises the first plurality of bone conduction transducers and a second earpad of the headphones comprises the second plurality of bone conduction transducers.

[0087] (14) The head-mounted equipment of (11) or (12), further comprising headphones configured to output sounds related to the virtual reality environment, wherein the first plurality of bone conduction transducers and the second plurality of bone conduction transducers are provided separate from the headphones.

[0088] (15) A method for stimulating the vestibular system of a user, comprising: controlling, based on a target directional motion to be perceived by the vestibular system of the user, emission of vibrations by a first plurality of bone conduction transducers and a second plurality of bone conduction transducers, wherein the first plurality of bone conduction transducers are configured to emit vibrations into a first area of the user’s skull for stimulating the vestibular system, and wherein the second plurality of bone conduction transducers are configured to emit vibrations into a second area of the user’s skull for stimulating the vestibular system.

[0089] (16) A non-transitory machine-readable medium having stored thereon a program having a program code for performing the method according to (15), when the program is executed on a processor or a programmable hardware.

[0090] (17) A program having a program code for performing the method according to (15), when the program is executed on a processor or a programmable hardware.

[0091] The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example. Examples may further be or relate to a (computer) program including a program code to execute one or more of the above methods when the program is executed on a computer, processor or other programmable hardware component. Thus, steps, operations or processes of different ones of the methods described above may also be executed by programmed computers, processors or other programmable hardware components. Examples may also cover program storage devices, such as digital data storage media, which are machine-, processor- or computer- readable and encode and / or contain machine-executable, processor-executable or computerexecutable programs and instructions. Program storage devices may include or be digital storage devices, magnetic storage media such as magnetic disks and magnetic tapes, hard disk drives, or optically readable digital data storage media, for example. Other examples may also include computers, processors, control units, (field) programmable logic arrays ((F)PLAs), (field) programmable gate arrays ((F)PGAs), graphics processor units (GPU), ASICs, integrated circuits (ICs) or system-on-a-chip (SoCs) systems programmed to execute the steps of the methods described above.

[0092] It is further understood that the disclosure of several steps, processes, operations or functions disclosed in the description or claims shall not be construed to imply that these operations are necessarily dependent on the order described, unless explicitly stated in the individual case or necessary for technical reasons. Therefore, the previous description does not limit the execution of several steps or functions to a certain order. Furthermore, in further examples, a single step, function, process or operation may include and / or be broken up into several sub-steps, - functions, -processes or -operations.

[0093] If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.

[0094] The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.

Claims

ClaimsWhat is claimed is:

1. An apparatus for stimulating the vestibular system of a user, comprising: a first plurality of bone conduction transducers configured to emit vibrations into a first area of the user’s skull for stimulating the vestibular system; a second plurality of bone conduction transducers configured to emit vibrations into a second area of the user’s skull for stimulating the vestibular system; and control circuitry configured to control, based on a target directional motion to be perceived by the vestibular system of the user, emission of the vibrations by the first plurality of bone conduction transducers and the second plurality of bone conduction transducers.

2. The apparatus of claim 1, wherein the control circuitry is configured to: determine, based on the target directional motion, a respective amplitude for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers; and control the first plurality of bone conduction transducers and the second plurality of bone conduction transducers to emit the vibrations according to the determined respective amplitude.

3. The apparatus of claim 2, wherein the control circuitry is configured to: determine, based on the target directional motion, a relative amplitude difference between the vibrations emitted by the first plurality of bone conduction transducers and the vibrations emitted by the second plurality of bone conduction transducers; and determine the respective amplitude for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers based on the relative amplitude difference.

4. The apparatus of claim 1, wherein the control circuitry is configured to:determine, based on the target directional motion, a respective phase for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers; and control the first plurality of bone conduction transducers and the second plurality of bone conduction transducers to emit the vibrations according to the determined respective phase.

5. The apparatus of claim 4, wherein the control circuitry is configured to: determine, based on the target directional motion, a first relative phase delay between the vibrations emitted by two or more of the first plurality of bone conduction transducers and a second relative phase delay between the vibrations emitted by two or more of the second plurality of bone conduction transducers; and determine the respective phase for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers based on the first relative phase delay and the second relative phase delay.

6. The apparatus of claim 1, wherein the control circuitry is configured to: determine, based on the target directional motion, a respective frequency for the vibrations emitted by each of the first plurality of bone conduction transducers and the vibrations emitted by each of the second plurality of bone conduction transducers; and control the first plurality of bone conduction transducers and the second plurality of bone conduction transducers to emit the vibrations according to the determined respective frequency.

7. The apparatus of claim 6, wherein the control circuitry is configured to adjust, based on the target directional motion, a frequency range for the vibrations emitted by one of the first plurality of bone conduction transducers and the second plurality of bone conduction transducers different from the frequency range for the vibrations emitted by the other one of the first plurality of bone conduction transducers and the second plurality of bone conduction transducers.

8. The apparatus of claim 6, wherein the control circuitry is configured to:select, based on the target directional motion, a pair of head-related transfer functions from a set of predetermined head-related transfer functions; filter the determined frequencies for the vibrations emitted by each of the first plurality of bone conduction transducers with one of the selected head-related transfer functions; and filter the determined frequencies for the vibrations emitted by each of the second plurality of bone conduction transducers with the other one of the selected head-related transfer functions.

9. The apparatus of claim 1, wherein the first area of the user’s skull and the second area of the user’ s skull are opposite to each other.

10. The apparatus of claim 1, wherein the first area of the user’s skull is the mastoid part of one of the temporal bones of the user’s head, and wherein the second area of the user’s skull is the mastoid part of the other one of the temporal bones.

11. A head-mounted equipment, comprising: a display configured to graphically output a virtual reality environment; and an apparatus for stimulating the vestibular system of a user according to claim 1, wherein the target directional motion is based on motion in the virtual reality environment graphically output to the user.

12. The head-mounted equipment of claim 11, further comprising processing circuitry configured to determine the target directional motion based on the change of the motion in the virtual reality environment graphically output to the user.

13. The head-mounted equipment of claim 11, further comprising headphones configured to output sounds related to the virtual reality environment, wherein a first earpad of the headphones comprises the first plurality of bone conduction transducers and a second earpad of the headphones comprises the second plurality of bone conduction transducers.

14. The head-mounted equipment of claim 11, further comprising headphones configured to output sounds related to the virtual reality environment, wherein the first plurality of bone conduction transducers and the second plurality of bone conduction transducers are provided separate from the headphones.

15. A method for stimulating the vestibular system of a user, comprising: controlling, based on a target directional motion to be perceived by the vestibular system of the user, emission of vibrations by a first plurality of bone conduction transducers and a second plurality of bone conduction transducers, wherein the first plurality of bone conduction transducers are configured to emit vibrations into a first area of the user’s skull for stimulating the vestibular system, and wherein the second plurality of bone conduction transducers are configured to emit vibrations into a second area of the user’s skull for stimulating the vestibular system.

16. A non-transitory machine-readable medium having stored thereon a program having a program code for performing the method according to claim 15, when the program is executed on a processor or a programmable hardware.

17. A program having a program code for performing the method according to claim 15, when the program is executed on a processor or a programmable hardware.

Images