Apparatus and method for stimulating the vestibular system of a user, and head-mounted equipment

Abstract

Provided is an apparatus for stimulating the vestibular system of a user. The apparatus includes a first stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation and a second stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation. Additionally, the apparatus includes control circuitry configured to determine a respective stimulation effectiveness for each of the different types of vestibular stimulation. The control circuitry is further configured to determine a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses. In addition, the control circuitry is configured to control the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights.

Description

[0001] APPARATUS AND METHOD FOR STIMULATING THE VESTIBULAR SYSTEM OF A USER, AND HEAD-MOUNTED EQUIPMENT

[0002] Field

[0003] The present disclosure relates to stimulation of the vestibular system. In particular, examples of the present disclosure relate to an apparatus and a method for stimulating the vestibular system of a user, a head-mounted equipment, a non-transitory machine-readable medium and a program.

[0004] Background

[0005] The vestibular system of a human being is a sensory system that creates the sense of balance and spatial orientation for the purpose of coordinating movement with balance. Together with the respective cochlea, a part of the auditory system, the vestibular apparatuses of the vestibular system constitute the labyrinths of the inner ears. The vestibular system serves as an organ that detects acceleration along all three axes on both sides of the head through its semicircular canals. The semicircular canals are filled with fluid. The fluid moves with inertia as the head changes position. The movement of fluid pushes on a structure called the cupula which contains hair cells that transduce the mechanical movement to electrical signals. The electrical signals activate the vestibular nerve. This, in turn, stimulates various regions of the brain, including the vestibular cortex, to recognize and respond to the interpreted motion.

[0006] For example, if the vestibular system does not perceive the same motion as the eyes when viewing motion through a Virtual Reality (VR) headset, a user will experience Visually Induced Motion Sickness (VIMS). Even if this is not the case, the user does not have the feeling of being fully immersed in the content.

[0007] Various techniques such as Galvanic Vestibular Stimulation (GVS), magnetic vestibular stimulation or auditory vestibular stimulation may be used to induce motion perception in users or to selectively inhibit the functioning of the vestibular system in order to alleviate VIMS. However, the individual techniques only provide adequate stimulation of the vestibular system under certain conditions.

[0008] Hence, there may be a demand for improved stimulation of the vestibular system

[0009] Summary

[0010] This demand is met by an apparatus and a method for stimulating the vestibular system of a user, a head-mounted equipment, a non-transitory machine-readable medium and a program in accordance with the independent claims. Advantageous embodiments are defined by the dependent claims.

[0011] According to a first aspect, the present disclosure provides an apparatus for stimulating the vestibular system of a user. The apparatus comprises a first stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation and a second stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation. Additionally, the apparatus comprises control circuitry configured to determine a respective stimulation effectiveness for each of the different types of vestibular stimulation. The control circuitry is further configured to determine a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses. In addition, the control circuitry is configured to control the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights.

[0012] According to a second aspect, the present disclosure provides a method for stimulating the vestibular system of a user. The method comprises determining a respective stimulation effectiveness for each of multiple different types of vestibular stimulation supported by each of a first stimulator and a second stimulator for stimulating the vestibular system. Further, the method comprises determining a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses. In addition, the method comprises controlling the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights. According to a third aspect, the present disclosure provides a head-mounted equipment. The head-mounted equipment comprises a display configured to graphically output a VR environment. Additionally, the head-mounted equipment comprises an apparatus for stimulating the vestibular system of a user according to the first aspect. The control circuitry is further configured to control the stimulation of the vestibular system by the first stimulator and the second stimulator based on motion in the VR environment graphically output to the user.

[0013] According to a fourth aspect, the present disclosure provides a non-transitory machine- readable medium having stored thereon a program having a program code for performing the method according to the second aspect, when the program is executed on a processor or a programmable hardware.

[0014] According to a fifth aspect, the present disclosure provides a program having a program code for performing the method according to the second aspect, when the program is executed on a processor or a programmable hardware.

[0015] Brief description of the Figures

[0016] Some examples of apparatuses and / or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

[0017] Fig. 1 illustrates a first exemplary apparatus for stimulating the vestibular system of a user;

[0018] Fig. 2 illustrates a second exemplary apparatus for stimulating the vestibular system of a user;

[0019] Fig. 3 illustrates an exemplary head-mounted equipment; and

[0020] Fig. 4 illustrates a flowchart of an example of a method for stimulating the vestibular system of a user.

[0021] Detailed Description Some examples are now described in more detail with reference to the enclosed figures. However, other possible examples are not limited to the features of these embodiments described in detail. Other examples may include modifications of the features as well as equivalents and alternatives to the features. Furthermore, the terminology used herein to describe certain examples should not be restrictive of further possible examples.

[0022] Throughout the description of the figures same or similar reference numerals refer to same or similar elements and / or features, which may be identical or implemented in a modified form while providing the same or a similar function. The thickness of lines, layers and / or areas in the figures may also be exaggerated for clarification.

[0023] When two elements A and B are combined using an “or”, this is to be understood as disclosing all possible combinations, i.e. only A, only B as well as A and B, unless expressly defined otherwise in the individual case. As an alternative wording for the same combinations, "at least one of A and B" or "A and / or B" may be used. This applies equivalently to combinations of more than two elements.

[0024] If a singular form, such as “a”, “an” and “the” is used and the use of only a single element is not defined as mandatory either explicitly or implicitly, further examples may also use several elements to implement the same function. If a function is described below as implemented using multiple elements, further examples may implement the same function using a single element or a single processing entity. It is further understood that the terms "include", "including", "comprise" and / or "comprising", when used, describe the presence of the specified features, integers, steps, operations, processes, elements, components and / or a group thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components and / or a group thereof.

[0025] Fig- 1 illustrates an apparatus 100 for stimulating the vestibular system of a user. As a reference, the head 150 of the user - including the ears 151 and 152 at the left and the right side of the head 150 - is schematically illustrated in Fig. 1. The vestibular system of the user comprises the central system in the brain and the brainstem as well as the peripheral system in the inner ears (and the pathways to the brainstem). For reasons of simplicity, only the vestibular apparatuses 141 and 142 in the inner ears are schematically illustrated in Fig. 1. The vestibular apparatus 141 is arranged behind the left ear 151 in the inner ear of the left ear 151. Analogously, the vestibular apparatus 142 is arranged behind the right ear 152 in the inner ear of the right ear 152.

[0026] The apparatus 100 comprises a first stimulator 110 capable of stimulating (able to stimulate) the vestibular system by multiple different types of vestibular stimulation and a second stimulator 120 capable of stimulating (able to stimulate) the vestibular system by multiple different types of vestibular stimulation. In other words, each of the first stimulator 110 and the second stimulator 120 is capable of stimulating the vestibular system by at least two (two or more) different types of vestibular stimulation. That is, each of the first stimulator 110 and the second stimulator 120 is capable of affecting (influencing) the vestibular system of the user such that the perception of motion is manipulated or induced, by using at least two different types (mechanisms) of vestibular stimulation. The types of vestibular stimulation supported by the first stimulator 110 may be the same as or be different from the types of vestibular stimulation supported by the second stimulator 120. For example, the different types of vestibular stimulation may comprise two or more of electrical (vestibular) stimulation, vibrational (vestibular) stimulation, magnetic (vestibular) stimulation, thermal (vestibular) stimulation and acoustic (vestibular) stimulation.

[0027] Electrical (vestibular) stimulation refers to the application of controlled (predetermined) electrical power or currents to the user’s vestibular system. The applied electrical power or currents allow to modulate or manipulate the electrical signals within the vestibular system and, hence, to manipulate or induce the perception of motion in the vestibular system of the user. The amplitude of the applied currents may, e.g., range from 0.1 mA to 2mA depending on the specific application. For example, each of the first stimulator 110 and the second stimulator 120 may comprise one or more electrodes configured to selectively apply electrical power or an electrical current 111 to the user’s skull for modulating or manipulating the electrical signals within the vestibular system of the user. The first stimulator 110 may comprise the same number of electrodes like the second stimulator 120. However, it is to be noted that the present disclosure is not limited thereto. In other examples, the first stimulator 110 may comprise a different number of electrodes than the second stimulator 120. GVS is a special type of electrical (vestibular) stimulation that utilizes direct current (galvanic current). Vibrational (vestibular) stimulation refers to the application of (mechanical) vibrations to the user’s vestibular system. The vibrations 111 and 121 output by the first stimulator 110 and the second stimulator 120 are transmitted through the bones of the user’s skull (e.g., the cheekbones or the temporal bones, in particular the mastoid parts of the temporal bones) to the inner ear. The organs in the inner ear such as the vestibular apparatuses 141 and 142 or the respective cochlea translate these vibrations into electrical signals that the brain interprets. . The vestibular system of the user is stimulated by the vibrations rather than the auditory systems. In other words, the first stimulator 110 and the second stimulator 120 are each configured to emit the vibrations for the purpose of stimulating the vestibular system of the user rather than the auditory systems. Accordingly, the frequencies of the vibrations 111 and 121 emitted by the first stimulator 110 and the second stimulator 120 are such that the vestibular apparatuses 141 and 142 can recognize the vibrations. In particular, the frequencies of the vibrations 111 and 121 emitted by the first stimulator 110 and the second stimulator 120 are such that the perception of motion in the vestibular system is manipulated or induced when the vestibular apparatuses 141 and 142 translate these vibrations. The vestibular apparatuses 141 and 142 are particularly sensitive for vibrations at a frequency of approx. 250 Hz. For example, the frequency or frequencies of the vibrations emitted by the first stimulator 110 and the second stimulator 120 may be at least 10 Hz, 20 Hz or 50 Hz. On the other hand, the frequency or frequencies of the vibrations emitted by the first stimulator 110 and the second stimulator 120 may be at maximum 1000 Hz, 750 Hz or 500 Hz. In some examples, the frequency or frequencies of the vibrations emitted by the first stimulator 110 and the second stimulator 120 may be between 50 Hz and 500 Hz, and in particular between 200 Hz and 300 Hz. Each of the first stimulator 110 and the second stimulator 120 may comprise one or more bone conduction transducers (e.g., arranged in a respective array) configured to emit the vibrations for stimulating the user’s vestibular system. The first stimulator 110 may comprise the same number of bone conduction transducers like the second stimulator 120. However, it is to be noted that the present disclosure is not limited thereto. In other examples, the first stimulator 110 may comprise a different number of bone conduction transducers than the second stimulator 120. For example, a bone conduction transducer may comprise one or more electromagnets attracting and repelling one or more vibration plates for emitting vibrations. In other examples, a bone conduction transducer may comprise one or more piezoelectric vibrators for emitting vibrations. In still other examples, a bone conduction transducer may capacitively attract and repel one or more vibration plates for emitting vibrations. A bone conduction transducer may be a Micro-ElectroMechanical Systems (MEMS) device.

[0028] Magnetic (vestibular) stimulation refers to the application of magnetic fields to the user’s vestibular system. The applied magnetic fields allow to induce electrical currents (signals) in the vestibular system by electromagnetic induction. The induced electrical currents (signals) allow to modulate or manipulate the electrical signals within the vestibular system and, hence, to manipulate or induce the perception of motion in the vestibular system of the user. The amplitude of the applied magnetic fields may, e.g., range from micro Tesla to milli Tesla depending on the specific application. The frequencies of the applied magnetic fields may, e.g., range from below 1 Hz to several hundred Hertz. For example, each of the first stimulator 110 and the second stimulator 120 may comprise one or more magnetic coils configured to selectively apply a respective magnetic field to the user’s head for inducing electrical currents (signals) in the vestibular system of the user. The first stimulator 110 may comprise the same number of magnetic coils like the second stimulator 120. However, it is to be noted that the present disclosure is not limited thereto. In other examples, the first stimulator 110 may comprise a different number of magnetic coils than the second stimulator 120.

[0029] Thermal (vestibular) stimulation (also known as caloric vestibular stimulation) refers to the application of thermal stimuli to the user’s vestibular system. The thermal stimuli lead to alterations in the fluid dynamics within the semicircular canals of the vestibular apparatuses 141 and 142. This, in turn, affects the firing patterns of vestibular nerve fibers, influencing the signals sent to the brain about the body's position and motion. Accordingly, perception of motion in the vestibular system of the user may be manipulated or induced. For example, each of the first stimulator 110 and the second stimulator 120 may comprise means for delivering warm or cool air, water, or other thermal agents to the ear canals or other areas around the vestibular apparatuses 141 and 142.

[0030] Acoustic (vestibular) stimulation refers to the application of sound or acoustic stimuli to the user’s vestibular system. The characteristics of the sound or acoustic stimuli, including frequency, intensity, and spatial properties, can influence the vestibular system. The sound waves transmitted through the auditory system can lead to vibrations in the inner ear structures, including the fluid-filled semicircular canals. These vibrations, in turn, affect the firing patterns of vestibular nerve fibers, influencing the signals sent to the brain about the body's position and motion. Accordingly, perception of motion in the vestibular system of the user may be manipulated or induced. For example, clicks or tone bursts may be used to stimulate the user’s vestibular system. Each of the first stimulator 110 and the second stimulator 120 may, e.g., comprise one or more speakers (sound output devices) configured to selectively output sound or acoustic stimuli for inducing vibrations in the vestibular apparatuses 141 and 142. The first stimulator 110 may comprise the same number of speakers like the second stimulator 120. However, it is to be noted that the present disclosure is not limited thereto. In other examples, the first stimulator 110 may comprise a different number of speakers than the second stimulator 120. The sound or acoustic stimuli used for acoustic (vestibular) stimulation may exhibit frequencies in the hearing range (i.e., between 20 Hz and 20 kHz).

[0031] However, it is to be noted that the present disclosure is not limited to the aforementioned types of vestibular stimulation. Other types of vestibular stimulation may be used additionally or alternatively.

[0032] The first stimulator 110 is configured to contact a first area of the user’s skull for stimulating the vestibular system. The second stimulator 120 is configured to contact a second area of the user’s skull for stimulating the vestibular system. The second area of the user’s skull is different from the first area of the user’s skull. In the example of Fig. 1, the first stimulator 110 applies an electrical power or current 111 to and emits vibrations 112 into the left part of the user’s skill, whereas the stimulator 120 applies an electrical power or current 121 to and emits vibrations 122 into the right part of the user’s skill. Electrical power / current and vibrations are illustrated in Fig. 1 as exemplary means for electrical and vibrational stimulation of the vestibular system. However, as indicated above, alternative or additional types of vestibular stimulation (vestibular stimulation techniques) may be used. The first area of the user’s skull may, e.g., be the mastoid part of one of the temporal bones of the user’s head (such as the mastoid part of the left temporal bone of the user’s head), and the second area of the user’s skull may be the mastoid part of the other one of the temporal bones (such as the mastoid part of the right temporal bone of the user’s head). However, it is to be noted that the present disclosure is not limited thereto. Other parts / areas of the skull may be used as well (e.g., the parietal bones). In some examples, the first area of the user’s skull and the second area of the user’s skull are opposite to each other. In other words, the first area of the user’s skull is directly contrary to the second area of the user’s skull. In still other words, the first area of the user’ s skull and the second area of the user’ s skull are corresponding (the same, matching, congruent) areas on different sides of the user’s skull such as the mastoid part of the left temporal bone and the mastoid part of the right temporal bone of the user’s head.

[0033] The first stimulator 110 and the second stimulator 120 may be configured for directly contacting the user’s head (e.g., contacting the user’s scalp). In other examples, a contact material may be provided between the user’s head and at least part of the respective one of the first stimulator 110 and the second stimulator 120. The contact material is a material enabling transmission of various or selected types of energy from the first stimulator 110 and the second stimulator 120 into the user’s skull. For example, the contact material may be a skin-compatible material (a material that is comfortable to the user and does not irritate the skin) such as leather or cloth. The contact material may be part of the apparatus 100 or be part of a device (e.g., a head-mounted equipment) comprising the apparatus 100.

[0034] The apparatus 100 further comprises control circuitry 130 coupled to each of the first stimulator 110 and the second stimulator 120. For example, the control circuitry 130 may be a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which or all of which may be shared, a digital signal processor (DSP) hardware, an application specific integrated circuit (ASIC), a system-on-a-chip (SoC) a neuromorphic processor or a field programmable gate array (FPGA). The control circuitry 130 may optionally be coupled to, e.g., memory such as read only memory (ROM) for storing software, random access memory (RAM) and / or non-volatile memory. For example, the apparatus 100 may comprise memory configured to store instructions, which when executed by the control circuitry 130, cause the control circuitry 130 to perform the steps and methods described herein.

[0035] The control circuitry 130 is configured to determine a respective stimulation effectiveness for each of the different types of vestibular stimulation (used and / or supported by the stimulators 110 and 120). The stimulation effectiveness of a given type of vestibular stimulation refers to the degree to which the given type of vestibular stimulation produces or causes the desired (targeted, intended) stimulation of the vestibular system. In particular, the stimulation effectiveness of a given type of vestibular stimulation denotes the ability of the given type of vestibular stimulation to induce or manipulate perception of motion in the vestibular system of the user. The respective stimulation effectiveness may be determined based on various inputs and parameters. Exemplary inputs and parameters for determining the respec- tive stimulation effectiveness for the different types of vestibular stimulation will be described below in greater detail.

[0036] The control circuitry 130 is further configured to determine a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses. The determined weights for the different types of vestibular stimulation determine a relative strength of the respective type of vestibular stimulation in the total vestibular stimulation by the first stimulator 110 and the second stimulator 120. The control circuitry 130 may, e.g., be configured to adjust (set) the weight or relative strength of the respective type of vestibular stimulation according to the determined stimulation effectivenesses for the different types of vestibular stimulation. In other words, the weight (relative strength) of the respective type of vestibular stimulation may scale with the determined stimulation effectiveness for the respective type of vestibular stimulation. For example, the control circuitry may set a larg- er / greater weight (i.e., a greater relative strength) for a first type of vestibular stimulation than for a second type of vestibular stimulation if the determined stimulation effectiveness for the first type of vestibular stimulation is higher than for the second type of vestibular stimulation.

[0037] In addition, the control circuitry 130 is configured to control the first stimulator 110 and the second stimulator 120 to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights. For example, the control circuitry 130 may generate control signals for the first stimulator 110 and the second stimulator 120 according to the determined weights. The control circuitry 130 may further supply the respective control signal to the first stimulator 110 and the second stimulator 120. Based on the control signals, the first stimulator 110 and the second stimulator 120 stimulate the vestibular system by one or more of the different types of vestibular stimulation.

[0038] The apparatus 100 allows improved stimulation of the user’s vestibular system. In particular, the apparatus 100 allows for increasing the relative strength(s) of the supported types of vestibular stimulation that exhibit (show) higher stimulation effectiveness(es) compared to those showing lower stimulation effectiveness(es). In other words, supported types of vestibular stimulation with higher stimulation effectiveness(es) are preferred over supported types of vestibular stimulation with lower stimulation effectiveness(es). The apparatus 100 allows to leverage the advantages of the different supported types of vestibular stimulation as the weights (i.e., the relative strength) of the most effective types of vestibular stimulation are increased compared to the less effective types of vestibular stimulation. In other words, (e.g., temporary) failure or disadvantages of some of the supported types of vestibular stimulation are compensated by the weighting according to the determined stimulation effectivenesses.

[0039] As described above, respective stimulation effectiveness for the different types of vestibular stimulation may be determined based on various inputs and parameters. In the following, a few examples will be described.

[0040] For example, the control circuitry 130 may be configured to receive first sensor data 101. The first sensor data 101 are indicative of one or more measured stimulation parameters for the different types of vestibular stimulation. A stimulation parameter is a parameter that indicates the quality and / or efficiency (effectiveness) of a given type of vestibular stimulation. The control circuitry 130 may further be configured to determine the respective stimulation effectiveness for the different types of vestibular stimulation based on the first sensor data 101 (i.e., based on the one or more measured stimulation parameters).

[0041] For example, for electrical vestibular stimulation (e.g., for GVS), the stimulation effectiveness is mainly determined by the electrical resistance of the stimulation path. For example, a measured stimulation parameter may be the current(s) at one or more electrodes of the stimulators 110 and 120. It may, e.g., be determined based on the measured currents whether the targeted (intended, selected) current is flowing through the electrodes of the stimulators 110 and 120. For example, for a constant current source setup, it may be determined how the voltage that is applied to the electrodes is rising close to the supply (the current source raises its voltage to be able to provide the same current). These parameters indicate the effectiveness of electrical vestibular stimulation (e.g., for GVS). Similarly, applying a small electrical stimulation via one or both of the stimulators 110 and 120 and measure the resistance of the user’s scalp may be a good indicator if electrical vestibular stimulation will be effective. The presence of humidity on the user’s scalp (e.g., sweat) may indicate an increased electrical conductivity and, hence, high effectiveness of electrical vestibular stimulation. Accordingly, the humidity on the user’s scalp may be a measured stimulation parameter (e.g., via the user’s skin conductivity). Also the contact pressure of one or more electrodes of the stimulators 110 and 120 may be a measured as stimulation parameter since the contact pressure indicates if the electrode(s) is (are) attached (or detached) from the user’s scalp, which will affect the effectiveness of the electrical vestibular stimulation.

[0042] For magnetic vestibular stimulation, the stimulation effectiveness may be determined by (based on) the respective distance of one or both of the stimulators 110 and 120 to the user’s head and, hence, the user’s vestibular system. For example, the stimulation effectiveness may decrease over distance. The respective distance of one or both of the stimulators 110 and 120 to the user’s head may, e.g., be measured as a stimulation parameter as the distance affects the effectiveness of the magnetic vestibular stimulation. The respective distance of one or both of the stimulators 110 and 120 to the user’s vestibular system may be calculated based on the measured respective distance of one or both of the stimulators 110 and 120 to the user’s head (e.g., according to predefined mathematical expressions or equations).

[0043] For vibrational or thermal vestibular stimulation, good contact of the stimulators 110 and 120 with the user’s scalp is important. For example, the contact pressure of one or more bone conduction transducers of one or both of the stimulators 110 and 120 or the contact pressure of the means for delivering warm or cool air, water, or other thermal agents to the ear canals of one or both of the stimulators 110 and 120 may be measured as stimulation parameter as it provides feedback on the contact surface. For example, a contact pressure above a predefined threshold may indicate sufficient / good contact between the user’s head / scalp and the respective one of the stimulators 110 and 120. On the other hand, a contact pressure below the predefined threshold may indicate that one or both of the stimulators 110 and 120 are not in good contact with the user’s head / scalp (e.g., due to detachment) Similarly, if a small electrical stimulation is applied to the user’s scalp, the skin resistance may be measured to determine if the area is too fatty to be effective for bone vibration due to underlying soft tissue. Further, the effectiveness or quality of vibrational vestibular stimulation may be determined based on measured bone vibrations transmitted by the user’s skull (or parts thereof). If the Signal-to-Noise Ratio (SNR) of the measured bone vibrations degrades, the vibrational vestibular stimulation becomes less effective.

[0044] Analogously to the exemplary stimulation parameters described above for selected types of vestibular stimulation, one or more stimulation parameters may be measured for any type of vestibular stimulation. For example, a score for the stimulation effectiveness of the respective type of vestibular stimulation may be determined based on one or more of the measured stimulation parameters (e.g., one or more of the examples described above). The score is a value indicating the stimulation effectiveness of the respective type of vestibular stimulation. The minimum and maximum values of the score are within a predefined value range. For example, the score may be expressed as a percentage value that can range from 0 % to 100 % to indicate the stimulation effectiveness of the respective type of vestibular stimulation. In other examples, the score may be expressed as digit X relative to a reference digit Y (e.g., X out of 10, i.e., with Y being 10 and the predefined value range ranging from 0 to 10) to indicate the stimulation effectiveness of the respective type of vestibular stimulation.

[0045] As described above, according to the quality or effectiveness of the measured stimulation, the weighting of the type or modality of the stimulation is adjusted. According to example, one type of stimulation is favored over the other. The weights (i.e., the relative strengths) of the different type of vestibular stimulation (i.e., the relative strength of the vibrational vestibular stimulation) may, e.g., be adjusted according to their respective score indicating the respective stimulation effectiveness. For example, the amplitude of the emitted vibrations (i.e., the relative strength of the vibrational vestibular stimulation) may be increased if GVS is determined to be less effective due to the detachment of one or more electrodes from the user’s scalp. Similarly, the amplitude of the emitted magnetic field(s) may be increased if magnetic (vestibular) stimulation is determined to be more effective than vibrational (vestibular) stimulation due a high SNR of measured bone vibrations or because a measured skin resistance indicates that a contact area contacting one or both of the stimulators 110 and 120 is too fatty to be effective for bone vibration due to underlying soft tissue. In other examples, electrical currents used for GVS may be reduced if it is determined that GVS is less effective than vibrational (vestibular) stimulation or acoustic (vestibular) stimulation as the measured electrical resistance an area of the user’s scalp, which contacts one or both of the stimulators 110 and 120, is high. It is to be noted that the adaptation of the weighting described in the foregoing examples are selected for illustrative purposes. Other adaptations may be used alternatively or in addition depending on the types of vestibular stimulation supported by the stimulators 110 and 120.

[0046] In some examples, the apparatus 100 comprises one or more first sensors configured to measure the one or more stimulation parameters for the different types of vestibular stimula- tion and generate the first sensor data 101 based on the one or more measured stimulation parameters. In the example of Fig. 1, only a single first sensor 160 is illustrated for reasons of simplicity. For example, one or more of a Time-of-Flight (ToF) sensor an ultrasonic sensor or an infrared proximity sensor may be used to measure the distance of one or both of the stimulators 110 and 120 to the user’s scalp. A bone vibration sensor may be used to measure bone vibrations transmitted by the user’s skull (or parts thereof). Capacitive or resistive pressure sensors may, e.g., be used to measure a contact pressure as described above.

[0047] However, it is to be noted that the one or more first sensors for measuring the one or more stimulation parameters for the different types of vestibular stimulation need not be part of the apparatus 100. In some examples, the one or more first sensors for measuring the one or more stimulation parameters for the different types of vestibular stimulation may be external to the apparatus 100. For example, the one or more first sensors for measuring the one or more stimulation parameters for the different types of vestibular stimulation may be part of a separate device.

[0048] Alternatively or additionally, the control circuitry 130 may be configured to receive second sensor data 102. The second sensor data 102 are indicative of at least one measured physiological property of the user. The physiological property of the user is a property (quantity, characteristic) describing the physiology of the user. In other words, the physiological property is a property describing one or more functions and / or mechanisms in the user’s body. For example, the physiological property may be one or more of a stress level of the user, a heart (pulse) rate of the user, a heart rate variability of the user, a cardiac cycle of the user, a respiration of the user, a blood pressure of the user, a level or concentration of a hormone (e.g., a hormone indicating stress such as cortisol), a sweat level of the user (e.g., indicated by the user’s skin conductivity), an electrical activity on the scalp of the user (e.g., an electroencephalogram indicative of the user’s brain activity), an electrical activity of the skeletal muscles of the user (e.g., an electromyogram), etc. The control circuitry 130 may further be configured to determine the respective stimulation effectiveness for the different types of vestibular stimulation based on the second sensor data 102 (i.e., based on the at least one measured physiological property of the user).

[0049] Vital signs like the aforementioned physiological properties allow to determine that the user is perceiving motion. For example, skeletal muscles contract if the user perceives an acceler- ation. The electrical activity due to the muscle contraction may, e.g., be measured by means of an electromyogram. Similarly, vital signs like the aforementioned physiological properties allow to determine whether a type of vestibular stimulation is ineffective or even counterproductive. For example, an increased heart rate of the user or and increased skin conductivity due to increased sweating of the user may indicate stress of the user due to ineffective or even counter-productive vestibular stimulation. Similarly, a high level or concentration of a hormone such as cortisol may indicate stress of the user due to ineffective or even counterproductive vestibular stimulation.

[0050] For example, the score for the stimulation effectiveness of the respective type of vestibular stimulation may be determined based on one or more of the measured physiological properties (e.g., one or more of the examples described above).

[0051] As described above, the weighting of the type of the stimulation is adjusted according to the stimulation effectiveness. According to examples, one type of stimulation is favored over the other. The weights (i.e., the relative strengths) of the different type of vestibular stimulation may, e.g., be adjusted according to their respective score indicating the respective stimulation effectiveness. For example, the weights (i.e., the relative strengths) of the different type of vestibular stimulation may be adjusted to reduce or minimize one or more of the measured physiological parameters. For example, the weights may be adjusted to reduce or minimize one or more of the user’s heart rate, the user’s sweat level or the user’s cortisol level. Accordingly, a desired vestibular stimulation and, hence, a good user experience may be achieved.

[0052] In some examples, the apparatus 100 comprises one or more second sensors (e.g., one or more physiological sensors) configured to measure the at least one physiological property of the user and generate the second sensor data 102 based on the measured at least one physiological property of the user. In the example of Fig. 1, a single second sensor 170 is illustrated for reasons of simplicity. For example, the one or more second sensors may be one or more of a PhotoPlethysmoGraphy (PPG) sensor, an ElectroCardioGraphy (ECG) sensor, an Elec- troEncephaloGraphy (EEG) sensor, an ElectroMyoGraphy (EMG) sensor, a Galvanic Skin Response (GSR) sensor (e.g., for sweat detection) and a sensor for detecting cortisol (being an indicator for stress) or a bio-sensor for detecting other hormones. However, it is to be noted that the one or more second sensors for measuring the at least one physiological property of the user need not be part of the apparatus 100. In some examples, the one or more second sensors for measuring the at least one physiological property of the user may be external to the apparatus 100. For example, the one or more second sensors for measuring the at least one physiological property of the user may be part of a separate device.

[0053] The apparatus 100 may be used for various applications such as medical applications or VR applications. For example, the apparatus 100 may be used in vertigo treatment. The apparatus 100 may further be used in VR applications to avoid VIMS and / or improve the feeling of immersion.

[0054] The different types of vestibular stimulation have advantages and disadvantages in inducing perception of certain types of motion in the vestibular system of the user. Hence, if the apparatus 100 is used for VR applications, the motion in the VR environment graphically output to the user may further determine the weights for the different supported types of vestibular stimulation. For example, electrical vestibular stimulation such as GVS may be very effective in conveying rotational motion perception. Accordingly, electrical vestibular stimulation such as GVS may be advantageous for strong motions in a VR environment graphically output to the user. On the other hand, vibrational vestibular stimulation is less invasive than electrical vestibular stimulation (e.g., noisy GVS) and very effective to reduce VIMS for small motions in the VR motion. Hence, vibrational vestibular stimulation may be advantageous for small motions in the VR environment graphically output to the user. In view of the foregoing explanations, the control circuitry 130 may be configured to determine the respective stimulation effectiveness for the different types of vestibular stimulation based on a type of motion in the VR environment graphically output to the user. For example, depending on rotational or linear motions in the VR environment graphically output to the user, the type of vestibular stimulation most effective for the particular scenario may be chosen and the weights may be adjusted accordingly. For example, the score for the stimulation effectiveness of the respective type of vestibular stimulation may be determined based on the type of motion in the VR environment graphically output to the user (e.g., according to one or more of the examples described above). The weights (i.e., the relative strengths) of the different type of vestibular stimulation may, e.g., be adjusted according to their respective score indicating the respective stimulation effectiveness. For example, vibrational vestibular stimula- tion may be used / favored (i.e., the weight is increased) to reduce VIMS for small motions in the VR environment graphically output to the user, while electrical vestibular stimulation such as GVS may be used / favored (i.e., the weight is increased) to reduce VIMS for strong motions in the VR environment graphically output to the user.

[0055] The control circuitry 130 may be provided with information about the type of motion in the VR environment graphically output to the user in various ways. For example, the control circuitry 130 may be configured to receive (e.g., wirelessly or wired) data 104 indicating (encoded with information about) the type of motion in the VR environment graphically output to the user from an external source (device). For example, the control circuitry 130 may receive the data 104 from an external device such as a server, a computer, a game console or processing circuitry of a head-mounted equipment that controls the graphically output of the VR environment at a display of the head-mounted equipment. In other examples, the control circuitry 130 may be configured to determine the type of motion in the VR environment graphically output to the user based on data 105 indicating (encoded with information about) the VR environment graphically output to the user. For example, the data 105 may be received from an external device such as a server, a computer, a game console or processing circuitry of a head-mounted equipment that controls the graphically output of the VR environment at a display of the head-mounted equipment. The control circuitry 130 may, e.g., analyze the VR environment graphically output the user and determine the type of motion in the VR environment based on the optical flow of the VR environment graphically output to the user. This may be done on the spot, i.e., online (live, at runtime) during the graphical output of the VR environment to the user.

[0056] The respective stimulation effectiveness for the different types of vestibular stimulation as well as the respective weight for the different types of vestibular stimulation may be determined rule-based. For example, the processing circuitry 130 may be configured to execute a program having program code specifying the rules for determining the respective stimulation effectiveness for the different types of vestibular stimulation and determining the respective weight for the different types of vestibular stimulation. Instead of using one or more rulebased algorithms, the processing circuitry 130 may in alternative examples be configured to use a trained machine-learning model for at least one of determining the respective stimulation effectiveness for the different types of vestibular stimulation and determining the respective weight for the different types of vestibular stimulation. The machine-learning model is a data structure and / or set of rules representing a statistical model that the processing circuitry 120 uses to determine the respective stimulation effectiveness for each of the different types of vestibular stimulation and / or to determine the respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses without using explicit instructions or rules, instead relying on models and inference. The data structure and / or set of rules represents learned knowledge (e.g., based on training performed by a machine-learning algorithm). In machine-learning, instead of a rule-based transformation of data, a transformation of data may be used, that is inferred from an analysis of training data.

[0057] The machine-learning model is trained by a machine-learning algorithm. The term "machine-learning algorithm" denotes a set of instructions that are used to create, train or use a machine-learning model. For the machine-learning model to determine the respective stimulation effectiveness for each of the different types of vestibular stimulation, the machinelearning model may be trained using training data such as one or more of first sensor data indicative of measured stimulation parameters for the different types of vestibular stimulation, second sensor data indicative of at least one measured physiological property of the user, and data indicating a type of motion in VR environment graphically output to the user as input. Further, predefined (known) stimulation effectiveness for each of the different types of vestibular stimulation may be used as target output for the training data. By training the machine-learning model with a large set of training data and associated training content information, the machine-learning model "learns" how to determine the respective stimulation effectiveness for each of the different types of vestibular stimulation, so that stimulation effectiveness for the different types of vestibular stimulation can be obtained using the machine-learning model. Similarly, for the machine-learning model to determine the respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses, the machine-learning model may be trained using training data such as stimulation effectivenesses for different types of vestibular stimulation and predefined (known) weights for the different types of vestibular stimulation as target output. By training the machine-learning model with a large set of training data and associated training content information, the machine-learning model "learns" how to determine the respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses, so that weights for the different types of vestibular stimulation can be obtained using the machine-learning model. By training the machine-learning model using the respective type of training data and respective associated training content information, the machine-learning model "learns" a transformation between the respective training data and the desired output, which can be used to provide an output based on non-training data provided to the machinelearning model. For example, one or more of first sensor data indicative of measured stimulation parameters for the different types of vestibular stimulation, second sensor data indicative of at least one measured physiological property of the user, and data indicating a type of motion in VR environment graphically output to the user may be provided to the machinelearning model as input for determining the respective stimulation effectiveness for each of the different types of vestibular stimulation. Similarly, stimulation effectivenesses for different types of vestibular stimulation may be provided to the machine-learning model as input for determining the respective weight for the different types of vestibular stimulation.

[0058] The machine-learning model may be trained using training input data (e.g., the training data described in the foregoing paragraph). For example, the machine-learning model may be trained using a training method called "supervised learning". In supervised learning, the machine-learning model is trained using a plurality of training samples, wherein each sample may comprise a plurality of input data values, and a plurality of desired output values, i.e., each training sample is associated with a desired output value. By specifying both training samples and desired output values, the machine-learning model "learns" which output value to provide based on an input sample that is similar to the samples provided during the training. For example, a training sample may comprise one or more of first sensor data indicative of measured stimulation parameters for the different types of vestibular stimulation, second sensor data indicative of at least one measured physiological property of the user, and data indicating a type of motion in VR environment graphically output to the user as input data and stimulation effectiveness for each of the different types of vestibular stimulation as desired output data. Similarly, a training sample may comprise stimulation effectivenesses for different types of vestibular stimulation as input data and the respective weight for the different types of vestibular stimulation as desired output data.

[0059] Apart from supervised learning, semi-supervised learning may be used. In semi-supervised learning, some of the training samples lack a corresponding desired output value. Supervised learning may be based on a supervised learning algorithm (e.g., a classification algorithm or a similarity learning algorithm). Classification algorithms may be used as the desired outputs of the trained machine-learning model are restricted to a limited set of values (categorical variables), i.e., the input is classified to one of the limited set of values (e.g., only first sensor data indicative of measured stimulation parameters for the different types of vestibular stimulation or only data indicating a type of motion in VR environment graphically output to the user; only stimulation effectivenesses for certain types of vestibular stimulation or stimulation effectivenesses within a certain value range). Similarity learning algorithms are similar to classification algorithms but are based on learning from examples using a similarity function that measures how similar or related two objects are.

[0060] Apart from supervised or semi-supervised learning, unsupervised learning may be used to train the machine-learning model. In unsupervised learning, (only) input data are supplied and an unsupervised learning algorithm is used to find structure in the input data such as one or more of first sensor data indicative of measured stimulation parameters for the different types of vestibular stimulation, second sensor data indicative of at least one measured physiological property of the user, and data indicating a type of motion in VR environment graphically output to the user as input data for training the machine-learning model to determine the respective stimulation effectiveness for each of the different types of vestibular stimulation. Analogously, input data such as stimulation effectivenesses for different types of vestibular stimulation may be used for training the machine-learning model to determine the respective weight for the different types of vestibular stimulation.

[0061] Reinforcement learning is a third group of machine-learning algorithms. In other words, reinforcement learning may be used to train the machine-learning model. In reinforcement learning, one or more software actors (called "software agents") are trained to take actions in an environment. Based on the taken actions, a reward is calculated. Reinforcement learning is based on training the one or more software agents to choose the actions such that the cumulative reward is increased, leading to software agents that become better at the task they are given (as evidenced by increasing rewards).

[0062] Furthermore, additional techniques may be applied to some of the machine-learning algorithms. For example, feature learning may be used. In other words, the machine-learning model may at least partially be trained using feature learning, and / or the machine-learning algorithm may comprise a feature learning component. Feature learning algorithms, which may be called representation learning algorithms, may preserve the information in their input but also transform it in a way that makes it useful, often as a pre-processing step before performing classification or predictions. Feature learning may be based on principal components analysis or cluster analysis, for example.

[0063] For example, the machine-learning model may be an Artificial Neural Network (ANN). ANNs are systems that are inspired by biological neural networks, such as can be found in a retina or a brain. ANNs comprise a plurality of interconnected nodes and a plurality of connections, so-called edges, between the nodes. There are usually three types of nodes, input nodes that are receiving input values (e.g., the first sensor data, the second sensor data or determined stimulation effectivenesses), hidden nodes that are (only) connected to other nodes, and output nodes that provide output values (e.g., the determined stimulation effectivenesses or the determined weights). Each node may represent an artificial neuron. Each edge may transmit information from one node to another. The output of a node may be defined as a (non-linear) function of its inputs (e.g. of the sum of its inputs). The inputs of a node may be used in the function based on a "weight" of the edge or of the node that provides the input. The weight of nodes and / or of edges may be adjusted in the learning process. In other words, the training of an ANN may comprise adjusting the weights of the nodes and / or edges of the ANN, i.e., to achieve a desired output for a given input. It is to be noted that the weight of nodes and / or of edges of the ANN are different from the aforementioned weights for the different types of vestibular stimulation.

[0064] Alternatively, the machine-learning model may be a support vector machine, a random forest model or a gradient boosting model. Support vector machines (i.e. support vector networks) are supervised learning models with associated learning algorithms that may be used to analyze data (e.g. in classification or regression analysis). Support vector machines may be trained by providing an input with a plurality of training input values (e.g., the first sensor data or stimulation effectivenesses) that belong to one of two categories (e.g., different measured stimulation parameters for the first sensor data or different types of vestibular stimulation for the stimulation effectivenesses). The support vector machine may be trained to assign a new input value to one of the two categories. Alternatively, the machine-learning model may be a Bayesian network, which is a probabilistic directed acyclic graphical model. A Bayesian network may represent a set of random variables and their conditional dependencies using a directed acyclic graph. Alternatively, the machine-learning model may be based on a genetic algorithm, which is a search algorithm and heuristic technique that mimics the process of natural selection.

[0065] In some examples, the machine-learning model may be a combination of the above examples.

[0066] The machine-learning model may further be trained on user feedback. For example, the control circuitry 130 may be configured to receive feedback data 106 indicating a user feedback on the effectiveness of the stimulation of the vestibular system. Further, the control circuitry 130 may be configured to further train the trained machine-learning model based on the user feedback. For example, the results of a questionnaire such as the Simulator Sickness Questionnaire (SSQ) may be used as user feedback. Accordingly, the machine-learning model may be personalized based on the actual experiences and feelings of the user. The further training of the machine-learning model based on the user feedback may be performed based on one of the machine-learning approaches described above. For example, the control circuitry 130 may be configured to change parameters (e.g., weights between nodes and / or edges of the machine-learning model) based on or similarly to stochastic gradient descent.

[0067] Another exemplary apparatus 200 for stimulating the vestibular system of a user is illustrated in Fig- 2. The apparatus 200 supports electrical vestibular stimulation (e.g., GVS) and vibrational vestibular stimulation. The apparatus uses a specific configuration of the stimulators 110 and 120 compared to the apparatus 100,. Apart from that, the apparatus 200 is identical to the apparatus 100 described above.

[0068] The first stimulator 110 comprises one or more bone conduction transducers. A respective electrically conductive layer is formed on at least one of the one or more bone conduction transducers for contacting the user’s head 150 (e.g., the scalp of the user’s head 150). The one or more electrically conductive layers serve as electrodes for applying electrical current or power via the first stimulator 110 for the electrical vestibular stimulation. In the example of Fig. 2, one bone conduction transducer 210 is illustrated for reasons of simplicity. Accordingly, the electrically conductive layer 215 is formed such on the bone conduction transducer 210 that the electrically conductive layer 215 contacts the user’s head 150. The second stimulator 120 is formed analogously. That is, the first stimulator 120 comprises one or more bone conduction transducers. A respective electrically conductive layer is formed on at least one of the one or more bone conduction transducers for contacting the user’s head 150 (e.g., the scalp of the user’s head 150). The one or more electrically conductive layers serve as electrodes for applying electrical current or power via the second stimulator 120 for the electrical vestibular stimulation. In the example of Fig. 2, one bone conduction transducer 220 is illustrated for reasons of simplicity. Accordingly, the electrically conductive layer 225 is formed such on the bone conduction transducer 220 that the electrically conductive layer 225 contacts the user’s head 150.

[0069] The control circuitry 130 determines the weights (i.e., the relative strengths) for the electrical vestibular stimulation and the vibrational vestibular stimulation as described above for the apparatus 100. The control circuitry 130 is configured to control, based on the determined weights, a respective electrical potential applied to the respective electrically conductive layer of the first stimulator 110 for electrical stimulation of the vestibular system. Accordingly, a defined electrical power or current 111 for electrical stimulation of the vestibular system is applied to the user’s head 150. The control circuitry 130 is further configured to control, based on the determined weights, emission of vibrations 112 by the one or more bone conduction transducers for vibrational stimulation of the vestibular system. The second stimulator 120 is controlled analogously by the control circuitry 130 to control the application of the electrical power or current 121 and the emission of the vibrations 122.

[0070] The apparatus 200 provides a hybrid system for vestibular stimulation that not only allows to compensate disadvantages, short-comings or (e.g., temporary) failures of one of the supported types of vestibular stimulation but allows to leverage the advantages of the supported types of vestibular stimulation.

[0071] As stated above, the apparatuses 100 and 200 may be used for VR applications. Fig. 3 illustrates a side-view of an exemplary head-mounted equipment 300 using one of the proposed apparatuses 100 and 200 for stimulating the vestibular system of a user.

[0072] The head-mounted equipment 300 comprises a display (e.g., a stereoscopic display) 310 configured to graphically output a VR environment (scene). The head-mounted equipment 300 comprises a fastening means (structure) 340 such as one or more straps or headbands for mounting the display 310 and other elements of the head-mounted equipment 300 to the user’s head.

[0073] The VR environment (scene) is an environment (scene) in a virtual world, which is simulated to give the user an immersive feel of the virtual world. For example, the virtual reality environment may be an environment in a video game or a metaverse. The user is visually perceiving the VR environment via the display 310. Depending on the user’s viewing direction and the user’s viewing angle, the user visually perceives a certain part of the VR environment. The viewing direction of the user is the sight direction of the user along which the user is viewing to perceive the VR environment. The viewing angle of the user is the angular range of the VR environment which the user is viewing (that is visible to the user) to perceive the VR environment.

[0074] The VR environment may be generated internally by the head-mounted equipment 300 or be received from an external device such as a server, a computer or a game console. For example, processing circuitry 320 of the head-mounted equipment 300 may be configured to generate the VR environment. In other examples, the head-mounted equipment 300 may comprise a transmitter or transceiver (not illustrated in Fig. 3) configured to receive (e.g., wirelessly or wired) data related the VR environment and the display 310 may be configured to graphically output the VR environment based on the received data.

[0075] The user’s viewing direction and / or viewing angle in the VR environment may be adjusted in various ways. For example, an operator of the VR environment or a VR engine may change the viewing direction and / or viewing angle (e.g., according to a script). In other examples, the viewing direction and / or viewing angle may be changed based on one or more inputs at one or more controllers operable / operated by the user via his / her hands. Additionally or alternatively, the head-mounted equipment 300 may comprise one or more sensors (e.g., one or more accelerometers or one or more gyroscopes; not illustrated in Fig. 3) configured to measure the motion of the user’s head and the viewing direction and / or viewing angle is changed according to the tracked motion of the user’s head.

[0076] Irrespective of how the user’s viewing direction and / or viewing angle in the VR environment is adjusted, there may be a discrepancy between the motion perceived by the user’s eyes from the graphical output of the VR environment and the motion perceived by the user’s vestibular system. For example, if the user’s viewing direction and / or viewing angle in the VR environment is changed based on the tracked motion of the user’s head, the change in the viewing direction and / or viewing angle perceived by the user’s eyes from the graphical output of the VR environment may be more or less than the motion perceived by the user’s vestibular system due to the actual motion of the user’s head in the real world. Similarly, if an avatar of the user moves in the VR environment (e.g., because the avatar is walk- ing / running in the VR environment or because the avatar is located in a moving object such as a vehicle / car or a rollercoaster in the VR environment), the change in position and / or orientation perceived by the user’s eyes from the graphical output of the VR environment may be more or less than the motion perceived by the user’s vestibular system due to the actual motion of the user’s head in the real world. This may cause VIMS or at least reduce the feeling of immersion.

[0077] The apparatus 100 or 200 allows to align the motion perceived by the user’s vestibular system with the motion perceived by the user’s eyes from the graphical output of the VR environment by stimulating the user’s vestibular system via one or more of the supported types of vestibular system. The stimulation of the user’s vestibular system by the apparatus 100 or 200 is performed in accordance with the motion in the VR environment graphically output to the user. In other words, the control circuitry 130 of the apparatus 100 or 200 is configured to control the stimulation of the vestibular system by the first stimulator and the second stimulator based on motion in the VR environment graphically output to the user. For example, the motion in the VR environment may be a change of the viewing direction in the VR environment, a change of the viewing angle in the VR environment, motion (movement) of the user’s avatar of the user in the VR environment, or any combination thereof.

[0078] The control circuitry 130 may, e.g., be configured to control the stimulation of the vestibular system by the first stimulator and the second stimulator such that the vestibular system of the user perceives a target directional motion (movement) in alignment (accordance) with the motion in the VR environment graphically output to the user. The target directional motion is motion in a specific direction that the vestibular system of the user is (supposed) to perceive. In other words, the target directional motion is a change in position and / or orientation along a particular path or trajectory that the vestibular system of the user is (supposed) to perceive. For example, target directional motions may be a rotation of the user’s head (e.g., a rotation of the whole body of the user, a motion of the user’s head from left to right, from right to left, from bottom to top, head from top to bottom, a tilt of the user’s head to the left, a tilt of the user’s head to the right, or any combination thereof), a linear movement of the user’s head without rotation of the head (as the whole body of the user moves; e.g., sideways, frontwards or backwards), or any combination thereof. The rotation of the user’s head may be along any (suitable) axis of rotation. That is, the control circuitry 130 is configured to control the first stimulator 110 and the second stimulator 120 such that they induce perception of the target directional motion in the vestibular system of the user. For example, the control circuitry 130 may determine parameters and weights for the respective types of vestibular stimulation supported by the first stimulator 110 and the second stimulator 120 based on the target directional motion and control the first stimulator 110 and the second stimulator 120 to stimulate the user’s vestibular system according to the determined parameters.

[0079] The weights and, hence, the relative strength of the different types of vestibular stimulation of the first stimulator 110 and the second stimulator 120 are adjusted as described above. Additionally, the information about the motion in the VR environment graphically output to the user is used to adjust the parameters of the different types of vestibular stimulation by the first stimulator 110 and the second stimulator 120 to induce perception of the target directional motion in the vestibular system of the user.

[0080] For example, the processing circuitry 320 or the control circuitry 130 may be configured to determine the target directional motion based on the motion in the VR environment graphically output the user. For example, the processing circuitry 320 or the control circuitry 130 may be configured to determine the target directional motion based on the change of the viewing direction in the VR environment, the change of the viewing angle in the VR environment, the motion of the user’s avatar of the user in the VR environment, or any combination thereof. The processing circuitry 320 or the control circuitry 130 may compare the motion in the VR environment graphically output the user to an actual motion of the user’s head to determine the target directional motion (e.g., based on a discrepancy or non-alignment between the actual motion of the user’s head and the motion in the VR environment). For example, the processing circuitry 320 or the control circuitry 130 may compare a change of the viewing direction in the VR environment, a change of the viewing angle in the VR environment, motion of the user’s avatar of the user in the VR environment, or any combination thereof to actual motion of the user’s head to determine the target directional motion (e.g., based on a discrepancy or non-alignment between the actual motion of the user’s head and the one or more of the change of the viewing direction in the VR environment, the change of the viewing angle in the VR environment and the motion of the user’s avatar of the user in the VR environment). The processing circuitry 320 or the control circuitry 130 may, e.g., map the motion in the VR environment graphically output the user to a corresponding target directional motion. For example, the processing circuitry 320 or the control circuitry 130 may refer to a Look-Up Table (LUT) for mapping the motion in the VR environment graphically output the user to a corresponding target directional motion. The LUT comprises a plurality of entries denoting a corresponding target directional motion for a given motion in the VR environment. The LUT may be previously learnt. The LUT may be adjusted over time in closed-loop with optimization techniques or machine-learning techniques. However, it is to be noted that the present disclosure is not limited to mapping based on LUTs. In other examples, the processing circuitry 320 or the control circuitry 130 may, e.g., analyze the VR environment graphically output the user and determine the motion in the VR environment based on the optical flow of the VR environment graphically output to the user. This may be done on the spot, i.e., online (live, at runtime) during the graphical output of the VR environment to the user.

[0081] In case the target directional motion is determined by the processing circuitry 320, the target directional motion may be encoded to data output by the processing circuitry 320 and provided to the control circuitry 130. However, it is to be noted that the present disclosure is not limited thereto. In other examples, data indicating the target directional motion may be received from an external device such as a server, a computer or a game console. For example, the external device may provide the data indicating the target directional motion together with the data related the VR environment. The external device may generate the data indicating the target directional motion analogously to what is described above for the processing circuitry 320 and the control circuitry 130.

[0082] The stimulators of the apparatus 100 or 200 may, e.g., be integrated into headphones 330 of the head-mounted equipment 300. The headphones 330 are configured to output sounds related to the VR environment and may be integrated into the same device as the other headmounted equipment. However, in some examples, the headphones 330 may be provided separately from other components of the head-mounted equipment 300. The headphones 330 comprise two earpads for the two ears of the user. One of the earpads is exemplarily illustrated in Fig. 3. In the example of Fig. 3, the earpad 335 of the headphones comprises the first stimulator 110. Analogously, the other earpad of the headphones 330, which is not depicted in Fig. 3, comprises the second stimulator. In other examples, the stimulators of the apparatus 100 or 200 may be integrated into another part of the headphones 330 of the headmounted equipment 300 other than the respective earpads. For example, the stimulators of the apparatus 100 or 200 may be integrated into a support structure (e.g. a headband securing the earpads to the user’s head) of the headphones 330.

[0083] However, it is to be noted that the stimulators of the apparatus 100 or 200 need not be integrated into the headphones 330 of the head-mounted equipment 300. In some examples, the first stimulator and the second stimulator may be provided separate from the headphones 330.

[0084] In some examples, the head-mounted equipment 300 does not comprise the headphones 330.

[0085] The processing circuitry 320 may be a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which or all of which may be shared, a DSP hardware, an ASIC, a SoC, a neuromorphic processor or a FPGA. The processing circuitry 320 may optionally be coupled to, e.g., memory such as ROM for storing software, RAM and / or non-volatile memory. For example, the head-mounted equipment 300 may comprise memory configured to store instructions, which when executed by the processing circuitry 320, cause the processing circuitry 320 to perform the steps and methods described herein.

[0086] The control circuitry 130 of the apparatus 100 or 200 and the processing circuitry 320 may be separate elements as illustrated in Fig. 3. In alternative examples, the control circuitry 130 and the processing circuitry 320 may be the same circuitry. In other words, the functionalities of the control circuitry 130 and the processing circuitry 320 may be integrated into a single circuitry.

[0087] The head-mounted equipment 300 may optionally comprise further elements such as one or more sensors (e.g., for eye-tracking) or a haptic feedback device.

[0088] For further highlighting the stimulation of a user’s vestibular system described above, Fig. 4 illustrates a flowchart of a method 400 for stimulating the vestibular system of a user. The method 400 comprises determining 402 a respective stimulation effectiveness for each of multiple different types of vestibular stimulation supported by each of a first stimulator and a second stimulator for stimulating the vestibular system. Further, the method 400 comprises determining 404 a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses. In addition, the method 400 comprises controlling 406 the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights.

[0089] Analogously to what is described above, the method 400 provides improved stimulation of the user’s vestibular system. In particular, the method 400 allows to leverage the advantages of the different supported types of vestibular stimulation.

[0090] More details and aspects of the method 400 are explained in connection with the proposed technique or one or more examples described above (e.g., Fig. 1 to Fig. 3). The method 400 may comprise one or more additional optional features corresponding to one or more aspects of the proposed technique or one or more examples described above.

[0091] The proposed technique provides multi-modal (e.g., hybrid) vestibular stimulation.

[0092] The following examples pertain to further embodiments:

[0093] (1) An apparatus for stimulating the vestibular system of a user, comprising: a first stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation; a second stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation; and control circuitry configured to: determine a respective stimulation effectiveness for each of the different types of vestibular stimulation; determine a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses; and control the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights. (2) The apparatus of (1), wherein the different types of vestibular stimulation comprise two or more of electrical stimulation, vibrational stimulation, magnetic stimulation, thermal stimulation and acoustic stimulation.

[0094] (3) The apparatus of (1) or (2), wherein the determined weights for the different types of vestibular stimulation determine a relative strength of the respective type of vestibular stimulation in the total vestibular stimulation by the first stimulator and the second stimulator.

[0095] (4) The apparatus of any one of (1) to (3), wherein the control circuitry is further configured to: receive first sensor data, the first sensor data being indicative of one or more measured stimulation parameters for the different types of vestibular stimulation; and determine the respective stimulation effectiveness for the different types of vestibular stimulation based on the first sensor data.

[0096] (5) The apparatus of (4), further comprising one or more first sensors configured to: measure the one or more stimulation parameters for the different types of vestibular stimulation; and generate the first sensor data based on the one or more measured stimulation parameters.

[0097] (6) The apparatus of any one of (1) to (5), wherein the control circuitry is further configured to: receive second sensor data, the second sensor data being indicative of at least one measured physiological property of the user; and determine the respective stimulation effectiveness for the different types of vestibular stimulation based on the second sensor data.

[0098] (7) The apparatus of (6), further comprising one or more second sensors configured to: measure the at least one physiological property of the user; and generate the second sensor data based on the measured at least one physiological property of the user. (8) The apparatus of any one of (1) to (7), wherein the control circuitry is further configured to determine the respective stimulation effectiveness for the different types of vestibular stimulation based on a type of motion in a virtual reality environment graphically output to the user.

[0099] (9) The apparatus of (8), wherein: the control circuitry is further configured to receive data indicating the type of motion in the virtual reality environment graphically output to the user; or the control circuitry is further configured to determine the type of motion in the virtual reality environment graphically output to the user based on data indicating the virtual reality environment graphically output to the user.

[0100] (10) The apparatus of any one of (1) to (9), wherein the control circuitry is configured to use a trained machine-learning model for at least one of determining the respective stimulation effectiveness for the different types of vestibular stimulation and determining the respective weight for the different types of vestibular stimulation.

[0101] (11) The apparatus of (10), wherein the control circuitry is further configured to: receive feedback data indicating a user feedback on the effectiveness of the stimulation of the vestibular system; and further train the trained machine-learning model based on the user feedback.

[0102] (12) The apparatus of any one of (1) to (11), wherein the first stimulator comprises one or more bone conduction transducers, wherein a respective electrically conductive layer is formed on at least one of the one or more bone conduction transducers for contacting the user’s head, and wherein the control circuitry is further configured to: control, based on the determined weights, a respective electrical potential applied to the respective electrically conductive layer for electrical stimulation of the vestibular system; and control, based on the determined weights, emission of vibrations by the one or more bone conduction transducers for vibrational stimulation of the vestibular system.

[0103] (13) The apparatus of any one of (1) to (12), wherein the first stimulator is configured to contact a first area of the user’s skull for stimulating the vestibular system, wherein the second stimulator is configured to contact a second area of the user’s skull for stimulating the vestibular system, and wherein the first area of the user’s skull and the second area of the user’ s skull are opposite to each other.

[0104] (14) The apparatus of (13), wherein the first area of the user’s skull is the mastoid part of one of the temporal bones of the user’s head, and wherein the second area of the user’s skull is the mastoid part of the other one of the temporal bones.

[0105] (15) A head-mounted equipment, comprising: a display configured to graphically output a virtual reality environment; and an apparatus for stimulating the vestibular system of a user according to any one of (1) to (14), wherein the control circuitry is further configured to control the stimulation of the vestibular system by the first stimulator and the second stimulator based on motion in the virtual reality environment graphically output to the user.

[0106] (16) The head-mounted equipment of (15), further comprising headphones configured to output sounds related to the virtual reality environment, wherein a first earpad of the headphones comprises the first stimulator and a second earpad of the headphones comprises the second stimulator.

[0107] (17) The head-mounted equipment of (15), further comprising headphones configured to output sounds related to the virtual reality environment, wherein the first stimulator and the second stimulator are provided separate from the headphones.

[0108] (18) A method for stimulating the vestibular system of a user, comprising: determining a respective stimulation effectiveness for each of multiple different types of vestibular stimulation supported by each of a first stimulator and a second stimulator for stimulating the vestibular system; determining a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses; and controlling the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights. (19) A non-transitory machine-readable medium having stored thereon a program having a program code for performing the method according to (18), when the program is executed on a processor or a programmable hardware.

[0109] (20) A program having a program code for performing the method according to (18), when the program is executed on a processor or a programmable hardware.

[0110] The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example.

[0111] Examples may further be or relate to a (computer) program including a program code to execute one or more of the above methods when the program is executed on a computer, processor or other programmable hardware component. Thus, steps, operations or processes of different ones of the methods described above may also be executed by programmed computers, processors or other programmable hardware components. Examples may also cover program storage devices, such as digital data storage media, which are machine-, processor- or computer-readable and encode and / or contain machine-executable, processor-executable or computer-executable programs and instructions. Program storage devices may include or be digital storage devices, magnetic storage media such as magnetic disks and magnetic tapes, hard disk drives, or optically readable digital data storage media, for example. Other examples may also include computers, processors, control units, (field) programmable logic arrays ((F)PLAs), (field) programmable gate arrays ((F)PGAs), graphics processor units (GPU), ASICs, integrated circuits (ICs) or SoC systems programmed to execute the steps of the methods described above.

[0112] It is further understood that the disclosure of several steps, processes, operations or functions disclosed in the description or claims shall not be construed to imply that these operations are necessarily dependent on the order described, unless explicitly stated in the individual case or necessary for technical reasons. Therefore, the previous description does not limit the execution of several steps or functions to a certain order. Furthermore, in further examples, a single step, function, process or operation may include and / or be broken up into several substeps, -functions, -processes or -operations. If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system. The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.

Claims

ClaimsWhat is claimed is:

1. An apparatus for stimulating the vestibular system of a user, comprising: a first stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation; a second stimulator capable of stimulating the vestibular system by multiple different types of vestibular stimulation; and control circuitry configured to: determine a respective stimulation effectiveness for each of the different types of vestibular stimulation; determine a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses; and control the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights.

2. The apparatus of claim 1, wherein the different types of vestibular stimulation comprise two or more of electrical stimulation, vibrational stimulation, magnetic stimulation, thermal stimulation and acoustic stimulation.

3. The apparatus of claim 1, wherein the determined weights for the different types of vestibular stimulation determine a relative strength of the respective type of vestibular stimulation in the total vestibular stimulation by the first stimulator and the second stimulator.

4. The apparatus of claim 1, wherein the control circuitry is further configured to: receive first sensor data, the first sensor data being indicative of one or more measured stimulation parameters for the different types of vestibular stimulation; anddetermine the respective stimulation effectiveness for the different types of vestibular stimulation based on the first sensor data.

5. The apparatus of claim 4, further comprising one or more first sensors configured to: measure the one or more stimulation parameters for the different types of vestibular stimulation; and generate the first sensor data based on the one or more measured stimulation parameters.

6. The apparatus of claim 1, wherein the control circuitry is further configured to: receive second sensor data, the second sensor data being indicative of at least one measured physiological property of the user; and determine the respective stimulation effectiveness for the different types of vestibular stimulation based on the second sensor data.

7. The apparatus of claim 6, further comprising one or more second sensors configured to: measure the at least one physiological property of the user; and generate the second sensor data based on the measured at least one physiological property of the user.

8. The apparatus of claim 1, wherein the control circuitry is further configured to determine the respective stimulation effectiveness for the different types of vestibular stimulation based on a type of motion in a virtual reality environment graphically output to the user.

9. The apparatus of claim 8, wherein: the control circuitry is further configured to receive data indicating the type of motion in the virtual reality environment graphically output to the user; or the control circuitry is further configured to determine the type of motion in the virtual reality environment graphically output to the user based on data indicating the virtual reality environment graphically output to the user.

10. The apparatus of claim 1, wherein the control circuitry is configured to use a trained machine-learning model for at least one of determining the respective stimulation effectiveness for the different types of vestibular stimulation and determining the respective weight for the different types of vestibular stimulation.

11. The apparatus of claim 10, wherein the control circuitry is further configured to: receive feedback data indicating a user feedback on the effectiveness of the stimulation of the vestibular system; and further train the trained machine-learning model based on the user feedback.

12. The apparatus of claim 1, wherein the first stimulator comprises one or more bone conduction transducers, wherein a respective electrically conductive layer is formed on at least one of the one or more bone conduction transducers for contacting the user’s head, and wherein the control circuitry is further configured to: control, based on the determined weights, a respective electrical potential applied to the respective electrically conductive layer for electrical stimulation of the vestibular system; and control, based on the determined weights, emission of vibrations by the one or more bone conduction transducers for vibrational stimulation of the vestibular system.

13. The apparatus of claim 1, wherein the first stimulator is configured to contact a first area of the user’s skull for stimulating the vestibular system, wherein the second stimulator is configured to contact a second area of the user’s skull for stimulating the vestibular system, and wherein the first area of the user’ s skull and the second area of the user’ s skull are opposite to each other.

14. The apparatus of claim 13, wherein the first area of the user’s skull is the mastoid part of one of the temporal bones of the user’s head, and wherein the second area of the user’s skull is the mastoid part of the other one of the temporal bones.

15. A head-mounted equipment, comprising: a display configured to graphically output a virtual reality environment; and an apparatus for stimulating the vestibular system of a user according to claim 1,wherein the control circuitry is further configured to control the stimulation of the vestibular system by the first stimulator and the second stimulator based on motion in the virtual reality environment graphically output to the user.

16. The head-mounted equipment of claim 15, further comprising headphones configured to output sounds related to the virtual reality environment, wherein a first earpad of the headphones comprises the first stimulator and a second earpad of the headphones comprises the second stimulator.

17. The head-mounted equipment of claim 15, further comprising headphones configured to output sounds related to the virtual reality environment, wherein the first stimulator and the second stimulator are provided separate from the headphones.

18. A method for stimulating the vestibular system of a user, comprising: determining a respective stimulation effectiveness for each of multiple different types of vestibular stimulation supported by each of a first stimulator and a second stimulator for stimulating the vestibular system; determining a respective weight for the different types of vestibular stimulation based on the determined stimulation effectivenesses; and controlling the first stimulator and the second stimulator to stimulate the vestibular system by one or more of the different types of vestibular stimulation according to the determined weights.

19. A non-transitory machine-readable medium having stored thereon a program having a program code for performing the method according to claim 18, when the program is executed on a processor or a programmable hardware.

20. A program having a program code for performing the method according to claim 18, when the program is executed on a processor or a programmable hardware.

Images