Method for operating a listening system

The hearing system with motion sensors and acoustic feedback addresses the risk of falls in hearing-impaired individuals by enhancing movement perception and rehabilitation, improving safety and reducing healthcare costs through multisensory integration.

EP4080904B1Active Publication Date: 2024-11-20SIVANTOS PTE LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
EP2022164425
Authority / Receiving Office
EP · EP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-04-22
Filing Date
2022-03-25
Publication Date
2024-11-20
Estimated Expiration
2042-03-25

AI Technical Summary

Technical Problem

Hearing-impaired individuals face increased risks of falls and related injuries due to reduced ability to perceive self-motion, which complicates rehabilitation and increases healthcare costs, especially when combined with age-related or illness-induced cognitive decline.

Method used

A hearing system with integrated motion sensors and acoustic feedback, where the hearing aid detects and analyzes movement patterns, comparing them to target patterns to generate audio signals that assist users in aligning their movements, thereby providing real-time support for improved motor skills and safety.

Benefits of technology

The system enhances the user's ability to perceive and adjust their movements, reducing the risk of falls, improving physical condition, and facilitating independent living by providing multisensory integration of somatosensory and acoustic feedback, which is particularly effective for hearing-impaired individuals.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure IMGF0001
    Figure IMGF0001
  • Figure IMGF0002
    Figure IMGF0002
  • Figure IMGF0003
    Figure IMGF0003
Patent Text Reader

Abstract

The invention relates to a method (40) for operating a hearing system (2) comprising a hearing aid (4) with at least one input transducer (12) and with an output transducer (20) as well as with a motion sensor (24), wherein an actual movement of the hearing system user is detected as motion sensor data (26) of the motion sensor (24) and / or as motion noise data (44) of the input transducer (12), wherein an actual movement pattern (48) for the actual movement is determined on the basis of the motion sensor data (26) and / or the motion noise data (44), wherein the actual movement pattern (48) is compared with a target movement pattern (54), wherein, depending on the comparison, an audio signal (66) with a perceptible signal level is generated by means of the output transducer (20), which supports the hearing system user in executing the actual movement, and wherein the motion noises detected by the input transducer (12) are used as the audio signal (66).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The invention relates to a method for operating a hearing system comprising a hearing aid with at least one microphone, a receiver, and a motion sensor. The invention further relates to a hearing system, in particular a hearing aid device, for implementing the method.

[0002] Hearing aids are portable hearing aids designed to provide hearing assistance to people with hearing loss or hard of hearing. To meet a wide range of individual needs, different types of hearing aids are available, such as behind-the-ear (BTE) hearing aids, receiver-in-the-canal (RIC) hearing aids, and in-the-ear (ITE) hearing aids, such as in-the-ear (concha) hearing aids or in-the-canal (ITE) hearing aids (CIC: Completely-In-Channel), or invisible-in-the-Channel (IIC). The hearing aids listed here are worn on the outer ear or in the ear canal of the hearing aid user. Bone conduction, implantable, and vibrotactile hearing aids are also available on the market. These devices stimulate the impaired hearing either mechanically or electrically.

[0003] Such hearing aids generally have an input transducer, an amplifier, and an output transducer as their essential components. The input transducer is usually an acoustoelectrical transducer, such as a microphone, and / or an electromagnetic receiver, such as an induction coil or a (radio frequency, RF) antenna. The output transducer is usually implemented as an electroacoustic transducer, such as a miniature loudspeaker (earpiece), or as an electromechanical transducer, such as a bone conduction receiver. The amplifier is usually integrated into a signal processing unit. Power is typically supplied by a battery or rechargeable accumulator.

[0004] In a so-called binaural hearing aid device, two hearing aids of this type are worn by one user, with a communication link between the hearing aids. During operation, data, possibly large amounts of data, are exchanged wirelessly between the hearing aid in the right and left ear. The exchanged data and information enable particularly effective adaptation of the hearing aids to the respective acoustic environment. In particular, this provides the user with a particularly authentic surround sound and improves speech understanding, even in noisy environments.

[0005] As people age, they are at increasing risk of developing diseases or comorbidities, particularly those affecting the autonomic nervous system (ANS), such as hearing loss, dementia, or Parkinson's disease. In particular, several such diseases often occur, so that people with Parkinson's disease, for example, regularly also experience a hearing impairment or hearing loss.

[0006] Many of these diseases and disorders also lead to a higher risk of falls and, ultimately, to falls in the affected individual, which can result in serious injuries. Psychological consequences or effects of falls, such as post-fall anxiety, must also be considered. A fall can thus lead to a reduced quality of life and increased healthcare costs.

[0007] Furthermore, a hearing impairment or hearing loss is associated with a reduced ability to perceive one's own movement. This can lead, especially in combination with other conditions, to a diminished sense of "safety," and thus to a vicious cycle of decreasing physical activity, further decline in self-confidence, and an increased risk of falls. Furthermore, rehabilitation measures after falls and associated injuries are generally time-consuming, costly, and often difficult to implement if the affected individuals also suffer from hearing loss and age- or illness-related cognitive decline.

[0008] To reduce the risk of falls and / or as a rehabilitation measure, sonification or sonication can be used to support and improve self-movement. Sonification refers specifically to the use of non-verbal audio signals to convey information or perceive data. Auditory perception offers advantages in temporal, spatial, amplitude, and frequency resolution, opening up possibilities as an alternative or complement to visualization techniques.

[0009] For example, so-called rhythmic auditory cueing is possible, which is defined as the application of repeating isosynchronous beats with the goal of synchronizing the motor execution of a movement with this rhythm. This is usually implemented in training sessions, for example, on a treadmill. Additionally or alternatively, real-time "sonification" of movement parameters, which translates into acoustic feedback, is also possible in special training sessions, for example, to make information about the length and force of a movement audible.

[0010] Document WO 2020 / 206155 A1 discloses a monitoring system comprising two or more sensors capable of detecting a characteristic of a subject of the system and generating data representative of the characteristic. The system disclosed in WO 2020 / 206155 A1 may further comprise a controller operatively connected to the first and second sensors and capable of determining statistics about an overall condition of the subject over a monitoring period based on the confirmation of one or more condition subconditions. The system may be implemented in a hearing aid.

[0011] The invention is based on the object of specifying a particularly suitable method for operating a hearing system. In particular, to avoid or reduce the risk of falls, simple and reliable support for self-movement should be enabled, which is preferably applicable in everyday life. The invention is further based on the object of specifying a particularly suitable hearing system for carrying out the method. With regard to the method, the object is achieved according to the invention with the features of claim 1 and with regard to the hearing system with the features of claim 8. Advantageous embodiments and further developments are the subject of the dependent claims. The advantages and embodiments cited with regard to the method can also be transferred mutatis mutandis to the hearing system and vice versa.

[0012] If method steps are described below, advantageous embodiments for the hearing system result in particular from the fact that it is designed to carry out one or more of these method steps.

[0013] The method according to the invention is intended for operating a hearing system, in particular a hearing aid device, and is suitable and designed for this purpose. The hearing system comprises a hearing aid.

[0014] The hearing aid is particularly designed to provide hearing aid for a hearing-impaired user (hearing system user). The hearing aid is designed to receive sound signals from the environment and output them to the user of the hearing aid. For this purpose, the hearing aid has at least one acousto-electrical input transducer, in particular a microphone, and at least one electro-acoustic output transducer, for example, an earpiece. During operation of the hearing aid, the input transducer receives sound signals (noises, tones, speech, etc.) from the environment and converts them into an electrical input signal (acoustic data). An electrical output signal is generated from the electrical input signal by modifying the input signal in a signal processing system. Signal processing, for example, is a part of the hearing aid.The signal processing unit comprises, for example, an (output) amplifier for (selectively) amplifying the input signal or for adjusting the signal level of the output signal. The output transducer generates an acoustic signal from the output signal. The input transducer and the output transducer, as well as, if applicable, the signal processing unit, are housed in a housing of the hearing aid. The housing is designed such that it can be worn by the user on the head and near the ear, e.g., in the ear, on the ear, or behind the ear. The hearing aid is preferably designed as a BTE hearing aid, ITO hearing aid, or RIC hearing aid.

[0015] The hearing aid can be designed binaurally and, for this purpose, comprise two individual devices, each of which has at least one input transducer and at least one output transducer, and is thus configured to receive sound signals from the environment and output them to a hearing aid user. The method described below can be performed in one of the individual devices, in both individual devices, or jointly by both individual devices.

[0016] For example, a wireless interface is provided for data exchange between the two individual devices. The wireless interface can be a Bluetooth interface, a WLAN interface, or an MI interface (MI-Link, MI: Magnetic Induction). The Bluetooth interface can be a regular or low-energy Bluetooth interface.

[0017] As an alternative to a binaural hearing aid, a monaural hearing aid with a single device is also suitable. The explanations regarding a monaural hearing aid apply analogously to a binaural hearing aid, and vice versa.

[0018] The hearing aid further comprises a motion sensor for detecting movements of the hearing system user. The motion sensor is intended and configured to detect three-dimensional (body) movements or movement events, in particular translational and / or rotational movements. The motion sensor is designed, for example, as an accelerometer and / or a gyroscope, i.e., as a gyroscopic (position) sensor. Alternatively, the motion sensor can also be a pulse, blood pressure, or light sensor. A combination of accelerometer and / or gyroscope and / or pulse sensor and / or blood pressure sensor and / or light sensor is also possible. The motion sensor is preferably integrated into the hearing aid, in particular into its housing. In a binaural hearing aid, a motion sensor is preferably integrated into each individual device.

[0019] The conjunction "and / or" is to be understood here and in the following in such a way that the features linked by this conjunction can be formed both together and as alternatives to one another.

[0020] In this application, the hearing aid is worn on the head and near the ear of the hearing aid user, so that the motion sensor can detect, in particular, head movements of the hearing aid user. In particular, walking or gait movements, i.e., body movements resulting from walking or running, can also be detected, at least indirectly, via the hearing aid's motion sensor.

[0021] A "movement" is understood here and in the following to mean a particularly translational and / or rotational body movement of the hearing system user, i.e. a movement action or a (temporal) sequence of movement actions (movement sequence).

[0022] The term "movement pattern" here and below refers in particular to a characterization of the respective (body) movement based on characteristic (movement) variables (movement parameters), which are determined, for example, based on recorded data. For example, the movement pattern of a walking movement can be characterized by a step frequency, a step length, and step synchrony. A "movement pattern" is understood in particular to mean a (temporal) signal sequence characterizing the respective (body) movement, or a signal pattern or signal curve of the recorded data or the characteristic variables derived from it.

[0023] According to the method according to the invention, during operation of the hearing system, an actual movement of the hearing system user is recorded as motion sensor data of the motion sensor and as movement noise data of the input transducer.

[0024] An "actual movement" of the hearing system user is understood to mean, in particular, the hearing system user's own movement (body movement).

[0025] "Motion sensor data" refers in particular to direction-dependent accelerations and / or rotations that are recorded by the motion sensor.

[0026] "Motion sound data" refers in particular to acoustic signals in the signal captured by the input transducer (acoustic signal, acoustic data) that can be directly or indirectly attributed to the body or self-movement of the hearing system user (motion sounds). The analysis or capture of the motion sounds or motion sound data from the input transducer data / signals (acoustic data) can be carried out, for example, using a motion sound classifier. Such a classifier can be, for example, an artificial intelligence or a neural network, in particular a deep neural network (DNN), which is trained with various motion sounds.

[0027] The motion sensor data and / or motion sound data thus provide a measure of the body movements performed by the hearing system user. Based on the motion sensor data and / or motion sound data, an actual movement pattern is determined for the actual movement. In other words, the actual movement, or the motion sensor data and / or motion sound data, is analyzed during the execution of the actual movement, thus characterizing the actual movement based on the recorded signal curves / signal patterns.

[0028] For example, a target movement pattern is determined for the hearing system user's movement detected by the hearing system. For example, a desired step frequency and / or step length, or similar, is determined for a walking movement at a certain walking pace. If the hearing system user suffers from an ANS disorder, the target movement pattern can also be an externally specified or stored movement pattern. Alternatively, the target movement pattern can also be a stored movement pattern of a previous movement of the hearing system user themselves, which serves, for example, as an individual baseline.

[0029] The actual movement pattern is compared with the desired movement pattern, and based on the comparison, an audio signal with a perceptible signal level is generated by the output transducer, which supports the hearing system user in executing the movement. In other words, deviations between the variables or movement parameters characterizing the movement are determined based on the actual and desired movement patterns, and based on this, a perceptible, acoustic audio signal is output to the hearing system user in order to adapt the hearing system user's (actual) movement to a desired (desired) movement. The signal level of the audio signal refers in particular to the volume of the acoustic audio signal. For example, the movement patterns or deviations from the movement patterns act as triggers for the audio signal.The audio signal acts as acoustic movement feedback, which helps the hearing system user perceive their own movement and adapt it to a desired movement. This provides a particularly suitable method for operating a hearing system. In particular, the hearing system is used to monitor and control the hearing system user's own movement.

[0030] The motion sensor in the hearing aid, which is worn to compensate for hearing loss, is thus used according to the invention to monitor and support the movement patterns of the hearing aid wearer. A core concept of the invention is therefore, in particular, to record the movements of the hearing aid user (e.g., gait, turning, etc.) using the hearing aid, as well as to analyze the recorded movements and compare them with a predetermined or desired movement, and to provide the hearing aid user with an acoustic signal for adjusting or verifying their movement. Based on the movement analysis, for example, the current walking (gait) or general movement is identified and compared with an ideal (target) movement pattern.

[0031] Providing audible or perceptible audio signals during a movement can help the hearing aid user to learn new movements or to optimize certain movements. This means performing a movement with an appropriate force or duration, which is indicated to the hearing aid user by acoustic aspects of the audio signal (volume, pitch, rhythm, etc.).

[0032] This procedure improves the hearing aid user's self-perceived movement and extends it to everyday use. In particular, it is possible, for example, to use the hearing aid to provide rhythmic auditory cues or movement sonification, thereby expanding the availability and intensity of motor rehabilitation without hindering the hearing aid user's ability to perceive their surroundings and remain responsive in daily life.

[0033] Acoustic feedback can be more precise for a hearing aid user than visual feedback, particularly, but not exclusively, for hearing aid users with hearing loss or hearing impairment. Thus, this design also offers advantages beyond applications for hearing-impaired users, particularly in the field of physical rehabilitation through applied sonification. Furthermore, sonification is known to stimulate subcortical learning and is therefore superior to verbal and visual instructions in general, and especially in cases of age- or disease-related cognitive impairment.

[0034] Furthermore, the method according to the invention eliminates the need for the hearing system user to carry an additional device in the event of an ANS disorder. In other words, there is no additional burden from using any other device in addition to the hearing aid. This can therefore also help convince people with only mild hearing impairments to use a hearing aid to improve their quality of life, prevent injuries, and / or monitor their own movement.

[0035] From a medical perspective, the method according to the invention contributes to safety, reducing the risk of falls, fewer injuries, and lower costs for the healthcare system. The hearing system user achieves improved physical fitness, which translates into optimal gait symmetry, safe walking speed, etc. This allows the hearing system user to, for example, retrain stride length and / or arm and leg synchronization, and even learn new complex movements that help them lead a high-quality, independent life.

[0036] From a neurological perspective, the method according to the invention offers advantages regarding the multisensory integration of somatosensory and acoustic feedback on movement patterns in general and gait patterns in particular. Multisensory integration is a basis for learning in the brain, and thus a basis for relearning safe walking and for compensating for the loss of self-perceived movement in hearing impairment due to the reduced acoustic feedback in hearing impairment.

[0037] More generally, the present invention makes it possible to offer additional benefits of hearing systems beyond hearing alone. Therefore, people may be willing to use hearing aids even if they only suffer from mild hearing impairment because they receive additional (health-related) benefits. By extending the functionality of the hearing system to provide acoustic feedback for a movement, learning or relearning movements is not limited to special training sessions but can also take place in everyday life. This increases the training frequency and duration compared to regular training sessions, resulting in better learning outcomes for the hearing system user.

[0038] For the method according to the invention, preferably three applications or use cases, or combinations thereof, are possible.

[0039] In a first application, the hearing system essentially functions like a pacemaker or metronome, i.e., as an extended feedback system that provides rhythmic acoustic feedback or sonification of kinematic movement pattern parameters. This enables the hearing system user to adjust or synchronize their current (suboptimal) movement pattern with the acoustically reproduced (ideal, optimal) movement pattern. This reduces, for example, the risk of falls and improves the hearing system user's execution of the desired movement.

[0040] In this application, the hearing system preferably continuously compares the current actual movement pattern with the desired movement pattern, with the acoustic audio signal being adjusted as optimally as possible in each situation. For example, it is conceivable that the audio signal is generated only depending on the situation. In other words, a specific situation is classified, for example, and the acoustic feedback is generated via the audio signal or not depending on this.

[0041] In a second application, the method according to the invention is used, for example, for the specific amplification of gait or movement sounds, e.g., footstep sounds when walking, cycling, swimming in water, or other complex movements. This supports the processes of multisensory integration in the brain by somatosensory feedback from the respective sensory cells in the feet and the acoustic feedback of the movement sounds (e.g., footstep sounds, arm sounds, or swimming sounds).

[0042] In a third application, the method according to the invention is used for training purposes to relearn or improve movements or motor skills (learning application). For example, the "shape" of the audio signal is changed using the hearing system to trigger changes in the body movement of the hearing system user. A change in body movement here is understood to mean, for example, a change in the temporal sequence of movement (timing), a (movement) force, or other details of the movement.

[0043] To start the process, a desired application is preferably specified. The determination or selection of the target movement pattern is suitably carried out depending on the respective application. In other words, the optimal movement pattern for the respective application is determined as the target movement pattern. For example, for a given gait, the desired walking speed, walking rhythm, and step length are determined. The generated audio signal can vary depending on the selected application.

[0044] The selection of the application or the specification of the target movement can be made, for example, during an adjustment or fine-tuning of the hearing aid. Preferably, the target movement or application is specified via a user interface of an additional device that is signal-linked to the hearing aid.

[0045] The additional device is preferably a mobile operating and display device, for example a mobile phone, in particular a mobile phone with a computer function, or a smartphone, or even a tablet computer. The additional device suitably has stored application software (operating software). For this purpose, the application software can be installed or is installed on the operating and display device, preferably as a so-called app or mobile app (mobile application, smartphone app).

[0046] This embodiment is based on the consideration that modern control and display devices, such as smartphones or tablet computers in particular, are widespread in today's society and are generally available and accessible to users at any time. In particular, the user of the hearing system is highly likely to have at least one such control and display device in their home.

[0047] Modern smartphones are still equipped with a variety of different near-field and far-field communication devices as standard, making it fundamentally easy to establish a communication or signal connection to the hearing aid. The application software is preferably also suitable and configured for adjusting the hearing aid's operating parameters, such as volume. This eliminates the need for the user to have an additional, separate control system to monitor and adjust the hearing aid. Instead, by (subsequently) downloading and / or installing the application software, they can use their existing smartphone to specify the target movement and evaluate the movement data. This advantageously reduces user costs.

[0048] The surfaces of smartphones or tablet computers, typically designed as touchscreens (displays), allow for particularly simple and intuitive operation of the application software of the resulting additional device. This makes it particularly cost-effective to retrofit a smartphone or tablet computer for self-motion monitoring.

[0049] The operating and display device comprises an internal controller which is formed at least in its core by a microcontroller with a processor and a data memory in which the functionality for carrying out the method is implemented in the form of the application software, so that the method or the monitoring of the self-movement - if necessary in interaction with the user - is carried out automatically when the application software is executed in the microcontroller.

[0050] The actual movement pattern is preferably determined using both the motion sensor data and the movement noise data. The movement noise data from the input transducer or microphone can be used, for example, to determine the quality of movement, such as periodicity. With regard to a walking movement, for example, different footstep noises from step to step can indicate a balance disorder. The deviation from a periodic (target) footstep noise can thus be used as a measure for adjusting the audio signal. During a swimming movement, for example, the duration of the individual swimming strokes, and thus the corresponding movement noise duration, can vary. The duration of the swimming stroke thus provides information that can be taken into account when generating the audio signal.

[0051] The audio signal generated according to the method has a perceptible signal level, meaning that the audio signal has a signal level that is perceptible to the hearing system user in the acoustic signal generated by the output transducer. The acoustically generated signal is, for example, an amplified signal from the input transducer for supplying a hearing-impaired hearing system user. The signal level of the audio signal can be adjusted by the hearing system user, for example, via a remote control or an additional device. It is conceivable, for example, that the signal level of the audio signal can be adjusted separately by the hearing system user.

[0052] The rhythmic audio signal, for example, is provided by the input transducer signal. In a binaural hearing aid, the audio signal can be generated binaurally, monaurally, or alternating monaurally.

[0053] In one embodiment, it is conceivable, for example, that a time frame or schedule for carrying out the method can be specified, i.e., that the method is started at a specific time and ended at a further specific time. For example, it is possible for a hearing system user to specify the intervals or frequency at which self-movement should be checked, monitored, or supported. It is also conceivable, for example, that a desired schedule be stored in the hearing system, for example, a time period per hour, day, week, etc.

[0054] In a particularly suitable embodiment, the audio signal is generated for a predetermined period of time when the deviation between the actual movement pattern and the desired movement pattern reaches or exceeds a predetermined threshold. This means that the audio signal is generated when the movement pattern indicates that there is a health or safety risk for the hearing system user.

[0055] For example, it is possible for the hearing aid user to manually adjust, influence, or change the volume of the audio signal, for example, via an additional device. It is also possible, for example, for the hearing aid user to manually start, pause, and end the procedure.

[0056] In a preferred development of the method, a probability for a future deviation of the actual movement pattern from the target movement pattern of the hearing system user is determined based on the recorded motion sensor data and / or motion noise data. To determine the probability, a method is used, for example, as described in the applicant's international application dated December 16, 2020, with the file number PCT / EP2020 / 086518, which was still unpublished at the time of filing. The disclosure content of this application, in particular its claims (with associated explanations), are hereby expressly incorporated into the present application. Special reference is made to claims 1 and 2, with the associated explanations, specifically on pages 8 to 14 of the description.

[0057] The deviation of the actual movement pattern from the target movement pattern can, for example, be realized in the form of a fall or tumble event, so that, in particular, a fall probability is determined as in PCT / EP2020 / 086518. Additionally or alternatively, however, other deviations can also be predicted, with the audio signals generated according to the method intervening in a supportive manner to reduce or prevent such a predicted deviation.

[0058] According to this training, the target movement pattern is adjusted based on probability. In particular, the current target movement pattern is adjusted in such a way that the probability of a deviation or a fall is reduced or minimized as much as possible. In other words, the target movement pattern is updated to take the current actual movement pattern into account and thus avoid larger, undesirable deviations in the future. For example, if the risk of falls is to be minimized by optimizing gait / walking behavior, the necessary walking speed, step synchronization, and other patterns are determined and used as triggers for the audio signal. For example, a rhythm of the audio signal is changed to reduce the risk of falls for the hearing system user or to maximize the safety of the hearing system user.

[0059] It is also conceivable, for example, that a threshold comparison of the probability could be used as a trigger criterion for movement monitoring / adaptation. For example, the audio signal would only be generated if the probability reaches or exceeds a predefined threshold, i.e., if there is a sufficiently high risk of a fall.

[0060] In a suitable training, a signal property, particularly the signal level, of the audio signal is adjusted based on the determined probability. In other words, the signal level of the audio signal is changed as a measure to reduce the risk of falls. Alternatively, it is possible, for example, to change the rhythm of the audio signal to reduce the risk of falls for the hearing aid user or to maximize the hearing aid user's safety. It is also possible, for example, to change the "pitch" or frequency response of the audio signal, for example, statically or as a "sweep" (either ascending or descending).

[0061] In one conceivable embodiment, the audio signal is used as a sonification of the target movement pattern. In other words, the target movement pattern is rendered into sound; for example, a rhythmic audio signal is generated as a beat or trigger sound for a walking or swimming movement. In a yoga movement, the target movement pattern or the audio signal can also be a measure or sonification of a desired breathing movement; in particular, the audio signal can thus also be used to support meditation exercises. Sonification of the target movement pattern ensures that the hearing system user is supported in executing a desired target movement.

[0062] According to the invention, the movement sounds detected by the input transducer are used as the audio signal. This means that the movement sounds, i.e., the sounds or sound signals caused by the user's own movement (movement sound data), are detected by the input transducer and specifically amplified to generate the audio signal. For example, to support a walking movement, footstep sounds are amplified more strongly than other signals from the input transducer, so that the hearing system user can reliably perceive the footstep sounds. This improves the hearing system user's self-perceived movement.

[0063] This means that real or actual motion sound is used as the audio signal. For this purpose, the input transducer signal (acoustic data) is analyzed and specifically processed with respect to the motion sound data. Depending on the application, this processing can lead to increased audibility, adjustments to the frequency shape, sound character, duration, or other effects.

[0064] An additional or further aspect of the invention provides that the signal level of the audio signal is adjusted proportionally to a deviation between the desired movement pattern and the actual movement pattern. In other words, the volume of the audio signal is adjusted depending on the comparison. In particular, the signal level is reduced when the actual and desired movement patterns match and increased when the actual and desired movement patterns differ. Thus, the volume of the perceived audio signal acts as additional acoustic feedback for the deviation from the desired movement execution.

[0065] In a useful further development, an environmental situation is determined based on the acoustic data of the input transducer, whereby a signal property or signal characteristic, in particular the signal level, of the audio signal is adjusted based on the specific environmental situation. The optimal output gain for the audio signal is thus adjusted based on the specific environmental situation. Additionally or alternatively, a decision regarding the appropriateness of generating the audio signal can also be made based on the specific environmental situation. For example, the volume of the audio signal is increased in noisy environments, such as when walking along a busy road. It is also possible for the generation of the audio signal to be paused or interrupted if, for example, a speech situation or a conversation of the hearing system user is detected.

[0066] An environmental situation is understood here, in particular, to be an acoustic or auditory situation. The environmental situation is identified and described, for example, using situation recognition and / or at least one level measurement and / or at least one algorithm of the hearing aid or signal processing. For example, the environmental situation is classified according to certain criteria, and each of these classes is assigned a specific setting of the hearing aid parameters and / or hearing aid performance.

[0067] The signal level, or volume, of the audio signal should ideally always be audible to the hearing aid user. To achieve optimal audibility, the hearing aid analyzes the ambient level (internal level meter), takes into account, for example, the hearing aid wearer's hearing loss (audiogram information or derived from hearing aid fitting), and ensures audibility, particularly by automatically adjusting the output gain of the audio signal. Relative changes in the audibility of the audio signal can be used to indicate different "qualities" of movement sequences. For example, the relative level of a footstep sound can indicate different step sizes or left-right asymmetries. In this example, the optimum for the user would be achieved if all audio signals were presented at the same level.In another example, the frequency shape or pitch of the audio signal can indicate the length of a swimming stroke. The goal of the hearing aid user would be to obtain the correct pitch that indicates optimal stroke length.

[0068] In an advantageous embodiment, the actual and / or target movement patterns are stored in a memory. This makes it possible, for example, to record and monitor progress or changes in the movement patterns. This data can be made available to medical experts and / or the hearing system user, for example, by visualization on an external device. Additionally or alternatively, it is possible, for example, to derive changes in the risk of falls or falling based on the stored movement patterns and to assess the quality of the supported self-movement.

[0069] The hearing system according to the invention is designed in particular as a hearing aid device and comprises a hearing aid. The hearing aid comprises at least one input transducer for receiving an acoustic ambient signal, an output transducer for outputting an acoustic signal, and a motion sensor for detecting body movement of a hearing system user.

[0070] The hearing system also includes a controller, i.e., a control unit. The controller is integrated into the hearing aid, for example, and is part of a signal processing system. Additionally or alternatively, it is also conceivable for the controller to be part of an additional device, particularly a smartphone, that is or can be connected to the hearing aid for signal transmission.

[0071] The controller is generally configured—in terms of programming and / or circuitry—to carry out the method according to the invention described above. The controller is thus specifically configured to analyze or characterize a user movement or a movement event and to generate a signal that supports the movement.

[0072] In a preferred embodiment, the controller is formed, at least in its core, by a microcontroller with a processor and a data memory, in which the functionality for carrying out the method according to the invention is implemented in the form of operating software (firmware), so that the method - possibly in interaction with a hearing system user - is carried out automatically when the operating software is executed in the microcontroller. Within the scope of the invention, however, the controller can alternatively also be formed by a non-programmable electronic component, such as an application-specific integrated circuit (ASIC), in which the functionality for carrying out the method according to the invention is implemented using circuitry.

[0073] Exemplary embodiments of the invention are explained in more detail below with reference to a drawing. In these, schematic and simplified representations: Fig. 1a hearing system with a binaural hearing aid, Fig. 2the hearing system according to Fig. 1 , in which the hearing aid is signal-coupled to a mobile additional device, and Fig. 3 a flowchart of a method for operating the hearing system.

[0074] Corresponding parts and sizes are always provided with the same reference symbols in all figures.

[0075] The Fig. 1 shows the basic structure of a hearing system 2 according to the invention. In this exemplary embodiment, the hearing system 2 is designed as a hearing aid device with a binaural hearing aid 4 with two signal-coupled hearing aids or individual devices 6a, 6b. The individual devices 6a, 6b are designed, for example, as behind-the-ear hearing aids (BTE). The individual devices 6a, 6b are or can be coupled to each other via a wireless communication link 8.

[0076] The communication connection 8 is, for example, an inductive coupling between the individual devices 6a and 6b; alternatively, the communication connection 8 is designed, for example, as a radio connection, in particular as a Bluetooth or RFID connection, between the individual devices 6a and 6b.

[0077] The structure of the individual devices 6a, 6b is explained below using the example of the individual device 6a. The individual device 6a comprises, as shown in the Fig. 1 schematically depicted is a device housing 10 into which one or more microphones, also referred to as acoustoelectrical input transducers 12, are built. The input transducers 12 record a sound or acoustic signal in the environment of the hearing system 2 and convert it into electrical acoustic data 14.

[0078] The acoustic data 14 is processed by a controller 15 of a signal processing device 16, which is also arranged in the device housing 10. Based on the acoustic data 14, the signal processing device 16 generates an output signal 18, which is passed to a loudspeaker or receiver 20. The receiver 20 is designed as an electro-acoustic output transducer 20, which converts the electrical output signal 18 into an acoustic signal and outputs it. In the individual BTE device 6a, the acoustic signal is transmitted to the eardrum of a hearing system user via a sound tube (not shown in detail) or an external receiver, which is fitted with an earmold seated in the ear canal. However, an electromechanical output transducer is also conceivable as the receiver 20, for example, as in a bone conduction receiver.

[0079] The power supply of the individual device 6a and in particular of the signal processing device 16 is provided by a battery 22 accommodated in the device housing 10.

[0080] The signal processing device 16 is coupled to a motion sensor 24 of the individual device 6a. During operation, the motion sensor 24 detects acceleration and / or rotational movements of the individual device 6a and transmits these as motion sensor data 26 to the signal processing device 16. The motion sensor 24 is embodied, for example, as a 3D acceleration sensor. Additionally or alternatively, the motion sensor 24 is embodied, for example, as a position sensor, in particular as a gyroscopic sensor.

[0081] The signal processing device 16 is further connected to a first transceiver 28 and a second transceiver 30 of the individual device 6a. The transceiver 28 is used to transmit and receive wireless signals via the communication link 8, and the transceiver 30 is used to transmit and receive wireless signals via a communication link 32 to an additional device 34 external to the hearing aid ( Fig. 2 ). For example, it is also conceivable that only one transceiver is provided for both communication connections 8, 32.

[0082] In the embodiment of the Fig. 2 The additional device 34 is designed as a separate, mobile, operating and display device, which is or can be coupled to the hearing aid 4 by means of the communication connection 32. In the Fig. 2 The schematically illustrated additional device 34 is, in particular, a smartphone. The additional device 34, also referred to below as a smartphone, has a touch-sensitive display unit 36, which is also referred to below as a touchscreen. The smartphone 34 is expediently placed within the transmission range of the communication connection 32. The signal coupling between the smartphone 34 and the transceivers 30 of the individual devices 6a and 6b is effected via a corresponding integrated transceiver (not further designated), for example, a radio antenna, of the smartphone 34.

[0083] The smartphone 34 has an integrated controller, which is essentially formed by a microcontroller with implemented application software 38 for the programmatic evaluation of signals and data transmitted via the communication connection 32. The application software 38 is preferably a mobile app or a smartphone app stored in a data memory of the controller. During operation, the controller displays the application software 38 on the display unit 36, which is designed as a touchscreen. The application software 38 can be operated by a hearing system user using the touch-sensitive surface of the display unit 36.

[0084] Based on the Fig. 3 A method 40 according to the invention for operating the hearing system 2 is explained in more detail below using the flow chart shown.

[0085] During operation of the hearing system 2, an actual movement of the hearing system user is recorded as motion sensor data 26 of the motion sensor 24 and as movement noise data 44 of the input transducer 12. The analysis or recording of the movement noise or movement noise data 44 from the acoustic data 14 can be carried out, for example, by means of a movement noise classifier 46 of the controller 15. In the Fig. 3 For example, the classifier 46 is part of the input converter 12, but the classifier 46 can also be designed separately from the input converter 12.

[0086] Based on the motion sensor data 26 and the motion noise data 44, an actual movement pattern 48 for the actual movement or self-movement of the hearing system user is determined. The analysis of the motion sensor data 26 and the motion noise data 44, or the determination of the actual movement pattern 48 derived therefrom, is carried out, for example, using a classifier or a (first) evaluation unit 50 of the controller 15. Based on the movement of the hearing system user characterized by the actual movement pattern 48, a (second) evaluation unit 52 of the controller 15 determines a desired movement pattern 54. The evaluation unit 52 compares the actual movement pattern 48 with the desired movement pattern 54, and a control signal 56 for a sound generator 58 is generated based on the comparison.The tone generator 58 generates an electrical signal 60, which is amplified by an amplifier 62 and sent as an amplified signal 64 to the output transducer 20, which converts the signal 64 into a perceivable audio signal 66.

[0087] The audio signal 66 or the signals 60 and 64 can be a sonification of the target movement pattern 54. The particularly rhythmic audio signal 66 supports a walking movement of the hearing system user, for example, as a beat or trigger sound.

[0088] The signal level or volume of the audio signal 66 is preferably always audible to the hearing system user. To achieve optimal audibility, the controller 15 uses a level meter or an environment classifier 68 to analyze the ambient level, which results from the acoustic data 14 of the input transducers 12. In addition, for example, a hearing loss or hearing impairment of the hearing system user (audiogram information or derived from the hearing aid fitting) is taken into account, and a signal level signal 70 is generated. The signal level signal 70 sets a signal level of the audio signal 66. In the Fig. 3 In the embodiment shown, for example, the signal level of signal 60 is adjusted, so that the signal level of audio signal 66 is subsequently varied. Alternatively, signal level signal 70 can also be used to adjust the amplifier or signal 64.

[0089] The actual and / or desired movement patterns 48, 54 are stored in a memory 72 of the controller 15. This makes it possible, for example, to record and monitor progress or changes in the movement patterns 48, 54. This data can be made available, for example, to medical experts and / or the hearing system user, for example by visualization on the display unit 36 ​​of the additional device 34.

[0090] Preferably, an application case 74 is transmitted to the hearing aid 4 by means of the additional device 34 or by means of the application software 38. Depending on the selected application case 74, for example, the evaluation unit 52 and the tone generator 58 are controlled, thereby changing the desired movement pattern 54 and / or the audio signal 66.

[0091] In an embodiment not shown in detail, the controller 15 determines a probability of a future fall or fall event of the hearing system user based on the recorded motion sensor data 26 and / or motion noise data 44. The probability can be determined, for example, in the evaluation unit 50.

[0092] Preferably, the target movement pattern 56 is adjusted depending on the determined probability. In particular, the target movement pattern 56 or the audio signal 66 is adjusted such that the probability of a fall or trip is reduced or minimized as much as possible. For example, a rhythm of the audio signal 66 is changed to reduce the risk of falling for the hearing system user. For example, the signal level of the signal 60 or the audio signal 66 is adjusted based on the determined probability.

[0093] During operation, the individual devices 6a, 6b of the binaural hearing aid 4 output the audio signal 66, for example, synchronously with each other. For example, it is conceivable that the same audio signal 66 is output at different volumes (depending on the hearing system user's hearing loss and / or depending on the deviation between the actual and desired movement) at the same time.

[0094] It is also possible, for example, that the same audio signal 66 is generated at different times, for example the individual device worn on the left ear generates the audio signal 66 when the left foot is taken forward, while the individual device worn on the right ear generates the audio signal 66 analogously when the right foot is taken forward.

[0095] In particular, in the case of impaired balance or one-sided movement impairments (e.g. limping with one leg), it is still possible that the individual devices 6a, 6b generate different audio signals 66 or differently manipulated audio signals 66 for the right and left ear.

[0096] The movement sounds detected by the input transducer 12 are used as the audio signal 66. This means that the movement sounds or the movement sound data 44 are specifically amplified in the tone generator 58 to generate the audio signal 66. The tone generator 58 can influence a signal property of the movement sound data 44, such as a signal level, a rhythm, or a frequency response of the movement sound data 44. Alternatively, the tone generator 58 can be omitted and the amplifier 62 used directly.

[0097] For example, to support a walking movement, footstep sounds are amplified more strongly than other signals in the acoustic data 14, so that the hearing system user can reliably perceive the footstep sounds. The signal level of the audio signal 66 is adjusted, for example, depending on the deviation of the actual movement pattern 48 from the target movement pattern 54.

[0098] The claimed invention is not limited to the embodiments described above. Rather, other variants of the invention may be derived from the above by those skilled in the art within the scope of the disclosed claims without departing from the subject matter of the claimed invention. List of reference symbols

[0099] 2 Hearing system 4 Hearing aid 6a, 6b Individual device 8 Communication connection 10 Device housing 12 Input transducer 14 Acoustic data 15 Controller 16 Signal processing unit 18 Output signal 20 Output transducer 22 Battery 24 Motion sensor 26 Motion data 28, 30 Transceiver 32 Communication connection 34 Additional device / smartphone 36 Display unit 38 Application software 40 Procedure 44 Motion sound data 46 Motion sound classifier 48 Actual movement pattern 50 Evaluation unit 52 Evaluation unit 54 Target movement pattern 56 Control signal 58 Tone generator 60 Signal 62 Amplifier 64 Signal 66 Audio signal 68 Level meter / environment classifier 70 Signal level signal 72 Memory

Claims

1. A method (40) for operating a listening system (2), which comprises a hearing aid (4) having at least one input transducer (12) and having an output transducer (20) as well as having a movement sensor (24), - wherein an actual movement of the listening system user is detected as movement sensor data (26) of the movement sensor (24) and as movement noise data (44) of the input transducer (12), - wherein an actual movement pattern (48) for the actual movement is determined on the basis of the movement sensor data (26) and / or the movement noise data (44), - wherein the actual movement pattern (48) is compared with a target movement pattern (54), - wherein an audio signal (66) having a perceptible signal level is generated by means of the output transducer (20) in dependence on the comparison, characterized in that the audio signal (66) assists the listening system user in the execution of the actual movement, and - wherein the movement noise data (44) detected by the input transducer (12) are used as the audio signal (66).

2. The method (40) according to Claim 1, characterized in that a probability of a future deviation of the actual movement pattern (48) from the target movement pattern (54) of the listening system user is determined on the basis of the detected movement sensor data (26) and / or movement noise data (44), wherein the target movement pattern (54) is adjusted on the basis of the probability.

3. The method (40) according to Claim 2, characterized in that the signal level of the audio signal (66) is adjusted on the basis of the determined probability.

4. The method (40) according to any one of Claims 1 to 3, characterized in that the movement noise data (44) are determined as acoustic data (14) of the input transducer (12).

5. The method (40) according to any one of Claims 1 to 4, characterized in that the signal level of the audio signal (66) is adjusted proportionally to a deviation between the target movement pattern (54) and the actual movement pattern (48).

6. The method (40) according to any one of Claims 1 to 5, characterized in that a surroundings situation of the listening system user is determined on the basis of acoustic data (14) of the input transducer (12), and that the signal level of the audio signal (66) is adjusted on the basis of the surroundings situation.

7. The method (40) according to any one of Claims 1 to 6, characterized in that the actual movement pattern (48) and / or the target movement pattern (54) is stored in a memory (72).

8. A listening system (2), in particular a hearing aid device, comprising a hearing aid (4) having at least one input transducer (12) for receiving an acoustic surroundings signal, and having an output transducer (20) for outputting an acoustic signal, as well as having a movement sensor (24) for detecting a movement of a listening system user and having a controller (15) for carrying out a method according to any one of Claims 1 to 7.

Citation Information

Patent Citations

  • Monitoring system and method of using same

    WO2020206155A1

  • Hearing aid system and method for operating same

    EP3585072A1

  • Hearing aid and method for controlling hearing aid

    US20150010177A1

  • EP2020086518W