Hearing device and method of adjusting an input audio signal to a hearing device

Abstract

The invention relates to a hearing device (100) comprising at least one microphone (1) configured to receive ambient audio; an audio signal processing unit (2) configured to receive an input audio signal comprising at least the ambient audio from the microphone and to generate an output audio signal based on the input audio signal; and a receiver (3) configured to be supplied with the output audio signal. The hearing device (100) further comprises at least an infrared, IR, sensor (4) configured to detect human presence (300). A controller (5) of the hearing device (100) is configured to adjust the input audio signal to the audio signal processing unit (2), based on the detected human presence (300). The invention also relates to a method of adjusting an input audio signal implemented in the hearing device (100).

Description

[0001] HEARING DEVICE AND METHOD OF ADJUSTING AN INPUT AUDIO SIGNAL TO A HEARING DEVICE

[0002] TECHNICAL FIELD

[0003] Embodiments presented herein relate to a hearing device, preferably a wearable electronic hearing device, comprising a controller configured to adjust the input audio signal thereto, as well as to a method of adjusting an input audio signal to a hearing device.

[0004] BACKGROUND

[0005] In the context of the present invention, electronic hearing devices are generally electronic devices capable of stimulating a user's hearing and adapted to be worn on one or more body parts by the user, by way of example but not necessarily at (the vicinity of) an ear or partially within an ear canal of a user.

[0006] A possible application of hearing devices is to support and / or improve the individual hearing capacity of users, for instance hearing impaired users. In such cases, the hearing devices are preferably miniaturized and typically referred to as hearing instruments or hearing aids or hearing prostheses.

[0007] Other possible uses of hearing devices pertain, by way of example, to augmenting the hearing of normal hearing persons, for instance by means of noise suppression; or to the provision of audio signals originating from remote sources, e.g. within the context of audio communication.

[0008] Hearing devices such as hearing aids can therefore be provided with different types of earpieces for coupling to the ear and / or to the ear canal of a user; with earplugs; with headsets or other wearable elements, preferably head-worn elements.

[0009] Non-limiting examples of electronic hearing devices are in-ear, or at least partially in- ear, headsets (also referred to as in-ear monitors); on-ear headsets (also referred to as earbuds, or the like); over-ear headsets and eyewear-mounted (such as smart glasses for VR, AR or generally XR applications). It will be recognized that the inventive features of the present invention are substantially compatible with any model of above-mentioned hearing devices.

[0010] Currently, many hearing devices like headphones offer easy ways to turn active noise cancellation (ANC) on or off, for instance by a dedicated button on the headphones themselves, wherein a single press might cycle through modes like ANC on, ANC off, and transparency (which lets in some external sound). Alternatively, some models require that the user press and hold a dedicated button to change in between modes. More modern headphones use touch-sensitive areas on the earcups, allowing to control ANC functions with various taps or swiping gestures.

[0011] For finer control, premium headphones with ANC cooperate with companion smartphone applications or apps. These apps offer detailed ANC settings, including turning ANC on or off, sometimes even adjusting the cancellation intensity level and / or the cancellation frequency. Certain deeply integrated headphones, like AirPods with Apple devices, may let a user control ANC directly from the device's settings, such as within the Bluetooth menu.

[0012] Moreover, for convenient control, many headphones now support voice-activated commands, e.g., through assistants like Siri, Google Assistant, or Alexa, which enables turning ANC on or off hands-free.

[0013] Some headphones can detect a user's activity by embedded sensors, for example by IMUs (Inertial Measurement Units), and autonomously determine, based on the sensors' detections, when an ANC-on or a transparency mode is deemed most suitable. Such smart switching may encompass functions like activating noise cancellation when a user starts walking down a busy street and switching to transparency mode if the user stops to have a conversation. Some existing headphones offer a transparency mode using microphone input that lets you easily stay aware of your surroundings by mixing in external sounds, trying to adapt to situations where a user may still need to hear public announcements or have an extemporaneous conversation.

[0014] More in particular, in the present technical field, several noise cancelling modes are known. Passive Noise Cancellation relies on the physical design of headphones or earbuds to block noise. Proper Active Noise Cancellation (ANC) is a technology used in hearing devices such as headphones and earbuds to reduce unwanted background noise by way of microphones that listen to the ambient noise emitted around the user. Electronics inside the hearing devices analyze the incoming noise and create an "anti-noise" countersignal, essentially a mirror image of the original noise wave, with the peaks and troughs flipped. The anti-noise signal is then combined with music or audio to be enjoyed by the user. When the original noise wave and the anti-noise wave meet, they effectively cancel each other out, reducing the amount of background noise that reaches the user's ears.

[0015] Transparency mode on headphones or earbuds, like that implemented on AirPods Pro, is a functional feature that blends external sounds with the audio destined to the user. These hearing devices use tiny microphones to capture the sounds around the user. The audio is then processed and mixed with what the user is expressly listening to, delivered through speakers at a low volume, allowing the user to hear important announcements, have conversations, or simply be aware of potential hazards while still enjoying music or podcasts.

[0016] While using ambient sound, as above described, as a way to activate or deactivate ANC seems like a convenient idea, it comes with a few significant drawbacks and technological challenges. One major problem is that sound-based triggers are susceptible to false activation. Environmental noises like a sudden bark or loud clatter could be easily misinterpreted by the system as a command, unintentionally turning ANC on, or even off. Additionally, relying on sound for control lacks specificity. It's difficult to design a system that recognizes a specific voice command without also potentially reacting to similar-sounding noises.

[0017] Furthermore, there are situations where using voice commands might be intrusive or disruptive, such as in a quiet meeting or a library.

[0018] Privacy is also a concern, since using a user's voice to control ANC means the headphones need to have microphones constantly listening and analyzing content, which might be unsettling. Additionally, on a technical level, isolating actual command sounds from environmental sound or noise is difficult and prone to causing false triggers. This technology also needs reliable sound pattern recognition in order to distinguish between a specific command and general noise. This requires advanced signal processing and potentially machine learning, adding in complexity and processing power. Constant audio analysis for commands can also significantly drain the battery life of wireless hearing devices. Finally, some delay between issuing a sound command and the ANC responding is inevitable, so minimizing that delay is important for a good user experience.

[0019] Thus, there exists a need for a hearing device which enables a prompt, reliable and efficient transition between a mode wherein an ANC, or active noise cancelling, function is on, and a mode wherein the ANC functions is off or set to a given level of transparency.

[0020] SUMMARY

[0021] An object of embodiments herein is to address the above issues.

[0022] A particular object is enabling a more robust and user-friendly solution to switching between different ANC modes in a hearing device.

[0023] Another particular object is enabling a (preferably wearable) electronic hearing device to precisely and reliably determine when (as well as the extent to which) background and / or surrounding noise needs to be reduced, or rather maintained, without requiring manual input.

[0024] According to a first aspect there is presented a hearing device comprising at least one microphone configured to receive ambient audio; an audio signal processing unit configured to receive an input audio signal comprising at least the ambient audio from the microphone and to generate an output audio signal based on the input audio signal; a receiver configured to be supplied with the output audio signal. The hearing device according to the present invention further comprises at least an infrared, IR, sensor configured to detect human presence (other than the hearing device user) by detection of radiation emitted by a human body. Additionally, the hearing device according to the present invention comprises a controller, in communication with the infrared, IR, sensor and with the audio signal processing unit, configured to adjust the input audio signal to the audio signal processing unit, based on the detected human presence. The controller can be additionally configured to adjust the output audio signal transmitted to the receiver, based on the detected human presence. Alternatively, the controller is configured to adjust solely the output audio signal to the receiver, based on the detected human presence.

[0025] By adjusting an input audio signal to a hearing device, it is hereby meant adjusting the way one or multiple audio signals are processed, before being transmitted to the receiver for ultimate conversion into sound for the hearing device user. Such an adjustment can be carried out at any stage. For instance, it can be an adjustment of the input audio signal to the audio signal processing unit of the hearing device; or an adjustment of the output audio signal to the receiver of the hearing device; or a combination thereof.

[0026] As used herein, “ambient audio” relates to ambient sound audible in the physical environment where the hearing device user is presently located. The input audio signal received by the audio signal processing unit can further comprise other, distinct components which do not originate from the ambient or environment around the user. The source(s) of these further, distinct components are therefore alternative and different from the physical location of the heating device user, and can be for instance playback audio sources producing playback audio, as better defined in the following.

[0027] The hearing device is preferably head-worn. The audio signal processing unit is operatively connected to the microphone and to the receiver, or loudspeaker, and can performs various signal processing functions, including amplification, beamforming, feedback cancelling, frequency lowering or enhancing, sound type classification, tone control, etc., besides carrying out background noise reduction or more specifically ANC. The controller is communicatively coupled with the infrared, IR, sensor and with the audio signal processing unit, either wired or wirelessly.

[0028] According to a second aspect there is presented a method for adjusting an input audio signal to a hearing device, comprising detecting human presence by detection of radiation emitted by a human body, by a at least an infrared, IR, sensor 4; and, by a controller, in communication with the infrared, IR, sensor and with the hearing device, adjusting the input audio signal to the hearing device, based on the detected human presence.

[0029] In some embodiments, at least a microphone of the hearing device is configured to receive ambient audio; an audio signal processing unit of the hearing device is configured to receive an input audio signal comprising at least the ambient audio from the microphone and to generate an output audio signal based on the input audio signal; and a receiver of the hearing device is configured to be supplied with the output audio signal. The method comprises detecting human presence by detection of radiation emitted by a human body, by a at least an infrared, IR, sensor. The method then preferably comprises, by a controller in communication with the infrared, IR, sensor and with the audio signal processing unit, adjusting the input audio signal to the audio signal processing unit, based on the detected human presence.

[0030] The method can be performed, in part or even in full, in the hearing device. The IR sensor and the controller can be thus part of the hearing device. In some embodiments, in order to enable a smoother implementation of the above method, the IR sensor and the controller are preferably part of the hearing device.

[0031] The controller can be additionally configured to adjust the output audio signal to the receiver, based on the detected human presence. Alternatively, the controller can be configured to adjust solely the output audio signal to the receiver, based on the detected human presence.

[0032] According to a third aspect there is presented a computer program for adjusting an input audio signal to a hearing device, the computer program comprising computer code which, when run on processing circuitry of a controller, causes the controller to implement the above method.

[0033] Advantageously, these aspects do not suffer from the drawbacks and issues disclosed above.

[0034] In fact, advantageously these aspects enable the hearing device according to the present invention to carry out a prompt, consistent and efficient transition between a mode wherein an active noise cancelling (ANC) function is on, and a mode wherein the ANC functions is instead turned off, or set to a given level of transparency, in view of a likely conversation happening with a fellow speaking person.

[0035] Advantageously, these aspects enable to adjust the input audio signal to the hearing device (e.g., the input audio signal to the audio signal processing unit of the hearing device and / or the output audio signal to the receiver of the hearing device) automatically, without need for a manual input by the hearing device user.

[0036] Other objectives, features and advantages of the enclosed embodiments will be apparent from the following detailed disclosure, from the attached dependent claims as well as from the drawings.

[0037] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / an / the element, apparatus, component, means, module, step, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, module, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

[0038] BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The inventive concept is now described, by way of example, with reference to the accompanying drawings, in which:

[0040] Fig. 1 schematically illustrates a hearing device according to an embodiment;

[0041] Fig. 2 and Fig. 3 schematically illustrate a use case and the working of a hearing device according to an embodiment;

[0042] Fig. 4 schematically illustrates the working of a hearing device according to a further embodiment;

[0043] Fig. 5 is a flowchart of a method according to embodiments;

[0044] Fig. 6 is a schematic diagram showing functional units of a controller according to an embodiment; and Fig. 7 shows one example of a computer program product comprising computer readable storage medium according to an embodiment.

[0045] DETAILED DESCRIPTION

[0046] The inventive concept will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments of the inventive concept are shown. This inventive concept may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art. Like numbers refer to like elements throughout the description. Any step or feature illustrated by dashed lines should be regarded as optional.

[0047] As noted above, there is still a need for enabling a hearing device, preferably a wearable electronic hearing device, to automatically adjust the input audio signal to take into account current circumstances of a hearing device user, such as the convenience, or need, to communicate with a distinct individual who is, or becomes, present in the vicinity of the hearing device user. Particularly, there is an outstanding need to allow a hearing device user to still intelligibly receive messages from a third person intending to engage in conversation, or in general to communicate, with the hearing device user, selectively on instances when it is expedient and in a way unaffected by background ambient or environment noise of different origin.

[0048] Current technologies adopted in hearing devices to deal with background noise, in fact, are unable to consistently discern between occasions when a total block or cancellation of actual unwanted ambient sound is desirable for a hearing device user and occasions when, instead, human speech, however set in a noisy context, is to be conveyed to the hearing aid user. Accordingly, known hearing devices fail to reliably determine human presence in a hearing device user's proximity, particularly in a proximity suitable, and conducive, to an exchange of information, especially spoken, between a third party (or distinct individual) and the user of the hearing device.

[0049] Ultimately, carrying out active noise cancelling (solely) on the base of miniature microphones that acquire audio information of the surroundings is liable to false triggers and wrong assumptions on the circumstances experienced by a hearing device user.

[0050] These problems are solved through a hearing device 100 and a method of adjusting an input audio signal to the hearing device 100 according to the present invention, wherein human presence 300 is effectively detected.

[0051] The embodiments disclosed herein therefore relate to techniques for adjusting the input audio signal to hearing devices 100 (e.g., for adjusting the input audio signal to audio signal processing units 2 of hearing devices 100 and / or for adjusting the output audio signal to receivers 3 of hearing devices 100), based on an accurate and timely detection of human presence 300 in the proximity of a hearing device user 200.

[0052] Fig. 1 illustrates a hearing device 100 as worn by a user 200. In the exemplary illustration, the hearing device 100 is depicted in the form of on-the-ear headset and comprises head-worn elements such as a headband and ear cups that fully encompass the user's ears.

[0053] Other designs of the hearing device 100 according to the present invention are possible, with respective fit and compactness, such as over-the-ear or an at least partially in-the-ear headset. Alternatively, the hearing device of the present invention can also comprise different earpieces for coupling to the user's ear canal and take the form of earbuds resting at the edge of the ear canal of a user; or possibly in-ear headphones, appropriately customized.

[0054] The hearing device 100 comprises at least one microphone 1 configured to receive ambient audio and an audio signal processing unit 2.

[0055] The audio signal processing unit 2 is configured to receive an input audio signal comprising at least the ambient audio from the microphone 1. The audio signal processing unit 2 is also configured to consequently generate an output audio signal based on the input audio signal. The audio signal processing unit 2 can take the form a CPU such as low-power ARM -based processor specifically designed for embedded systems and wearables. This central processing unit, CPU, can execute system instructions, running applications and processing input / output data. The hearing device 100 further comprises a receiver, or loudspeaker or speaker, 3 configured to be supplied with the output audio signal which is output by the audio signal processing unit 2. The loudspeaker, preferably a micro-speaker optimized for size and power efficiency, acts as an electroacoustic transducer and converts the electrical output audio signal from the signal processing unit 2 back into an acoustic signal transmitted as audio into the ear of user 200, thus enabling crucial interactions like receiving instructions, hearing alerts or notifications, and playing music or other media.

[0056] The audio signal processing unit 2 is thus operatively connected to the microphone 1 and to the receiver 3.

[0057] The hearing device 100 further comprise at least an infrared, IR, sensor 4 configured to detect human presence 300 by detection of radiation emitted by a human body.

[0058] A human body is a source of infrared radiation, its temperature being typically about 37 C or 98 F, which constantly exchanges heat with the environment. The radiation characteristics of a human body, analyzed using the black-body radiation curve governed by Planck's Law, show that essentially all of the radiation is in the infrared region with the peak radiation occurring at 9.55 pm. The average human frame radiates about 100 W / m2 of heat power to the environment. Infrared sensors, or detectors, 4 can be made sensitive to the thermal pattern, or signature, of a human body, for instance in a range of 8-14 pm, and thus able to detect humans within a reasonable distance.

[0059] The hearing device 100 further comprises controller 5, in communication with the IR, sensor 4 and with the audio signal processing unit 2, configured to adjust the input audio signal to the audio signal processing unit 2, based on the detected human presence 300. The controller 5 can be communicatively coupled with signal processing unit 2 and IR sensor 4 either by wires or wirelessly. The controller 5 can instruct the audio signal processing unit 2 to carry out specific audio signal tasks. The controller 5 preferably comprises a digital signal processor (DSP), programmed to instruct execution of functional tasks of dedicated signal processing algorithms, such as the tasks shown in Fig. 5. The DSP of the controller 5 can thus comprise instructions, rules and parameters for performing the specific tasks of Fig. 5, including implementing adaptive directional functions and automatic mode switching depending on nature of the input sound.

[0060] The controller 5 may also take the form of a CPU or of a system-on-chip (SoC).

[0061] In some embodiments, the audio signal processing unit 2 can be part of the controller 5 and be therein comprised.

[0062] In some embodiments, the audio signal processing unit 2 and the controller 5 can be incorporated in the same component, substantially coinciding.

[0063] Alternatively, the audio signal processing unit 2 can be part of the microphone transducer.

[0064] The hearing device 100 can thus support functionalities able to adjust the input audio signal to the audio signal processing unit 2, like Active Noise Canceling (ANC) or a so-called transparency mode, and can dynamically switch between such functionalities or modes, depending on the detected environmental interactions.

[0065] Based on the definition of adjustment of an input audio signal to a hearing device above given, the hearing device 100 can analogously support functionalities (e.g., ANC or transparency mode) able to adjust the output audio signal to the receiver 3

[0066] In fact, the controller 5 can be additionally configured to instruct the adjustment of the output audio signal to the receiver 3, based on the detected human presence, by way of the audio signal processing unit 2. Alternatively, the controller 5 can be configured to adjust solely the output audio signal to the receiver 3, based on the detected human presence.

[0067] The microphone 1 can be a MEMS (Micro-Electro-Mechanical Systems) microphone, for instance enabling voice commands, recording voice memos, or capturing environmental audio for analysis. MEMS are small, energy-efficient and suitable to be incorporated in wearables. The microphone 1 can also be made according to alternative technologies, for instance the at least one microphonei can be an ECM microphone, a piezoelectric microphone, a capacitive microphone or possibly an optical microphone. The hearing device 100 preferably comprises a memory 6, such as a combination of RAM (for active processes) and non-volatile storage, like flash memory (for long-term storage). RAM can thus provide working space for running application, while Flash memory can store the operating system, user files, and other persistent data.

[0068] Moreover, the hearing device 100 can comprise IMU (Inertial Measurement Unit) sensors (not shown in the Figures). An IMU sensor can contain a 3-axis accelerometer, measuring linear acceleration; a 3-axis gyroscope, measuring rotational rate; and a 3-axis magnetometer, measuring magnetic fields. By combining these sensor outputs, an IMU sensor can accurately determine the device's orientation (roll, pitch, yaw) and track complex motions in 3D space.

[0069] Preferably, the at least an infrared, IR, sensor 4 is a sensor capable of detecting mid and long-wave infrared, IR radiation, such as a thermopile sensor, or a bolometric sensor, or a micro bolometric array sensor, or a pyroelectric sensor.

[0070] Thermopile sensors operate based on the Seebeck effect, where multiple thermocouples generate a voltage proportional to the temperature difference between their junctions. When infrared radiation falls on it, the thermopile sensor generates heat, creating a temperature difference across the thermocouples and producing an electrical signal. Thermopile sensors offer fast response times, high sensitivity, and wide temperature measurement ranges, suitable for measuring temperature from a distance and withstanding harsh environments.

[0071] Bolometric Sensors detect infrared radiation by measuring changes in the electrical resistance or other properties of a sensitive material when exposed to radiation. When infrared radiation is absorbed by sensitive material of a bolometric sensor, this causes a change in temperature, leading to a measurable change in resistance.

[0072] Bolometric sensors offer high sensitivity and can detect a wide range of infrared wavelengths, suitable for capturing detailed thermal images and operating over a broad temperature range.

[0073] Microbolometers are similar to bolometric sensors but are fabricated using microfabrication techniques, allowing for the creation of arrays of miniaturized resistive elements. Each miniaturized resistive element acts as a pixel in a thermal imaging sensor, with changes in resistance corresponding to changes in temperature caused by incident infrared radiation. Microbolometer arrays are offer high sensitivity, low power consumption, and compact size, suitable for integration into portable and handheld thermal imaging devices. They provide real-time thermal imaging capabilities and can detect temperature variations over a wide area.

[0074] The IR sensor 4 could also take the form of pyroelectric sensors, detecting changes in temperature by measuring the voltage generated when certain materials (pyroelectric materials) undergo a change in polarization in response to temperature fluctuations. Pyroelectric sensors operate in the mid to long-infrared range and can be used for detecting changes in infrared radiation, even from a distance. Since they are more suitable for detecting changes in the IR environment rather than measuring the exact temperature of distant objects, pyroelectric sensors might have to cooperate with complementary sound algorithms.

[0075] With reference to Fig. 1, in one embodiment, the hearing device 100 can comprise a lens, or a transmissive optical element, 41 which is configured to shape a field of view, or FOV, of the IR sensor 4. Thus, advantageously the FOV can be made sufficiently narrow, down to a few degrees, and accordingly the limit range of width and / or distance of detection can be limited. By such lens 41, an optimal definition of, or focusing on, an area for detection of IR radiation can be achieved.

[0076] Preferably, but not necessarily, the lens 41 is made of a material transparent to wavelengths of radiation emitted by a human body, particularly to wavelengths of about 9 to 12 micrometers. The lens 41 can therefore be made of Germanium and / or Zinc Selenide and / or Zinc Sulfide and / or Chalcogenide Glasses and / or Silicon.

[0077] As exemplified in Figs. 2, 3 and 4, in some embodiments the IR sensor 4 can be positioned in the hearing device 100 so that the field of view, FOV, 40 thereof is in the direction of the nose of a device user 200, or in the direction of the device user's gaze. In this configuration, the sensor detects whatever the user 200 is turning his head to.

[0078] Alternatively, or additionally, the at least an IR sensor 4 can be directed to the left and / or right and / or back of the user's head, thus being able to detect if someone is on either side of the user 200. Alternatively, or additionally, an array of infrared, IR, sensors 4 can be positioned on the hearing device 100 such that the FOV 40 of each of the infrared sensors 4 in the array overlaps with the field of view of adjacent sensor(s) 4 in the array, for complete coverage and guaranteed detection of a possible third person willing to communicate to user 200, possibly for a whole 360 degree range around user 200, as illustrated at Fig. 4. With reference to Fig. 4, the headset of the hearing device 100 can be provided with two or more couples of IR sensors 4, for instance a first couple of IR sensors 4 being deployed on a right ear cup of the headset worn by the user 200 and second couple of IR sensors 4 being deployed on a left ear cup of the headset worn by the user 200. Each couple of IR sensors 4 comprises a first forward facing IR sensor 4, positioned frontally with respect to the head of the user 200; and a second back facing IR sensor 4, positioned towards the rear of the head of the user 200. Each couple of IR sensors 4 can have overlapping FOVs, so as to be able to concurrently cover, and potentially detect human presence in, three positions. For instance, in Fig. 4, the forward-facing IR sensor 4 on the right ear cup covers an area A; the back facing IR sensor on the same right ear cup covers an area E; and both the forward facing and the back facing IR sensors 4 can cooperate to cover intermediate area F, thanks to overlapping FOVs. Analogous considerations apply to the front facing and back facing IR sensors 4 on the left ear cup of the hearing device 100, covering respectively area B and area D as well as sharing coverage of the intermediate area C.

[0079] A hearing device 100 provided with a headset comprising multiple IR sensors 4, arranged on the headset so as to cover the surroundings of the user 200 on multiple sides, can achieve to track a nearby individual 300 as the individual moves around, and changes position with respect to, the user 200.

[0080] In some embodiments, the IR sensor 4 can be a single-pixel photodetector, advantageously providing a simple, cost-effective, solution with low power consumption but still with high sensitivity to IR light and fast response times.

[0081] Alternatively, the IR sensor 4 can be a multipixel detector. An IR-based multipixel sensor captures infrared light across multiple points, allowing it to form detailed images or provide spatially resolved data. This makes an IR-based multipixel sensor suitable also for tasks like thermal imaging, gesture recognition, or environment mapping. In some embodiments, the hearing device 100 can comprise other components for enabling extended user interaction, such as touch sensors, and / or buttons.

[0082] Preferably, the controller 5 of the hearing device 100 according to the present invention is configured to activate a noise cancelling function, to at least partially cancel an ambient component of the input audio signal to the audio signal processing unit 2 based on the received ambient audio, if no human presence 300 is detected by the IR sensor 4. If, instead, the IR sensor 4 detects a human presence 300, the controller 5 of the hearing device 100 is preferably configured to deactivate a noise cancelling function, in order to at least partially allow an ambient component of the input audio signal to the audio signal processing unit 2 based on the received ambient audio. In fact, in the latter situation wherein a human presence 300 is detected, it might well be advantageous for the hearing device user 200 to be able to hear what the detected person in the proximity might have to say, substantially having the hearing device 100 set in a transparency mode wherein potentially interesting ambient audio signal is let in.

[0083] In some embodiments, situations are envisaged when the hearing device 100 reproduces, or plays back, digital media comprising audio data. The digital media audio data can be, for instance, audio data previously recorded and stored in a digital format. The digital media audio data can be in turn transmitted by an audio source (e.g., by an alternative audio source like audio streaming services or by a distinct playback device) to the audio signal processing unit 2 of the hearing device 100. In these situations, the user 200 accordingly receives, and listens to, playback audio originating from the abovementioned audio source. Accounting for these situations, the controller 5 can be configured to at least partially allow a playback component of the input audio signal to the audio signal processing unit 2 based on the received playback audio, if no human presence 300 is detected. In fact, the user 200 does not need to dedicate any part of his / her attention to anything else than the streaming digital media, in absence of a potential person 300 with whom to converse.

[0084] Otherwise, if a human presence 300 is detected, the controller 5 can be configured to at least partially cancel a playback component of the input audio signal to the audio signal processing unit 2 based on the received playback audio, thus allowing the user 100 to better focus on speech from the person 300 detected, at least to a lesser extent disturbed by the playback component. In some embodiments, the audio signal processing unit 2 of the hearing device 100 can be fed with an input audio signal wherein the ambient audio from the microphone and the playback audio components each have been previously weighted, to form respectively a weighted ambient audio signal and a weighted playback audio signal, and subsequently mixed to produce a combined audio signal, optimized for intelligibility by the user 200.

[0085] In some embodiments, the controller 5 can be configured to activate an audio steering function, for instance incorporate in the audio signal processing unit 2, to adjust a directionality of the microphone(s) 1. Thus, the microphone 1, or an array of microphones 1, can pick up sound, or audio, preferentially from the direction in which the human presence 300 is detected. In fact, the position of the detected human presence 300 can, in turn, provide information enabling a multiplicity of microphones 1 of the hearing device 100 (arranged, for instance, in an array of microphones) to preferentially pick up sound from the direction in which the human presence 300 has just been detected and rather block noise from other, different directions, according to a beamforming microphone array configuration.

[0086] In some embodiments, the controller 5 may be configured to identify or detect human speech, for instance by selecting bands of frequencies typical of human speech. For enhancing recognition of human speech, band-pass filtering can be carried out; or different sound equalization techniques (including boosting, cutting, shelving or peaking) can be applied. Also, the controller 5 can be configured to instruct reduction of amplitude of frequencies outside bands typical human speech.

[0087] Enhancement of human speech frequencies and reduction of non-human speech frequencies can be adjusted in a combined fashion.

[0088] In some embodiments, the at least an IR sensor 4 is programmed to actively detect human presence 300 conditional to the microphone 1 picking up a trigger like human voice sound in the ambient audio. This allows to carry out the method according to the present invention in a power efficient way, without draining the battery of the hearing device 100 for activating the IR sensor 4 when it is least likely to yield a positive result of detection of a human presence 300. According to a second aspect of the present invention there is provided a method of adjusting an input audio signal to a hearing device 100 as above described. The method can be performed, in part or in full, in the hearing device 100.

[0089] With reference to Figs. 1, 2 and 5, the method comprises detecting, at S13, human presence 300, through detection of radiation emitted by a human body, by way of at least an infrared, IR, sensor 4. At step S13, in fig. 5, the IR sensor, for instance a thermopile sensor, senses, or detects, a “heat signature” in its direction, that is radiation emitted in a range of about 9 to 12 pm. At step S14, information about the detected “heat signature” is added to the data that the controller 5 processes, being thereto transmitted. At step S15, the controller 5 can, subsequently, determine that a human individual 300 is in proximity of the hearing device user 200, potentially willing to speak, or already in the act of speaking, to the user 200.

[0090] The input audio signal adjusting method further comprises adjusting, S16, S17, the input audio signal to the audio signal processing unit 2 of the hearing device 100, based on the above described detection of human presence 300. The adjustment is carried out by the controller 5 which is in communication with the IR, sensor 4 and with the audio signal processing unit 2.

[0091] In some embodiments, based on detection of the ambient audio and in absence of a detected “thermal signature” representative or characteristic of a human presence 300, the method can further comprise activating, or keeping active, si6, a noise cancelling function, such as an ANC function. By keeping activated an ANC function, the method allows to at least partially cancel an ambient component of the input audio signal to the audio signal processing unit 2, in case the at least one IR sensor 4 doesn’t detect any human presence 300. That is, given that the presence of a third person willing to interact with the hearing device user 200 is deemed very low based on (absence of) detection, the hearing device user 200 is let fully focus on specific content, without having to be aware of ambient audio information.

[0092] Otherwise, in case the at least one IR sensor 4 detects a human presence 300, the method can comprise (at least partially) deactivating, S17, a noise cancelling function. By deactivating an ANC function, the method allows, at least partially, transmission of an ambient component of the input audio signal to the audio signal processing unit 2, based on the received ambient audio. That is, given that the presence of a third person 300 willing to interact with the hearing device user 200 is deemed high based on actual detection thereof, the hearing device user 200 is made aware of ambient audio information, therefore making speech from such third person 300 intelligible to the user 200. This equals, in the wording of step S17 in Fig. 5, to enabling a transparent mode on the hearing device 100.

[0093] In some embodiments, the method may further comprise, by the controller 5, detecting, in the audio and / or in the input signal based thereof which is received by the audio signal processing unit 2, playback audio. Such playback audio can be transmitted by an audio source of the hearing device or of a distinct playback device. As already mentioned above, playback audio encompasses digital media comprising audio data, thus digital media audio data can be, for instance, audio data previously recorded and stored in a digital format. The digital media audio data can be in turn transmitted by an audio source (e.g., by an alternative audio source like audio streaming services or by a distinct playback device) to the audio signal processing unit 2 of the hearing device 100.

[0094] In these circumstances, the method can comprise at least partially (or in total) allowing processing of a playback component of the input audio signal to the audio signal processing unit 2 based on the received playback audio, if no human presence 300 is detected. It is thus assumed that, under these circumstances, the hearting device user 200 does not need to turn his attention to any other individual and can fully enjoy the playback audio that is being broadcasted to him / her.

[0095] Alternatively, the method can comprise at least partially (on in full) cancelling, or blocking, processing of a playback component of the input audio signal to the audio signal processing unit 2 based on the received playback audio, if a human presence 300 is detected. Focus by the hearing device user 200 on (potential) communication from the detected individual is in this instance favored.

[0096] In some embodiments, the method may comprise activating an audio steering function to adjust a directionality of the at least one microphone 1, such that the microphone 1 picks up sound, or audio information, preferentially from the direction in which the human presence 300 is detected. In this case, it is advantageous to employ a multiplicity of microphones 1, or an array thereof, in order to effectively carry out the selective sound pick up, preferentially from the direction in which the human presence 300 has just been detected, and to rather block noise from different directions. Such a beamforming microphone array configuration can optimize the above audio steering function.

[0097] In some embodiments, the position of the human presence 300 can be derived by calculating it based on the positioning, on the hearing device 100, of the at least an IR sensor 4. If an array of IR sensors 4 is employed to carry out the detection of human body radiation, the localization, or position, of the human presence 300 can be computed based on the positioning, on the hearing device 100, of the infrared sensors 4 of the array, for instance by way of triangulation.

[0098] In some embodiments, when human presence 300 has been detected by detection, S13 - S15, of radiation emitted by a human body, band-pass filtering of frequencies typical of human speech can be carried out, thus enhancing speech recognition. Additionally, or alternatively, the method can comprise reducing amplitude of frequencies outside typical human speech.

[0099] Detecting, S13, human presence 300 by detection of radiation emitted by a human body can be carried out, as above explained, by any one, or a combination, of: a thermopile or a bolometric sensor or a micro bolometric array sensor or a pyroelectric sensor.

[0100] In some embodiments, the method comprises shaping a FOV 40 of the IR sensor 4 by an interposition of a lens 41, or of another transmissive optical element, which is made transparent to wavelengths of radiation emitted by a human body, preferably to wavelengths of about 9-12 pm. Such shaping can advantageously achieve that the FOV be sufficiently narrow, possibly down to a few degrees. Accordingly, the limit range of width and / or distance of human presence detection can be defined by the specific design of the lens 41 and an optimal definition of, or an optimal focus on, an area of interest for detection of IR radiation can be achieved.

[0101] In some embodiments, actively detecting, S13, human presence by the IR sensor(s) 4 can be made conditional to a distinct sensor, different from the IR sensor(s), picking up a given, preferably pre-set trigger. For instance, with reference to Fig. 5, the microphone 1 picking up, at sn, human voice sound in the ambient audio can work as such a trigger. As already explained, this solution has the advantage of automatically switching on the IR sensor 4 when it is most relevant, without the need of unnecessarily keeping the IR sensor in an on state, with ensuing energy efficiency. Only if the set condition on the trigger event is met, is the IR sensor 4, for instance a thermopile sensor, activated, at S12.

[0102] In some embodiments, the microphone 1 of the hearing device 100 can receive ambient audio continuously; and the active noise cancelling, ANC, function is kept activated, at S16, until the controller 5 detects, S11, human voice sound in the received ambient audio, preferably until the IR sensor 4 confirms, at S13-S15, to the controller 5 that a “heat signature” characteristic of a human presence 300 is detected, signalling a high likelihood that an individual 300 intends to talk, or actually already is talking, to the hearing device user 200.

[0103] In some embodiments, the transparent mode entered at S17 -allowing for filtered, useful ambient audio information to be transmitted to the user 200- is kept until the controller 5 gets an information that the condition of a trigger activating the IR sensor 4 is still met e.g., that human speech is detected by the microphone 1; and / or an information that the at least one IR sensor 4 actually keeps on detecting a human presence 300. If, instead, as exemplified at step S18 of Fig. 5, the sensors picking up the trigger for an activation of the IR sensors indicate that speech communication, as in a conversation, has ended; and / or the IR sensors indicate that, for instance upon end of a conversation, an individual 300 previously interacting with the hearing device user 200 has left, the controller 5 can instruct, at S19, that an ANC functionality mode be turned on and reinstated, e.g., a last known / used ANC functionality mode. Such ANC functionality mode can thus be maintained at least up to the point that, as at S11, speech is detected yet gain in proximity of the hearing aid user 200 by the microphone 1, triggering anew an activation of the IR sensor 4 for ultimate verification of a human presence 300.

[0104] Fig. 7 schematically exemplifies, as a number of functional units, the components of a controller 5 according to an embodiment. Processing circuitry 510 is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions stored in a computer program product 610 (as in Fig. 7), e.g. in the form of a storage medium 530. The processing circuitry 510 may further be provided as at least one application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

[0105] Particularly, the processing circuitry 510 is configured to cause the controller 5 to perform a set of operations, or steps, as disclosed above. For example, the storage medium 530 may store the set of operations, and the processing circuitry 510 maybe configured to retrieve the set of operations from the storage medium 530 to cause the controller 5 to perform the set of operations. The set of operations may be provided as a set of executable instructions.

[0106] Thus, the processing circuitry 510 is thereby arranged to execute methods as herein disclosed. The storage medium 530 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory. The controller 5 may further comprise a communications (comm.) interface 520 at least configured for communications with other entities, functions, nodes, and devices. As such the communications interface 520 may comprise one or more transmitters and receivers, comprising analogue and digital components. The processing circuitry 510 controls the general operation of the controller 5 e.g. by sending data and control signals to the communications interface 520 and the storage medium 530, by receiving data and reports from the communications interface 520, and by retrieving data and instructions from the storage medium 530. Other components, as well as the related functionality, of the controller 5 are omitted in order not to obscure the concepts presented herein.

[0107] Fig. 7 shows one example of a computer program product 610 comprising computer readable storage medium 630. On this computer readable storage medium 630, a computer program 620 can be stored, which computer program 620 can cause the processing circuitry 510 and thereto operatively coupled entities and devices, such as the communications interface 520 and the storage medium 530, to execute methods according to embodiments described herein. The computer program 620 and / or computer program product 610 may thus provide means for performing any steps as herein disclosed.

[0108] In the example of Fig. 7, the computer program product 610 is illustrated as an optical disc. The computer program product 610 could also be embodied as a memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM) and more particularly as a non-volatile storage medium of a device in an external memory such as a USB (Universal Serial Bus) memory or a Flash memory, such as a compact Flash memory. Thus, while the computer program 620 is here schematically shown as a track on the depicted optical disk, the computer program 620 can be stored in any way which is suitable for the computer program product 610.

[0109] The inventive concept has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the inventive concept, as defined by the appended patent claims.

Claims

CLAIMS1. A hearing device (100) comprising: at least one microphone (1) configured to receive ambient audio; an audio signal processing unit (2) configured to receive an input audio signal comprising at least the ambient audio from the microphone and to generate an output audio signal based on the input audio signal; a receiver (3) configured to be supplied with the output audio signal; characterized in that the hearing device (100) further comprises at least an infrared, IR, sensor (4) configured to detect human presence (300) by detection of radiation emitted by a human body; and a controller (5), in communication with the infrared, IR, sensor (4) and with the audio signal processing unit (2), configured to adjust the input audio signal to the audio signal processing unit (2) and / or to adjust the output audio signal to the receiver (3), based on the detected human presence (300).

2. The hearing device (100) according to claim 1, wherein the controller (5) is configured to activate a noise cancelling function, to at least partially cancel an ambient component of the input audio signal to the audio signal processing unit (2), the ambient component being based on the received ambient audio, if no human presence (300) is detected; or to deactivate a noise cancelling function, to at least partially allow an ambient component of the input audio signal to the audio signal processing unit (2) based on the received ambient audio, if a human presence (300) is detected.

3. The hearing device (100) according to claim 1 or 2, wherein playback audio is transmitted by an audio source of the hearing device (100) or of a distinct playback device to the audio signal processing unit (2), and wherein the controller (5) is configuredto at least partially allow a playback component of the input audio signal to the audio signal processing unit (2) based on the received playback audio, if no human presence (300) is detected; or to at least partially cancel a playback component of the input audio signal to the audio signal processing unit (2) based on the received playback audio, if a human presence (300) is detected.

4. The hearing device (100) according to any one of claims 1 to 3, wherein controller (5) is configured to activate an audio steering function to adjust a directionality of the microphone (1), such that the microphone (1) picks up sound, or audio, preferentially from the direction in which the human presence (300) is detected.

5. The hearing device (100) according to any one of claims 1 to 4, wherein controller (5) is configured to detect human speech, for instance by selecting bands of frequencies typical of human speech, such as band-pass filtering or applying sound equalization techniques; and / or to reduce amplitude of frequencies outside bands typical of human speech.

6. The hearing device (100) according to any one of claims 1 to 5, wherein the at least an infrared, IR, sensor (4) is a thermopile or a bolometric sensor or a micro bolometric array sensor or a pyroelectric sensor.

7. The hearing device (100) according to any one of claims 1 to 6, comprising a lens or a transmissive optical element (41) configured to shape a field of view of the infrared, IR, sensor (4).

8. The hearing device (100) according to claim 7, wherein the lens (41) is of a material transparent to wavelengths of radiation emitted by a human body, preferably to wavelengths of about 12 micrometers.

9. The hearing device (100) according to claim 7 or 8, wherein lens (41) is made of Germanium and / or Zinc Selenide and / or Zinc Sulfide and / or Chalcogenide Glasses and / or Silicon.

10. The hearing device (100) according to any one of claims 1 to 9, wherein the at least an infrared, IR, sensor (4) actively detects human presence (300) conditional to the microphone (1) picking up human voice sound in the ambient audio.

11. The hearing device (100) according to any one of claims 1 to 10, wherein the at least an infrared, IR, sensor (4) is positioned in the hearing device so that the field of view(4o), FOV, thereof is directed in the direction of a nose of a user (200) of the hearing device; and / or wherein the at least an infrared, IR, sensor (4) is directed to the left and / or right and / or back of the user's head; and / or an array of infrared, IR, sensors (4) is positioned on the hearing device (100) such that the field of view (40) of each of the infrared sensors (4) in the array overlaps with the field of view of adjacent sensor (s) (4) in the array.

12. The hearing device (100) according to any one of claims 1 to 11, wherein the infrared, IR, sensor (4) is a single-pixel photodetector; or a multipixel detector.

13. The hearing device (100) according to any one of claims 1 to 12, comprising an on-the-ear; an over-the-ear or an at least partially in-the-ear headset; and / or comprising other wearable elements, preferably head-worn elements.

14. A method of adjusting an input audio signal to a hearing device (100), comprising(S13) detecting human presence (300) by detection of radiation emitted by a human body, by a at least an infrared, IR, sensor (4); and by a controller (5), in communication with the infrared, IR, sensor (4) and with the hearing device (100), adjusting (S16, S17) the input audio signal to the hearing device (100), based on the detected human presence (300).

15. The method of adjusting an input audio signal to a hearing device (100) of claim 14, wherein a microphone (1) of the hearing device (100) is configured to receive ambient audio; an audio signal processing unit (2) of the hearing device (100) is configured to receive an input audio signal comprising at least the ambient audio from themicrophone (1) and to generate an output audio signal based on the input audio signal; and a receiver (3) of the hearing device (100) is configured to be supplied with the output audio signal; the method comprising:(S13) detecting human presence (300) by detection of radiation emitted by a human body, by the at least an infrared, IR, sensor (4); and by the controller (5), in communication with the infrared, IR, sensor (4) and with the audio signal processing unit (2), adjusting (S16, S17) the input audio signal to the audio signal processing unit (2) and / or adjusting the output audio signal to the receiver (3), based on the detected human presence (300).

16. The method of claim 15, further comprising by the controller (5), detecting ambient audio received by the microphone (1), and based on detection of the ambient audio, activating, or keeping active, a noise cancelling function (si6), to at least partially cancel an ambient component of the input audio signal to the audio signal processing unit (2) based on the received ambient audio, if no human presence (300) is detected; or deactivating a noise cancelling function (S17), to at least partially allow an ambient component of the input audio signal to the audio signal processing unit (2) based on the received ambient audio, if a human presence (300) is detected.

17. The method of claim 15 or 16, further comprising by the controller (5),detecting, in the audio and / or in the input signal based thereof which is received by the audio signal processing unit (2), playback audio, transmitted by an audio source of the hearing device or of a distinct playback device, and at least partially allowing processing of a playback component of the input audio signal to the audio signal processing unit (2) based on the received playback audio, if no human presence (300) is detected; or at least partially cancelling processing of a playback component of the input audio signal to the audio signal processing unit (2) based on the received playback audio, if a human presence (300) is detected.

18. The method of any one of claims 15 to 17, comprising activating an audio steering function to adjust a directionality of the at least one microphone (1), such that the microphone (1) picks up sound, or audio, preferentially from the direction in which the human presence (300) is detected.

19. The method according to claim 18, wherein the position of the human presence (300) is calculated based on the positioning on the hearing device (100) of the at least an infrared, IR, sensor (4) measuring the human presence (300); or, if an array of infrared, IR, sensors (4) is employed to carry out the detection of human body radiation, the position of the human presence (300) is calculated based on the positioning on the hearing device (100) of the infrared sensors (4) of the array measuring the human presence (300).

20. The method according to any one of claims 15 to 19, comprising, when human presence (300) has been detected by detection (S13 - S15) of radiation emitted by a human body, band-pass filtering frequencies typical of human speech, in order to enhance speech recognition; and / or reduce amplitude of frequencies outside typical human speech.

21. The method according to any one of claims 15 to 20, wherein detecting (S13) human presence (300) by detection of radiation emitted by a human body, is carried out by any one, or a combination, of: a thermopile or a bolometric sensor or a micro bolometric array sensor or a pyroelectric sensor.

22. The method according to any one of claims 15 to 21, comprising shaping a field of view (40), FOV, of the infrared, IR, sensor (4) by a lens (41) transparent to wavelengths of radiation emitted by a human body, preferably to wavelengths of about 12 micrometers.

23. The method according to any one of claims 15 to 22, wherein actively detecting (S13) human presence by the at least an infrared, IR, sensor (4) is conditional to a distinct sensor picking up a trigger, such as the microphone (1) picking up (sn) human voice sound in the ambient audio.

24. The method according to claim 23 when dependent from claim 16, wherein the microphone (1) of the hearing device (100) receives ambient audio continuously and the noise cancelling function is kept activated (S16) until the controller (5) detects (Sn) human voice sound in the received ambient audio.

25. A computer program (620) for adjusting an input audio signal to a hearing device (100), the computer program comprising computer code which, when run on processing circuitry (510) of a controller (5), causes the controller (5) to: instruct an infrared, IR, sensor (4), in communication with the controller (5), to detect (S13) human presence (300) by detection of radiation emitted by a human body, and adjust (S16, S17) the input audio signal to the hearing device (100), based on the detected human presence (300).

26. A computer program product (610) comprising a computer program (620) according to claim 25, and a computer readable storage medium (630) on which the computer program is stored.

Images