Event triggered audio forwarding arrangement and wireless communication device

Abstract

The invention relates to an event triggered audio forwarding arrangement (500) comprising a voice assistant device (502) configured to connect with a remote caregiver. The arrangement comprises: a first wireless communication device (200) having a speaker (202) arranged at a first location at a first distance from a microphone (504) of a voice assistant device (502); a second wireless communication device (100) having a microphone (102) arranged at a second position at a second distance, and comprising an event ascertaining unit (104) configured to determine an occurrence of a body event associated with the person (101), and an audio data forwarding unit (106) configured to forward audio data to the first wireless communication device (100), the second wireless communication device (100) being arranged at a second position at a second distance, and the audio data forwarding unit (106) being configured to forward audio data to the second wireless communication device (100). An audio data forwarding unit (106) is configured to record a first audio signal (A1) and to provide first audio data (AD1) to a first wireless communication device configured to play a second audio signal (A2) indicative of the first audio data (AD1) to be picked up by a voice assistant device, thereby increasing its effective spatial hearing range.

Description

Technical Field

[0001] The present invention is directed to a wireless communication device, an event-triggered audio forwarding arrangement, a corresponding method for controlling the operation thereof, and a computer program. Background Art

[0002] Document US2021 / 0160675 A1 describes a method, system, and apparatus for smart speaker phone emergency monitoring (e.g., for fall detection). The method includes detecting sounds at one or more multiple locations on a property, determining that the sounds indicate an emergency, sending an indication of the emergency to a monitoring station or call center, receiving a two-way voice call from the monitoring station, and broadcasting the two-way call to the one or more multiple locations. Each speaker phone is configured to communicate with a hub device using Digital Enhanced Cordless Telecommunications (DECT) signals. Summary of the Invention

[0003] According to the Centers for Disease Control and Prevention, falls as a type of physical event are one of the leading causes of injury and death among adults over 65 years old. Inconsistent wearing of monitoring wearable devices by seniors requires automated systems for fall prevention and fall detection. Leading IoT sensing companies in the fields of senior care and fall detection have concluded that their AI-driven fall detection services require human-in-the-loop. This is understandable, considering that there are many types of falls, some of which are difficult to detect even for advanced AI algorithms. Given the need for human-in-the-loop to make decisions about fall prevention and / or fall detection, voice assistant devices (such as, for example, Amazon Echo devices) are becoming increasingly common in homes, and major players in the field of senior care are introducing voice assistant speakers as part of their senior care solutions. In the event of an emergency (such as a fall), a user can interact with the voice assistant device to communicate with a remote caregiver (such as an external call center, a monitoring station, or a family member or acquaintance of the user). A typical arrangement of only two voice assistant devices per home will generally not work, especially if the person has fallen in a location in the house or apartment that is outside the significant audio coverage or hearing range of the voice assistant device, i.e., the spatial range in which a sound of a given intensity can be recorded by a microphone with at least a predetermined quality. However, including corresponding voice assistant devices in more rooms (e.g., bedroom, kitchen, living room, bathroom, etc.) significantly increases the cost of the elder care solution.

[0004] It would be beneficial to increase the operational reach of an event monitoring arrangement connected to a remote caregiver, such as an external call center or monitoring station, while minimizing incremental costs.

[0005] A first aspect of the invention is formed by an event-triggered audio forwarding arrangement according to claim 1 .

[0006] An event-triggered audio forwarding arrangement (also referred to as the arrangement) includes a voice assistant device including a microphone and a speaker and configured to connect to a remote caregiver that is external to the arrangement. The remote caregiver may include, for example, a call center or a monitoring station. The connection is achieved via a communication channel.

[0007] The arrangement also includes a first wireless communication device including a speaker, the first wireless communication device being arranged at a first distance from the microphone of the voice assistant device.

[0008] The arrangement also includes one or more second wireless communication devices including a microphone, the second wireless communication devices being arranged at a second distance from the microphone of the voice assistant device.

[0009] The second wireless communication device includes: an event ascertaining unit, which is configured to ascertain the occurrence of a predefined physical event related to a person; and an audio data forwarding unit, which is configured to operate a microphone to record a first audio signal, and when the predefined physical event is ascertained, provide first audio data indicating the first audio signal to the first wireless communication device via an audio data communication channel. In the arrangement of the first aspect of the present invention, the first wireless communication device is configured to operate its speaker to play a second audio signal upon receiving the first audio data, and the second audio signal indicates the first audio data to be picked up or recorded by the microphone of the voice assistant device. The voice assistant device is also configured to provide second audio data indicating the second audio signal to an external caregiver via a communication channel.

[0010] Thus, the event-triggered audio forwarding arrangement of the first aspect of the present invention effectively extends the effective spatial hearing range or sound pickup area of a voice assistant device by enabling an audio data communication channel between two wireless communication devices, wherein one of the communication devices (i.e., the first wireless communication device) is arranged at a first position at a first distance from a microphone of the voice assistant device and is effectively arranged within the sound pickup area of the microphone of the voice assistant device (e.g., in the same room, or at a relatively short distance), and the other communication device (i.e., the second wireless communication device) is arranged at a second position at a second distance from the microphone of the voice assistant device and is therefore effectively arranged outside the sound pickup area. The sound pickup area can be defined as a spatial area or volume in which sounds or audio signals having a predetermined sound intensity (e.g., 60 dB, or a normal conversation level) can be captured or recorded by the microphone of the voice assistant device with at least a given signal-to-noise ratio or a predefined sound quality. The sound pickup area of the voice assistant device can be predetermined. For example, a user can determine which wireless devices are within the sound pickup area of the voice assistant device, wireless devices can determine whether they are within the sound pickup area by, for example, playing a test sound, the roles of the wireless devices (i.e., whether they are within the sound pickup area) can be stored in a memory, and so on. The first distance and the second distance (or, in other words, the first and second positions) are such that the first sound intensity picked up by the microphone of the voice assistant device from a reference sound signal of a given intensity generated at the first distance (or first position) is higher than the second sound intensity picked up by the microphone of the voice assistant device from a reference sound signal of the same given intensity generated at the second distance (or second position). Typically, the first distance is shorter than the second distance, such that the first wireless communication device is closer to the voice assistant device than the second wireless communication device. However, and depending on the spatial orientation of the microphone and speaker and on the environment (e.g., the presence of objects (such as walls)), the second wireless communication device may in some cases be closer to the voice assistant device than the first communication device, i.e., the second distance is shorter than the first distance. A more relevant aspect is that a predetermined sound (referred to as a reference sound and having a predefined intensity) is recorded or picked up by the microphone of the voice assistant device with a higher intensity or quality or a lower signal-to-noise ratio when provided at the location of the first wireless communication device than when the same reference sound is provided at the location of the second wireless communication device.

[0011] Therefore, the spatial extension of the sound pickup area depends on the sound intensity, the sensitivity of the microphone, and the environment (e.g., the presence of walls between the sound emitter and the microphone). Of course, the louder the sound, the larger the sound pickup area. However, the sound pickup area is preferably defined as a sound intensity that is realistic for domestic use (e.g., between 30 and 80 dB) and is used in a given environment (home, nursing home, warehouse, etc.).

[0012] The audio data forwarding unit operates the microphone of the second wireless communication device to record sounds (referred to as audio signals) originating from its vicinity, which may include, for example, a person calling for help or a person groaning in pain. After the event ascertainment unit ascertains an event (referred to as a physical event of a person and may include movement, particularly a fall, posture, or vital signs such as breathing rate or heartbeat), an audio data communication channel with the first wireless communication device is enabled or established for providing the recorded first audio signal to the first wireless communication device in the form of first audio data. The audio data forwarding unit is therefore configured to generate audio data indicating the recorded audio signal. The first audio data is typically unprocessed with respect to the second wireless communication device's voice recognition. In other words, unlike some voice assistant devices, the second wireless communication device does not perform any voice recognition, but instead records the original audio data or a compressed version of the original audio data and forwards the original audio data or a compressed version of the original audio data to the first wireless communication device. Furthermore, upon receiving the first audio data, the first wireless communication device is configured to operate its microphone and play the first audio data so that it is picked up or recorded by the microphone of the voice assistant device. Thus, a second audio signal related to the first audio signal from the user is picked up by the microphone of the voice assistant device and provided to the remote caregiver via the communication channel. As possible examples of remote caregivers, an operator of a call center or monitoring station, or a family member or supervisor then becomes aware of what is happening in the vicinity of a second wireless communication device, which is not within the original sound pickup area of the voice assistant device. Therefore, the event-triggered audio forwarding arrangement, in addition to requiring a reduced number of voice assistant devices, also enhances the privacy of the environment because audio data is only forwarded after a predefined event has been detected. Furthermore, not all wireless communication devices have to be directly connected to the remote caregiver. Furthermore, the wireless communication devices can utilize very lightweight microcontrollers because they only need to capture audio or replay audio, but unlike the voice assistant device, do not need to understand wake-up words or perform audio analysis.

[0013] In the context of this specification, an audio signal refers to a sound wave in the audible frequency range, which is generated by a user in the case of a first audio signal, or by a loudspeaker in the case of a second audio signal. Audio data refers to an encoded data signal indicative of the audio signal, which can then be transmitted over a wired or wireless communication link according to a predetermined communication protocol.

[0014] In the following, embodiments of the event-triggered audio forwarding arrangement of the first aspect of the invention will be described.

[0015] In a preferred embodiment, the event-triggered audio forwarding arrangement is configured as a fall-triggered or gesture-triggered audio forwarding arrangement, wherein the event ascertaining unit is configured to ascertain event data indicative of a fall or a predetermined gesture of a person.

[0016] In a preferred embodiment, the event ascertainment unit, configured as a fall or gesture ascertainment unit, comprises a radio frequency-based fall or gesture sensing arrangement, or receives fall or gesture data from a radio frequency-based fall or gesture sensing arrangement, which does not rely on audio input to determine the occurrence of a fall or gesture and thus respects the privacy of conversations within an environment (e.g., a house, apartment, etc.). The radio frequency-based fall or gesture sensing arrangement is configured to determine a fall or gesture, or any other predetermined activity including vital signs, by using signal data indicative of one or more known signal parameters that may be associated with a predefined event, such as, for example, a received signal strength indicator of channel state information. The radio frequency-based fall or gesture sensing arrangement may be part of a broader activity sensing arrangement that is advantageously configured to detect not only the occurrence of a fall or gesture, but also the presence, movement, gait, posture, or vital signs (such as heartbeat or breathing rate) of a user within a given sensing volume.

[0017] Preferably, in one embodiment, the audio data communication channel between the first wireless communication device and the second wireless communication device is based on a wireless communication protocol, in particular WiFi, or a power line communication protocol or a LiFi communication protocol.

[0018] In a preferred embodiment, the communication channel between the audio assistant device and the remote caregiver is a digital communication channel, wherein digital signals including audio data are transmitted via said communication channel. In an alternative embodiment, the communication channel is an analog communication channel.

[0019] In one embodiment, the second wireless communication device is configured to record audio signals during a time span of a predetermined duration upon detecting a fall, and to provide first audio data once the predetermined duration has elapsed. In this embodiment, the audio data is not provided in real time or near real time. Additionally or alternatively, in another embodiment, the audio forwarding unit is configured to record the first audio signals and, upon detecting the occurrence of a predefined physical event, provide audio data to the first wireless communication device, the audio data indicating the audio signals recorded during a first time span prior to the occurrence of the predefined physical event (e.g., a fall, a posture, a movement, a vital sign indicating a potential risk, etc.) upon detecting the occurrence of the fall event. For example, a microphone continuously records audio, and once an event, such as a fall or a predetermined posture, has been detected or data indicating the event has been received, the audio data forwarding unit provides audio data indicating the audio signals recorded during the predetermined time span prior to the occurrence of the event. Thus, the recorded audio is transmitted to the remote caregiver, who can better assess the situation by listening to what occurred before the event occurred. Additionally or alternatively, the audio forwarding unit is configured to record the first audio signal and, upon ascertaining the occurrence of the predefined physical event, provide audio data to the first wireless communication device, the audio data indicating the audio signal recorded during a second time span after ascertaining the occurrence of the predefined physical event.

[0020] In one embodiment, the audio data communication channel established between the first and second wireless communication devices may be a dedicated audio data communication channel for audio data, or a shared communication channel where transmission of audio data uses the same channel as other types of data provided by the wireless communication devices.

[0021] Also preferably, in another embodiment, the first wireless communication device and the second wireless communication device are wirelessly controllable lighting devices of a wirelessly controllable lighting arrangement. In particular, the first and second wireless communication devices are lighting devices or luminaires that can be wirelessly controlled via a router, a hub, a bridge, or any other suitable control device that provides wireless signals including instructions or status data for operating the lighting arrangement. Non-limiting examples of such lighting arrangements are Philips Hue systems or Wavelinx systems used in nursing homes. The first and / or second wireless communication devices can also be other devices of the lighting arrangement in addition to lighting devices, such as, for example, associated sensors and / or switches.

[0022] The inventors have recognized that the density of wireless communication devices (e.g., WiFi-equipped devices) in a lighting arrangement is typically much higher than the density of voice assistant devices. Accordingly, an exemplary event-triggered audio forwarding arrangement comprising a wirelessly controllable lighting arrangement uses a first luminaire having a microphone and located near the suspected fall location, which transmits an audio stream as audio data via a WiFi radio signal to a second luminaire including a speaker and located near the voice assistant device. The second luminaire plays the audio stream via its speaker, and the audio is captured by the voice assistant device. Thus, a remote caregiver (e.g., staff at an external call center or monitoring station, or a family member or acquaintance of the user) will be aware of the situation of a person who has fallen, and will be able to communicate with the person.

[0023] It is worth noting that compared to adding a wireless communication device (such as a WiFi lighting device), the basic ability to record and relay audio signals in the form of audio data between two devices and have the audio signals replayed by the receiving device is much less expensive and computationally intensive than integrating full voice assistant functionality (including voice recognition) into each wireless communication device. The wireless communication device (particularly the wireless controllable lighting device) of this arrangement does not need to understand the meaning of the audio message or even recognize the individual words. In addition, the wireless communication device can have a memory for buffering audio data. For example, 20 seconds after no fall is detected, the stored audio data is overwritten in the memory.

[0024] In one embodiment, the event-triggered audio forwarding arrangement includes fall detection functionality, particularly radio frequency sensing functionality, which uses signal parameters such as channel state information (CSI) or received signal strength indicator (RSSI) to infer the location or activity of objects within the sensing volume. For example, embodiments of event-triggered audio forwarding arrangements based on Zigbee sensing typically use RSSI because CSI data becomes unavailable. Embodiments of arrangements using WiFi chips can use both CSI and RSSI data. The sensed activity can include a fall, which can be included in one of the following fall categories:

[0025] Trip and fall: This is a rapid fall from a walking position to the ground.

[0026] Falls occur when entering or exiting a chair (static chair or wheelchair). Thus, this is a rapid fall from a sitting position to the ground.

[0027] Soft falls: The person manages to grab onto furniture to (somewhat) slow the fall to the floor.

[0028] Stroke falls: First, fall slowly from a standing position to one knee and then fall to the ground.

[0029] Floor fall: A very long fall (up to two minutes!) that occurs when an older adult tries to pick something up from the floor and does not have the strength to get back up and ends up falling slowly to the ground.

[0030] Other types of activity that may be sensed are described below.

[0031] Thus, in one embodiment, the fall ascertainment unit uses fall detection functionality, in particular radio frequency-based fall detection functionality, to determine or detect whether a fall has occurred. In an alternative embodiment, the event-triggered audio forwarding arrangement receives a fall indication signal from an external device, which is received by the event ascertainment unit and indicates a fall of a person.

[0032] In another embodiment, the event ascertainment unit is further configured to ascertain (in other words, determine or receive) event data indicating that an event (e.g., a fall, movement, posture, a vital sign indicating a risk) has occurred within or outside a predetermined sound pickup area of the microphone of the voice assistant device. In this embodiment, the audio data forwarding unit is configured to operate the microphone to record a first audio signal and provide first audio data indicating the first audio signal via an audio data communication channel when it is ascertained that the fall has occurred outside the sound pickup area. Thus, if the fall location is considered to be within the sound pickup area of the voice assistant device, the audio forwarding unit is not activated to record the audio signal and provide the corresponding audio data to the first wireless communication device. However, audio data related to the context before the fall event can be forwarded by the communication device to the voice assistant device for the fall event itself and immediately after the fall (before two-way communication with the call center has been established). Forwarding of the fall context can be performed after the elderly person has been taken to the hospital (because the data is relevant to future fall prevention rather than organizing emergency care). When forwarding audio data about the context of a fall to the voice assistant, the communication device can add preamble work to wake up the voice assistant and inform it that the audio data it is about to receive is a historical record of the fall event.

[0033] In an exemplary embodiment, the communication device can continuously record audio signals and generate audio data, but if no predefined event (e.g., a fall or gesture) is detected, the audio data is deleted after a predetermined time (such as after 30 seconds). Similarly, the audio file can be recorded and stored just after the event has been detected, for example before a communication channel (preferably a two-way communication channel) has been established between the person and the remote caregiver. Therefore, in a further developed embodiment, the audio forwarding service can also be used to forward historical audio streams based on the detection of an event, where the historical audio stream is related to the event (e.g., immediately before the event, simultaneously with the event, or after the event). Preferably, the forwarded audio stream is not processed by the lighting device, i.e., there is no speech recognition. The audio stream is not necessarily part of a conversation between the remote caregiver and the user. To this end, a time period before the event is selected and the audio data of the selected time period is forwarded to allow the caregiver to infer the context (cause) of the event.

[0034] In order to infer the context (e.g., cause or origin) of an event (e.g., a fall) from historical audio data recorded by a wireless communication device, a time span prior to the fall is deliberately selected and the audio data for the selected time span is forwarded. This audio data will enable the caregiver to diagnose what caused the fall (e.g., the person got up from the bed without assistance) and what happened (e.g., the person hit the floor very hard and cursed in pain after the fall) before two-way communication was established.

[0035] In an embodiment configured as a fall-triggered audio forwarding arrangement and using, for example, WiFi sensing data, the probability of each fall type is determined. For example, for the first fall event, the fall detection algorithm may infer the highest probability of trips and falls (e.g., over 90%). For trips and falls, because the context of the fall (i.e., the elderly person's activities prior to the fall) is less relevant to preventing future falls, a short time span (e.g., 1-5 seconds) is selected so that the transmitted audio data begins only 1 to 5 seconds before the fall. However, because trips and falls are serious fall events that may cause significant pain and / or injury, audio data obtained after the fall within another predetermined time span (e.g., 1-5 minutes) after the fall event is also transmitted. This allows the caregiver to hear the person's prolonged expressions of pain, conversations, cries for help, swearing, etc. It also helps the caregiver determine how long it takes the person to get up again after, for example, they tell the caregiver that they do not need any help.

[0036] For the second fall, the fall detection algorithm can infer the highest probability of a soft fall. For a soft fall event, the context of the fall (i.e., the person's activities before the fall) may be relevant to preventing future fall events, or may not be relevant to preventing future fall events. A soft fall is defined as the person trying to grab furniture to (to some extent) slow down the fall to the floor or fall onto the sofa. However, a soft fall may be caused by tripping over an object on the floor (a context that is not very relevant to the caregiver), or may be caused by the person becoming dizzy while lacing his shoelaces (a context that is very relevant to the fall prevention system). Therefore, the transmitted audio file is selected to start a moderate time span (e.g., one minute) before the fall. However, since a soft fall may not cause obvious pain and / or injury, we only transmit audio data indicating audio data a few seconds (e.g., 5 to 10 seconds) after the fall event. Alternatively, the transmission of audio data can be adaptively stopped once the person has gotten up again after the soft fall. Since the person is most likely uninjured, there is little value in forwarding and monitoring the audio data after the person is already up again.

[0037] For the third fall, the fall detection algorithm can infer the highest probability of falls associated with entering and exiting a chair (static chair or wheelchair). It is believed that 41% of falls in the frail elderly population are caused by inappropriate sit-to-stand transitions. For chair falls, the context of the fall (i.e., the elderly's activities before the fall) is highly relevant to preventing future fall events. For example, the elderly person falls from a chair because she had fallen asleep a few minutes ago. Therefore, as contextual information, it is important to know whether the elderly person falls because he or she is trying to get up after sitting in a chair for a long period of time and thus has low blood pressure (it is also known that one of the factors that increases the risk of falls in the elderly is low blood pressure associated with changes in the elderly person's body position. This effect is called orthostatic hypotension and occurs when your blood pressure drops when you go from lying down to sitting up or from sitting to standing.

[0038] For chair falls, the transmitted audio files are selected to start a long time span (e.g., 30 to 60 minutes) before the fall. Since chair falls may or may not result in injury, the transmission of audio data can be stopped at a moderate time after the fall (e.g., 1 to 5 minutes, such as 2 minutes). Preferably, once the elderly person has stood up again after the chair fall, the transmission of audio data is non-adaptively stopped. Since the elderly person may not be injured, we monitor the pain symptoms (e.g., audible pain when walking) after the elderly person stands up again.

[0039] For the fourth fall, the fall detection algorithm can infer the highest probability of the fall associated with stroke. This is first falling slowly from a standing position to kneeling on one knee, and then falling to the ground. Since this type of fall is not related to the activities carried out by the elderly before the fall, we therefore choose to start transmitting audio files a short time (e.g., 5 to 15 seconds, such as 10 seconds) before the fall. Since stroke falls are related to medical emergencies, audio signals are recorded and audio data is forwarded until an ambulance has arrived. Preferably, the interaction with medical staff is also recorded until the room is vacated.

[0040] For the fifth fall, the fall detection algorithm can infer the highest probability of the fall associated with picking up something from the floor. This type of fall is a very long fall (up to two minutes) that occurs when the elderly try to pick up something from the floor and have no strength to get up again and eventually fall slowly to the ground. For chair falls, the context of the fall (i.e. the elderly's activities before the fall) is highly relevant to preventing future fall events. For falls from picking up something from the floor, we therefore select the audio file of the transmission of the fall event to start a considerable time span (e.g., 5 to 15 minutes, such as 10 minutes) before the fall. This makes it possible to infer what activities the elderly were doing before the fall. Since chair falls may or may not cause injury, the transmission of audio data can be stopped after a moderate time span (e.g., 1 to 5 minutes) after the fall.

[0041] For the sixth fall, the fall detection algorithm can infer the highest probability of the fall associated with tying shoelaces or a dementia patient playing with something on the floor for a long period of time. For example, it is known that dementia patients like to play with extension cords on the floor, and when bending forward for a long time to reach the cable, they become dizzy. Therefore, the audio file (or audio data) transmitted is selected to start a moderate time span (e.g., 2 to 10 minutes, such as five minutes) before the fall. Since this type of fall may or may not cause injury, the transmission of audio data can be stopped after a moderate time span (e.g., 1 to 5 minutes, such as two minutes) after the fall.

[0042] In another embodiment, the event-triggered audio forwarding arrangement of the first aspect of the present invention further includes an auditory range inference unit configured to: ascertain contextual information related to the current state of the environment in which the event-triggered audio forwarding arrangement is arranged; and use the ascertained contextual information to determine whether a given wireless communication device is within the current sound pickup area of the voice assistant device. This enables the current sound pickup area to be inferred relative to the location of the event (e.g., a suspected fall) and not necessarily relative to the location of the second wireless communication device. The contextual information is related to the current state of the environment (e.g., a house or apartment or a relevant part thereof). For example, the contextual information may indicate whether a door is open or closed, which affects the current sound pickup area of the voice assistant device. In another example, the contextual information may be provided by a user, for example, a user may identify which devices are in the sound pickup area of the voice assistant device, or a user may provide location data indicating the location of the wireless device in a space (e.g., a room), and the auditory range inference unit may use the location data to determine the sound pickup area. Similarly, and using recent historical or real-time sensor data, it can be assessed whether the microphone of the second wireless communication device can record or hear sounds from the room where the first wireless communication device and / or voice assistant device is located. If so, the event-triggered audio forwarding arrangement concludes that the user may be within the sound pickup area of the voice assistant device. In some cases, after a person has fallen, he or she may only be able to speak with a weaker voice than usual, and therefore the voice assistant device may no longer be within the sound pickup area of the person who fell, so the event-triggered audio forwarding functionality is still needed. The hearing range inference unit may also include a radar unit or a time-of-flight sensing unit configured to ascertain at least a portion of the contextual information. Typically, the first wireless communication device should be avoided from repeating what the user said when the user's voice can be heard directly at the voice assistant device, as this will cause interference. If some spoken audio signals from the user are still heard at the voice assistant, the event-triggered audio forwarding arrangement can advantageously be configured to detect this fact and provide instructions to the first wireless communication device to increase the volume of the audio signal played by the speaker to prevail over the direct path audio between the user and the voice assistant device, so that the voice assistant device prioritizes the audio signal from the wireless communication device over the weaker human voice.

[0043] Alternatively or additionally, in another embodiment, the event ascertainment unit is further configured to ascertain whether an event (such as a fall) has occurred within or outside a predefined sound pickup area of the microphone of the voice assistant device, and the audio data forwarding unit is configured to operate the microphone of the second wireless communication device to record the first audio signal, and provide the first audio data via the audio data communication channel only if the event ascertainment unit has indicated that the event has occurred outside the predefined sound pickup area.

[0044] Preferably, in another embodiment, the first wireless communication device further comprises a microphone and an audio data forwarding unit for providing response audio data indicative of a corresponding response audio signal recorded by its microphone. In addition, the second wireless communication device further comprises a speaker configured to play a second response audio signal indicative of received response audio data recorded by and received from the first wireless communication device. In this embodiment, the event-triggered audio forwarding arrangement is further configured to enable a two-way audio data communication channel between the first wireless communication device and the second wireless communication device for exchanging audio data therebetween upon ascertaining event data indicative of a predetermined physical event (fall, posture, etc.). In this way, a call center or monitoring station is able to conduct a two-way conversation to verify with the person who has fallen whether he or she is in good condition or needs help.

[0045] In an exemplary embodiment, multiple second wireless communication devices are included near the location of the identified incident. It may be advantageous to utilize one of the communication devices to record an audio signal from the person and forward it to the first wireless communication device, while another communication device is used to transmit a response audio signal (from the caregiver via the voice assistant device and the first wireless communication device) to the person lying on the floor. The second wireless communication devices (e.g., two lighting devices) are located in the room where the person has fallen and may be equipped with low-cost microphones and speakers, which may reduce interference between the speakers and microphones.

[0046] Similarly, this arrangement will distribute the computational load for bidirectional communication between a first lighting device recording an audio signal and a second lighting device reproducing the response. This is advantageous because lighting devices are typically computationally resource-constrained (particularly if the downlight is currently performing another task, such as rendering a dynamic light effect or performing an over-the-air firmware update).

[0047] In another embodiment of the event-triggered audio forwarding arrangement of the first aspect of the present invention, the voice assistant device is configured to enable or establish a communication channel for providing second audio data to a remote caregiver upon recognition of a predetermined wake-up word or trigger word, i.e., a predetermined or pre-agreed word, phrase, or command that, when recognized by the voice assistant device, activates a communication channel with the remote caregiver, call center, or monitoring station. In this particular embodiment, the second wireless communication device includes a word recognition unit configured to recognize the predetermined wake-up word, and an audio data forwarding unit configured to further enable or establish an audio data communication channel upon recognition of the predetermined wake-up word by the word recognition unit. In one embodiment, recognition of the predetermined or pre-agreed wake-up word is performed by an edge-AI audio analysis unit included in the second wireless communication device. In one embodiment, upon ascertaining fall data indicative of a fall, audio forwarding functionality for providing recorded audio data to the first wireless communication device is activated depending on a response or non-response to a pre-recorded audio message played by the second wireless communication device. For example, in one embodiment, the audio forwarding functionality is activated when the user fails to respond to the pre-recorded audio message in a pre-agreed manner. In another embodiment, the audio forwarding functionality is activated when the user responds affirmatively with a pre-agreed wake-up word or phrase to a pre-recorded audio message regarding whether the user needs assistance. The pre-agreed wake-up word can be easily identified using a lightweight edge-AI algorithm that is applicable to common commodity microcontrollers used in wireless communication devices (such as WiFi lighting devices).

[0048] For example, in one embodiment, the response audio signal is enabled to be transmitted before the voice assistant device has picked up the wake word. This allows the caregiver to ask the person about their current situation, but the channel from the second wireless communication channel to the remote caregiver (via the first wireless communication device and the voice assistant device) is not activated until the user has spoken the wake word.

[0049] In one embodiment, a second wake-up word is introduced to indicate that the user does not want to start the audio channel. Although the system can automatically activate the audio data communication channel to the first wireless communication device and then automatically activate the audio data communication channel to the remote caregiver after a predetermined waiting time (e.g., two minutes) without receiving the wake-up word (because the user may have forgotten the wake-up word), if the user has already spoken the second wake-up word (which indicates that he does not want to transmit his audio data to the caregiver), the system will not activate the audio data communication channel.

[0050] Optionally, in one embodiment, the second wireless communication device is configured to determine whether the user's attempt to contact the voice assistant device was unsuccessful, for example, where the sound level of the words spoken by the user is too low to be recorded by the voice assistant device.

[0051] In a particular embodiment, wherein the voice assistant device is configured to enable or establish a communication channel for providing second audio data to a remote caregiver upon recognizing a predetermined wake-up word, the first wireless communication device, upon receiving the first audio data, is configured to provide a pre-recorded wake-up word audio signal via its speaker, the pre-recorded wake-up word audio signal including the predetermined wake-up word when played.

[0052] For example, in the event that a user is unfortunately no longer able to talk after a suspected fall event, an embodiment of the event-triggered audio forwarding arrangement of the present invention is configured to record faint audio indicating that the user is in pain and forward the audio data indicative thereof to the first wireless communication device. The first wireless communication device is advantageously configured to add a synthesized or pre-recorded wake-up word preamble to the audio data for activation of the voice assistant device. In one embodiment, the first wireless communication device is configured to determine whether the audio encoded in the audio data includes a wake-up word and, if not, to add a pre-recorded or synthesized wake-up word preamble.

[0053] In another embodiment, the event-triggered audio forwarding arrangement is also configured as a radio frequency-based event detection arrangement. To this end, in the event-triggered audio forwarding arrangement, the audio data communication channel between the first wireless communication device and the second wireless communication device is based on a wireless communication protocol, such as WiFi, Bluetooth, Zigbee, Thread, LoRa, or any other suitable wireless communication protocol. The arrangement also includes an event or activity sensing unit that is configured to use the wireless channel of the wirelessly transmitted audio data to determine wireless signal parameters related to activities (events) of objects within a sensing volume defined by the relative positions of the first wireless communication device and one or more second wireless communication devices, and provide activity data indicative of the wireless signal parameters.

[0054] Thus, in this particular embodiment, the forwarding of audio data in the form of radio frequency signals (preferably bidirectional forwarding (i.e., from the second wireless communication device to the first wireless communication device, but also from the first wireless communication device to the second wireless communication device)) is coordinated so that the wireless signals sent to forward the audio data are also used for RF-based sensing of the subject's activity, such as for monitoring the position, movement, posture, or vital signs of a fallen user. Signal transmission parameters (frequency, directionality, signal strength, etc.) can be varied to better perform the activity sensing functionality.

[0055] In another embodiment, the voice assistant device further comprises an event ascertainment unit configured to receive event data indicating a physical event of the person, and upon ascertaining the occurrence of a predetermined event (such as a fall, a predetermined posture, a vital sign indicating a risk) using the event data, the voice assistant device is configured to initiate a communication channel with a remote caregiver or monitoring station. The event-triggered audio forwarding arrangement is configured to forward the audio message reproduced by the voice assistant device to the wireless communication device closest to the location of the fall to ensure that no response from the user is not due to the user being out of audio range of the voice assistant device.

[0056] Non-limiting examples of predefined physical events and related interactions with the remote caregiver may include the following:

[0057] a) falls, such as any of the above types of falls;

[0058] b) gait posture that indicates an increased risk of falls (the remote caregiver can then utilize an event-triggered audio forwarding arrangement to communicate with the person (e.g., an elderly person) to try to minimize the risk of falls, such as asking the person to sit down);

[0059] c) Sitting up in bed: A person (such as an elderly or sick person who should not be able to walk on his own) sits up in bed during the night. This indicates that the person may have tried to get up from the bed to go to the toilet during the night. The remote caregiver can then use the event-triggered audio forwarding arrangement to communicate with the person and find out if help or assistance is needed;

[0060] d) A person is bent over for an extended period of time, such as when tying their shoes. This indicates that the person does not have the strength to straighten up again (and is therefore experiencing a prolonged fall event). The remote caregiver can then utilize an event-triggered audio forwarding arrangement to communicate with the person, for example to ask the elderly person to maintain the bent position as help is on the way):

[0061] e) A person is performing a prohibited or not recommended activity that is considered risky to that person, such as bending over to play with cables on the floor, doing push-ups on the floor, or lowering themselves to the floor;

[0062] f) A person is physically interacting with another person, such as physically fighting with another person (e.g., in a senior citizen's home).

[0063] g) A person is showing abnormal vital signs, e.g. as recorded from a distance by an RF-based sensing arrangement. For example, if a high respiratory rate is detected, the remote caregiver can then utilize an event-triggered audio forwarding arrangement to communicate with the person and, for example, check whether the person is doing some physical exercise or is feeling unwell.

[0064] h) A person suffering from dementia or otherwise incapacitated is attended to by a second person. The remote caregiver can then remotely check in using an event-triggered audio forwarding arrangement to monitor for possible abuse of that person.

[0065] i) A person (e.g. an elderly person) is sitting on the toilet and stays there for a long time. The remote caregiver can then use the event-triggered audio forwarding arrangement to communicate with the person and ask what happened.

[0066] j) A person may exhibit a lack of physical movement, such as sitting in a chair for a long time. The caregiver can then communicate with the person using an event-triggered audio forwarding arrangement, for example, and advise against getting up (as this may lead to a high risk of dizziness and falls).

[0067] A second aspect of the present invention is formed by a wireless communication device for use in an event-triggered audio forwarding arrangement according to the first aspect, in particular as a second wireless communication device. The wireless communication device comprises: a microphone; an event ascertainment unit configured to ascertain the occurrence of a predefined physical event associated with a person (such as a posture, a fall, a movement, a vital sign indicating a dangerous situation, etc.); and an audio data forwarding unit configured to operate the microphone to record a first audio signal and, upon ascertaining the event, provide first audio data indicating the first audio signal to another wireless communication device via an audio data communication channel. Preferably, in one embodiment, the event ascertainment unit includes or receives event data from a radio frequency-based event or activity sensing arrangement that does not rely on audio input to determine the occurrence of a fall and therefore respects the privacy of conversations within the environment.

[0068] In a preferred embodiment, the wireless communication device of the second aspect of the present invention also includes a speaker, which is configured to play an audio signal indicative of received audio data, and wherein the event-triggered audio forwarding arrangement is further configured to enable or establish a two-way audio data communication channel with another wireless device when an event is identified for exchanging audio data therebetween.

[0069] Furthermore, in one embodiment, the wireless communication device is a wirelessly controllable lighting device (such as a luminaire or lamp), the operation of which can be controlled using the wireless communication device according to a predetermined wireless communication protocol (such as, for example, WiFi, Bluetooth or Zigbee).

[0070] A third aspect of the present invention is formed by a method for operating a wireless communication device according to the second aspect. The method comprises:

[0071] - using an RF-based sensing arrangement to pinpoint the occurrence of predefined physical events associated with a person;

[0072] - operating the microphone to record first audio data, and upon ascertaining the occurrence of a physical event associated with the person (e.g., a fall, movement, posture, vital signs indicating danger, etc.), providing first audio data indicating a first audio signal to another wireless communication device.

[0073] Therefore, the method of the third aspect shares the advantages of the wireless communication device of the second aspect of the present invention.

[0074] In certain embodiments, the method further comprises:

[0075] - upon ascertaining a physical event, enabling a two-way audio data communication channel with another wireless device for exchanging recorded audio data therebetween; and

[0076] - playing, using a loudspeaker, a second audio signal indicative of the received first audio data.

[0077] A fourth aspect of the present invention is formed by a method for operating an event-triggered audio forwarding arrangement according to the first aspect. The method comprises:

[0078] - performing the method for operating the wireless communication device of the third aspect for the second wireless communication device. As a result, the second wireless communication device provides the first audio data to the first wireless communication device. The method further comprises:

[0079] - the first wireless communication device receiving the first audio data plays a second audio signal indicative of the first audio data to be picked up by a microphone of a voice assistant device;

[0080] - recording a second audio signal by a microphone of the voice assistant device;

[0081] - establishing a communication channel between the voice assistant device and the remote caregiver, and providing second audio data indicative of a second audio signal to the remote caregiver via the communication channel.

[0082] The method for operating an event triggered audio forwarding arrangement according to the invention shares the advantages of the event triggered audio forwarding arrangement of the first aspect.

[0083] Another aspect of the present invention is formed by a corresponding computer program, which includes instructions that, when executed by a processor of a wireless communication device according to the second aspect or a processor of an event-triggered audio forwarding arrangement according to the first aspect, causes the wireless communication device or the event-triggered audio forwarding arrangement to respectively perform the method of the third aspect or the method of the fourth aspect.

[0084] It should be understood that the event-triggered audio forwarding arrangement of claim 1, the wireless communication device of claim 10, the corresponding methods of claims 12 and 14, and the computer program of claim 15 have similar and / or identical preferred embodiments, in particular as defined in the dependent claims.

[0085] It shall be understood that a preferred embodiment of the invention can also be any combination of the dependent claims or the above-described embodiments with the respective independent claim.

[0086] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In the following drawings:

[0088] Figure 1 shows a schematic block diagram of an event-triggered audio forwarding arrangement according to an embodiment of the present invention (particularly implemented as a fall-triggered audio forwarding arrangement);

[0089] Figure 2 shows a schematic block diagram of an event-triggered audio forwarding arrangement (particularly implemented as a fall-triggered audio forwarding arrangement) according to another embodiment of the present invention;

[0090] Figure 3 shows a schematic representation of an event-triggered audio forwarding arrangement, in particular implemented as a fall-triggered audio forwarding arrangement within a wirelessly controllable lighting arrangement in an apartment;

[0091] Figure 4 A schematic block diagram showing a wireless communication device according to an embodiment of the present invention; and

[0092] Figure 5 A flow chart illustrating a method for controlling the operation of an event-triggered audio forwarding arrangement according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0093] Figure 1A schematic block diagram of an event-triggered audio forwarding arrangement 500 is shown. The event-triggered audio forwarding arrangement 500 (also referred to as the arrangement for simplicity) includes a voice assistant device 502. The voice assistant device 503 includes a microphone 504 and, optionally, a speaker 506. The voice assistant device is configured to connect to a remote caregiver 600 (e.g., an external call center) and enable or establish a communication channel 602 with the remote caregiver 600. The communication channel is suitable for transmitting audio data AD2 indicative of an audio signal A2 recorded by the microphone 504 according to any suitable communication protocol, which may be a wireless communication protocol, such as WiFi, Thread, Bluetooth, LiFi communication protocol; or a wired communication protocol, including a power line communication protocol. The voice assistant device 502 is configured to generate audio data AD2 using the recorded audio signal A2 and provide the audio data AD2 to the remote caregiver 600 via the communication channel 602.

[0094] The arrangement 500 also includes a first wireless communication device 200, which includes a speaker 202. The first wireless communication device 200 is arranged at a first location L1, which is a first distance D1 from the voice assistant device and within a sound pickup area 508 of a microphone 504 of the voice assistant device 502. The sound pickup area 508 is a volume in space in which a sound of a given intensity can be recorded with a given quality by a microphone 504 having a given sensitivity.

[0095] The arrangement 500 also includes a second wireless communication device 100. Other arrangements (see Figure 3 ) may include multiple second wireless communication devices. The second wireless communication device 100 includes a microphone 102 and is arranged at a second position L2, which is a second distance D2 from the voice assistant device and outside the sound pickup area 508 of the microphone 504 of the voice assistant device 502 (for example, in another room). The first and second positions at the first and second distances cause the first sound intensity picked up by the microphone of the voice assistant device from a reference sound signal of a given intensity generated at the first position to be higher than the second sound intensity picked up by the microphone of the voice assistant device from the reference sound signal of the same given intensity generated at the second position.

[0096] For example, the voice assistant device and the first wireless communication device are located in the bedroom 603, while the second wireless communication device is arranged in the bathroom 604, separated from the bedroom 603 by a wall 605. The second wireless communication device includes an event ascertainment unit 104, which is configured to ascertain (i.e., receive or determine) event data FD indicating a physical event associated with the person 101. Physical events include, for example, falls, postures, predetermined movements, vital signs indicating potential risks, etc. Preferably, the ascertainment of event data is accomplished using radio frequency-based sensing technology, where a given signal quality parameter (e.g., RSSI, CSI, etc.) is related to the current state of the person in the sensing volume. The second wireless communication device also includes an audio data forwarding unit 106, which is configured to operate the microphone 102 to record a first audio signal A1 when the event data FD (e.g., fall data FD indicating a fall) is ascertained, and provide first audio data AD1 indicating the first audio signal to the first wireless communication device 200 via the audio data communication channel 108.

[0097] In arrangement 500, the first wireless communication device is configured to operate the speaker 202 to play the second audio signal A2 indicating the first audio data AD1 within a predetermined sound pickup area of the voice assistant device when receiving the first audio data AD1, so that the audio signal A2 is picked up or recorded by the microphone 504 of the voice assistant device. The voice assistant device 502 is also configured to generate second audio data AD2 indicating the second audio signal A2 and provide the second audio data AD2 indicating the second audio signal A2 to the remote caregiver 600 via the communication channel 602.

[0098] Optionally, the event ascertainment unit 104 is further configured to ascertain whether a predefined physical event associated with the subject 101 has occurred within or outside the predetermined sound pickup area 508, and the audio data forwarding unit is configured to operate the microphone to record the first audio signal and enable the audio data communication channel when it is ascertained that a fall has occurred outside the sound pickup area. Therefore, if a fall has occurred within the sound pickup area of the voice assistant device, the audio forwarding functionality is not activated.

[0099] Additionally or alternatively, the arrangement 500 comprises an auditory range inference unit, which Figure 1exemplarily depicted as a unit 110 of the second wireless communication device. The hearing range inference unit is configured to: ascertain contextual information related to the current state of the environment in which the event-triggered audio forwarding arrangement is arranged; and use the ascertained contextual information to determine whether a given wireless communication device is within the current sound pickup area of the voice assistant device. This particular embodiment enables the current sound pickup area to be inferred relative to the location of the suspected fall and not necessarily relative to the location of the second wireless communication device. The contextual information is related to the current state of the environment (e.g., a house or apartment or a relevant part thereof). For example, the contextual information may indicate whether a door in a wall 605 between a room 603 in which the voice assistant device is arranged and a room 604 in which the second wireless communication device 100 is located is open or closed, which affects the current sound pickup area of the voice assistant device. In another example, the user may provide location data indicating the locations of the first and second wireless devices (L1 and L2), and the hearing range inference unit may determine the sound pickup area based on the location data. Similarly, and using recent historical or real-time sensor data, it can be assessed whether the microphone 102 of the second wireless communication device 100 can record or hear sounds from the room where the first wireless communication device and / or voice assistant device is located. For example, the first wireless device 200 can play a first test audio signal, and the hearing range inference unit can determine whether the first test audio signal has been picked up by the voice assistant device. If so, the event-triggered audio forwarding arrangement concludes that the user may be within the sound pickup area of the voice assistant device 502. The second wireless device 100 may also include a speaker and play a second test audio signal. The hearing range inference unit can determine whether the second test audio signal has been picked up by the voice assistant device, and if not, determine that the second wireless device 100 is outside the sound pickup area of the voice assistant device. In some cases, after a person has fallen, they may only be able to speak with a weaker voice than usual, and thus the voice assistant device may no longer be within the sound pickup area of the fallen person, thus still requiring event-triggered audio forwarding functionality. The hearing range inference unit may also include a radar unit configured to ascertain at least a portion of the contextual information.

[0100] Figure 2 FIG2 shows a schematic block diagram of an event-triggered audio forwarding arrangement 500a according to an embodiment of the present invention. The following discussion will focus on the arrangement 500a that is different from the Figure 1 Features having the same or similar functionality will be used in Figure 1 The same reference numerals used in Figure 2In the arrangement 500a, the first wireless communication device 200a further includes a microphone 203 and an audio data forwarding unit 204 for generating and providing response audio data RAD2 indicative of the corresponding response audio signal RA1 recorded by the microphone 302. The second wireless communication device 100a further includes a speaker 103, which is configured to play a second response audio signal RA2 indicative of the received response audio data RAD2. The event-triggered audio forwarding arrangement is further configured to enable a two-way audio data communication channel 108a between the first wireless communication device 200a and the second wireless communication device 100a for exchanging audio data therebetween when event data FD indicative of a predefined physical event is ascertained. Thus, for example, in the arrangement 500a, when the user falls in the room 604, the event ascertainment unit 104 of the second wireless communication device 100a ascertains (e.g., receives or determines) fall data FD indicative of the fall. When the occurrence of a fall has been detected, the microphone 102 records an audio signal A1 (in particular, an audio signal A1 from the fallen user), which is then converted into first audio data AD1 and sent to the first wireless communication device via the audio data communication channel 108a (in particular, a wireless channel, such as, for example, a WiFi communication channel). The first wireless communication device 200a receives the first audio data AD1, and the first wireless communication device 200a then plays the audio data using the speaker 202. The played audio data is referred to as the second audio signal A2, which is related to the first audio signal. The second audio signal A2 is played within the sound pickup area 508 of the voice assistant device 502, and the voice assistant device 502 then provides the second audio data AD2 indicating the second audio signal A2 to the remote caregiver 600 via the communication channel 602a. The operations described so far are also represented by Figure 1 However, in Figure 2In the arrangement 500a of FIG. 5 , the communication channel between the voice assistant device and the communication channel 602a and the audio data communication channel 108a is a two-way communication channel that allows a remote caregiver (e.g., a call center or monitoring station) to establish a conversation with the user in the room 604. To this end, and in response to the second audio data AD2, the remote caregiver provides response audio data RAD1, for example, indicating a query about the user's status. The response audio signal RA1 indicating the response is reproduced by the speaker 506 of the voice assistant device 502. This audio signal, including the response from the call center, is recorded by the microphone 203 of the first wireless communication device 100a, which generates corresponding second response audio data RAD2, which is provided by the audio forwarding unit 204 to the second wireless communication device 100a, for example, via a WiFi radio link. The second wireless communication device 100a outputs the second response audio signal RA2 via the integrated speaker 103, thereby enabling two-way communication between the user and the remote caregiver, which allows the caregiver to assess the situation of the fallen user 101.

[0101] Optionally, as indicated by the dotted line, the second wireless communication device 100a further comprises a word recognition unit 107 configured to recognize a predetermined wake-up word, and wherein the audio data forwarding unit 106 is further configured to enable an audio data communication channel and thereby provide the first audio data AD1 when the predetermined wake-up word is recognized by the word recognition unit 107. This is particularly advantageous for arrangements including a voice assistant device configured to enable a communication channel with a remote caregiver when the predetermined wake-up word is recognized, such as, for example, an Amazon Echo device.

[0102] Figure 3A schematic representation of an event-triggered audio forwarding arrangement 500 implemented within an exemplary wirelessly controllable lighting arrangement 550 in an apartment 650 is shown. The apartment has a wirelessly controllable lighting arrangement 550 that includes a plurality of wirelessly controllable lighting devices 100a and, in this particular example, a bridge 150. With respect to the wirelessly controllable lighting devices 100a, a hallway 651 includes two luminaires L, a first bedroom 652 and a second bedroom 653 each include a luminaire L and two bedside lamps B, a living room 654 includes two luminaires L and a floor lamp T, a kitchen 655 includes a luminaire L, a pantry or pantry 656 includes a luminaire L, and a bathroom 657 includes a luminaire L and a bathroom mirror light M. These lighting devices are controlled using wireless communication signals exchanged between the lighting devices and the bridge using a suitable wireless communication protocol, such as, but not limited to, WiFi or Zigbee. Additionally, the wireless communication signals are used to determine the activity of objects within the apartment, in particular by performing known radio frequency-based sensing functions, which are advantageously used to determine whether a physical event associated with a person, such as a fall, has occurred in any room. In addition to the lighting devices configured as the required wireless communication devices 100a, the event-triggered audio forwarding arrangement 500 also includes two voice assistant devices 502 placed in the master bedroom 653 and the living room 654. For simplicity, the predetermined sound pickup area of the respective voice assistant device is initially limited to its room. According to the present invention, the lighting device 200a closest to the respective voice assistant device 502 is considered to be the first lighting device. Therefore, it is clear that the density of wirelessly controllable lighting devices 100a in the apartment 650 is much higher than the density of voice assistant devices.

[0103] exist Figure 3 In this particular arrangement 500 of FIGURE 5, the voice assistant device is configured to enable a communication channel with the remote caregiver upon recognition of a predetermined wake-up word (e.g., "activate"). If a fall is detected in the living room 654 or the master bedroom 653, the arrangement assumes that the fall has occurred within the sound pickup area of the voice assistant device 502 and takes no further action. However, if the fall occurs in another room (such as in the pantry 656, as indicated by the Figure 3), the luminaire in the pantry ascertains the fall data indicating it and activates the audio forwarding functionality to record the audio signal A1 in the pantry using the microphone in the lighting device 100a. In this audio signal, the user utters the wake-up word "activate" and declares that there has been an accident and assistance is needed. The audio data forwarding unit in the luminaire generates first audio data AD1 indicative of the recorded audio signal A1 and provides said first audio data AD1 to one or both of the lighting devices 200a located in each of the master bedroom 653 and the living room. In this case, the audio data is provided to the voice assistant device in the living room. Depending on the arrangement, the provision can be performed directly or via the bridge 150. Both possibilities are Figure 3 As shown in FIG. One or both of the lighting devices 200a use a speaker to generate a second audio signal A2 that reproduces the first audio signal, namely, the wake-up word "activate" and a statement about needing help. This second audio signal is played within the sound pickup area of the voice assistant device 502. The microphone of the voice assistant device picks up the wake-up word "activate" and initiates a communication channel with the remote caregiver to provide second audio data indicating a request for assistance.

[0104] In a variation of this arrangement, the event-triggered audio forwarding arrangement 500 further includes an auditory range inference unit configured to: ascertain contextual information related to the current state of the environment in which the event-triggered audio forwarding arrangement is arranged; and use the ascertained contextual information to determine whether a given wireless communication device is within the current sound pickup area of the voice assistant device. For example, if a fall has occurred in the kitchen 655 and the door 658 between the kitchen 655 and the living room 654 is open, the auditory range inference unit is configured to ascertain contextual information indicating the state of the door and the historical capacity of the voice assistant device in the living room 654 to hear audio signals originating in the kitchen 655 when the door is open, to infer that under the current conditions, the voice assistant device is capable of recording audio signals originating in the kitchen.

[0105] In the preferred arrangement 500, the first wireless communication device 200a further includes a microphone and an audio data forwarding unit for providing response audio data indicative of a corresponding response audio signal recorded by its microphone 203, and the second wireless communication device 100a further includes a speaker configured to play a second response audio signal (RA2) indicative of the received response audio data. Here, the event-triggered audio forwarding arrangement 500 is further configured to enable a two-way audio data communication channel between the first wireless communication device and the second wireless communication device for exchanging audio data therebetween upon ascertaining fall data indicative of a fall. Thus, the monitoring center's response to the fallen user's request for assistance is played by the voice assistant device, picked up by the nearest wireless communication device 200a, forwarded to the lighting device closest to the fallen user, and reproduced there to ensure that the user is aware that the message has been received in order to gather further information or confirm that assistance is in progress.

[0106] Figure 4 A schematic block diagram of a wireless communication device 100 (which is configured as a wirelessly controllable lighting arrangement) according to an embodiment of the present invention is shown. The operation of the lighting unit 105 is wirelessly controlled by means of wireless communication signals according to a predetermined wireless communication protocol. Signals are provided and received via a dedicated antenna 107, which is signally connected to a processing unit 140 of the lighting device. The lighting device also includes an event ascertainment unit 104, which is configured to use the received wireless communication signal to determine fall data FD indicating that a person has fallen. When it is determined that a fall has occurred, the microphone 102 records a first audio signal A1, and the recorded audio signal is sampled at the audio data forwarding unit 106 and converted into audio data, which is provided as part of the wireless communication signal via the antenna for reception by another wireless communication device.

[0107] Optionally, Figure 4The wireless communication device 100 includes an auditory range inference unit 110, which is configured to: ascertain contextual information related to the current state of the environment in which the event-triggered audio forwarding arrangement is deployed; and use the ascertained contextual information to determine whether a given wireless communication device is within the current sound pickup area of the voice assistant device. To infer the current sound pickup area of the voice assistant device relative to the location of the suspected fall, the lighting system utilizes contextual information about the current state of the apartment; for example, the radar sensor 111 is configured to determine whether the door between the location of the fall and the room where the voice assistant device is located is currently open or closed. Similarly, and based on, for example, recent historical or real-time sensor data, an assessment is made as to whether the microphone of the lighting device closest to the fall location can record ambient noise or noise from the room where the voice assistant device is located. If so, the event-triggered audio forwarding arrangement concludes that the person who fell is likely within the sound pickup area of the voice assistant device. Note that after the elderly person has fallen, he may only be able to speak with a weaker voice than usual, and therefore the voice assistant device kitchen may suddenly no longer be within the audio range of the fallen person, so event-triggered audio forwarding functionality is still required.

[0108] Alternatively or additionally, the wireless communication device may further include a word recognition unit 112 configured to recognize a predetermined wake-up word or phrase. In this particular example, the audio data forwarding unit is configured to further enable the audio data communication channel to provide audio data when the word recognition unit 112 recognizes the predetermined wake-up word.

[0109] Figure 4 The wireless communication device 100 further includes a speaker 103 configured to play a second response audio signal indicating the received response audio data via the antenna. The wireless communication device 100 is further configured to enable a two-way audio data communication channel between the two wireless communication devices for exchanging audio data therebetween upon ascertaining fall data indicating a fall.

[0110] Preferably, all wireless communication devices in the exemplary event-triggered audio forwarding arrangement are configured to Figure 4 Wireless communication device 100.

[0111] Figure 5A flow chart of a method 800 for operating an event-triggered audio forwarding arrangement according to an embodiment of the present invention is shown. The method 800 includes, in step 702, ascertaining fall data indicating a person falling using an RF-based sensing arrangement. When the fall data indicating a fall is ascertained, the method includes, in step 704, operating a microphone to record a first audio signal, and in step 706, providing first audio data indicating the first audio signal to another wireless communication device via an audio data communication channel. These steps also form the method 700 of a wireless communication device for operating an event-triggered audio forwarding arrangement. The method 700 may also include, in step 705, enabling a two-way audio data communication channel with another wireless device for exchanging audio data therebetween, and in response to receiving audio data from the other device, playing a second response audio signal indicating the received response audio data using a speaker.

[0112] Method 800 also includes, in step 802, playing a second audio signal indicating the first audio data within a sound pickup area of the voice assistant device by another wireless communication device that has received the audio data. The method also includes, in step 804, recording the second audio signal by a microphone of the voice assistant device. Then, the method includes, in step 806, enabling or establishing a communication channel between the voice assistant device and the remote caregiver, and in step 808, providing second audio data indicating the second audio signal to the remote caregiver via the communication channel.

[0113] Thus, according to the present invention, and because the audio data is forwarded between the second and first wireless communication devices, with the first communication device replaying the audio data so that the message can be picked up by the microphone of the voice assistant device located away from the person who has fallen, the call center will be able to hear the person who has fallen and preferably also have a two-way conversation with him or her, for example to verify whether the person is okay or needs assistance. The essence of the present invention is that the wireless communication devices do not need to be directly connected to the call center, as they only have to pass the audio data to a dedicated gateway, i.e., the voice assistant device.

[0114] In another exemplary arrangement, the call center can activate audio forwarding functionality. Fall data indicating a fall event can be directed to the call center. Upon receiving the notification, the call center is configured to initiate a conversation with the person to check whether the person is okay or needs assistance. If the person is detected outside the range of the voice assistant device, the audio signal provided by the speaker of the voice assistant device is picked up or recorded by the microphone of the wireless communication device located closest to the voice assistant device, and audio data indicative of the audio signal is provided via wireless transmission to the wireless communication device closer to the fallen person, which then plays the audio signal indicative of the received audio data via the speaker.

[0115] In summary, the present invention relates to an event-triggered audio forwarding arrangement comprising a voice assistant device configured to connect to a remote caregiver. The arrangement comprises: a first wireless communication device having a speaker, the first wireless communication device being arranged at a first position at a first distance from a microphone of the voice assistant device; a second wireless communication device having a microphone, the second wireless communication device being arranged at a second position at a second distance, and an event ascertaining unit and an audio data forwarding unit, the event ascertaining unit being configured to ascertain the occurrence of a physical event associated with a person, the audio data forwarding unit being configured to record a first audio signal and provide first audio data to the first wireless communication device, the first wireless communication device being configured to play a second audio signal indicative of the first audio data to be picked up by the voice assistant device, thereby increasing its effective spatial hearing range.

[0116] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0117] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0118] A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0119] The processes performed by one or several units or devices (e.g., processing of detection signals for determining a fall, authentication, control of the fall monitoring system, etc.) may be performed by any other number of units or devices. These processes (in particular, control of the fall monitoring system according to a control method executed by an event-triggered audio forwarding arrangement) may be implemented as program code means of a computer program and / or as dedicated hardware.

[0120] The computer program may be stored / distributed on a suitable medium, such as optical storage media or solid-state media, provided together with or as part of other hardware; but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0121] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. An event-triggered audio forwarding arrangement (500) for conditionally forwarding audio received at a second wireless communication device outside a sound pickup area of a voice assistant device to a first wireless communication device for presentation within the sound pickup area; the arrangement comprising: - the first wireless communication device (200) comprising a speaker (202), the first wireless communication device (200) being arranged at a first distance (D1) from a microphone (504) of a voice assistant device (502) and within the sound pickup area of the microphone of the voice assistant device (502); and - the second wireless communication device (100) comprising a microphone (102), the second wireless communication device being arranged at a second distance from the microphone of the voice assistant device and outside the sound pickup area, the second wireless communication device further comprising: - an event ascertaining unit (104) configured to ascertain the occurrence of a predefined physical event associated with the person (101); and - an audio data forwarding unit (106) configured to operate the microphone (102) to record the first audio signal (A1), and to provide first audio data (AD1) indicative of the first audio signal to the first wireless communication device (200) via the audio data communication channel (108) when the physical event is ascertained; and wherein -The first wireless communication device is configured to operate a speaker (202) to play a second audio signal (A2) when receiving the first audio data (AD1), the second audio signal (A2) being indicative of the first audio data (AD1) to be picked up by a microphone of the voice assistant device.

2. An event-triggered audio forwarding arrangement according to claim 1, wherein the event ascertainment unit is further configured to ascertain whether the event has occurred within or outside a predefined sound pickup area of the microphone of the voice assistant device; and wherein the audio data forwarding unit is further configured to operate the microphone of the second wireless communication device to record the first audio signal and provide the first audio data via the audio data communication channel only when the event ascertainment unit has indicated that the event has occurred outside the predefined sound pickup area.

3. An event-triggered audio forwarding arrangement according to claim 1 or 2, wherein the audio forwarding unit is configured to record a first audio signal and provide audio data to the first wireless communication device when the occurrence of a predefined physical event is ascertained, wherein the audio data indicates the audio signal recorded during a first time span before the occurrence of the predefined physical event is ascertained and / or during a second time span after the occurrence of the predefined physical event is ascertained.

4. The event-triggered audio forwarding arrangement according to any of the preceding claims further includes an auditory range inference unit (110), which is configured to: ascertain contextual information related to the current state of the environment in which the event-triggered audio forwarding arrangement is arranged; and use the ascertained contextual information to determine whether a given wireless communication device is within the current sound pickup area of the microphone of the voice assistant device.

5. An event-triggered audio forwarding arrangement (500a) according to any one of the preceding claims, wherein the first wireless communication device (200a) further includes a microphone and an audio data forwarding unit for providing response audio data (RAD2) indicative of a corresponding response audio signal (RA1) recorded by the microphone (203) of the first wireless communication device, and wherein the second wireless communication device further includes a speaker configured to play a second response audio signal (RA2) indicative of the received response audio data (RAD2), the event-triggered audio forwarding arrangement being further configured to enable a two-way audio data communication channel (108a) between the first wireless communication device (200a) and the second wireless communication device (100a) for exchanging audio data (AD1, RAD2) therebetween when an occurrence of a predefined physical event is ascertained.

6. The event-triggered audio forwarding arrangement (500) according to any one of the preceding claims, further comprising a voice assistant device (502) configured to connect to a remote caregiver (600) via a communication channel (602), wherein the voice assistant device is configured to provide second audio data (AD2) indicating a second audio signal (A2) to the remote caregiver via the communication channel, and wherein the voice assistant device is configured to establish a communication channel for providing the second audio data to the remote caregiver when a predetermined wake-up word is recognized, and wherein the second wireless communication device (100) includes a word recognition unit (112) configured to recognize a predetermined wake-up word, and wherein the audio data forwarding unit is configured to further enable the audio data communication channel when the predetermined wake-up word is recognized by the word recognition unit.

7. An event-triggered audio forwarding arrangement according to any of the preceding claims, further comprising a voice assistant device (502) configured to connect to a remote caregiver (600) via a communication channel (602), wherein the voice assistant device is configured to provide second audio data (AD2) indicative of a second audio signal (A2) to the remote caregiver via the communication channel, and wherein the voice assistant device is configured to establish a communication channel for providing the second audio data to the remote caregiver when a predetermined wake-up word is recognized, and wherein the first wireless communication device is configured to provide a pre-recorded wake-up word audio signal via its speaker upon receiving the first audio data, the pre-recorded wake-up word audio signal including the predetermined wake-up word when played.

8. An event-triggered audio forwarding arrangement according to any one of the preceding claims, wherein the audio data communication channel between the first wireless communication device and the second wireless communication device is based on a wireless communication protocol or a power line communication protocol or a LiFi communication protocol.

9. An event-triggered audio forwarding arrangement according to any one of the preceding claims, wherein the audio communication channel is based on a wireless communication protocol, the event-triggered audio forwarding arrangement further comprising a radio frequency-based activity sensing unit (140), the radio frequency-based activity sensing unit (140) being configured to use wirelessly transmitted audio data (AD1, RAD2) to determine signal parameters associated with activity of an object within a sensing volume defined by the relative positions of the first wireless communication device and the second wireless communication device, and to provide activity data indicative of the signal parameters.

10. A wireless communication device (100) for use in an event-triggered audio forwarding arrangement according to any one of the preceding claims, the wireless communication device comprising: -microphone; - an event ascertaining unit configured to ascertain the occurrence of a predefined physical event associated with a person; and - an audio data forwarding unit configured to operate the microphone to record a first audio signal and, upon ascertaining an event, provide first audio data indicative of the first audio signal to another wireless communication device via an audio data communication channel.

11. The wireless communication device according to claim 10 further includes a speaker, which is configured to play a second response audio signal (RA2) indicating response audio data (RAD2), and wherein the wireless communication device is further configured to enable a two-way audio data communication channel with another wireless communication device for exchanging audio data therebetween when an event is identified.

12. A method (700) for operating a wireless communication device (100) according to claim 10 or 11, the method comprising: - ascertaining (702) the occurrence of a predefined physical event associated with a person using an RF-based sensing arrangement; - operating (704) a microphone to record a first audio signal, and providing (706) first audio data indicative of the first audio signal to another wireless communication device via an audio data communication channel upon ascertaining the physical event.

13. The method (700) of claim 12, further comprising: - upon ascertaining the physical event, enabling (705) a two-way audio data communication channel with another wireless device for exchanging audio data therebetween; and - playing, using the loudspeaker, a second response audio signal (RA2) indicative of received response audio data (RAD2) provided by the other wireless communication device (200a).

14. A method (800) for operating an event-triggered audio forwarding arrangement according to any one of claims 1 to 9, the method comprising: - performing a method (700) for operating a wireless communication device according to claim 11; - the wireless communication device receiving the first audio data plays (802) a second audio signal indicative of the first audio data to be picked up by a microphone of the voice assistant device (502); - Recording (804) a second audio signal by the microphone of the voice assistant device.

15. A computer program comprising instructions which, when executed by a processor of a wireless communication device or a processor of an event-triggered audio forwarding arrangement, cause the wireless communication device or the event-triggered audio forwarding arrangement to perform the methods according to claims 12 and 13 or to perform the method according to claim 14, respectively.

Images