Household appliance with acoustic interface

Abstract

A household appliance (110,115) comprises a control unit (130) with a first wireless interface (135); and an acoustic interaction unit (125) with a second wireless (140) interface, wherein the interfaces (135, 140) are adapted to provide a wireless audio stream connection (180) between the units (125, 130). At least one of the interfaces (135, 140) is adapted to simultaneously provide another wireless connection (180) to an interface outside the household appliance (110,115).

Description

[0001] Household Appliance with Acoustic Interface

[0002] Present invention concerns a household appliance. More specifically, present invention concerns a household appliance that can be voice-operated.

[0003] A household appliance has a control unit that is adapted to control operation of the appliance. The control unit connects to a microphone and is adapted to scan audio data in a vicinity of the appliance. An utterance of a user may be recognized and a voice command may be identified. Depending on a current state of operation, a function corresponding to the command may be executed. In the opposite direction, the household appliance may provide acoustic user feedback by means of a spoken text.

[0004] In some cases, ambient sound may keep a user from operating the acoustic interface. Should for instance the appliance stand in the vicinity of another appliance that emits significant sound, for example a ventilation hood or a vacuum cleaner, both speech input and speech output between the user and the appliance may not be possible.

[0005] An object underlying present invention lies in providing an improved technique for operating a household appliance by voice. The invention solves above-mentioned object through the subject matter of the independent claims. Dependent claims describe preferred embodiments.

[0006] According to a first aspect of present invention, a household appliance comprises a control unit with a first wireless interface; and an acoustic interaction unit with a second wireless interface, wherein the interfaces are adapted to provide a wireless audio stream connection between the units. At least one of the interfaces is adapted to simultaneously provide another wireless connection to an interface outside the household appliance.

[0007] The household appliance may connect to an external acoustic interface so that a possible disturbance between a user and the internal acoustic interface can be circumvented. In this, parallel operation is possible so that both acoustic interfaces are supplied with the same audio data. Likewise, the internal acoustic interface can be made available to another household appliance. The proposed technique enables cooperation between several household appliances with acoustic interfaces, thus enabling improved acoustic control through a user. It is also possible to connect a standalone acoustic interface to the household appliance, for example in the shape of a headset, a wireless microphone or in-ear speakers.

[0008] The acoustic interface may convert acoustic data into an audio stream or vice versa. The audio stream may especially comprise speech data. The acoustic interface is furthermore preferred to have sound processing capabilities. These capabilities may encompass:

[0009] Far-field voice pickup: user can interact with the home appliance by voice from at least 2-3m distance

[0010] Wake-word (\I\N\T) based activation of the voice dialogue: user can “wake up” the home appliance by uttering the system’s specific wake-word, e.g., “Alexa” or “Hey Bosch”. Wake-word activation must be able to operate in highly constrained power consumption scenarios, e.g., (Networked) Stand-By state of the host device or home appliance.

[0011] Acoustic Echo Cancellation (AEC): user can interrupt any currently on-going audio output, e.g., music playback or voice feedback, by uttering the wakeword. AEC is a type of Software algorithm that aims at “subtracting” the current audio output from current audio input received from the microphones, to effectively remove any echo or feedback loop from the system. The AEC algorithm is a part of the Audio Front-End (AFE) solution provided by some Integrated Circuit (IC) on the Voice Processing Hardware, called Voice Control Module (VCM).

[0012] Digital Signal Processing (DSP) algorithms like Beam Forming, Blind Source Separation, Noise Reduction etc., to optimize the audio input signal originating from the user’s mouth in the sense of rendering a mostly “clean”, noise- free mono audio signal are used in the audio processing. These algorithms typically run in embedded software or hardware in the host device (e.g., a smart speaker or a home appliance). The DSP algorithms are another part of the Audio Front-End (AFE) solution provided by some Integrated Circuit on the VCM. Cloud-based Automatic Speech Recognition (ASR), Natural Language Understanding (NLU) and Text-to-Speech (TTS) software, yielding state-of-the- art voice command recognition rates and voice response correctness I quality Full-duplex audio transmission: System allows simultaneous transmission of audio-in (originating from microphones) and audio-out (originating from cloud software and being played back on VCM’s loudspeaker) Over-the-air software updates for WW and AFE firmware.

[0013] Software-based control of WW and AFE (both located on “Voice Control Module" (VCM)) from main electronic control module.

[0014] Software-based control of audio volume (implemented in Digital-Analogue Converter (DAC) or Power Amplifier (PA), both realized by an Integrated Circuit located on “Voice Control Module" (VCM) from main electronic control module.

[0015] However, speech processing capabilities of the acoustic interaction unit are generally limited and it is preferred that the acoustic interaction unit passes a stream of audio data over the wireless interface to a unit that processes speech. This unit may especially comprise the control unit of the same or a different household appliance. The control unit may perform speech processing on its own or it may forward the audio data stream over another interface to a back-end service such as a server or a cloud-based service for processing. Such external processing may not be apparent to the acoustic interaction unit.

[0016] The acoustic interface unit is preferred to provide sound output via a loudspeaker and sound input via a microphone. However, in some cases only a simpler acoustic interface may be available which provides only sound output or only sound input. The loudspeaker and the microphone, respectively, may be part of the acoustic interface unit or the acoustic interface unit may be connected to these elements with an electric interface. In this case, the elements are preferred to be part of the household appliance that comprises the acoustic interface unit.

[0017] The wireless interfaces are preferred to comprise Bluetooth interfaces. For the purpose at hand, both Bluetooth Classic (BT) and Bluetooth Low Energy (BLE) may be used. It is furthermore preferred that the interfaces support Bluetooth Au- dio LE which offer features like asynchronous broadcasting and multicast / multistream connections. Bluetooth is a well-understood and widespread wireless technology and highly integrated Bluetooth endpoints are available at competitive prices. Power consumption is low, depending on an operating mode, and an achievable range should be sufficient for required transmission paths inside one household.

[0018] It is generally preferred that wireless connections between the acoustic interaction unit of the household appliance and another device or between the control unit of the household device and another acoustic interaction unit are limited to one household or one room within one household. In case a connection is made between different household appliances, the appliances may be of the same or of different types.

[0019] More specifically, the acoustic interaction unit is preferred to be adapted to scan audio data in the vicinity of the household appliance; and the second interface may be adapted to provide a stream of scanned audio data to several control units. Control units receiving the audio data stream may comprise the control unit of the same household appliance and / or one or more control units of another household appliance. The acoustic interaction unit may be operated as an audio stream provider for more than one audio stream consumers. The audio data stream may also be supplied to another consumer like a wireless headset or another acoustic human interface device (HID) like a headphone, a microphone, a wireless speaker or a headset.

[0020] In complementary operation, the acoustic interaction unit may be adapted to output audio data at the household appliance; and the second interface may be adapted to accept a stream of audio data from a stream provider. The acoustic interaction unit may thus be one of several audio consumers which are connected to one common audio stream provider. The stream provider may be comprised by another household appliance. The stream provider may also come in the shape of a standalone acoustic user interface device. The control unit may be adapted to provide a stream of audio data to several acoustic interface units. An acoustic interface unit acting as consumer to such a stream of audio data may be part of the same or another household appliance.

[0021] The control unit may also be adapted to accept streams of audio data from several acoustic interface units. An acoustic interface unit acting as provider for such a stream of audio data may be part of the same or another household appliance.

[0022] The control unit may be adapted to select from the streams one with the best signal to noise ratio (SNR). This way, audio input can be taken from an acoustic interface unit that is closest to a source of sound or that is least disturbed, for instance by another sound source.

[0023] The control unit is adapted to control operation of the household appliance. The appliance may have a main function that is controlled by the control unit. For instance, a main function of a dishwasher is to wash dishes, a task for which a number of sensors and actuators are controlled by the control unit. Similarly, a control unit of a washing machine is adapted to control the machine so that garment inside the machine is cleaned. The control unit may also provide a classic user interface, for instance via a display or buttons. In another embodiment, such a classic user interface may be implemented by a dedicated control unit connected to the control unit.

[0024] The control unit is preferred to be adapted to perform speech recognition on a stream of audio data. It is furthermore preferred that the control unit is adapted to control said household appliance depending on recognized speech inside received audio data. The control unit may also provide speech output through an acoustic interface unit. In one variant, the control unit may provide an audio stream that is output through an acoustic interaction unit; in another variant the control unit may pass other data, especially text data, to the acoustic interaction unit which may convert the data into spoken text and output it locally.

[0025] The appliance is especially preferred to comprise a kitchen appliance, a dish care appliance or a laundry treating appliance. Examples for a kitchen appliance in- elude an oven, a ventilation hood, a refrigerator, a deep freezer, a blender or a food processor. A dish appliance may especially comprise a dishwasher. Laundry care may for instance be performed with a washing machine, a tumble dryer, a laundry press or a flat iron.

[0026] According to another embodiment of present invention, a system comprises at least two household appliances according to one of the above claims. Acoustic interaction units and control units of the household appliances may share one or more wireless multistream connections. More than two household appliances may be comprised by the system. Connections may follow a 1-to-N, N-to-1 or N-to-M scheme, wherein N and M are integers greater than one.

[0027] Non-restricting embodiments of the invention will now be discussed in more detail with reference to the enclosed drawings in which: figure 1 shows a system; and figures 2-5 show constellations with at least one household appliance and at least one other audio element inside a common household.

[0028] Figure 1 shows a system 100 inside a household 105. The system 100 comprises a first household appliance 110 and a second household appliance 115 which essentially share the same structure with respect of a technique proposed herein. In the following, reference will primarily be made to the first household appliance 110 and given details may be valid for the second household appliance 115.

[0029] The household appliance 110 comprises a control system 120 with an acoustic interface unit 125 and a control unit 130. The acoustic interface unit 125 is adapted to input and / or output acoustic or audio data at the household appliance 110 and the control unit 130 is adapted to control operation of the household appliance 110.

[0030] The control unit 130 has a first wireless interface 135 and the acoustic interface unit 125 has a second wireless interface 150, which may provide a wireless connection over which control data, one or more audio data streams or signalling data may be transmitted. Thus, the control unit 130 may control operation of the acoustic interface unit 125 and for instance enable or disable it, prompt it to wirelessly connect to another device or to give status information like an SNR. The acoustic interface unit 125 may send a signal to the control unit 130 over the connection, for instance when it detects an utterance in the vicinity of the household appliance 110 or when it detects a predetermined keyword or “wakeup” word inside said utterance.

[0031] The acoustic interface unit 125 is preferred to be attached to a microphone 145 and a loudspeaker 150. Inside the acoustic interface unit 125 microphone 145 data may be processed by an analog-to-digital converter (ADC) 155. Audio data to be output may be run through a digital-to-analog converter (DAC) 160 which may comprise a power amplifier (PA). A processing block 165 may be realized by a micro computer or micro controller and is preferred to comprise function blocks for an audio front-end (AFE), keyword spotting (KWS), text to speech conversion (TTS) and / or wakeup word detection (WW). Basically, the processing block converts and processes audio signals from the microphone 145 and speaker 150 and sends it over the wireless connection via the second wireless interface 140.

[0032] The control unit 130 is preferred to comprise a voice processor 170 for processing spoken words. The voice processor 170 may be implemented in software and the control unit 130 may feature a microcontroller or micro computer on which said software is running. The voice processor 170 may come in the shape of a software development kit (SDK) and there may be more software running on said micro computer or micro controller to use or extend functionality of the SDK.

[0033] There may be provided a third wireless interface 175 through which a connection may be made to an external unit or service. The third wireless interface 175 may comprise a Wi-Fi interface and this interface 175 may be connected to a hotspot of said household, the hotspot providing internet access. The service or server to connect to may be reachable over the internet and the control unit 130 may choose to exchange information with said server or service. In one embodiment, audio data the control unit 130 received from the acoustic interface unit 125 is passed on to the server or service for processing. A processing result may be passed back to the control unit 130 and the household appliance 110 may be controlled based on said result. In other cases, control or software update data may be provided by the external server or service and the control unit 130 and / or the acoustic interface unit 125 may receive a control or software update from there.

[0034] The first household device 110 may connect to the second household device 115 over a multicast / multistream connection 180. The connection 180 generally connects an acoustic interface unit 125 and a control unit 130, but there may be more than one acoustic interface units 125 and / or more than one control units 130 who take part in the connection 180. The connection 180 may span two or more household appliances 110, 115 inside the same household 110. The connection 180 may transport audio data, especially an audio data stream, between three or more participating devices 135, 140. More than one concurrent data stream may be supported. Besides audio data, control or signalling data may be transmitted over the connection 180. For this, a broadcast or multicast connection 180 as well as a traditional point-to-point connection may be used.

[0035] In some cases, capabilities of one of the household appliances 110, 115 cover only part of what is described above. For instance, a household appliance 110, 115 may come without a microphone 145 or without a loudspeaker 150 or the acoustic interface unit 125 may not be adapted to process wake word detection.

[0036] In some cases, a control unit 130 may also connect to a standalone Bluetooth device, especially a human interface device (HID) like a wireless microphone or a wireless headset or loudspeaker.

[0037] Figure 2 shows a system 100 in a first exemplary configuration in order to illustrate a first use case of present invention. A user 205 and two or more household appliances 110, 115 are in one household 105. There is a first exemplary household appliance 210 (an oven), a second household appliance 215 (a washing machine), a third exemplary household appliance 220 (a refrigerator) and a fourth exemplary household appliance 225 (a ventilation hood). Each of the appliances 210 - 225 may be equipped with a control system 120 as shown in figure 1 . The user 205 provides an utterance 230 which may comprise a wakeup word and / or a control word or phrase in order to control the oven 210. The oven 210 is disposed close to the ventilation hood 225 which may emit significant noise during high speed operation. The utterance 230 may therefore not be intelligible to the oven 210. It is to be noted that the noise or another acoustic disturbance does not necessarily involve a household appliance 110, 115.

[0038] The washing machine 215 or the refrigerator 220 may be less affected by outside noise and the utterance 230 may be received there. Optionally, processing may take place in one of the appliances 215, 220, especially wake word detection. An audio command that is comprised by or follows the utterance 230 may then be picked up at an appliance 215, 220 and be forwarded wirelessly as an audio stream to the oven 210. There, a voice command may be recognized in the utterance 230 and the oven 210 may be controlled accordingly.

[0039] Figure 3 shows a system 100 in a similar setup like in figure 1 . In contrast to figure 1 , an audio output 235 from the oven 210 is to be made to the user 205. The audio output 235 may not be intelligible to the user 205 as it is drowned out by noise from the ventilation hood 225 or another acoustic disturbance. The audio output may be wirelessly transferred over the connection 180 from the oven 210 to another household appliance like the washing machine 215 or the refrigerator 220.

[0040] Audio data may be output synchronised so that the user 205 has a much improved chance to understanding an output message.

[0041] Figure 4 shows another system 100 based on those of figures 2 and 3. Here, more than one user 205 may be present in the household 105 and each user 205 may wear a wireless audio device 240. The connection 180 may span one or more household appliances 210-225 as well as one or more wireless audio devices 240. Audio output 235 may be broadcast from one of the household appliances 2 IQ- 225 to acoustic interface units 125 of one or more household appliances 210-225 and one or more wireless audio devices 240.

[0042] Figure 5 shows a similar setup, in which a wireless audio device 240 provides an audio input which may be relayed over the connection 180 to acoustic interface units 125 or control units 130 of one or more household appliances 210-225. Bidirectional audio stream transfer is also possible.

[0043] Reference list (as part of the description)

[0044] 100 system

[0045] 105 household

[0046] 110 first household appliance

[0047] 115 second household appliance

[0048] 120 control system

[0049] 125 acoustic interface unit

[0050] 130 control unit

[0051] 135 first wireless interface

[0052] 140 second wireless interface

[0053] 145 microphone

[0054] 150 loudspeaker

[0055] 155 ADC

[0056] 160 DAC, PA

[0057] 165 processing block

[0058] 170 voice processor

[0059] 175 third wireless interface

[0060] 180 multicast I multistream connection

[0061] 205 user

[0062] 210 first household appliance (oven)

[0063] 215 second household appliance (washing machine)

[0064] 220 third household appliance (refrigerator)

[0065] 225 fourth household appliance (ventilation hood)

[0066] 230 utterance

[0067] 235 audio output

[0068] 240 wireless audio device

Claims

CLAIMS1 . A household appliance (110,115) comprises:- a control unit (130) with a first wireless interface (135);- an acoustic interaction (125) unit with a second wireless interface (140);- the interfaces (135, 140) being adapted to provide a wireless audio stream connection (180) between the units (125, 130);- wherein at least one of the interfaces (135, 140) is adapted to simultaneously provide another wireless audio stream connection (180) to an interface outside the household appliance (110,115).

2. Household appliance (110,115) according to claim 1 , wherein the wireless interfaces (135, 140) are Bluetooth interfaces.

3. Household appliance (110,115) according to claim 1 or 2, wherein the acoustic interaction unit (125) is adapted to scan audio data in the vicinity of the household appliance (110,115); and the second interface (140) is adapted to provide a stream of scanned audio data to several control units (130).

4. Household appliance (110,115) according to one of the above claims, wherein the acoustic interaction unit is adapted to output audio data at the household appliance (110,115); and the second interface (140) is adapted to accept a stream of audio data from a stream provider.

5. Household appliance (110,115) according to one of the above claims, wherein the control unit (130) is adapted to provide a stream of audio data to several acoustic interface units (125).

6. Household appliance (110,115) according to one of the above claims, wherein the control unit (130) is adapted to accept streams of audio data from several acoustic interface units (125).

7. Household appliance (110,115) according to claim 6, wherein the control unit (130) is adapted to select from the streams one with the best SNR.

8. Household appliance (110,115) according to one of the above claims, wherein the control unit (130) is adapted to perform speech recognition on a stream of audio data.

9. Household appliance (110,115) according to one of the above claims, wherein the appliance (110,115) comprises a kitchen appliance (110,115), a dish care appliance (110, 115) or a laundry treating appliance (110,115).

10. System (100), comprising at least two household appliances (110,115) accord- ing to one of the above claims.

Images