Processing an audio input signal with a trigger cue

Abstract

This disclosure relates to a method and system for processing and responding to an audio input signal including a trigger prompt. An example embodiment includes a computer-implemented method comprising: receiving (601) an audio input signal including a trigger prompt at a first device; processing (602) the audio input signal at the first device to determine a first quality metric of the audio input signal; transmitting (603) the first quality metric by the first device via a local network; monitoring (604) by the first device whether a higher second quality metric of the audio input signal is transmitted by a second device via the local network; and responding to the trigger prompt by the first device if the first device does not receive a higher second quality metric (608).

Description

Technical Field

[0001] This disclosure relates to a method and system for processing and responding to audio input signals, including trigger prompts. Background Technology

[0002] Voice user interfaces are now a popular way to interact with and control devices such as smartphones and smart speakers. A "wake-up word" is often used as the first step to cause the device to respond to subsequent voice commands. Wake-up words or trigger prompts can be of various types. Default trigger prompts could be, for example, "Hello Siri" for Apple devices, "Hello Google" for Google / Android devices, and "Alexa" for Amazon smart speakers and home automation systems. Trigger prompts can also be customized for specific devices or users.

[0003] Figure 1 An example scenario is illustrated where two users, Bob and Alice, each have a computer device, with Bob having a first device 101 and Alice having a second device 102. Devices 101 and 102 are mobile phones in this example, but could also be other types of computer devices. Bob provides a voice command 103, including a trigger prompt (“Hello NXP”), which is received by the first device 101 as a first audio signal 104 and also by the second device 102 as a second audio signal 105. Therefore, a trigger prompt from Bob may unintentionally trigger the second device 102 as well. Depending on the associated voice command, this can lead to confusion. Such multi-trigger scenarios also occur in situations where multiple people in close proximity wear wearable devices, and in smart home systems where multiple devices can be controlled by voice in a shared environment accessible to multiple users. Summary of the Invention

[0004] According to a first aspect, a computer-implemented method is provided, the computer-implemented method comprising: receiving an audio input signal including a trigger prompt at a first device; processing the audio input signal at the first device to determine a first quality metric of the audio input signal; transmitting the first quality metric by the first device via a local network; monitoring by the first device whether a higher second quality metric of the audio input signal is transmitted by a second device via the local network; and responding to the trigger prompt by the first device if the first device does not receive a higher second quality metric.

[0005] The audio input signal can be a voice command from the user.

[0006] The first quality metric may be an indicator of the sound quality of the audio input signal. The sound quality indicator may include the signal-to-noise ratio of the audio input signal.

[0007] The first quality metric may be an indicator of the signal amplitude of the audio input signal. The indicator of the signal amplitude may be the RMS amplitude of the audio input signal.

[0008] The first quality metric may include a combination of an indicator of the signal-to-noise ratio of the audio input signal and an indicator of the signal amplitude of the audio input signal. This combination may be defined by tuning parameters that depend on the acoustic environment.

[0009] The first quality metric can be wirelessly transmitted to the local network. The first quality metric can be transmitted along with a BLE announcement message.

[0010] If the first device does not receive a higher second quality metric within a predefined time period after transmitting the first quality metric via the local network, the first device may respond to a trigger prompt. The predefined time period is between approximately 50 ms and 200 ms, optionally approximately 100 ms.

[0011] According to a second aspect, a computer device is provided, the computer device comprising: a processor; an input / output interface; a microphone; and a network interface, wherein the processor is configured to: receive an audio input signal including a trigger prompt via the microphone and the input / output interface; process the audio input signal to determine a first quality metric of the audio input signal; transmit the first quality metric to a local network via the network interface; monitor via the network interface whether a higher second quality metric of the audio input signal is transmitted to the local network by another device; and respond to the trigger prompt if no higher second quality metric is received.

[0012] The processor may be configured to perform the other features described above that relate to the first aspect.

[0013] The computer device can be one of the following: a handheld portable electronic device, a mobile phone, a wearable electronic device, a smart home control unit, and a smart speaker.

[0014] According to a third aspect, a computer program is provided, the computer program including instructions to cause a computer processor to perform the method according to the first aspect.

[0015] A computer program may be provided that, when run on a computer, causes the computer to configure any device including the circuits, controllers, sensors, filters, or means disclosed herein, or to perform any of the methods disclosed herein. The computer program may be a software implementation, and the computer may be considered any suitable hardware, including digital signal processors, microcontrollers, and implementations in read-only memory (ROM), erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM), these being non-limiting examples. The software implementation may be an assembler.

[0016] Computer programs may be provided on a non-transient computer-readable medium or may be embodied as a transient signal. The non-transient computer-readable medium may be a physical computer-readable medium, such as an optical disc or storage device. This transient signal may be a network download, including an Internet download.

[0017] These and other aspects of the invention will be apparent from the embodiments described below, and will be illustrated with reference to those embodiments. Attached Figure Description

[0018] The embodiments are described using the reference drawings as examples only, wherein:

[0019] Figure 1 This is a schematic diagram illustrating an example of a trigger prompt received by the first and second devices;

[0020] Figure 2 This is a schematic diagram of an example triggered prompt received by a first and a second device, each configured to determine and compare quality metrics.

[0021] Figure 3 It is a graph comparing the sound quality measure (SQE) determined by the first and second devices;

[0022] Figure 4 This is an example drawing of the received audio input signal;

[0023] Figure 5 It is a plot of the root mean square (RMS) amplitude difference between the audio input signals received by the first and second devices;

[0024] Figure 6 This is a schematic flowchart illustrating a method implemented on an example computer; and

[0025] Figure 7 This is a schematic diagram of an example computer device.

[0026] It should be noted that the figures are illustrative and not drawn to scale. For clarity and convenience in the figures, the relative dimensions and scales of the parts have been shown as enlarged or reduced in size. The same reference numerals are generally used to refer to modified embodiments and corresponding or similar features in different embodiments. Detailed Implementation

[0027] Figure 2 The illustration schematically depicts a scenario where users Bob and Alice each have a corresponding first device 101 and a second device 102, where Bob issues a voice command 103 to his device 101. In this example, the voice command "Hello NXP, call Mom" ​​includes a trigger prompt (i.e., "Hello NXP") and an action to be performed by device 101 ("call Mom"). The action can be any other type of action that device 101 is capable of performing, such as playing specified music, performing an internet search for requested information, or sending a control signal to another device. Device 101 in this example is a handheld portable electronic device, in this case, in the form of a mobile phone. Alternatively, device 101 can be one of many different types of devices, such as wearable electronics, a smart home control unit, or a smart speaker.

[0028] First device 101 receives an audio input signal from Bob's voice command 103, but before responding to the command, it performs an execution check to determine whether the voice command is actually intended for execution by first device 101. As a first step, first device 101 analyzes the audio input signal to determine a first quality metric 201 of the audio input signal. Quality metric 201 may be, for example, a score of the quality of the received audio input signal, in this case a simple score from 0 to 5. In this example, first device 101 determines that quality metric 201 is 4 / 5, i.e., a relatively high quality metric. This first quality metric (QM) 201 is transmitted by first device 101 via a local network. The local network in this example is a wireless network, which may be, for example, a WiFi network (according to the IEEE 802.11x standard), a Bluetooth network, and / or a Bluetooth Low Energy (BLE) network. In some examples, the local network may be at least partially wired, such as in the case of a home automation system with one or more smart home control units.

[0029] Once the first device 101 has transmitted the first QM 201 via the local network, it monitors the local network to determine if any other device has transmitted a QM before taking any action. In this example, Alice's device 102, the second device, has also received an audio input signal from Bob's voice command 103 and, similarly configured to the first device 101, determines the QM of the audio input signal. This second QM 202 is also transmitted via the local network and received by the first device 201. Because Alice's device 102 is further away from Bob, the second QM 202 has a lower score than the first QM 201, in this example, 2 / 5. Alice's device 102 also monitors the local network after transmitting the second QM 202 and receives the first QM 201 transmitted from Bob's device 101.

[0030] After receiving the second QM 202, the first device 101 determines which QM is higher. In this example, the first QM 201 is higher than the second QM 202, indicating that the audio input signal is not intended for the second device 102. The first device 101 therefore determines that the voice command 103 is addressed to itself and responds to the trigger prompt along with any associated commands. After determining that the first QM 201 is higher than the second QM 202, the second device 102 determines that the voice command 103 in the received audio signal is not intended for itself and takes no action.

[0031] By using different quality metrics among devices that simultaneously receive the same voice command, and determining which device will respond to the voice command based on the higher (or highest) quality metric, this arrangement addresses the potential multi-triggering problem mentioned above.

[0032] To avoid perceptible delays in responding to voice commands, each device 101, 102 is configured to respond to a trigger cue if no higher second QM is received within a predefined time period or window following the transmission of the first QM over the local network. However, each device may begin processing the voice command before the end of the predefined time period, ensuring no delay between receiving the trigger cue and responding. If a higher second QM is received during the predefined time period, each device may cease processing. The predefined time period may be, for example, between approximately 50 ms and 200 ms, such as approximately 100 ms. This short time window allows different devices to detect the same voice command containing the trigger cue at slightly different times and to act on the same voice command. Each device configured to pause during this predefined time period allows detection by any other device that has also detected the same trigger cue and provided a higher quality metric. If no higher quality metric is received, or if any received quality metric is lower than the quality metric determined by the device, the device may continue verifying the trigger cue and proceed with the voice command. Any other device that has also received the voice command does not act and continues operating in listening mode.

[0033] Figure 3 This is a plot of the Sound Quality Estimate (SQE) of a pair of devices, with the SQE of the second device on the y-axis and the SQE of the first device on the x-axis. In this example, the SQE value is an indicator of the ratio of directly controlled speech beam energy to wide background noise during speech triggering, which can be used as a quality measure of the received audio input signal. A higher SQE indicates better speech capture by the device. The SQE measurement is normalized to any value between 0 and 1, where 1 is the maximum SQE. In the example implementation, the devices can use an absolute threshold to determine whether to take action in response to a detected trigger cue, for example, whether the SQE is higher than 0.95 in this case. In the first zone 301, the first device is prompted to respond to the detected trigger cue, while in the second zone 302, the second device is prompted to respond. In the third zone 303, where both devices have detected an SQE higher than the absolute threshold, the device that detected the higher SQE is prompted to respond to the trigger cue. A 'double trigger' event is caused only when both devices detect the same SQE, i.e., both devices respond to the trigger cue. When two devices detect different SQEs, a comparison between the different SQEs can be used to determine which device should respond to the trigger cue. The method described in this paper thus reduces double-triggering events.

[0034] Figure 4An audio input signal 401, representing a trigger prompt, is shown and plotted as its amplitude over time. For example, the amplitude of the audio input signal 401, measured as an average RMS amplitude, can be used to determine a quality metric of the audio input signal and to compare quality metrics between different devices. The quality metrics compared between different devices are relative rather than absolute, meaning that the absolute magnitude of the received signal does not affect the result.

[0035] Figure 5 A plot of RMS amplitude measurements 501 and 502 of a continuous audio input signal containing a trigger cue (or "wake-up word" / WW) received by corresponding first and second devices (TP device 1 and TP device 2, where TP is a true positive) is shown, where the measurements show the difference between the RMS amplitudes of the two devices. In the first measurement 503, the RMS index of the first device is 8.0563 dB greater than that of the second device, causing the first device to respond to the trigger cue. In the second measurement 504, the RMS index of the second device is 5.0125 dB greater than that of the first device, causing the second device to respond to the trigger cue.

[0036] Alternatively, in some cases, the quality metric can be a combination of the amplitude and tone quality estimation metrics mentioned above. It is expected that RMS amplitude-based quality metrics will tend to be more applicable in non-reverberant or free-field environments, while SQE quality metrics may be more applicable in reverberant environments.

[0037] In a general sense, speech quality metrics can be considered as varying with RMS amplitude and SQE metric, that is:

[0038]

[0039] in These are tuning parameters that can be set depending on the acoustic environment. For example, the tuning parameters can be set for a reverberant environment. And it is set for free field environments as .

[0040] Figure 6This is a schematic flowchart illustrating an example method for determining a response to an audio input signal including a trigger cue. In a first step 601, an audio input signal is received by a first device, the audio input signal including a trigger cue. In step 602, the first device processes the audio input signal to determine a first quality metric (QM) of the audio input signal. In step 603, the first device transmits this first QM via a local network and begins a predefined time window. In step 604, the first device monitors the local network for any QM that has been transmitted by any other device. The first device continues to monitor the local network until the predefined time window ends at step 605. The first device then determines at step 606 whether a higher QM has been received. If a higher QM has been received, then at step 607, the first device takes no action. If no higher QM has been received, then at step 608, the first device responds to the trigger cue, for example, by taking an action indicated by a command associated with the trigger cue.

[0041] Figure 7 This is a schematic diagram of an example computer device 700 configured to determine a response to an audio input signal, including a trigger prompt. The computer device 700 includes a processor 701, an input / output interface 702, a microphone 703, and a network interface 704. The input / output interface 702 is connected to receive audio input signals via the microphone 703, and the processor receives and transmits signals to and from the input / output interface 702. The processor 701 is also connected to the network interface 704 to send and receive signals to and from a local network. The network interface 704 may be a wired and / or wireless interface. The processor 701 may also be connected to a memory 705, and the input / output unit 702 may be connected to a speaker 706.

[0042] Processor 701 is configured to receive an audio input signal, including a trigger cue, via microphone 703 and input / output interface 702. The processor then processes the audio input signal to determine a first quality metric. The first quality metric is then transmitted to a local network via network interface 704. The processor monitors the local network via network interface 704 to see if a higher second quality metric of the audio input signal is transmitted to the local network from another device. If no higher second quality metric is received, processor 701 responds to the trigger cue, for example, by executing an action indicated by a voice command associated with the trigger cue.

[0043] The computer device 700 can be one of a handheld portable electronic device, a mobile phone, a wearable electronic device, a smart home control unit, and a smart speaker.

[0044] Various other optional features described above regarding the method for determining the response to an audio input signal can also be executed by the processor 701.

[0045] The advantage of the methods and apparatus disclosed herein is that devices closer to the user can be activated over other devices further away, without requiring user registration to differentiate between users. This functionality can be used, for example, in smart home environments where multiple devices can be operated by multiple users and users may want only one device to respond to voice commands. The functionality can also be used in wearable devices, such as portable voice-activated devices, where multiple users wearing such devices only need to activate their own devices in response to voice commands.

[0046] By reading this disclosure, those skilled in the art will understand other changes and modifications. Such changes and modifications may involve equivalents and other features known in the art of automated speech recognition systems that can replace or supplement the features described herein.

[0047] Although the appended claims are directed to specific combinations of features, it should be understood that the scope of the disclosure of this invention also includes any novel feature or combination of novel features or any generalized form thereof explicitly or implicitly disclosed herein, regardless of whether it relates to the same invention as claimed in any of the present claims or whether it alleviates the same technical problem as any or all of the technical problems alleviated by this invention.

[0048] Features described in the context of multiple embodiments may also be provided in combination in a single embodiment. Conversely, for the sake of brevity, various features described in the context of a single embodiment may also be provided individually or in any suitable sub-combination. The applicant hereby reminds that new claims may be formulated based on such features and / or combinations of such features during the examination of this application or any other application derived therefrom.

[0049] For the sake of completeness, it is also stipulated that the term "comprising" does not exclude other elements or steps, the term "a" does not exclude a plurality, a single processor or other unit can perform the functions of several components recited in the claims, and the reference numerals in the claims should not be interpreted as limiting the scope of the claims.

Claims

1. A computer-implemented method, characterized by, The method includes: An audio input signal including a trigger prompt is received (601) at the first device (101); The audio input signal is processed (602) at the first device (101) to determine a first quality metric (201) of the audio input signal; The first quality measure (201) is transmitted (603) by the first device (101) through the local network; The first device (101) monitors (604) whether a higher second quality metric (202) of the audio input signal is transmitted by the second device (102) through the local network; and If the first device (101) does not receive a higher second quality metric (202), the first device (101) responds to the trigger prompt (608).

2. The computer-implemented method of claim 1, wherein, The audio input signal is a voice command from the user (103).

3. The computer-implemented method of claim 1 or claim 2, wherein, The first quality metric (201) is an indicator of the sound quality of the audio input signal.

4. The computer-implemented method of claim 3, wherein, The audio quality metric includes the signal-to-noise ratio of the audio input signal.

5. The computer-implemented method of claim 1 or claim 2, wherein, The first quality metric (201) is an indicator of the signal amplitude of the audio input signal.

6. The computer-implemented method of claim 1 or claim 2, wherein, The first quality metric includes a combination of an index of the signal-to-noise ratio of the audio input signal and an index of the signal amplitude of the audio input signal.

7. The computer-implemented method of claim 6, wherein, The combination is defined by tuning parameters that depend on the acoustic environment.

8. The computer-implemented method of any preceding claim, wherein, If the first device (101) does not receive a higher second quality metric (202) within a predefined time period after transmitting the first quality metric (201) through the local network, the first device (101) responds to the trigger prompt (608).

9. A computer arrangement (700) characterized by include: Processor (701); Input / output interface (702); Microphone (703); and Network interface (704). The processor (701) is configured to: Audio input signals, including trigger prompts, are received via the microphone (703) and input / output interface (702); The audio input signal is processed to determine a first quality metric of the audio input signal (201). The first quality metric is transmitted to the local network via the network interface (704); The network interface (704) monitors whether a higher second quality metric (202) of the audio input signal is transmitted to the local network by another device (102); as well as If a higher second quality metric (202) is not received, then respond to the trigger prompt.

10. A computer program, characterized in that, The computer program includes instructions for causing the computer device (700) to perform the method according to any one of claims 1 to 8.

Images