Method, apparatus and non-transitory medium for selecting a device for audio processing

Abstract

A method for selecting a device for audio processing may involve receiving a first wake-up word confidence metric from a first device including at least a first microphone and receiving a second wake-up word confidence metric from a second device including at least a second microphone. The first and second wake-up word confidence metrics may correspond to a first local maximum of a first plurality of wake-up word confidence values ​​determined by the first device and a second local maximum of a second plurality of wake-up word confidence values ​​determined by the second device. The method may include comparing the first wake-up word confidence metric with the second wake-up word confidence metric and selecting a device for subsequent audio processing based at least in part on the comparison of the first wake-up word confidence metric with the second wake-up word confidence metric.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 880,112, filed on July 30, 2019; and U.S. Provisional Patent Application No. 62 / 964,018, filed on January 21, 2020, which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to systems and methods for automatically selecting a device from among a plurality of devices in an environment for audio processing. Background Art

[0004] Audio devices, including but not limited to smart audio devices, have been widely deployed and are becoming a common feature of many homes. While existing systems and methods for locating audio devices provide benefits, improved systems and methods would be desirable.

[0005] Symbols and nomenclature

[0006] In this document, we use the expression "smart audio device" to refer to a smart device that is either a single-purpose audio device or a virtual assistant (e.g., a connected virtual assistant). A single-purpose audio device is a device (e.g., a smart speaker, a television (TV), or a mobile phone) that includes or is coupled to at least one microphone (and in some examples may also include or be coupled to at least one speaker) and is largely or primarily designed to achieve a single purpose. While TVs can typically play (and are considered capable of playing) audio from program material, in most cases modern TVs run some kind of operating system on which applications run locally, including applications for watching TV. Similarly, the audio input and output in a mobile phone can do a lot of things, but these are all served by applications running on the phone. In this sense, a single-purpose audio device with (one or more) speakers and (one or more) microphones is often configured to run local applications and / or services to directly use the (one or more) speakers and (one or more) microphones. Some single-purpose audio devices can be configured to be grouped together to enable audio to be played on zones or user-configured areas.

[0007] In this document, a "virtual assistant" (e.g., a connected virtual assistant) is a device (e.g., a smart speaker, smart display, or voice assistant integrated device) that includes or is coupled to at least one microphone (and optionally also includes or is coupled to at least one speaker) and that can provide the ability to use multiple devices (other than the virtual assistant) for applications that are in some sense supported by the cloud or otherwise not implemented in or on the virtual assistant itself. Virtual assistants can sometimes work together, for example, in very discrete and conditionally defined ways. For example, two or more virtual assistants can work together in the sense that one of them (i.e., the one that is most confident of hearing the wake word) responds to the word. The connected devices can form a constellation, which can be managed by a master application, which can be (or include or implement) a virtual assistant.

[0008] In this document, "wake-up word" is used in a broad sense to refer to any sound (e.g., a word uttered by a human, or some other sound) that a smart audio device is configured to wake up in response to detecting ("hearing") the sound (through use of at least one microphone included in or coupled to the smart audio device, or at least one other microphone). In this context, "waking up" means that the device enters a state in which it is waiting for (i.e., listening for) a voice command.

[0009] In this document, the expression "wake-up word detector" refers to a device (or software including instructions for configuring the device) that is configured to continuously search for alignment between real-time sound (e.g., speech) features and a trained model. Typically, a wake-up word event is triggered whenever the wake-up word detector determines that the probability that a wake-up word has been detected exceeds a predefined threshold. For example, the threshold can be a predetermined threshold that is tuned to give a good compromise between a false acceptance rate and a false rejection rate. After a wake-up word event, the device may enter a state (which may be referred to as an "awake" state or an "attention" state) in which it listens for commands and passes received commands to a larger, more computationally intensive recognizer.

[0010] Throughout this disclosure, including in the claims, "speaker" and "speaker" are used synonymously to refer to any sound-producing transducer (or collection of transducers) driven by a single speaker feed. A typical headphone set includes two speakers. A speaker may be implemented to include multiple transducers (e.g., a woofer and a tweeter), all driven by a single common speaker feed. In some cases, the speaker feed may undergo different processing in different circuitry branches coupled to different transducers.

[0011] Throughout this disclosure, including in the claims, statements that an operation is “performed on” a signal or data (e.g., filtering, scaling, transforming, or applying a gain to the signal or data) are used broadly to mean performing the operation directly on the signal or data, or on a processed version of the signal or data (e.g., a version of the signal that has been initially filtered or preprocessed before the operation is performed on it).

[0012] Throughout this disclosure, including in the claims, the expression "system" is used in a broad sense to refer to a device, system, or subsystem. For example, a subsystem that implements a decoder may be referred to as a decoder system, and a system including such a subsystem (e.g., a system that generates X output signals in response to multiple inputs, where the subsystem generates M inputs and the other XM inputs are received from external sources) may also be referred to as a decoder system.

[0013] Throughout this disclosure, including in the claims, the term "processor" is used in a broad sense to refer to a system or device that is programmable or otherwise configurable (e.g., using software or firmware) to perform operations on data (e.g., audio or video or other image data). Examples of processors include field programmable gate arrays (or other configurable integrated circuits or chipsets), digital signal processors that are programmed and / or otherwise configured to perform pipeline processing on audio or other sound data, programmable general purpose processors or computers, and programmable microprocessor chips or chipsets. Summary of the invention

[0014] At least some aspects of the present disclosure may be implemented via methods. Some methods may involve receiving a first wake-up word confidence metric from a first device including at least a first microphone. For example, the first wake-up word confidence metric may correspond to a first local maximum of a first plurality of wake-up word confidence values ​​determined by the first device. Some such methods may involve receiving a second wake-up word confidence metric from a second device including at least a second microphone. For example, the second wake-up word confidence metric may correspond to a second local maximum of a second plurality of wake-up word confidence values ​​determined by the second device. Some such methods may involve comparing the first wake-up word confidence metric with the second wake-up word confidence metric and selecting a device for subsequent audio processing based at least in part on the comparison of the first wake-up word confidence metric with the second wake-up word confidence metric.

[0015] In some examples, a method may involve receiving a third wake word confidence metric from a third device including at least a third microphone. The third wake word confidence metric may correspond to a third local maximum of a third plurality of wake word confidence values ​​determined by the third device. A method may involve comparing the third wake word confidence metric with the first wake word confidence metric and the second wake word confidence metric, and selecting a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric, the second wake word confidence metric, and the third wake word confidence metric.

[0016] According to some implementations, the subsequent audio processing may involve a speech recognition process. In some examples, the subsequent audio processing may involve a command recognition process. Some such methods may also involve controlling the selected device according to the command recognition process.

[0017] In some examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold. In some such examples, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold. According to some such implementations, a local maximum may be determined by detecting an audio frame after a previous wake word confidence value has exceeded the wake word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the wake-up word confidence value of the first device, the second device, or another device with a decrease compared to the wake-up word confidence value of the second device, where k is an integer. Some such examples may involve initiating a local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device exceeds the wake-up word detection start threshold with a rising edge. Some such examples may involve terminating the local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device drops below the wake-up word detection end threshold.

[0018] In some cases, the first device may sample the audio data received by the first microphone according to a first clock domain. In some such examples, the second device may sample the audio data received by the second microphone according to a second clock domain different from the first clock domain.

[0019] According to some implementations, the method may be performed by a device configured to at least partially perform subsequent audio processing. Alternatively or additionally, the method may be performed by a device configured to determine a wake-up word confidence metric corresponding to a local maximum of a plurality of wake-up word confidence values. However, in some cases, the method may be performed at least in part by a device that is not configured to perform subsequent audio processing.

[0020] At least some aspects of the present disclosure may be implemented via alternative methods. Some alternative methods may involve determining a first wake-up word confidence metric by a first device including a first microphone system having at least a first microphone. Determining the first wake-up word confidence metric may involve generating first audio data corresponding to the detected sound via the first microphone system and determining a first plurality of wake-up word confidence values ​​based on the first audio data. Determining the first wake-up word confidence metric may involve determining a first local maximum of the first plurality of wake-up word confidence values ​​and determining the first wake-up word confidence metric based on the first local maximum.

[0021] Some such methods may involve receiving a second wake word confidence metric from a second device including at least a second microphone. The second wake word confidence metric may correspond to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device. Some such methods may involve comparing the first wake word confidence metric with the second wake word confidence metric and selecting a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric with the second wake word confidence metric.

[0022] In some examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold. In some cases, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold. According to some implementations, a local maximum may be determined by detecting an audio frame after a previous wake word confidence value has exceeded the wake word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the decrease in the wake-up word confidence value to that of , where k is an integer.

[0023] Some such implementations may also involve initiating a local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device exceeds the wake-up word detection start threshold with a rising edge. According to some examples, the local maximum determination time interval may be at time A starts and can be A+K ) terminates, the moment ( A+K ) is the moment when the wake-up word confidence values ​​of the first device and the second device fall below the wake-up word detection end threshold. In some examples, the wake-up word detection end threshold may be less than or equal to the wake-up word detection start threshold. In some implementations, the wake-up word detection end threshold may be less than or equal to the wake-up word detection start threshold. Some such examples may involve terminating the local maximum determination time interval after a maximum value of K has been reached.

[0024] In some examples, the method may be performed by a device configured to at least partially perform subsequent audio processing. According to some implementations, the method may be performed by a first device.

[0025] Some implementations may involve receiving a first wake word confidence metric from a first device. In some such implementations, the method may be performed by a device that is not configured to determine a wake word confidence metric corresponding to a local maximum of a plurality of wake word confidence values.

[0026] Some or all of the operations, functions, and / or methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc. Thus, some innovative aspects of the subject matter described in this disclosure may be implemented in non-transitory media having software stored thereon.

[0027] For example, the software may include instructions for controlling one or more devices to perform a method involving receiving a first wake word confidence metric from a first device including at least a first microphone. For example, the first wake word confidence metric may correspond to a first local maximum of a first plurality of wake word confidence values ​​determined by the first device. Some such methods may involve receiving a second wake word confidence metric from a second device including at least a second microphone. For example, the second wake word confidence metric may correspond to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device. Some such methods may involve comparing the first wake word confidence metric with the second wake word confidence metric and selecting a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric with the second wake word confidence metric.

[0028] In some examples, a method may involve receiving a third wake word confidence metric from a third device including at least a third microphone. The third wake word confidence metric may correspond to a third local maximum of a third plurality of wake word confidence values ​​determined by the third device. A method may involve comparing the third wake word confidence metric with the first wake word confidence metric and the second wake word confidence metric, and selecting a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric, the second wake word confidence metric, and the third wake word confidence metric.

[0029] According to some implementations, the subsequent audio processing may involve a speech recognition process. In some examples, the subsequent audio processing may involve a command recognition process. Some such methods may also involve controlling the selected device according to the command recognition process.

[0030] In some examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold. In some such examples, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold. According to some such implementations, a local maximum may be determined by detecting an audio frame after a previous wake word confidence value has exceeded the wake word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the wake-up word confidence value of the first device, the second device, or another device with a decrease compared to the wake-up word confidence value of the second device, where k is an integer. Some such examples may involve initiating a local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device exceeds the wake-up word detection start threshold with a rising edge. Some such examples may involve terminating the local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device drops below the wake-up word detection end threshold.

[0031] In some cases, the first device may sample the audio data received by the first microphone according to a first clock domain. In some such examples, the second device may sample the audio data received by the second microphone according to a second clock domain different from the first clock domain.

[0032] According to some implementations, the method may be performed by a device configured to at least partially perform subsequent audio processing. Alternatively or additionally, the method may be performed by a device configured to determine a wake-up word confidence metric corresponding to a local maximum of a plurality of wake-up word confidence values. However, in some cases, the method may be performed at least in part by a device that is not configured to perform subsequent audio processing.

[0033] At least some aspects of the present disclosure may be implemented via alternative methods. Some alternative methods may involve determining a first wake-up word confidence metric by a first device including a first microphone system having at least a first microphone. Determining the first wake-up word confidence metric may involve generating first audio data corresponding to the detected sound via the first microphone system and determining a first plurality of wake-up word confidence values ​​based on the first audio data. Determining the first wake-up word confidence metric may involve determining a first local maximum of the first plurality of wake-up word confidence values ​​and determining the first wake-up word confidence metric based on the first local maximum.

[0034] Some such methods may involve receiving a second wake word confidence metric from a second device including at least a second microphone. The second wake word confidence metric may correspond to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device. Some such methods may involve comparing the first wake word confidence metric with the second wake word confidence metric and selecting a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric with the second wake word confidence metric.

[0035] In some examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold. In some cases, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold. According to some implementations, a local maximum may be determined by detecting an audio frame after a previous wake word confidence value has exceeded the wake word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the decrease in the wake-up word confidence value to that of , where k is an integer.

[0036] Some such implementations may also involve initiating a local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device exceeds the wake-up word detection start threshold with a rising edge. According to some examples, the local maximum determination time interval may be at time A starts and can be A+K ) terminates, the moment ( A+K ) is the moment when the wake-up word confidence values ​​of the first device and the second device fall below the wake-up word detection end threshold. In some examples, the wake-up word detection end threshold may be less than or equal to the wake-up word detection start threshold. In some implementations, the wake-up word detection end threshold may be less than or equal to the wake-up word detection start threshold. Some such examples may involve terminating the local maximum determination time interval after a maximum value of K has been reached.

[0037] In some examples, the method may be performed by a device configured to at least partially perform subsequent audio processing. According to some implementations, the method may be performed by a first device.

[0038] Some implementations may involve receiving a first wake word confidence metric from a first device. In some such implementations, the method may be performed by a device that is not configured to determine a wake word confidence metric corresponding to a local maximum of a plurality of wake word confidence values.

[0039] At least some aspects of the present disclosure may be implemented via an apparatus. For example, one or more devices may be capable of at least partially performing the methods disclosed herein. In some implementations, the apparatus may include an interface system and a control system. The control system may include one or more general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or combinations thereof.

[0040] According to some examples, a control system may be configured to receive a first wake-up word confidence metric from a first device including at least a first microphone. For example, the first wake-up word confidence metric may correspond to a first local maximum of a first plurality of wake-up word confidence values ​​determined by the first device. The control system may be configured to receive a second wake-up word confidence metric from a second device including at least a second microphone. For example, the second wake-up word confidence metric may correspond to a second local maximum of a second plurality of wake-up word confidence values ​​determined by the second device. The control system may be configured to compare the first wake-up word confidence metric with the second wake-up word confidence metric and select a device for subsequent audio processing based at least in part on the comparison of the first wake-up word confidence metric with the second wake-up word confidence metric.

[0041] In some examples, the control system can be configured to receive a third wake-up word confidence metric from a third device including at least a third microphone. The third wake-up word confidence metric can correspond to a third local maximum of a third plurality of wake-up word confidence values ​​determined by the third device. The control system can be configured to compare the third wake-up word confidence metric with the first wake-up word confidence metric and the second wake-up word confidence metric, and select a device for subsequent audio processing based at least in part on the comparison of the first wake-up word confidence metric, the second wake-up word confidence metric, and the third wake-up word confidence metric.

[0042] According to some implementations, the subsequent audio processing may involve a speech recognition process. In some examples, the subsequent audio processing may involve a command recognition process. In some implementations, the control system may be configured to control the selected device according to the command recognition process.

[0043] In some examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold. In some such examples, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold. According to some such implementations, a local maximum may be determined by detecting an audio frame after a previous wake word confidence value has exceeded the wake word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the wake-up word confidence value of the first device, the second device, or another device with the wake-up word confidence value of the second device, where k is an integer. Some such examples may involve initiating a local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device exceeds the wake-up word detection start threshold on a rising edge. Some such examples may involve terminating the local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device drops below the wake-up word detection end threshold.

[0044] In some cases, the first device may sample the audio data received by the first microphone according to a first clock domain. In some such examples, the second device may sample the audio data received by the second microphone according to a second clock domain different from the first clock domain.

[0045] According to some implementations, the control system functionality may be performed by a device configured to at least partially perform subsequent audio processing. Alternatively or additionally, the control system functionality may be performed by a device configured to determine a wake-up word confidence metric corresponding to a local maximum of a plurality of wake-up word confidence values. However, in some cases, the control system functionality may be performed at least in part by a device that is not configured to perform subsequent audio processing.

[0046] At least some aspects of the present disclosure may be implemented via alternative devices. The control systems of some alternative devices may be configured to determine a first wake-up word confidence metric by a first device including a first microphone system having at least a first microphone. Determining the first wake-up word confidence metric may involve generating first audio data corresponding to the detected sound via the first microphone system and determining a first plurality of wake-up word confidence values ​​based on the first audio data. Determining the first wake-up word confidence metric may involve determining a first local maximum of the first plurality of wake-up word confidence values ​​and determining the first wake-up word confidence metric based on the first local maximum.

[0047] The control system may be configured to receive a second wake word confidence metric from a second device including at least a second microphone. The second wake word confidence metric may correspond to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device. The control system may be configured to compare the first wake word confidence metric with the second wake word confidence metric and select a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric with the second wake word confidence metric.

[0048] In some examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold. In some cases, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold. According to some implementations, a local maximum may be determined by detecting an audio frame after a previous wake word confidence value has exceeded the wake word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the decrease in the wake-up word confidence value to that of , where k is an integer.

[0049] Some such implementations may also involve initiating a local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device exceeds the wake-up word detection start threshold with a rising edge. According to some examples, the local maximum determination time interval may be at time A starts and can be A+K ) terminates, the moment ( A+K ) is the moment when the wake-up word confidence values ​​of the first device and the second device fall below the wake-up word detection end threshold. In some examples, the wake-up word detection end threshold may be less than or equal to the wake-up word detection start threshold. In some implementations, the wake-up word detection end threshold may be less than or equal to the wake-up word detection start threshold. Some such examples may involve terminating the local maximum determination time interval after a maximum value of K has been reached.

[0050] In some examples, at least some of the above operations may be performed by a device configured to at least partially perform subsequent audio processing. According to some implementations, such operations may be performed by a first device.

[0051] Some implementations may involve receiving a first wake word confidence metric from a first device. In some such implementations, some operations may be performed by a device that is not configured to determine a wake word confidence metric corresponding to a local maximum of a plurality of wake word confidence values.

[0052] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Note that the relative dimensions of the drawings may not be drawn to scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1A Represents an environment according to an example.

[0054] Figure 1B Represents an environment according to another example.

[0055] Figure 2An example of wake-up word confidence value curves determined by three devices is shown.

[0056] Figure 3 is a block diagram illustrating an example of components of an apparatus capable of implementing various aspects of the present disclosure.

[0057] Figure 4 is an overview that can be represented by Figure 3 A flowchart of an example of a method performed by the apparatus shown in FIG.

[0058] Figure 5 is a block diagram illustrating an example of elements (and processing steps implemented thereby) according to one disclosed embodiment.

[0059] Figure 6 is an overview that can be represented by Figure 3 Flowchart of another example of a method executed by the apparatus shown in .

[0060] Like reference numbers and designations in the various drawings indicate like elements.

[0061] Specific implementation method

[0062] An integrated system consisting of multiple smart audio devices can be configured to determine when a "wake word" (as defined above) is detected from a user. At least some devices of such a system can be configured to listen for commands from the user.

[0063] Figure 1A is a schematic diagram of an environment (living space), which includes a system including a set of smart audio devices (device 1.1) for audio interaction, speakers (1.3) for audio output, microphones 1.5 and controllable lights (1.2). As with other figures of this application, Figure 1A The specific elements and arrangements of elements shown in are by way of example only. All of these features may not be required to perform the various disclosed implementations. For example, for at least some disclosed implementations, controllable lights 1.2, speakers 1.3, etc. are optional. In some cases, one or more of the microphones 1.5 may be part of or associated with one of the devices 1.1, lights 1.2, or speakers 1.3. Alternatively or additionally, one or more of the microphones 1.5 may be attached to another part of the environment, for example, to a wall, ceiling, furniture, appliance, or another device of the environment. In the example, each smart audio device 1.1 includes at least one microphone 1.5 (and / or is configured to communicate therewith). Figure 1A The system can be configured to implement the embodiments of the present disclosure. Using various methods, it is possible to Figure 1A The microphones 1.5 collectively obtain information and provide the information to a device configured to provide an estimate of the position of the user who spoke the wake-up word.

[0064] In living spaces (e.g. Figure 1A In the space of a person, there is a set of natural activity zones where a person will perform tasks or activities, or cross thresholds. In some examples, these areas may be referred to herein as user zones and may be defined by the user without specifying coordinates or other markers of geometric locations. Figure 1A In the example shown in , a user zone may include:

[0065] 1. Kitchen sink and food preparation area (in the upper left area of ​​the living space);

[0066] 2. Refrigerator door (to the right of the sink and food preparation area);

[0067] 3. Dining area (in the lower left area of ​​the living space);

[0068] 4. The open area of ​​the living space (to the right of the sink and food preparation and dining areas);

[0069] 5. TV sofa (on the right side of the open area);

[0070] 6. TV itself;

[0071] 7. Table; and

[0072] 8. Door area or porch (in the upper right area of ​​the living space).

[0073] According to some embodiments, a system that estimates where a sound (e.g., a wake word or other attention-getting signal) occurs or originates can have some determined confidence (or multiple hypotheses) in that estimate. For example, if a user happens to be near a boundary between zones of the system's environment, then the uncertain estimate of the user's location can include a determined confidence that the user is in each zone. In some conventional implementations of voice interfaces, it may be required that the voice assistant's speech be uttered from only one location at a time, which forces a single selection of a single location (e.g., one of eight speaker locations, 1.1 and 1.3, in Figure 1A However, based on simple hypothetical role playing, it is apparent that (in such conventional implementations) the selected location of the source of the assistant's speech (e.g., including the location of a speaker in or configured for communicating with the assistant) may be less likely to serve as a focal point or a natural return response expressing attention.

[0074] Next, refer to Figure 1B, we describe an example of a system including another environment 100 (acoustic space) of a user (101) uttering direct speech 102, and a set of smart audio devices (103, 105, and 107), speakers for audio output, and microphones. The system can be configured according to embodiments of the present disclosure. The speech uttered by the user 101 (sometimes referred to herein as a speaker) can be recognized as a wake-up word by (one or more) elements of the system.

[0075] More specifically, Figure 1B The system components include:

[0076] 102: direct local speech (generated by user 101);

[0077] 103: voice assistant device (coupled to one or more speakers). Device 103 is closer to user 101 than device 105 or device 107, so device 103 is sometimes referred to as a "near" device, device 105 can be referred to as a "mid-range" device, and device 107 can be referred to as a "far" device;

[0078] 104: a plurality of microphones in (or coupled to) the proximity device 103;

[0079] 105: mid-range voice assistant device (coupled to one or more speakers);

[0080] 106: a plurality of microphones in (or coupled to) the mid-range device 105;

[0081] 107: a remote voice assistant device (coupled to one or more speakers);

[0082] 108: a plurality of microphones in (or coupled to) the remote device 107;

[0083] 109: Household appliances (e.g., lamps); and

[0084] 110: Multiple microphones in (or coupled to) household appliance 109. In some examples, each microphone 110 can be configured to communicate with a device configured to implement one or more of the disclosed methods, which in some cases can be at least one of devices 103, 105, or 107.

[0085] When a speaker 101 emits a sound 102 indicating a wake word in an acoustic space, the sound is received by a nearby device 103, a mid-range device 105, and a far-away device 107. In this example, each of devices 103, 105, and 107 is (or includes) a wake word detector, and each of devices 103, 105, and 107 is configured to determine when the wake word likelihood (the probability that the wake word is detected by the device) exceeds a predefined threshold. Over time, the wake word likelihood determined by each device can be plotted as a function of time.

[0086] Figure 2 An example of wake-up word confidence value curves determined by three devices is shown. Figure 2 The dotted curve 205a shown in FIG. 1 indicates the wake word likelihood as a function of time, as determined by the near device 103. The dashed curve 205b indicates the wake word likelihood as a function of time, as determined by the mid-range device 105. The solid curve 205c indicates the wake word likelihood as a function of time, as determined by the far device 107.

[0087] If from inspection Figure 2 It can be seen that over time, the likelihood of the wake word determined by each of devices 103, 105, and 107 increases and then decreases (e.g., as it passes into and out of the history buffer of the relevant device). In some cases, the wake word confidence of the distant device ( Figure 2 The solid line in the figure) may be the wake-up word confidence of the medium-distance device ( Figure 2 The wake-up word confidence of the medium-distance device may also exceed the threshold before the wake-up word confidence of the near device ( Figure 2 When the wake-up word confidence of the nearby device reaches its local maximum (e.g., Figure 2 ), this event is usually ignored (by conventional methods) in favor of selecting the device whose wake word confidence (wake word likelihood) first exceeds the threshold (in Figure 2 Remote device in the example).

[0088] Return to Figure 1B, the system may include at least one device configured to implement one or more methods of selecting a device for audio processing disclosed herein. For example, device 103, device 105, and / or device 107 may be configured to implement one or more such methods. Alternatively or additionally, another device configured to communicate with device 103, device 105, and / or device 107 may be configured to implement one or more such methods. In some examples, one or more disclosed methods may be implemented by another local device (e.g., a device within environment 100), while in other examples, one or more disclosed methods may be implemented by a remote device (e.g., a server) located outside environment 100.

[0089] Figure 3 300 is a block diagram illustrating an example of components of an apparatus capable of implementing various aspects of the present disclosure. According to some examples, apparatus 300 may be or may include an intelligent audio device configured to perform at least some of the methods disclosed herein. In other implementations, apparatus 300 may be or may include another device configured to perform at least some of the methods disclosed herein. In some such implementations, apparatus 300 may be or may include a server.

[0090] In this example, apparatus 300 includes an interface system 305 and a control system 310. In some implementations, interface system 305 can be configured to receive input from each of a plurality of microphones in an environment. Interface system 305 can include one or more network interfaces and / or one or more external device interfaces (such as one or more universal serial bus (USB) interfaces). According to some implementations, interface system 305 can include one or more wireless interfaces. Interface system 305 can include one or more devices for implementing a user interface, such as one or more microphones, one or more speakers, a display system, a touch sensor system, and / or a gesture sensor system. In some examples, interface system 305 can include control system 310 and a memory system (such as Figure 3 However, control system 310 may include a memory system.

[0091] The control system 310 may include, for example, a general purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, and / or discrete hardware components. In some implementations, the control system 310 may reside in more than one device. For example, a portion of the control system 310 may reside in Figure 1A and 1B, while another portion of the control system 310 may reside in a device outside the environment, such as a server, a mobile device (e.g., a smart phone or tablet), etc. In some such examples, the interface system 305 may also reside in more than one device.

[0092] In some implementations, the control system 310 may be configured to perform, at least in part, the methods disclosed herein. According to some examples, the control system 310 may be configured to implement a method of selecting a device for audio processing, such as, for example, the methods disclosed herein. In some such examples, the control system 310 may be configured to select a device for audio processing based, at least in part, on a comparison of multiple wake word confidence metrics.

[0093] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc. One or more non-transitory media may reside, for example, on Figure 3 315 and / or in the control system 310. Thus, various innovative aspects of the subject matter described in the present disclosure may be implemented in one or more non-transitory media having software stored thereon. For example, the software may include instructions for controlling at least one device to process audio data. For example, the software may be implemented by one or more components of the control system (such as Figure 3 The control system 310 is executed.

[0094] In some examples, apparatus 300 may include Figure 3 Optional microphone system shown in . Optional microphone system may include one or more microphones. According to some such examples, device 300 may be or may include a smart audio device. In some such implementations, device 300 may be or may include a wake-up word detector. For example, device 300 may be or may include a virtual assistant.

[0095] Figure 4 is an overview that can be represented by Figure 3 Flowchart of an example of a method performed by the apparatus shown in . The blocks of method 400, like other methods described herein, are not necessarily executed in the order indicated. Moreover, such methods may include more or fewer blocks than shown and / or described. In this implementation, method 400 involves selecting a device for audio processing. According to some examples, method 400 may be performed by a device configured to perform audio processing. However, in some examples, method 400 may be performed by a device that is not configured to perform audio processing.

[0096] In this example, block 405 involves receiving a first wake word confidence metric from a first device including at least a first microphone. According to this example, the first wake word confidence metric corresponds to a first local maximum of a first plurality of wake word confidence values ​​determined by the first device.

[0097] In this implementation, block 410 involves receiving a second wake word confidence metric from a second device including at least a second microphone. According to this example, the second wake word confidence metric corresponds to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device. In this example, the first device and the second device are in the same environment, which may be, for example, Figure 1A or Figure 1B The environment shown in .

[0098] However, based on the specific implementation, the first microphone and the second microphone may or may not be synchronized microphones. As used herein, microphones may be referred to as "synchronized" if the sounds detected by the microphones are digitally sampled using the same sampling clock or synchronized sampling clocks. For example, a first microphone among a plurality of microphones within an environment may sample audio data according to a first sampling clock, and a second microphone among the plurality of microphones may sample audio data according to the first sampling clock.

[0099] According to some alternative implementations, at least some microphones or microphone systems of an environment may be "asynchronous". As used herein, microphones may be referred to as "asynchronous" if the sounds detected by the microphones are digitally sampled using different sampling clocks. For example, a first microphone of a plurality of microphones within an environment may sample audio data according to a first sampling clock, and a second microphone of the plurality of microphones may sample audio data according to a second sampling clock. In some cases, the microphones in the environment may be randomly positioned, or at least may be distributed within the environment in an irregular and / or asymmetric manner.

[0100] Reference again Figure 2 , curve 205a provides an example of a first plurality of wake-up word confidence values ​​determined by the first device, and curve 205b provides an example of a second plurality of wake-up word confidence values ​​determined by the second device. Local maximum 210a provides an example of a first local maximum of the first plurality of wake-up word confidence values ​​determined by the first device, and local maximum 210b provides an example of a second local maximum of the second plurality of wake-up word confidence values ​​determined by the second device. In some examples, local maximum 210a may correspond to a first wake-up word confidence metric, and local maximum 210b may correspond to a second wake-up word confidence metric.

[0101] according to Figure 4 In the example shown in , block 415 involves comparing the first wake word confidence metric to the second wake word confidence metric. In this example, block 420 involves selecting a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric to the second wake word confidence metric. For example, block 420 may involve selecting a device that determines a higher wake word confidence metric.

[0102] According to some implementations, the subsequent audio processing may be or may include a speech recognition process. For example, the subsequent audio processing may be or may include a command recognition process. In some cases, method 400 may involve controlling a selected device according to the command recognition process. For example, method 400 may involve controlling a virtual assistant according to the command recognition process. In some such examples, method 400 may involve controlling a virtual assistant to initiate a phone call, controlling a virtual assistant to perform an Internet search, controlling a virtual assistant to provide instructions to another device (such as a television, a sound system controller, or another device in the environment).

[0103] In some examples, method 400 may involve receiving wake-up word confidence metrics from more than two devices in an environment. Some such examples may involve receiving a third wake-up word confidence metric from a third device including at least a third microphone. The third wake-up word confidence metric may correspond to a third local maximum of a third plurality of wake-up word confidence values ​​determined by the third device. In some such examples, method 400 may involve comparing the third wake-up word confidence metric with the first wake-up word confidence metric and the second wake-up word confidence metric and selecting a device for subsequent audio processing based at least in part on a comparison of the first wake-up word confidence metric, the second wake-up word confidence metric, and the third wake-up word confidence metric.

[0104] According to some examples, method 400 may involve performing a process from a first to a second step in an environment. N The device receives the first to N Wake-up word confidence measurement. N The wake-up word confidence measure can be compared with the first to the N The first to the second wake-up word confidence values ​​determined by the device N In some such examples, method 400 may involve comparing the first to the second N A wake word confidence measure and based at least in part on the first to the N Comparison of wake-word confidence metrics to select a device for subsequent audio processing.

[0105] In some implementations, blocks 405 and 410 may involve receiving a first wake-up word confidence metric and a second wake-up word confidence metric by a third device configured to determine a wake-up word confidence value and determine a local maximum of the wake-up word confidence value. In some such implementations, the third device may be configured to perform at least blocks 415 and 420 of method 400. In some implementations, the third device may be a local device. In some such implementations, all three devices may be or may include a wake-up word detector. One or more of the devices may be or may include a virtual assistant. However, in other implementations, the third device may be a remote device, such as a server.

[0106] According to some examples, the local maximum value may be determined after determining that the wake-up word confidence value exceeds a wake-up word detection start threshold, which may be a predetermined threshold. Figure 2 In some such examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold 215a. In some such examples, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold.

[0107] In some such implementations, a local maximum may be determined by a decrease in a wake-word confidence value of an audio frame detected after the previous wake-word confidence value has exceeded the wake-word detection start threshold compared to a wake-word confidence value of a previous audio frame (which may be the most recent audio frame or one of the most recent audio frames in some cases). For example, a local maximum may be determined by detecting an audio frame after the previous wake-word confidence value has exceeded the wake-word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the decrease in the wake-up word confidence value to that of , where k is an integer.

[0108] According to some such implementations, some methods may involve initiating a local maximum determination time interval after a wake-up word confidence value of a first device, a second device, or another device exceeds a wake-up word detection start threshold with a rising edge. Some such methods may involve terminating a local maximum determination time interval after a wake-up word confidence value of a first device, a second device, or another device falls below a wake-up word detection end threshold.

[0109] For example, refer again to Figure 2 In some such examples, when the wake-up word confidence value corresponding to any device in a group of devices exceeds the wake-up word detection start threshold 215a, the wake-up word detection may be started at the start time. AInitiate local maximum value determination time interval. In this example, the remote device first has a wake-up word confidence value that exceeds the wake-up word detection start threshold, and its time A is the time when curve 205c exceeds wake-up word detection start threshold 215a. According to this example, threshold 215b is the wake-up word detection end threshold. In this example, wake-up word detection end threshold 215b is less than (lower than) wake-up word detection start threshold 215a. In some alternative examples, wake-up word detection end threshold 215b may be equal to wake-up word detection start threshold 215a. In still other examples, wake-up word detection end threshold 215b may be greater than wake-up word detection start threshold 215a.

[0110] According to some examples, the local maximum value determination time interval may be terminated after the wake-up word confidence values ​​of all devices in the group fall below the wake-up word detection end threshold 215b. Figure 2 , when the wake-up word confidence value of the nearby device drops below the wake-up word detection end threshold 215b, the local maximum determination time interval can be equal to K time units and can be A+K End at the end time. A+K When the wake-up word confidence values ​​of the distant device and the medium-distance device have dropped below the wake-up word detection end threshold 215b. According to some examples, the local maximum determination time interval can end when the wake-up word confidence values ​​of all devices in the group drop below the wake-up word detection end threshold 215b or after the maximum value time interval has passed (whichever arrives first).

[0111] Figure 5 is a block diagram illustrating an example of components (and processing steps implemented thereby) according to one disclosed embodiment. Figure 5 In the example shown in , multiple microphones 501A-501N feed corresponding multiple wake-up word detectors 502A-502N. According to some such implementations, each device includes at least one of a microphone 501, a corresponding wake-up word detector 502, and a corresponding local maximum detector 503. For example, the wake-up word detectors 502A-502N may be Figure 1B Implementation of the wake-up word detection subsystem of devices 103, 105 and 107.

[0112] According to this example, a sequence of wake-up word confidence values ​​is determined by each of detectors 502A-502N, and each such sequence is fed to one of a plurality of local maximum detectors 503A-503N. In some such examples, each such value is ,in represents the number of wake-up word detectors 502, irepresents the detector index and n = represents the frame index. At some point after the wake-up word confidence (determined by one of the detectors 502A-502N) exceeds the predefined wake-up word detection start threshold, the wake-up word confidence typically begins to decrease. For example, one of the local maximum detectors 503A-503N may determine ,in k In one such implementation, one of the local maximum detectors 503A-503N may determine When the wake-up word confidence starts to decrease, in some implementations, the local maximum confidence value up to this point can be recorded. y i In some implementations, ,in Indicates the length of the associated history buffer.

[0113] According to some such implementations, each such local maximum confidence value may be provided to an element of a system implementing a device selector. Figure 5 In the example shown in , each of the local maximum detectors 503A-503N provides a corresponding one of the local maximum confidence values ​​504A-504N to the device selector 506. According to some examples, the device selector 506 can be implemented by a local device that implements one of the wake-up word detectors 502A-502N. In an alternative implementation, the device selector 506 can be implemented by a local device (e.g., a laptop, a phone, etc.) that does not implement one of the wake-up word detectors 502A-502N. In some examples, the device selector 506 can be implemented by a remote device (such as a server of a cloud-based service provider).

[0114] According to some examples, the maximum confidence level has been generated on all devices Afterwards, the index of the most trusted device is selected for subsequent voice capture. , which is the maximum confidence value For example, if the wake-up word detector closest to the user generates the largest confidence value, then the smart audio device in which (for which) this detector is implemented is caused to enter a focus state (and may assert an appropriate focus indication to the user) in which it awaits a subsequent voice command, and then, in response to such a voice command, the device may perform at least one predetermined action.

[0115] Figure 6 is an overview that can be represented by Figure 3Flowchart of another example of a method performed by the apparatus shown in . As with other methods described herein, the blocks of method 600 are not necessarily executed in the order indicated. Moreover, such methods may include more or fewer blocks than shown and / or described. In this implementation, method 600 involves selecting a device for audio processing. In this example, method 600 is performed by a device configured to determine a wake-up word confidence metric corresponding to a local maximum of a plurality of wake-up word confidence values. According to some examples, method 600 may be performed by a device configured to perform audio processing. However, in some examples, method 600 may be performed by a device that is not configured to perform audio processing.

[0116] In this example, box 605 involves determining a first wake-up word confidence metric by a first device including a first microphone system having at least a first microphone. In this example, determining the first wake-up word confidence metric involves generating first audio data corresponding to the detected sound via the first microphone system. According to this example, determining the first wake-up word confidence metric involves determining a first plurality of wake-up word confidence values ​​based on the first audio data and determining a first local maximum of the first plurality of wake-up word confidence values. In this implementation, determining the first wake-up word confidence metric involves determining the first wake-up word confidence metric based on the first local maximum. For example, determining the first wake-up word confidence metric may involve making the first wake-up word confidence metric equal to the first local maximum.

[0117] In this implementation, block 610 involves receiving a second wake word confidence metric from a second device including at least a second microphone. According to this example, the second wake word confidence metric corresponds to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device. In this example, the first device and the second device are in the same environment, which may be similar to Figure 1A or Figure 1B The environment shown in .

[0118] However, based on the specific implementation, the first microphone and the second microphone may or may not be synchronized microphones.According to some examples, a first microphone among the plurality of microphones within the environment may sample the audio data according to a first sampling clock, and a second microphone among the plurality of microphones may sample the audio data according to a second sampling clock.

[0119] according to Figure 6, block 615 involves comparing the first wake word confidence metric to the second wake word confidence metric. In this example, block 620 involves selecting a device for subsequent audio processing based at least in part on the comparison of the first wake word confidence metric to the second wake word confidence metric. For example, block 620 may involve selecting a device that determines a higher wake word confidence metric.

[0120] According to some implementations, the subsequent audio processing may be or may include a speech recognition process. For example, the subsequent audio processing may be or may include a command recognition process. In some cases, method 600 may involve controlling a selected device according to the command recognition process. For example, method 600 may involve controlling a virtual assistant according to the command recognition process. In some such examples, method 600 may involve controlling a virtual assistant to initiate a phone call, controlling a virtual assistant to perform an Internet search, controlling a virtual assistant to provide instructions to another device (such as a television, a sound system controller, or another device in the environment).

[0121] In some examples, method 600 may involve receiving wake-up word confidence metrics from more than two devices in an environment. Some such examples may involve receiving a third wake-up word confidence metric from a third device including at least a third microphone. The third wake-up word confidence metric may correspond to a third local maximum of a third plurality of wake-up word confidence values ​​determined by the third device. In some such examples, method 600 may involve comparing the third wake-up word confidence metric with the first wake-up word confidence metric and the second wake-up word confidence metric and selecting a device for subsequent audio processing based at least in part on a comparison of the first wake-up word confidence metric and the second wake-up word confidence metric with the third wake-up word confidence metric.

[0122] According to some examples, method 600 may involve performing a step of ... a step of performing a step of a step of performing a step of a step of performing a step of a step of a step of performing a step of a step of a step of a step N The device receives the first to N Wake-up word confidence measurement. N The wake-up word confidence measure can be compared with the first to the N The first to the second wake-up word confidence values ​​determined by the device N In some such examples, method 600 may involve comparing the first to the second N The wake-up word confidence metric is based at least in part on the first to the N Comparison of wake-word confidence metrics to select a device for subsequent audio processing.

[0123] In some implementations, method 600 may involve receiving a first wake-up word confidence metric and a second wake-up word confidence metric by a third device configured to determine a wake-up word confidence value and determine a local maximum of the wake-up word confidence value. In some such implementations, the third device may be configured to perform at least blocks 415 and 420 of method 400. In some implementations, the third device may be a local device. In some such implementations, all three devices may be or may include a wake-up word detector. One or more of the devices may be or may include a virtual assistant. However, in other implementations, the third device may be a local device that does not include a wake-up word detector and / or a device that is not configured to determine a wake-up word confidence metric corresponding to a local maximum of multiple wake-up word confidence values. According to some alternative implementations, the third device may be a remote device, such as a server.

[0124] According to some examples, the local maximum value may be determined after determining that the wake-up word confidence value exceeds a wake-up word detection start threshold, which may be a predetermined threshold. Figure 2 In some such examples, a local maximum may be determined after determining that the wake word confidence value exceeds the wake word detection start threshold 215a. In some such examples, a local maximum may be determined by detecting a decrease in the wake word confidence value after a previous wake word confidence value has exceeded the wake word detection start threshold.

[0125] In some such implementations, a local maximum may be determined by a decrease in a wake-word confidence value of an audio frame detected after the previous wake-word confidence value has exceeded the wake-word detection start threshold compared to a wake-word confidence value of a previous audio frame (which may be the most recent audio frame or one of the most recent audio frames in some cases). For example, a local maximum may be determined by detecting an audio frame after the previous wake-word confidence value has exceeded the wake-word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the decrease in the wake-up word confidence value to that of , where k is an integer.

[0126] According to some such implementations, some methods may involve initiating a local maximum determination time interval after a wake-up word confidence value of a first device, a second device, or another device exceeds a wake-up word detection start threshold with a rising edge. Some such methods may involve terminating a local maximum determination time interval after a wake-up word confidence value of a first device, a second device, or another device falls below a wake-up word detection end threshold.

[0127] According to some such methods, the local maximum determination time interval can be determined at time A starts and can be A+K) terminated. Figure 2 Some such methods are described. According to some examples, the moment ( A+K ) may be the time when the wake-up word confidence values ​​of the first device and the second device fall below the wake-up word detection end threshold. In some examples, the time ( A+K ) may be the time when the wake-up word confidence value of the first device, the second device, or another device falls below the wake-up word detection end threshold. In some examples, the wake-up word detection end threshold may be less than or equal to the wake-up word detection start threshold. Some implementations may involve K Some such implementations may involve terminating the local maximum determination time interval after the maximum value of has been reached. K The local maximum determination time interval is terminated after the maximum value of the wake-up word is reached, regardless of whether the wake-up word confidence value of the first device, the second device, or another device drops below the wake-up word detection end threshold.

[0128] Although specific embodiments and applications of the present disclosure have been described herein, it will be apparent to those of ordinary skill in the art that many changes may be made to the embodiments and applications described herein without departing from the scope of the present disclosure.

Claims

1. A method for selecting a device for audio processing, the method comprising: receiving a first wake word confidence metric from a first device including at least a first microphone, the first wake word confidence metric corresponding to a first local maximum of a first plurality of wake word confidence values ​​determined by the first device; receiving a second wake word confidence metric from a second device including at least a second microphone, the second wake word confidence metric corresponding to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device; comparing the first wake-up word confidence measure with the second wake-up word confidence measure; as well as selecting a device for subsequent audio processing based at least in part on a comparison of the first wake word confidence metric and the second wake word confidence metric, After determining that the wake-up word confidence value exceeds the wake-up word detection start threshold, a local maximum is determined. Wherein a local maximum is determined by detecting a decrease in the wake-up word confidence value after a previous wake-up word confidence value has exceeded a wake-up word detection start threshold.

2. The method of claim 1, further comprising: receiving a third wake word confidence metric from a third device including at least a third microphone, the third wake word confidence metric corresponding to a third local maximum of a third plurality of wake word confidence values ​​determined by the third device; comparing the third wake-up word confidence measure with the first wake-up word confidence measure and the second wake-up word confidence measure; as well as Based at least in part on a comparison of the first wake word confidence metric, the second wake word confidence metric, and the third wake word confidence metric, a device is selected for subsequent audio processing.

3. A method as claimed in claim 1 or claim 2, wherein the subsequent audio processing comprises a speech recognition process.

4. A method as claimed in claim 1 or claim 2, wherein the subsequent audio processing comprises a command recognition process.

5. The method of claim 4, further comprising controlling the selected device according to the command recognition process.

6. The method of claim 1, wherein the method comprises detecting an audio frame after a previous wake-up word confidence value has exceeded a wake-up word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the decrease in the wake-up word confidence value to that of , where k is an integer.

7. The method of claim 1, further comprising initiating a local maximum determination time interval after a wake-up word confidence value of the first device, the second device, or another device exceeds a wake-up word detection start threshold with a rising edge.

8. The method of claim 7 further includes terminating the local maximum determination time interval after the wake-up word confidence value of the first device, the second device, or another device drops below a wake-up word detection end threshold.

9. The method of claim 1 or claim 2, wherein: The first device samples audio data received by the first microphone according to the first clock domain; as well as The second device samples audio data received by the second microphone according to a second clock domain different from the first clock domain.

10. A method as claimed in claim 1 or claim 2, wherein the method is performed by a device configured to at least partially perform subsequent audio processing.

11. The method of claim 1 or claim 2, wherein the method is performed by a device configured to determine a wake-up word confidence metric corresponding to a local maximum of a plurality of wake-up word confidence values.

12. A method as claimed in claim 1 or claim 2, wherein the method is performed by a device that is not configured to perform subsequent audio processing.

13. A method for selecting a device for audio processing, the method comprising: Determining, by a first device including a first microphone system having at least a first microphone, a first wake word confidence metric, wherein determining the first wake word confidence metric includes: generating, via a first microphone system, first audio data corresponding to the detected sound; Determining a first plurality of wake-up word confidence values ​​based on the first audio data; Determining a first local maximum of a first plurality of wake-up word confidence values; and Determining a first wake-up word confidence measure based on the first local maximum; receiving a second wake word confidence metric from a second device including at least a second microphone, the second wake word confidence metric corresponding to a second local maximum of a second plurality of wake word confidence values ​​determined by the second device; comparing the first wake-up word confidence metric to the second wake-up word confidence metric; and selecting a device for subsequent audio processing based at least in part on a comparison of the first wake word confidence metric and the second wake word confidence metric, After determining that the wake-up word confidence value exceeds the wake-up word detection start threshold, a local maximum is determined. Wherein a local maximum is determined by detecting a decrease in the wake-up word confidence value after a previous wake-up word confidence value has exceeded a wake-up word detection start threshold.

14. The method of claim 13, wherein the method comprises detecting an audio frame after a previous wake-up word confidence value has exceeded a wake-up word detection start threshold. n The wake-up word confidence value and audio frame n The local maximum is determined by comparing the decrease in the wake-up word confidence value to that of , where k is an integer.

15. The method of claim 14, further comprising initiating a local maximum determination time interval after a wake-up word confidence value of the first device, the second device, or another device exceeds a wake-up word detection start threshold with a rising edge.

16. The method of claim 15, wherein the local maximum value determines the time interval at time A Start and at time ( A+K ) terminates, time ( A+K ) is the moment when the wake-up word confidence values ​​of the first device and the second device drop below the wake-up word detection end threshold.

17. The method of claim 15, wherein the local maximum value determines the time interval at time A Start and at time ( A +K ) terminates, time ( A+K ) is the moment when the wake-up word confidence value of the first device, the second device, or another device drops below the wake-up word detection end threshold.

18. The method of claim 16 or claim 17, wherein the wake-up word detection end threshold is less than or equal to the wake-up word detection start threshold.

19. The method of claim 16, further comprising terminating the local maximum determination time interval after a maximum value of K has been reached.

20. A method as claimed in claim 13 or claim 14, wherein the method is performed by a device configured to at least partially perform subsequent audio processing.

21. The method of claim 13 or claim 14, wherein the method is performed by a first device.

22. The method of claim 13 or claim 14 further comprises receiving a first wake-up word confidence metric from a first device, wherein the method is performed by a device that is not configured to determine a wake-up word confidence metric corresponding to a local maximum of multiple wake-up word confidence values.

23. An apparatus for selecting a device for audio processing, configured to perform the method according to any one of claims 1-22.

24. One or more non-transitory media storing software, the software comprising instructions for controlling one or more devices to perform the method of any one of claims 1-22.

Images