Sensor Management for Wireless Devices - Patent application

Abstract

A system and method for selecting an audio capture sensor of a wearable device when acquiring audio data includes acquiring signals associated with a user's voice at first and second wearable devices, comparing the energy levels of the first and second signals, and selecting one or more audio capture sensors based on the energy levels of each signal. Due to the symmetry of the acoustic energy generated by the user's voice at the first and second wearable devices, any difference in energy level between the total energy acquired by the first wearable device and the total energy acquired by the second wearable device can be attributed solely to ambient noise. Therefore, a device with a higher total energy has a lower signal-to-noise ratio, and selection of an audio capture sensor of another wearable device with a higher signal-to-noise ratio is provided to acquire forward-going audio data.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. patent application Ser. No. 17 / 126,630, filed Dec. 18, 2020, and entitled “Sensor Management for Wireless Devices,” the entire disclosure of which is incorporated herein by reference. (Technical field) The present invention relates to sensor management for wireless devices. [Background technology]

[0002] Aspects and implementations of the present disclosure are generally directed to systems and methods for managing sensor data, e.g., for managing the capture of audio data from one or more audio capture sensors between wearable devices.

[0003] Wearable wireless audio devices, e.g., wireless headphones, often utilize a paired connection to stream wireless audio data from a source device. Typically, each wireless headphone receives a separate stream of data specific to each wireless headphone, e.g., the source device generates one stream of data associated with the left headphone and one stream associated with the right headphone. In some situations, each headphone may also utilize one or more microphones to capture signals corresponding to the user's voice, e.g., when the user is engaged in a telephone conversation in which the headphones act (via a Bluetooth connection) as both a receiver and a transmitter for the conversation. In some examples, only one headphone of a pair of headphones is used to capture signals corresponding to the user's voice.

[0004] Additionally, at various times, the user's surrounding environment may generate excessive ambient noise, e.g., from wind, traffic, machinery, etc. If the user orients one of the headphones, e.g., the headphones responsible for obtaining signals corresponding to the user's voice, such that it is closer to a source of ambient noise, e.g., in the direct path of wind, the quality of the voice pickup for that headphone will be degraded. Summary of the Invention [Means for solving the problem]

[0005] The present disclosure relates to systems and methods, and computer program products, for selecting one or more audio capture sensors of a wearable device for use in acquiring voice data. Examples provided include acquiring signals associated with a user's voice at a first and a second wearable device, comparing the energy levels of the first and second signals (where the first wearable device is positioned at the user's right ear and the second wearable device is positioned at the user's left ear), and selecting one or more audio capture sensors based on the energy levels of each signal. Due to the symmetry of the acoustic energy generated by the user's voice to the first and second wearable devices (because the device at the user's right ear is equidistant from the device at the user's left ear), any difference in energy levels between the total energy acquired by the first wearable device and the total energy acquired by the second wearable device can be attributed solely to ambient noise. Thus, the signal-to-noise ratio of the device with the higher total energy is lower, and the selection of one or more audio capture sensors of the other wearable device with the higher signal-to-noise ratio is preferred for acquiring the voice data going forward.

[0006] In some examples, the systems and methods discussed herein utilize wireless data transmission, specifically wireless topologies for broadcasting and transmitting audio streams between devices. For example, Core Specification 5.2 ("The Core Specification"), released by the Bluetooth Special Interest Group (SIG) on January 6, 2020, defines new features related to Bluetooth Low Energy (BLE) topologies. One feature described in the 5.2 Core Specification is Broadcast Isochronous Streams, which utilizes unconnected isochronous communication. A similar feature described by the 5.2 Core Specification is LE Connected Isochronous Streams, which utilizes a connection-oriented isochronous channel to provide a point-to-point isochronous communication stream between two devices, for example, between peripheral device 104 and wearable devices 102A and 102B (discussed below). In one example, the systems, devices, and methods discussed herein utilize a Bluetooth Low-Energy audio topology enabled by the 5.2 Core Specification (referred to herein as "LE Audio"). For example, LE Audio enables unicast wireless topologies (referred to as "connected isochronous streams") that allow a single Bluetooth audio source device (e.g., a smartphone) to simultaneously transmit multiple audio streams to separate Bluetooth devices, e.g., wireless headphones. These topologies are intended to improve Bluetooth operation for wireless headphones.

[0007] Specifically, when using wireless headphones or wireless wearable devices (discussed below), algorithms may be used to improve sound pickup using input from multiple microphones on the wearable device. Some of these algorithms minimize ambient noise, e.g., wind noise, by reducing the gain of microphones directly in the wind path. Other algorithms use binaural beamforming (using microphones placed near both the left and right ear) to maximize the signal-to-noise ratio. Many of these algorithms cannot be realized on truly wireless earbud form factors due to the high latency between the earbuds. These truly wireless earbuds generally use Bluetooth to transmit data between the earbuds, which can take up to 100 ms to reliably transmit microphone data between the earbuds. This exceeds the acceptable latency that some sound pickup algorithms can tolerate.

[0008] In this disclosure, sensor inputs (e.g., microphone inputs) are monitored on both the left and right wearable devices to determine which sensor set provides the best signal-to-noise ratio. In some examples using a classic Bluetooth connection, the "master" earbud (which maintains the Bluetooth connection to the peripheral device) is the device configured to transmit microphone audio data to the peripheral device (e.g., smartphone) during voice interaction. If, through the communication data link between the two earbuds, it is determined that the ambient noise, such as wind noise, is high on the master bud but low on the puppet bud, the two buds can initiate a role switch midway. In some examples, the system and method utilize one or more classic Bluetooth chip solutions that enable "seamless" role switching over the classic Bluetooth connection that may allow this to switch roles. This allows the master bud to transfer responsibility for maintaining the Bluetooth connection to the smartphone to the puppet bud, allowing the puppet bud to use its microphone (which has lower wind noise) for voice pickup instead of the master bud's microphone. To do so without role switching would require the puppet's microphone audio to be sent via Bluetooth to the master bud and then forwarded to the phone, which would introduce unacceptably long latency for real-time voice conversation.

[0009] A similar scheme can be employed for LE Audio connections. In LE Audio, each bud can be synchronized to a separate Connected Isochronous Stream (CIS) that is part of the same Connected Isochronous Group (CIG). Each bud can have knowledge of the other bud's CIS, either by monitoring the connection setup of the CIS for the other bud, or by exchanging information over a communication data link between the buds. For example, when used for a call, one of the two buds may transmit microphone audio to a negotiated CIS that is sent to the phone. Upon detecting significant ambient noise on its microphone audio stream, e.g., wind noise, the bud can perform a handshake with the other bud and "hand off" responsibility for transmitting microphone audio to the other bud. This allows the second bud to transmit its microphone data to the phone (with less wind noise) without the phone knowing that a switch has occurred.

[0010] In both classic Bluetooth and LE Audio implementations, to prevent a user from noticing differences in microphone gain when microphones are switched, automatic gain control parameters for each microphone may be exchanged over a communications data link between the buds before performing a role switch between the microphones.

[0011] LE audio capabilities can be leveraged in additional ways to improve voice pickup for true wireless buds. In one implementation, raw (unprocessed) audio data from one or two microphones can be sent from each bud to a receiving device (such as a phone). The phone can receive audio from all microphones and apply beamforming or other voice pickup algorithms to the raw audio data. This can be done in one or more applications or utilize voice algorithm "plug-ins" to other third party VOIP applications. In other implementations, one or more of these algorithms can be integrated into the VPA application or integrated into a cloud service. The algorithms can be optimally tuned for different devices and selected by the mobile application or cloud service based on identifying information from the device. Performing the voice processing on the phone or cloud side allows multi-mic voice algorithms to be deployed using microphones from both sides of the head, which is much more difficult to achieve for truly wireless buds due to delays in audio transmission from bud to bud.

[0012] In other implementations, to conserve Bluetooth bandwidth, each bud can do local beamforming of the two microphones and send only a single channel of voice audio to the phone. The phone or cloud can then run an algorithm (similar to that described above) on the processed beamformed audio from each bud and combine them to produce improved voice pickup / signal-to-noise ratio for the audio signal.

[0013] In another implementation, an active noise reduction (ANR) feedback microphone (pointing toward the user's ear) can be used to capture audio from one earbud in windy conditions, while an outward-facing microphone can be used to capture audio in the other bud. The ANR feedback microphone is less susceptible to wind noise (or other high-noise environments such as babble noise), but cannot pick up many high-frequency components. The outward-facing microphone in the other bud, when mixed with the audio audio from the ANR feedback microphone from the first earbud, allows for an extended bandwidth of audio capture while minimizing wind noise. As wind conditions change dynamically, the buds can coordinate swapping of which bud uses the ANR feedback microphone and which bud uses the outward-facing microphone to capture audio.

[0014] Both buds can also capture audio using ANR feedback and outward-facing microphones simultaneously, compress the microphone audio, and send multiple microphone channels to the source device where mixing can take place. An algorithm running on the source device can then dynamically select which microphones are used for mixing based on the environmental noise picked up by the outward-facing microphones in each bud. Any audio algorithms running independently on each side can also send key information (e.g., through metadata) to the source device, which then uses that information to determine which microphones to utilize for mixing (microphones from one bud or another, or a mix of both). Non-standard codecs can also be used for microphone audio transmission with lower encoding / decoding algorithm delay or higher compression, as appropriate for a given use case.

[0015] In other use cases, the output of the microphone mixing scheme described above can be used to make decisions regarding the volume level or ANR setting of a product. For example, based on the noise level detected by a microphone mixing algorithm running on the source device, a signal can be sent back to the product telling the product to adjust the volume level to compensate for the environmental noise. Similarly, information about the environmental noise can be sent back as a signal to the product to automatically adjust the ANR level.

[0016] In one example, a method for selecting one or more audio capture sensors of a wearable device is provided, the method including: detecting a first signal corresponding to a user's voice using at least one audio capture sensor of a first wearable device; detecting a second signal corresponding to the user's voice using at least one audio capture sensor of a second wearable device, where the second wearable device is wirelessly connected to the first wearable device; determining an energy level of the first signal and an energy level of the second signal; and selecting the at least one audio capture sensor of the first wearable device and / or the at least one audio capture sensor of the second wearable device to acquire voice data based at least in part on the energy level of the first signal and the energy level of the second signal.

[0017] In one aspect, the first wearable device is configured to transmit a first signal to the peripheral device, and the second wearable device is configured to transmit a second signal to the peripheral device, and the peripheral device is configured to compare an energy level of the first signal with an energy level of the second signal to select at least one audio capture sensor of the first wearable device and / or at least one audio capture sensor of the second wearable device to acquire audio data.

[0018] In one aspect, the at least one audio capture sensor of the first wearable device comprises a first set of multiple audio capture sensors, the first signal is an average energy level of the first multiple audio capture sensors, and the first wearable device is configured to transmit the first signal to the peripheral device using a compression algorithm.

[0019] In one aspect, the first wearable device is further configured to transmit metadata to the first peripheral device for use with a mixing algorithm executed by the peripheral device.

[0020] In one aspect, the mixing algorithm is arranged to generate an output, the peripheral device is configured to send the output to the first wearable device, and the first wearable audio device is configured to automatically change volume levels or active noise reduction (ANR) settings using the output.

[0021] In one aspect, the second wearable device is configured to transmit a second signal to the first wearable device, and the first wearable device is configured to compare an energy level of the first signal with an energy level of the second signal to select at least one audio capture sensor of the first wearable device and / or at least one audio capture sensor of the second wearable device to acquire audio data.

[0022] In one aspect, the energy levels of the first and second signals are related to at least one of a signal-to-noise ratio (SNR), a presence of wind, a radio frequency (RF) performance, or an audio pickup.

[0023] In one aspect, the at least one audio capture sensor of the first wearable device and the at least one audio capture sensor of the second wearable device are selected from at least one of an external microphone, an internal microphone, a feedback microphone also used for acoustic noise reduction (ANR) purposes, a feedforward microphone also used for ANR purposes, or an accelerometer.

[0024] In one aspect, the at least one audio capture sensor of the first wearable device includes a first set of multiple audio capture sensors, and selecting the at least one audio capture sensor includes dynamically changing between selecting a first audio capture sensor of the first set of multiple audio capture sensors and a second audio capture sensor of the first set of multiple audio capture sensors.

[0025] In one aspect, the at least one audio capture sensor of the first wearable device includes a first set of multiple audio capture sensors, and the energy level of the first signal is an average energy level of the first set of the multiple audio capture sensors.

[0026] In one aspect, the at least one audio capture sensor of the second wearable device includes a second set of the multiple audio capture sensors, and the energy level of the second signal is an average energy level of the second set of the multiple audio capture sensors.

[0027] In one aspect, when the energy level of the first signal is less than the energy level of the second signal, only at least one audio capture sensor of the first wearable device is selected to acquire audio data, and at least one audio capture sensor of the second wearable device is not selected.

[0028] In one aspect, the voice data is sent to the peripheral device using a negotiated and connected isochronous stream.

[0029] In one aspect, when the selection causes a change from acquiring audio data using at least one audio capture sensor of the first wearable device to acquiring audio data using at least one audio capture sensor of the second wearable device, the first wearable device performs a handshake with the second wearable device to shift responsibility for acquiring audio data to the at least one audio capture sensor of the second wearable device.

[0030] In one aspect, when a selection causes a change from acquiring audio data using at least one audio capture sensor of the first wearable device to acquiring audio data using at least one audio capture sensor of the second wearable device, gain parameters associated with the at least one audio capture sensor of the first wearable device are shared with the second wearable device to be applied to the at least one audio capture sensor of the second wearable device.

[0031] In another example, a computer program product for selecting one or more audio capture sensors of a wearable device is provided, the computer program product including a set of non-transitory computer readable instructions that, when executed on a processor of at least one of a first wearable device, a second wearable device, or a peripheral device, is configured to cause the processor to: detect or receive a first signal corresponding to a user's voice captured by at least one audio capture sensor of the first wearable device; detect or receive a second signal corresponding to the user's voice captured by at least one audio capture sensor of a second wearable device, the second wearable device being wirelessly connected to the first wearable device; determine an energy level of the first signal; determine an energy level of the second signal; and select the at least one audio capture sensor of the first wearable device or the at least one audio capture sensor of the second wearable device to acquire voice data based at least in part on the energy level of the first signal and the energy level of the second signal.

[0032] In one aspect, a first wearable device is configured to transmit a first signal to a peripheral device, and a second wearable device is configured to transmit a second signal to the peripheral device, and the peripheral device is configured to compare an energy level of the first signal with an energy level of the second signal to select at least one audio capture sensor of the first wearable device or at least one audio capture sensor of the second wearable device to acquire audio data.

[0033] In one aspect, the second wearable device is configured to transmit a second signal to the first wearable device, and the first wearable device is configured to compare an energy level of the first signal with an energy level of the second signal to select at least one audio capture sensor of the first wearable device or at least one audio capture sensor of the second wearable device to acquire audio data.

[0034] In one aspect, the energy levels of the first and second signals are related to at least one of a signal-to-noise ratio (SNR), a presence of wind, a radio frequency (RF) performance, or an audio pickup.

[0035] In one aspect, the at least one audio capture sensor of the first wearable device and the at least one audio capture sensor of the second wearable device are selected from at least one of an external microphone, an internal microphone, a feedback microphone also used for acoustic noise reduction (ANR) purposes, a feedforward microphone also used for ANR purposes, or an accelerometer.

[0036] In one aspect, the at least one audio capture sensor of the first wireless wearable device includes a first set of multiple audio capture sensors and an energy level of the first signal is an average energy level of the first set of multiple audio capture sensors, and the at least one audio capture sensor of the second wearable device includes a second set of multiple audio capture sensors and an energy level of the second signal is an average energy level of the second set of audio capture sensors.

[0037] In one aspect, when the energy level of the first signal is less than the energy level of the second signal, only at least one audio capture sensor of the first wearable device is selected to acquire audio data, and at least one audio capture sensor of the second wearable device is not selected.

[0038] In one aspect, the selection causes a change from capturing audio data using at least one audio capture sensor of a first wearable device to capturing audio data using at least one audio capture sensor of a second wearable device, and the first wearable device performs a handshake with the second wearable device to shift responsibility for capturing the audio data to the at least one audio capture sensor of the second wearable device.

[0039] In one aspect, the selecting causes a change from acquiring audio data using at least one audio capture sensor of the first wearable device to acquiring audio data using at least one audio capture sensor of the second wearable device, and gain parameters associated with the at least one audio capture sensor of the first wearable device are shared with the second wearable device to be applied to the at least one audio capture sensor of the second wearable device.

[0040] These and other aspects of the various embodiments will be apparent from and elucidated with reference to the embodiments described hereinafter. In the drawings, like reference numbers generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of various embodiments. [Brief description of the drawings]

[0041] [Figure 1]FIG. 1 is a schematic diagram of a system according to the present disclosure. [Figure 2A] FIG. 1 is a schematic diagram of components of a first wearable device according to the present disclosure. [Figure 2B] FIG. 13 is a schematic diagram of components of a second wearable device according to the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of components of a peripheral device according to the present disclosure. [Figure 4A] FIG. 1 is a schematic top view of a system according to the present disclosure. [Figure 4B] FIG. 1 is a schematic top view of a system according to the present disclosure. [Diagram 5] 1 is a flow diagram illustrating steps of a method according to an aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0042] The present disclosure relates to systems and methods, and computer program products, for selecting one or more audio capture sensors of a wearable device for use in acquiring voice data. Examples provided include acquiring signals associated with a user's voice at a first and a second wearable device, comparing the energy levels of the first and second signals (where the first wearable device is positioned at the user's right ear and the second wearable device is positioned at the user's left ear), and selecting one or more audio capture sensors based on the energy levels of each signal. Due to the symmetry of the acoustic energy generated by the user's voice to the first and second wearable devices (because the device at the user's right ear is equidistant from the device at the user's left ear), any difference in energy levels between the total energy acquired by the first wearable device and the total energy acquired by the second wearable device can be attributed solely to ambient noise. Thus, the signal-to-noise ratio of the device with the higher total energy is lower, and the selection of one or more audio capture sensors of the other wearable device with the higher signal-to-noise ratio is preferred for acquiring the voice data going forward.

[0043] As used in this application, the term "wearable audio device" is intended to mean a device that fits around, on, in, or near the ear (including open-ear audio devices worn on the user's head or shoulders) and radiates acoustic energy into or toward the ear, in addition to including its ordinary meaning or meaning known to those skilled in the art. Wearable audio devices may also be referred to as headphones, earphones, earpieces, headsets, earbuds, or sports headphones, and may be wired or wireless. A wearable audio device includes an acoustic driver that converts audio signals into acoustic energy. The acoustic driver may be housed in an ear cup. Although some of the following figures and descriptions may show a single wearable audio device having a pair of ear cups (each including an acoustic driver), it should be understood that a wearable audio device may be a single standalone unit having only one ear cup. Each ear cup of a wearable audio device may be mechanically connected to another ear cup or headphone, for example, by a headband and / or by leads that carry audio signals to an acoustic transducer in the ear cup or headphone. Wearable audio devices may include components that wirelessly receive audio signals. Wearable audio devices may include components of an active noise reduction (ANR) system. Wearable audio devices may also include other features such as microphones so that they can function as headsets. While FIG. 1 shows an example of an in-ear headphone form factor, in other examples, the wearable audio device may be a supra-aural, circum-aural, post-aural, supra-aural, or near-aural headset, or may be an audio glasses form factor headset. In some examples, the wearable audio device may be an open-ear device that includes an acoustic driver that radiates acoustic energy toward the ear while opening the ear to the ear's environment and surroundings.

[0044] The term "connected isochronous stream" as used herein is intended to include its ordinary meaning or its meaning as known to one of ordinary skill in the art, as well as to refer to an isochronous data stream that utilizes a pre-established point-to-point communication link between a source device and an audio device or devices via LE Audio, for example. In other words, a connected isochronous stream can provide an isochronous audio stream that utilizes at least one established reliable communication channel and / or at least one acknowledged communication channel between a source device and any respective audio device.

[0045] The term "broadcast isochronous stream" as used herein is intended to include its ordinary meaning or its meaning known to those of skill in the art, and also to refer to an isochronous data stream that does not require a pre-established communications link to be established between a source device transmitting the data and an audio device receiving the data, and does not require positive or negative acknowledgments to be sent or received.

[0046] The following description should be read in light of Figures 1-4B. Figure 1 is a schematic diagram of a system 100 according to the present disclosure. The system 100 includes a plurality of wearable devices, e.g., a first wearable device 102A and a second wearable device 102B (collectively referred to herein as "wearable devices 102"), and a peripheral device 104. In the illustrated example, the first wearable device 102A and the second wearable device 102B are intended to be a pair of wearable audio devices, e.g., a pair of true wireless earbuds, with the first wearable device 102A and the second wearable device 102B positioned to be secured in proximity to or within a user's right and left ears, respectively. However, it should be understood that in some alternative examples, the first wearable device 102A and the second wearable device 102B may be selected from at least one of a hearing aid, a speaker, a portable speaker, a paired speaker, or a paired portable speaker. As illustrated, the system 100 includes a peripheral device 104, such as a smartphone or tablet, configured to establish a wireless data connection with the wearable device 102, which will be discussed in more detail below.

[0047] 2A, the first wearable device 102A includes a first circuit 106. The first circuit 106 includes a first processor 108 and a first memory 110 configured to execute and store a first set of non-transitory computer-readable instructions 112 to perform various functions of the first wearable device 102A and the first circuit 106, respectively, as described herein. The first circuit 106 also includes a first communication module 114 configured to transmit and / or receive wireless data, e.g., data associated with at least one of a plurality of communication data streams discussed below, e.g., communication data stream 162A. To that end, the first communication module 114 may include at least one radio or antenna, e.g., a first radio 116 capable of transmitting and receiving wireless data. In some examples, the first communication module 114 may include at least one radio (e.g., the first radio 116), as well as some form of automated gain control (AGC), a modulator and / or demodulator, and perhaps a separate processor for bit processing electrically connected to the first processor 108 and the first memory 110 to assist in transmitting and / or receiving wireless data. As discussed below, the first circuit 106 of the first wearable device 102A may also include a first speaker 118, e.g., a loudspeaker or acoustic transducer, electrically connected to the first processor 108 and the first memory 110 and configured to electromechanically convert the electrical signal into audible acoustic energy, e.g., audio reproduction, in the environment E. In some examples, the electrical signal and the audible acoustic energy are associated with data included in a plurality of communication data streams (discussed below).

[0048] The first wearable device 102A may further include a first set of audio capture sensors 120. In some examples, the first set of audio capture sensors includes only one audio capture sensor. In some examples, the first set of audio capture sensors 120 includes multiple audio capture sensors, e.g., a first audio capture sensor 120A and a second audio capture sensor 120B (collectively referred to as the "first set of audio capture sensors 120"). In other examples, the first set of audio capture sensors 120 may have more than two audio capture sensors, e.g., the set may include three, four, five, or more audio capture sensors. Each audio capture sensor in the first set of audio capture sensors 120 is intended to be a microphone or some other audio capture device (e.g., a unidirectional or directional micro-electro-mechanical system (MEMS) microphone disposed on, in, or near the first wearable device 102A). It should be appreciated that each audio capture sensor 120 may be configured as an external microphone (e.g., a microphone positioned to pick up or capture sound energy outside of the air cavity created between the wearable audio device 102A and the user's eardrum), an internal microphone (e.g., a microphone positioned to pick up or capture sound energy inside the air cavity created between the wearable audio device 102A and the user's eardrum), or an accelerometer. In some examples, the external microphone may be a feedforward microphone that is also used to capture or pick up external sound energy used in active noise reduction (ANR) applications. In other examples, the internal microphone may be a feedback microphone that is also used to capture or pick up internal sound energy used in ANR applications.It should be appreciated that each audio capture sensor can be configured to capture or pick up sound energy via air conduction and / or via bone conduction (through one or more bones of the user's body, e.g., the user's head or chin). It should also be appreciated that two or more audio capture sensors of the first set of audio capture sensors 120 can be arranged such that beamforming techniques using one or more algorithms can be utilized to improve the quality of audio pickup captured by the audio capture sensors, e.g., to enhance the voice data 164 (discussed below). In some implementations, multiple audio capture sensors of the same type are included on the wearable device, such as when there are two outward-facing microphones for audio capture sensors 120A and 120B on the first wearable device 102A. In some implementations, a mixture of different types of audio capture sensors are included on the wearable device, such as when the first audio capture sensor 120A is an outward-facing microphone and the second audio capture sensor 120B is an accelerometer.

[0049] As discussed below, each audio capture sensor of the first set of audio capture sensors 120 is configured to acquire sound energy from the environment surrounding the user and convert the sound energy into an electronic signal, i.e., first signal 122. Once acquired, the first signal 122 can be analyzed to determine a first energy level 124. The first energy level 124 is intended to be a measure of the total sound energy detected by the one or more audio capture sensors 120. This total sound energy can include sound energy generated by the user, the user's voice, etc., as well as other ambient sound energy from the surrounding environment, wind, conversation, traffic noise, machinery, etc. In some examples where the first set of audio capture sensors 120 includes two or more audio capture sensors, as described above, the first energy level 124 can be an average energy level acquired or picked up by all of the audio capture sensors of the first set of audio capture sensors 120, i.e., a first average energy level 126. For example, one or more external microphone signals can be averaged together to form the first energy level 124. In another example, one or more internal microphone signals may be averaged with one or more external microphones to form the first energy level 124.

[0050] 2B, the second wearable device 102B comprises a second circuit 128. The second circuit 128 includes a second processor 130 and a second memory 132 configured to execute and store a second plurality of non-transitory computer-readable instructions 134 to perform various functions of the second wearable device 102B and the second circuit 128, respectively, as described herein. The second circuit 128 also includes a second communication module 136 configured to transmit and / or receive wireless data, e.g., data related to a plurality of data streams, e.g., isochronous data stream 162B, discussed below. To that end, the second communication module 136 can include at least one radio or antenna, e.g., a second radio 138 capable of transmitting and receiving wireless data. In some examples, the second communication module 128 may include at least one radio (e.g., the first radio 138), as well as some form of automatic gain control (AGC), a modulator and / or demodulator, and perhaps a separate processor for bit processing electrically connected to the second processor 130 and the second memory 132 to assist in transmitting and / or receiving wireless data. As discussed below, the second circuit 128 of the second wearable device 102B may also include a second speaker 140, e.g., a loudspeaker or acoustic transducer, electrically connected to the second processor 130 and the second memory 132 and configured to electromechanically convert the electrical signal into audible acoustic energy, e.g., audio reproduction, in the environment E. In some examples, the electrical signal and the audible acoustic energy are associated with data included in multiple data streams (discussed below).

[0051] The second wearable device 102B may further include a second set of audio capture sensors 142. In some examples, the second set of audio capture sensors includes only one audio capture sensor. In some examples, the second set of audio capture sensors 142 includes multiple audio capture sensors, such as a first audio capture sensor 142A and a second audio capture sensor 142B (collectively referred to as the "second set of audio capture sensors 142"). In other examples, the second set of audio capture sensors 142 may have more than two audio capture sensors, for example, the set may include three, four, five, or more audio capture sensors. Each audio capture sensor of the second set of audio capture sensors 142 is intended to be a microphone or some other audio capture device (e.g., a unidirectional or directional microelectromechanical system (MEMS) microphone disposed on, in, or near the second wearable device 102B). It should be appreciated that each audio capture sensor 142 may be configured as an external microphone (e.g., a microphone positioned to pick up or capture sound energy outside of the air cavity created between the wearable audio device 102B and the user's eardrum), an internal microphone (e.g., a microphone positioned to pick up or capture sound energy inside the air cavity created between the wearable audio device 102B and the user's eardrum), or an accelerometer. In some examples, the external microphone may be a feedforward microphone that is also used to capture or pick up external sound energy used in active noise reduction (ANR) applications. In other examples, the internal microphone may be a feedback microphone that is also used to capture or pick up internal sound energy used in ANR applications.It should be appreciated that each audio capture sensor can be configured to acquire or pick up sound energy via air conduction and / or via bone conduction (through one or more bones of the user's body, e.g., the user's head or chin). It should also be appreciated that two or more audio capture sensors of the second set of audio capture sensors 142 can be arranged such that beamforming techniques using one or more algorithms can be utilized to improve the quality of audio pickup acquired by the audio capture sensors, e.g., to enhance the voice data 164 (discussed below). In some implementations, multiple audio capture sensors of the same type are included on the wearable device, such as when there are two outward-facing microphones for audio capture sensors 142A and 142B on the second wearable device 102B. In some implementations, a mixture of different types of audio capture sensors are included on the wearable device, such as when the first audio capture sensor 142A is an outward-facing microphone and the second audio capture sensor 142B is an accelerometer.

[0052] As discussed below, each audio capture sensor of the second set of audio capture sensors 142 is configured to acquire sound energy from the environment surrounding the user and convert the sound energy into an electronic signal, i.e., second signal 144. Once acquired, the second signal 144 can be analyzed to determine a second energy level 146. The second energy level 146 is intended to be a measure of the total sound energy detected by at least one audio capture sensor 142. This total sound energy may include sound energy generated by the user, the user's voice, etc., as well as other ambient sound energy from the surrounding environment, wind, conversation, traffic noise, machinery, etc. In some examples where the second set of audio capture sensors 142 includes more than one audio capture sensor, as described above, the second energy level 146 may be an average energy level acquired or picked up by all of the audio capture sensors of the second set of audio capture sensors 142, i.e., a second average energy level 148. For example, one or more external microphone signals may be averaged together to form the second energy level 146. In another example, one or more internal microphone signals may be averaged with one or more external microphones to form the second energy level 146 .

[0053] As illustrated in FIG. 3, the system 100 further includes a peripheral device 104. The peripheral device 104 is intended to be a wired or wireless device capable of transmitting and / or receiving data related to the multiple communication data streams 162A-162B to at least one wearable device, such as the first wearable device 102A and / or the second wearable device 102B. In one example, as illustrated in FIG. 1, the peripheral device 104 is a smartphone capable of transmitting data from the multiple data streams 162A-162B to the first wearable device 102A and / or the second wearable device 102B. Although not illustrated, it should be understood that the peripheral device 104 can also be selected from at least one of a personal computer, a mobile computing device, a tablet, a smart speaker, a smart speaker system, a smart hub, a smart television, or any other device capable of transmitting or receiving data from the multiple data streams 162A-162B (discussed below). In some examples, the peripheral device 104 is a remote device that is wirelessly paired with the first wearable device 102A and / or the second wearable device 102B. As such, the peripheral device 104 may comprise a peripheral circuit 150. The peripheral circuit 150 includes a peripheral processor 152 and a peripheral memory 154 configured to execute and store a plurality of non-transitory computer-readable instructions, e.g., peripheral instructions 156, respectively, to perform various functions of the peripheral device 104 and the peripheral circuit 150, as described herein. The peripheral circuit 150 also includes a peripheral communication module 158 configured to transmit and / or receive wireless data, e.g., data related to a plurality of data streams 162A-162B (discussed below), to and from the wearable device 102. To that end, the peripheral communication module 158 may include at least one radio or antenna, e.g., a peripheral radio 160, capable of transmitting and receiving wireless data.In some examples, the peripheral communications module 158 may include at least one radio (e.g., peripheral radio 160), as well as some form of automatic gain control (AGC), a modulator and / or demodulator, and perhaps a separate processor for bit processing electrically connected to the peripheral processor 152 and peripheral memory 154 to assist in transmitting and / or receiving wireless data. Additionally, the peripheral device 104 may include, within a set of non-transitory computer-readable instructions, one or more applications, e.g., mobile applications, that can interact and communicate with each wearable device in the system, i.e., at least the first wearable device 102A and the second wearable device 102B.

[0054] Each device of the system 100, i.e., each wearable device 102 and peripheral device 104, may use their respective communication modules to establish a communication data stream between each device. For example, as illustrated in FIG. 1, the system 100 may be configured to establish a first communication data stream 162A between the first wearable device 102A and the peripheral device 104, a second communication data stream 162B between the second wearable device 102B and the peripheral device 104, and a third communication data stream 162C between the first wearable device 102A and the second wearable device 102B. Each communication data stream, i.e., first communication data stream 162A, second communication data stream 162B, and third communication data stream 162C (collectively referred to as "communication data stream 162" or "multiple communication data streams 162"), may utilize a variety of wireless data protocols or transmission methods, such as a Bluetooth protocol, a Bluetooth Classic protocol, a Bluetooth Low-Energy protocol, an LE Audio protocol, an Asynchronous Connection-Oriented logical transport (ACL) protocol, a Radio Frequency (RF) communication protocol, a WiFi protocol, a Near-Field Magnetic Inductance (NFMI) communication, a LE Asynchronous Connection (LE ACL) logical transport protocol, or other transmission methods of wireless data suitable for transmitting and / or receiving audio and voice data streams. In one example, the multiple communication data streams 162 may utilize at least one negotiated isochronous data stream, such as a broadcast isochronous stream and / or one or more connected isochronous streams of the LE Audio protocol, and may also utilize the LC3 audio codec.In another example, the third communication data stream 162C established between the first wearable device 102A and the second wearable audio device 102B utilizes the Asynchronous Connection-Oriented (ACL) logical transport protocol. However, it should be understood that the third communication data stream 162C can be a broadcast isochronous stream or a connected isochronous stream between the first wearable device 102A and the second wearable device 102B. It should be understood that each communication data stream of the multiple communication data streams 162 can include a communication stream that utilizes at least one of the protocols listed above in any possible combination. Additionally, each device of the system 100 can be paired with at least one other device in the system. In one example, the first wearable device 102A and the second wearable device 102B are paired audio devices, such as paired true wireless earbuds or paired speakers. As used herein, the term "paired" is intended with its ordinary meaning to one of ordinary skill in the art to mean establishing a data connection between two devices based on a known relationship and / or identification of the devices. The devices may initially exchange authentication information, e.g., a Bluetooth passkey, between each other to establish a connection between the two devices that share the passkey. The exchange of authentication information may occur in a dedicated pairing mode of the two devices to indicate ownership of both devices and / or an intent to pair. Once the devices are paired, they may establish future connections based on the shared passkey and / or known identification of the devices. Similarly, one or more of the wearable devices 102 may pair with the peripheral device 104 to assist in establishing the first and second communication data streams 162A-162B discussed above.

[0055] During operation, a user of the system 100 may engage in various actions, activities, or applications that include or require the capture of audio data from the environment surrounding the user through one or more of the first set of audio capture sensors 120 and / or one or more of the second set of audio capture sensors 142. As discussed above, the audio data captured by one or more of the audio capture sensors of the first or second set of audio capture sensors 120, 142 represents electronic signals acquired by the one or more audio capture sensors and includes acoustic energy originating from sources such as wind, conversation, traffic noise, machinery, and the like, ambient acoustic energy from the surrounding environment. The audio data may also include voice data, i.e., voice data 164. The voice data 164 refers to portions of acoustic energy captured or picked up from the user's speech, e.g., vocalized from the user. It should thus be understood that the audio data includes not only ambient acoustic energy from noise occurring in the user's surrounding environment, but also acoustic energy generated by the user while speaking, i.e., voice data 164. Thus, as outlined herein, the signals, energy levels, and average energy levels discussed above can include energy corresponding to ambient acoustic energy as well as acoustic energy from a user's speech, i.e., voice data 164. For example, first signal 122 and first energy level 124 associated with acoustic energy picked up from at least one audio capture sensor of first set of audio capture sensors 120, and first average energy level 126 associated with acoustic energy averaged across two or more audio capture sensors of first set of audio capture sensors 120, can include energy attributable to ambient acoustic sources as well as human speech.Similarly, the second signal 144 and second energy level 146 associated with acoustic energy picked up from at least one audio capture sensor of the second set of audio capture sensors 142, as well as the second average energy level 148 associated with acoustic energy averaged across two or more audio capture sensors of the second set of audio capture sensors 142, can include energy attributable to ambient acoustic sources and human speech.

[0056] In some applications, the ambient noise surrounding the user is not omnidirectional or symmetrical; for example, significantly more ambient noise may be received closer to the user's right ear than the user's left ear. This may be caused by several factors, including wind directed at the user's right ear (typically less wind passing through the user's left ear because the user's head blocks direct wind energy). It may also be caused by the user's orientation closer to louder objects, vehicles, machinery, people, etc. In these applications and situations, the present system and method is configured to compare the first signal 122 (representing the total energy close to the right ear) with the second signal 144 (representing the total energy close to the left ear). If the comparison of these two signals results in one signal having a greater energy level (i.e., louder) than the other, the additional acoustic energy of the higher signal may be due to excessive ambient noise. For example, while a user is speaking, the first wearable device 102A (e.g., at the user's right ear) and the second wearable device 102B (e.g., at the user's left ear) are both equidistant from the user's mouth. Thus, the amount of acoustic energy acquired by the audio capture sensors of the first set of audio capture sensors 120 of the first wearable device 102A due to the voice data 164 and the amount of acoustic energy acquired by the audio capture sensors of the second set of audio capture sensors 142 of the second wearable device 102B due to the voice data 164 are substantially equal. Thus, any difference or discrepancy between the first signal 122 and the second signal 144 is due only to the difference or discrepancy in the ambient noise directed to the respective wearable device 102 having the higher (or louder) signal. In other words, a signal generated by the wearable device 102 with less total energy will have a higher signal-to-noise ratio, and a signal generated by the wearable device 102 with more total energy will have a lower signal-to-noise ratio.Thus, when choosing or selecting an audio capture device responsible for acquiring voice data 164 for various applications, e.g., voice pickup of a telephone call, the current system and method chooses or selects an audio capture sensor to pick up the voice data based on the audio capture sensor generating the lowest energy level (quietest) signal and therefore having a high signal-to-noise ratio. It should be understood that the first signal 122 and the second signal 144 used in the comparison below can be normalized, e.g., any automatic gain control parameters or settings applied to the first signal 122 or the second signal 144 can be taken into account when comparing the first energy level 124 and the second energy level 146. Conversely, the comparisons discussed herein may also compare the first signal 122 and the second signal 144 to determine, for example, whether the first energy level 124 is greater than or less than the second energy level 146 based on the raw unprocessed sensor data without taking into account any automatic gain control settings applied to the first signal 122 or the second signal 144.

[0057] 4A, the user of the system 100 is positioned such that the wind is directed toward the user's right side, i.e., toward the first wearable device 102A in the user's right ear. In this example, the first wearable device 102A is configured to obtain, via at least one audio capture sensor 120, a first signal 122 associated with a first energy level 124 or a first average energy level 126 representing an average energy level of two or more audio capture sensors of the first set of audio capture sensors 120, and to store the first signal 122, the first energy level 124, and / or the first average energy level 126 in its own memory (i.e., the first memory 110) or to transmit the first signal 122 via a first communication data stream 162A to the peripheral device 104, where the peripheral device 104 is configured to determine the first energy level 124 and / or the first average energy level 126. Also in this example, the second wearable device 102B is configured to acquire, via at least one audio capture sensor 142, a second signal 144 associated with a second energy level 146 or a second average energy level 148 representing an average energy level of two or more audio capture sensors of the second set of audio capture sensors 142, and store the second signal 144, the second energy level 146, and / or the second average energy level 148 in its own memory (i.e., the second memory 132) or transmit the second signal 144 via a second communication data stream 162B to the peripheral device 104, where the peripheral device 104 is configured to determine the second energy level 146 and / or the second average energy level 148.For example, if the first energy level 124 is less than the second energy level 146 (e.g., caused by wind increasing the total energy in the audio capture sensors of the second set of audio capture sensors 142), and it is determined that the first signal 122 has a lower energy level than the second signal 144, and thus has a higher signal-to-noise ratio, the system 100 may assign the first wearable device 102A to the role of primary sound capture device 166. Once assigned the role of primary sound capture device 166, the first wearable device 102A may utilize one or more audio capture sensors of the first set of audio capture sensors 120 to capture sound data 164 (e.g., capture or pick up and record a user's speech) and process and / or transmit the sound data 164 for use by one or more applications executable on the peripheral device 104, such as for use in a phone call.

[0058] Alternatively, as illustrated in FIG. 4B, a user of the system may be positioned such that the wind approaches the user's left side, i.e., is directed toward the second wearable device 102B in the user's left ear. As discussed above, the first signal 122 and the second signal 144 are acquired by the respective audio capture sensors of each device and compared. For example, if the first energy level 124 is greater than the second energy level 146 (e.g., caused by wind increasing the total energy in the audio capture sensors of the first set of audio capture sensors 120), the system 100 may assign the role of the primary audio capture device 166 to the second wearable device 102B upon determining that the first signal 122 has a higher energy level than the second signal 144 and thus a lower signal-to-noise ratio. When the second wearable device 102B is assigned the role of primary audio capture device 166, it utilizes one or more audio capture sensors in the second set of audio capture sensors 142 to capture audio data 164 (e.g., to capture or pick up and record a user's speech) and process and / or transmit the audio data 164 for use by one or more applications executable on the peripheral device 104, such as for use in a telephone call.

[0059] Once selected, the primary audio capture device 166 is responsible for compressing, encoding, and transmitting audio data 164 acquired from each of the audio capture sensors of the first and second audio capture sensor sets outlined above to the peripheral device 104 for decoding, mixing, and / or use in one or more applications executing on the peripheral device 104. In other words, the primary audio capture device 166 is responsible for collecting or acquiring audio data 164 for both wearable devices 102 and transmitting the audio data 164 to the peripheral device 104 for use in one or more applications. Thus, if the first wearable device 102A is selected as the primary audio capture device 166, the second wearable device 102B can utilize the third communication data stream 162C to transmit audio and audio data 164 to the first wearable device 102A for transfer to the peripheral device 104. Similarly, if the second wearable device 102B is selected as the primary audio capture device 166, the first wearable device 102A can use the third communication data stream 162C to transmit audio and voice data 164 to the second wearable device 102B for transfer to the peripheral device 104.

[0060] When multiple audio capture sensors (e.g., two or more audio capture sensors of a particular set of audio capture sensors) are used to acquire the audio data 164, it may be necessary to mix the audio data 164 acquired by the two or more audio capture sensors before transmitting the audio data 164 to the peripheral device 104. To that end, each device of the system 100, i.e., the first wearable device 102A, the second wearable device 102B, and the peripheral device 104, may each store and execute a mixing algorithm 168 on their respective memories and processors. The mixing algorithm 168 may be utilized to combine or mix the audio inputs of the two or more audio capture sensors on the selected primary audio capture device 166 so that they can be output, transmitted, or utilized by one or more applications executing on the peripheral device 104. It should be appreciated that the mixing algorithm may also include or utilize beamforming techniques for the inputs from the two or more audio capture sensors to improve the quality of the utilized audio data 164. It should also be appreciated that, in an alternative to mixing audio data from two or more audio capture sensors in the circuitry of the selected primary audio capture device 166, the primary audio capture device 166 can simply compress and transmit the audio data captured by each audio capture sensor to the peripheral device 104 for mixing by the peripheral device 104. Metadata 170 may be utilized by the primary audio capture device 166 (i.e., either the first wearable device 102A or the second wearable device 102B) when mixing the audio data captured by the two or more audio capture sensors of the respective device selected as the primary audio capture device 166.For example, metadata 170 may include the type of audio capture sensor (e.g., whether it is an internal or external microphone) and assist in how much or how little or suppressed a given signal is added to the final output. In one example, audio data from two or more audio capture sensors is sent to peripheral device 104 for mixing using a mixing algorithm 168. In this example, mixing algorithm 168 is configured to generate output 172 that may include or be affected by metadata 170. Peripheral device 104 may be configured to send output 172 to one or more of wearable device 102, which may be used to automatically adjust gain parameters 176 (discussed below), volume levels, or active noise reduction levels used by wearable device 102. Output 172 of aforementioned mixing algorithm 168 may be used to make decisions regarding volume levels or ANR settings of the wearable device. For example, based on the noise level detected by a microphone mixing algorithm 168 running on the peripheral device 104, a signal may be sent to the wearable device 102 instructing it to adjust the volume level to compensate for the environmental noise. Similarly, information about the environmental noise may be sent as a signal back to the wearable device 102 to automatically adjust the ANR level. As discussed below, the automatic adjustment of each of these parameters may depend on a role switch between the first wearable device 102A and the second wearable device 102B, e.g., a switch of which device is the primary audio capture device 166.

[0061] In analyzing which wearable device 102 is assigned the role of primary sound capture device 166, it should be understood that the audio capture sensors used may be any possible combination of the audio capture sensors listed above. For example, if a first wearable device 102A is initially selected as the primary sound capture device 166, the first set of audio capture sensors 120 may include a first audio capture sensor 120A and a second audio capture sensor 120B, where the first audio capture sensor 120A is an external feedforward microphone and where the second audio capture sensor 120B is an internal feedback microphone. In this example, the metadata 170 for each audio capture sensor may identify that the signal generated or captured by the first audio capture sensor 120A is an external microphone and the signal generated or captured by the second audio capture sensor 120B is an internal microphone, and the mixing algorithm 168 may choose to weight how much the audio captured by the external feedforward microphone should be weighted more heavily in the mixing process than the internal feedback microphone to produce a higher fidelity output. In some examples, the primary sound capture device 166 may use one audio capture sensor from the first set of audio capture sensors 120 and one audio capture sensor from the second set of audio capture sensors 142. For example, when a first wearable device 102A is initially selected as the primary audio capture device 166, the first set of audio capture sensors 120 may include a first audio capture sensor 120A and a second audio capture sensor 120B, where the first audio capture sensor 120A is an external feedforward microphone and where the second audio capture sensor 120B is an internal feedback microphone.Additionally, the second set of audio capture sensors 142 of the second wearable device 102B may include a first audio capture sensor 142A and a second audio capture sensor 142B, where the first audio capture sensor 142A is an external feedforward microphone and the second audio capture sensor 142B is an internal feedback microphone. In this example, since the first wearable device 102A is selected as the primary audio capture device, audio data captured from the first audio capture sensor 142A and / or the second audio capture sensor 142B of the second set of audio capture sensors 142 of the second wearable device 102B may be transmitted to the first wearable device 102A via the third communication data stream 162C, such that the mixing algorithm 168 may utilize at least one audio capture sensor from the first set of audio capture sensors 120 and at least one audio capture sensor of the second set of audio capture sensors 142.

[0062] Additionally, to aid in the transmission of audio data from two or more audio capture sensors to another device in the system 100, each device may also utilize a compression algorithm 174. The compression algorithm used may be a conventional compression algorithm designed for audio compression, such as a low-complexity subband codec (SBC), LC3 codec, advanced audio coding (AAC), or LDAC codec, or may be a non-conventional compression algorithm, such as OGG, MP3, M4A, etc. The compression algorithm 174 may use any of the aforementioned compression codecs or formats when compressing and transmitting the audio data from one device to another in the system, i.e., via the communication data streams 162A-162C.

[0063] During operation, the selection of the primary sound capture device 166 or the switching of the role of the primary sound capture device 166 may be a dynamic process. For example, in real time, the first wearable device 102A and the second wearable device 102B may utilize one or more algorithms to periodically or continuously acquire captured audio data from each of the audio capture sensors of the first set of audio capture sensors 120 and each audio capture second of the second audio capture sensor 142 and determine whether the first signal 122 or the second signal 144 has a greater energy level, i.e., whether the first energy level 124 or the second energy level 146 is greater. Thus, the selection of the audio capture sensor used to acquire the sound data 164 and transmit the sound data 164 to a peripheral device for use with one or more applications is also dynamic.

[0064] In one example, during operation, the system 100 can dynamically implement a switch of the primary audio capture device 155 from one wearable device 102 to another, where the peripheral device 104 is aware of the switch and the communication data stream being used utilizes a classic Bluetooth communication protocol. For example, the system 100 can be configured such that a first communication data stream 162A between the peripheral device 104 and the first wearable device 102A and a second communication data stream 162B between the peripheral device 104 and the second wearable device 102B are classic Bluetooth data streams. In this example, one of the wearable devices 102 is responsible for acquiring audio data 164 and transmitting it to the peripheral device 104 for use with one or more applications. If the signal-to-noise ratio of the primary audio capture device 166 is too low and audio data 164 needs to be acquired from another device, i.e., a device that is not the primary audio capture device. It is not desirable to obtain audio data from the other wearable device 102, transmit the audio data 164 from the other wearable device 102 to the primary audio capture device 166 via the third communication data stream 162C, and then transmit the audio data to the peripheral device via the first communication data stream 162A. The total latency of this relay of the audio data 164 is very long, resulting in a reduced user experience. Therefore, role switching of the primary audio capture device 166 is necessary. For example, one of the wearable devices 102, i.e., the primary audio capture device 166, is initially responsible for receiving the audio data 164 and transmitting the audio data 164 to the peripheral device for use with one or more applications. Additionally, in this example, the primary audio capture device 166 can be responsible for receiving any playback audio data for playback on both wearable devices 102 and transmitting the playback audio data to the other wearable device 102.In other words, the other wearable device 102 transmits and receives all audio data to the peripheral device 104 via a separate connection with the primary sound capture device 166, e.g., via the third communication data stream 162C. When the role of the primary sound capture device 166 is switched, e.g., from the first wearable device 102A to the second wearable device 102B, the responsibility of transmitting the sound data 164 to the peripheral device 104 is also switched. In this example, when a determination is made to switch the role of the primary sound capture device 166 from the first wearable device 102A to the second wearable device 102B (e.g., when the second energy level 146 is less than the first energy level 124), a request can be sent to the peripheral device 104 requesting that the peripheral device 104 acknowledge that a role switch is about to occur. Upon receiving a response to the request from the peripheral device 104, the first and second wearable devices 102 may perform a handshake and initiate a switch of the role of the primary audio capture device 166 from the first wearable device 102A to the second wearable device 102B. The second wearable device 102B thus becomes the new primary audio capture device 166 and is responsible for relaying all audio data to and from the first wearable device 102A to the peripheral device 104 and for transmitting audio data 164 to the peripheral device 104 for use with one or more applications.

[0065] In another example, during operation, the system 100 can dynamically implement a switch of the primary audio capture device role 166 from one wearable device 102 to another if the peripheral device recognizes the switch and the communication data stream being used utilizes a negotiated isochronous data stream. For example, the system 100 can be configured such that the first communication data stream 162A between the peripheral device 104 and the first wearable device 102A and the second communication data stream 162B between the peripheral device 104 and the second wearable device 102B are negotiated connected isochronous streams. Thus, the peripheral device 104 is configured to transmit to each wearable device 102 a data stream specific to each wearable device 102. In another example, there can be only one communication data stream, e.g., the first communication data stream 162A between the peripheral device 104 and the wearable device 102, i.e., the first communication data stream 162A is a broadcast isochronous stream. Thus, the peripheral device 104 is configured to transmit a single stream of data, where a portion of the stream is specific to the first wearable device 102A and a portion of the stream is specific to the wearable device 102B. In any of these examples, the peripheral device 104 is configured to initially receive the audio data 164 from the selected primary audio capture device 166, e.g., the first wearable device 102A. When the role of the primary audio capture device 166 is switched from the first wearable device 102A to the second wearable device 102B, the role of transmitting the audio data 164 from both wearable devices 102 may also be switched. In this example, once a determination is made to switch the role of the primary audio capture device 166 from the first wearable device 102A to the second wearable device 102B (e.g., when the second energy level 146 is less than the first energy level 124), a request may be sent to the peripheral device 104 requesting that the peripheral device 104 acknowledge that a role switch is about to occur.Upon receiving a response to the request from the peripheral device 104, the first and second wearable devices 102 can perform a handshake and initiate a switch of the role of the primary audio capture device 166 from the first wearable device 102A to the second wearable device 102B. In other words, the peripheral device 104 is configured to know that a switch has occurred or is about to occur, and adjusts its radio, e.g., the peripheral radio 160, accordingly, e.g., by adjusting the period during which it expects to receive audio data packets from the new primary audio capture device 166.

[0066] Alternatively, the dynamic selection process for selection of the primary sound capture device 166 can be accomplished without notifying or alerting the peripheral device 104, e.g., without the peripheral device "knowing" that a switch is about to occur or has occurred. For example, the decision to switch the role of the primary sound capture device 166 from one wearable device 102 to the other is made mid-stream within or between the two wearable devices 102, e.g., through the third communication data stream 162C, without providing an indication or request to the peripheral device 104 that a switch is occurring. In this example, the wearable devices switch roles completely, i.e., the first wearable device 102A receives and acknowledges data packets originally designated for the second wearable device 102B (assuming a connected isochronous stream), and the second wearable device 102B receives and acknowledges packets originally designated for the first wearable device 102A (assuming a connected isochronous stream), such that the peripheral device 104 is not aware that a change has occurred. When a single broadcast isochronous stream from the peripheral device 104 to the wearable device 102 is used, each device simply receives packets originally intended for the other device, without either device acknowledging receipt of the packets.

[0067] To aid in switching roles, each device may exchange specific access codes to grant access to certain portions of a given data stream designated for each device. For example, if the devices of system 100 utilize connected isochronous data streams, e.g., if one or more of communication data streams 162A-162C are connected isochronous data streams, the first wearable device 102A may utilize a first access code to authenticate or access and enable use of data provided by peripheral device 104 in first communication data stream 162A. Similarly, the second wearable device 102B may utilize a second access code to authenticate or access and enable use of data provided by peripheral device 104 in second communication data stream 162B. Before any role switching occurs between the first wearable device 102A and the second wearable device 102B, these devices may exchange first and second access codes via the third communication data stream 162C, so that after the role switching occurs, each device may seamlessly receive and access packets of the new data stream, e.g., after the switching, the first wearable device 102A may access, receive, and acknowledge packets transmitted in the second communication data stream 162B, and the second wearable device 102B may access, receive, and acknowledge packets transmitted in the first communication data stream 162A.

[0068] In one example, gain parameters 176, volume settings and / or active noise reduction (ANR) settings of each wearable device 102 may also be exchanged between the first wearable device 102A and the second wearable device 102B before switching roles. The gain parameters 176 are intended to be settings or values ​​that represent the gain applied to a particular signal, e.g., the first signal 122 or the second signal 144 associated with the voice data 164, or other audio data transmitted or received from the peripheral device 104, e.g., the voice of another speaker in a telephone conversation with the user or other media input. By exchanging these parameters and settings, for example, when a role switch occurs from a first wearable device 102A to a second wearable device 102B, the new primary audio capture device 166 (e.g., the second wearable device 102B) can apply the same gain parameters 176, volume settings, and / or ANR settings used by the first wearable device 102A such that when the switch is in effect, a user of the system does not perceive any change or interruption in voice or audio playback.

[0069] 5 illustrates a flow diagram corresponding to a method 200 according to the present disclosure. As shown, the method 200 may, for example, include detecting a first signal 122 corresponding to a user's voice using at least one audio capture sensor 120A of a first wearable device 102A (step 202), detecting a second signal 144 corresponding to the user's voice using at least one audio capture sensor 142A of a second wearable device 102B, the second wearable device 102B being wirelessly connected to the first wearable device 102A (step 204), and detecting an energy level of the first signal 122. The method may include determining an energy level 124 of the first signal 122 (step 206), determining an energy level 146 of the second signal 144 (step 208), and selecting at least one audio capture sensor (120A, 120B) of the first wearable device 102A and / or at least one audio capture sensor (142A, 142B) of the second wearable device 102B to acquire audio data 164 based at least in part on the energy level 124 of the first signal 122 and the energy level 146 of the second signal 144 (step 210).

[0070] All definitions and those used herein should be understood to control for any dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0071] The indefinite articles "a" and "an," as used in the specification and claims, unless expressly indicated otherwise, should be understood to mean "at least one."

[0072] The phrase "and / or" as used in the specification and claims should be understood to mean "either or both" of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with "and / or" should be construed in the same manner, i.e., "one or more" of the elements so conjoined. Other elements, whether related or unrelated to the elements specifically identified, may optionally be present other than the elements specifically identified by the "and / or" clause.

[0073] As used herein and in the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive, i.e., including at least one, but also including more than one, of a number or list of elements, and optionally including additional non-listed items. Only when terms such as "only one of" or "exactly one of," or, when used in the claims, "consisting of," refer to the inclusion of exactly one element of a number or list of elements. In general, the term "or" as used herein is only interpreted as indicating exclusive alternatives (i.e., "one or the other but not both") when preceded by terms of exclusivity such as "either," "one of," "only one of," or "exactly one of."

[0074] As used in this specification and claims, the phrase "at least one" in reference to a list of one or more elements should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed in the list of elements, and not excluding any combination of elements in the list of elements. This definition also allows for elements, whether related or unrelated to the specifically identified elements, to be optionally present other than the elements specifically identified in the list of elements to which the phrase "at least one" refers.

[0075] It is also to be understood that, unless expressly indicated otherwise, in any method claimed herein that includes more than one step or action, the order of the method steps or actions is not necessarily limited to the order in which the method steps or actions are recited.

[0076] In the claims, as well as in the above specification, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," "composed of," and the like, are to be understood to be open-ended, i.e., to mean including but not limited to. The transitional phrases "consisting of" and "consisting essentially of" only are closed or semi-closed transitional phrases, respectively.

[0077] The above-described examples of the described subject matter can be implemented in any of numerous ways. For example, some aspects may be implemented in hardware, software, or a combination thereof. If at least a portion of any aspect is implemented as software, the software code can be executed on any suitable processor, or collection of processors, whether provided on a single device or computer, or distributed across multiple devices / computers.

[0078] The present disclosure may be implemented as a system, method, and / or computer program product at any level of technical detail contemplated. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions that cause a processor to perform aspects of the present disclosure.

[0079] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, punch cards or raised structures in grooves, mechanically encoded devices having instructions recorded thereon, and any suitable combination of the above. As used herein, a computer-readable storage medium is not to be construed as a transient signal itself, such as freely propagating electromagnetic waves such as radio waves, electromagnetic waves propagating through a transmission medium such as a waveguide (e.g., light pulses passing through a fiber optic cable), or an electrical signal traveling down an electrical wire.

[0080] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card, or network interface, of each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0081] The computer readable program instructions for carrying out the operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state configuration data, integrated circuit configuration data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, procedural programming languages ​​such as the "C" programming language, or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet Service Provider). In some examples, electronic circuitry, including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer-readable program instructions by individualizing the electronic circuitry using state information of the computer-readable program instructions to perform aspects of the present disclosure.

[0082] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to examples of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0083] Computer-readable program instructions may be provided to a processor of a special purpose computer, or other programmable data processing device, to manufacture a machine, whereby the instructions executing via the processor of the computer or other programmable data processing device create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. Furthermore, these computer-readable program instructions may be stored on a computer-readable storage medium that can direct a computer, programmable data processing device, and / or other device to function in a particular manner, whereby the computer-readable storage medium on which the instructions are stored includes an article of manufacture having instructions that implement aspects of the functions / operations specified in the flowcharts and / or block diagrams or blocks.

[0084] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to generate a computer-implemented process for a series of operational steps to be performed on the computer, other programmable apparatus, or other device, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0085] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various examples of the present disclosure. In this regard, each block of the flowcharts or block diagrams may correspond to a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or in some cases, the blocks may be executed in reverse order depending on the functionality involved. Furthermore, it should be noted that each block of the block diagrams and / or flowchart illustrations, as well as combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented in a dedicated hardware-based system that performs a specific function or operates or executes a combination of dedicated hardware and computer instructions.

[0086] Other implementations are within the scope of the following claims as well as other claims to which the applicant may be entitled.

[0087] While various examples have been described and illustrated herein, those of ordinary skill in the art will readily conceive of various other means and / or structures for performing the functions and / or results and / or obtaining one or more of the advantages described herein, and each of such modifications and / or variations is deemed to be within the scope of the examples described herein. More generally, those of ordinary skill in the art will readily appreciate that all of the parameters, dimensions, materials, and configurations described herein are exemplary, and furthermore, that the actual parameters, dimensions, materials, and / or configurations will depend on the particular application or applications for which the teachings of the present invention are used. Those of ordinary skill in the art will recognize or be able to ascertain, using no more than routine experimentation, many equivalents to the specific examples described herein. Thus, it is to be understood that the foregoing examples are presented by way of example only, and that, within the scope of the appended claims and equivalents thereof, the examples may be practiced otherwise than as specifically described and claimed. Examples of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is within the scope of the present disclosure, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent. [Explanation of symbols]

[0088] 100 Systems 104 Peripheral Devices 102A, 102B Wearable Devices 162A, 162B, 162C Communication Data Stream 166 Primary Audio Capture Device

Claims

1. 1. A method for selecting one or more audio capture sensors of a wearable device, comprising: Detecting a first signal corresponding to a voice of a user using at least one audio capture sensor of a first wearable device; Detecting a second signal corresponding to a voice of the user using at least one audio capture sensor of a second wearable device, the second wearable device being wirelessly connected to the first wearable device; determining an energy level of the first signal; determining an energy level of the second signal; selecting at least one audio capture sensor of the first wearable device and / or at least one audio capture sensor of the second wearable device to capture audio data based at least in part on the energy level of the first signal and the energy level of the second signal; the first wearable device is configured to transmit the first signal to a peripheral device, the second wearable device is configured to transmit the second signal to a peripheral device, and the peripheral device is configured to select at least one audio capture sensor of the first wearable device and / or at least one audio capture sensor of the second wearable device to compare the energy level of the first signal with the energy level of the second signal to obtain audio data; The method, wherein the at least one audio capture sensor of the first wearable device comprises a first set of multiple audio capture sensors, the first signal is an average energy level of a first of the multiple audio capture sensors, and the first wearable device is configured to transmit the first signal to the peripheral device using a compression algorithm.

2. The method of claim 1 , wherein the first wearable device is further configured to transmit metadata to the first peripheral device for use with a mixing algorithm executed by the peripheral device.

3. 3. The method of claim 2, wherein the mixing algorithm is arranged to generate an output, the peripheral device is configured to send the output to the first wearable device, and the first wearable audio device is configured to use the output to automatically change volume levels or active noise reduction (ANR) settings.

4. 2. The method of claim 1, wherein the second wearable device is configured to transmit the second signal to the first wearable device, and the first wearable device is configured to select the at least one audio capture sensor of the first wearable device and / or the at least one audio capture sensor of the second wearable device to compare the energy level of the first signal with the energy level of the second signal to obtain audio data.

5. 2. The method of claim 1, wherein the energy levels of the first and second signals are related to at least one of a signal-to-noise ratio (SNR), a presence of wind, a radio frequency (RF) performance, or audio pickup.

6. 2. The method of claim 1, wherein the at least one audio capture sensor of the first wearable device and the at least one audio capture sensor of the second wearable device are selected from at least one of an external microphone, an internal microphone, a feedback microphone also used for acoustic noise reduction (ANR) purposes, a feedforward microphone also used for ANR purposes, or an accelerometer.

7. 2. The method of claim 1 , wherein the at least one audio capture sensor of the first wearable device includes a first set of multiple audio capture sensors, and wherein the selecting the at least one audio capture sensor includes dynamically changing between selecting a first audio capture sensor of the first set of multiple audio capture sensors and a second audio capture sensor of the first set of multiple audio capture sensors.

8. 2. The method of claim 1 , wherein the at least one audio capture sensor of the first wearable device includes a first set of multiple audio capture sensors, and the energy level of the first signal is an average energy level of the first set of multiple audio capture sensors.

9. 9. The method of claim 8, wherein the at least one audio capture sensor of the second wearable device includes a second set of multiple audio capture sensors, and the energy level of the second signal is an average energy level of the second set of multiple audio capture sensors.

10. 2. The method of claim 1, wherein when the energy level of the first signal is less than the energy level of the second signal, only the at least one audio capture sensor of the first wearable device is selected to acquire audio data, and the at least one audio capture sensor of the second wearable device is not selected.

11. The method of claim 1 , wherein the audio data is transmitted to the peripheral device using a negotiated connected isochronous stream.

12. The method of claim 11, wherein when the selecting causes a change from acquiring audio data using the at least one audio capture sensor of the first wearable device to acquiring audio data using the at least one audio capture sensor of the second wearable device, the first wearable device performs a handshake with the second wearable device to shift responsibility for acquiring audio data to the at least one audio capture sensor of the second wearable device.

13. The method of claim 1, wherein when the selecting causes a change from acquiring audio data using the at least one audio capture sensor of the first wearable device to acquiring audio data using the at least one audio capture sensor of the second wearable device, gain parameters associated with the at least one audio capture sensor of the first wearable device are shared with the second wearable device to be applied to the at least one audio capture sensor of the second wearable device.

14. 1. A computer program product for selecting one or more audio capture sensors of a wearable device, the computer program product comprising: a set of non-transitory computer readable instructions, the set of non-transitory computer readable instructions, when executed on a processor of at least one of a first wearable device, a second wearable device, or a peripheral device, causing the processor to: Detecting or receiving a first signal corresponding to a user's voice captured by at least one audio capture sensor of the first wearable device; Detecting or receiving a second signal corresponding to a user's voice captured by at least one audio capture sensor of the second wearable device, the second wearable device being wirelessly connected to the first wearable device; and determining an energy level of the first signal; determining an energy level of the second signal; and selecting at least one audio capture sensor of the first wearable device or at least one audio capture sensor of the second wearable device to capture audio data based at least in part on the energy level of the first signal and the energy level of the second signal; the first wearable device is configured to transmit the first signal to a peripheral device, the second wearable device is configured to transmit the second signal to a peripheral device, and the peripheral device is configured to select at least one audio capture sensor of the first wearable device and / or at least one audio capture sensor of the second wearable device to compare the energy level of the first signal with the energy level of the second signal to obtain audio data; 22. A computer program product comprising: the at least one audio capture sensor of the first wearable device comprising a first set of a plurality of audio capture sensors; the first signal is an average energy level of a first of the plurality of audio capture sensors; and the first wearable device configured to transmit the first signal to the peripheral device using a compression algorithm.

15. 15. The computer program product of claim 14, wherein the first wearable device is configured to transmit the first signal to the peripheral device, the second wearable device is configured to transmit the second signal to the peripheral device, and the peripheral device is configured to compare the energy level of the first signal with the energy level of the second signal to select the at least one audio capture sensor of the first wearable device or the at least one audio capture sensor of the second wearable device to acquire audio data.

16. 15. The computer program product of claim 14, wherein the second wearable device is configured to transmit the second signal to the first wearable device, and the first wearable device is configured to select the at least one audio capture sensor of the first wearable device or the at least one audio capture sensor of the second wearable device to compare the energy level of the first signal with the energy level of the second signal to obtain audio data.

17. 15. The computer program product of claim 14, wherein the energy levels of the first and second signals are related to at least one of a signal-to-noise ratio (SNR), a presence of wind, a radio frequency (RF) performance, or audio pickup.

18. 15. The computer program product of claim 14, wherein the at least one audio capture sensor of the first wearable device and the at least one audio capture sensor of the second wearable device are selected from at least one of an external microphone, an internal microphone, a feedback microphone also used for acoustic noise reduction (ANR) purposes, a feedforward microphone also used for ANR purposes, or an accelerometer.

19. 15. The computer program product of claim 14, wherein the at least one audio capture sensor of a first wireless wearable device includes a first set of multiple audio capture sensors, and the energy level of the first signal is an average energy level of the first set of multiple audio capture sensors, and the at least one audio capture sensor of the second wearable device includes a second set of multiple audio capture sensors, and the energy level of the second signal is an average energy level of the second set of audio capture sensors.

20. 15. The computer program product of claim 14, wherein when the energy level of the first signal is less than the energy level of the second signal, only the at least one audio capture sensor of the first wearable device is selected to acquire audio data, and the at least one audio capture sensor of the second wearable device is not selected.

21. 15. The computer program product of claim 14, wherein when the selecting causes a change from capturing audio data using the at least one audio capture sensor of the first wearable device to capturing audio data using the at least one audio capture sensor of the second wearable device, the first wearable device performs a handshake with the second wearable device to shift responsibility for capturing audio data to the at least one audio capture sensor of the second wearable device.

22. 15. The computer program product of claim 14, wherein when the selecting causes a change from acquiring audio data using the at least one audio capture sensor of the first wearable device to acquiring audio data using the at least one audio capture sensor of the second wearable device, gain parameters associated with the at least one audio capture sensor of the first wearable device are shared with the second wearable device to be applied to the at least one audio capture sensor of the second wearable device.

Images