Audio system with hybrid rendering audio enhancement

Abstract

Various implementations include audio systems and methods for hybrid rendering to enhance audio output. Some implementations include an audio system having: at least one far field speaker configured to output a first portion of an audio signal; and a pair of non-blocking near-field speakers configured to output a second portion of the audio signal in synchronization with the output of the first portion of the audio signal, where the first portion of the audio signal is different from the second portion of the audio signal.

Description

Technical Field

[0001] The present disclosure generally relates to audio systems. More specifically, the present disclosure relates to hybrid rendering audio enhancement in audio systems. Background Art

[0002] When multiple users are in a listening environment, conventional loud audio systems lack personalization. That is, these conventional systems provide the same loud audio content to all users in the environment. This conventional approach limits the user experience. Summary of the Invention

[0003] All examples and features mentioned below can be combined in any technically possible way.

[0004] Various embodiments include methods for enhancing audio output using hybrid rendering. Additional embodiments include audio systems configured to employ a hybrid rendering method to enhance audio output.

[0005] In some particular aspects, an audio system includes: at least one far-field speaker configured to output a first portion of an audio signal; and a pair of non-blocking near-field speakers configured to output a second portion of the audio signal synchronously with the output of the first portion of the audio signal, wherein the first portion of the audio signal is different from the second portion of the audio signal.

[0006] In additional particular aspects, a method of controlling an audio system includes: outputting a first portion of an audio signal to at least one far-field speaker; and outputting a second portion of the audio signal to a pair of non-blocking near-field speakers synchronously with the output of the first portion of the audio signal, wherein the first portion of the audio signal is different from the second portion of the audio signal.

[0007] Embodiments may include one of the following features, or any combination thereof.

[0008] In some cases, synchronously outputting the first portion of the audio signal and the second portion of the audio signal is performed within approximately 100 milliseconds (ms).

[0009] In certain embodiments, the first portion of the audio signal and the second portion of the audio signal include different channels of common content.

[0010] In particular aspects, the second portion of the audio signal is binaurally encoded and, when output synchronously with the output of the first portion of the audio signal, the second portion of the audio signal increases the externalization of the content in the audio signal.

[0011] In some cases, the second part of the audio signal enhances the speech content within the audio signal.

[0012] In a particular implementation, the second part of the audio signal increases the intelligibility of the speech content within the audio signal. In some examples, the speech content is removed from the audio track and then added back to the audio track. In certain cases, a near - field device (e.g., a wearable audio device) adds an audio detail level (i.e., clarity) that is not easily achieved by a far - field audio device (e.g., a loudspeaker several meters or more away), thereby improving the impulse response of the audio output. In some aspects, the far - field audio device can supplement the bass output at a non - occluding near - field device (e.g., an open - ear wearable audio device), which can strive to provide the desired bass without occlusion. In such examples, the near - field device and the far - field device have complementary audio output benefits.

[0013] In some aspects, the first part of the audio signal does not include speech content, and the second part of the audio signal includes speech content. In an example, the language of the speech content in the second part of the audio signal is selectable.

[0014] In a particular implementation, the first part of the audio signal and the second part of the audio signal include different frequency ranges from the audio signal. In some examples, the first part of the audio signal is output in a frequency range where the sound field is diffuse (or non - modal) (e.g., about 200 Hz or higher). In additional examples, the frequency limits at each device can be adjusted to enhance the output.

[0015] In some cases, i) the first part of the audio signal has a larger frequency range than the frequency range of the second part of the audio signal, or ii) the second part of the audio signal excludes frequencies below a pre - determined threshold.

[0016] In some aspects, the pair of non - occluding near - field speakers are housed in a wearable audio device. In some examples, the wearable audio device includes a pair of wirelessly coupled audio devices, such as wireless earbuds.

[0017] In an additional implementation, the pair of non - occluding near - field speakers are housed in a seat, such as in the neck support portion and / or the head support portion of the seat.

[0018] In a particular aspect, the first speaker of the pair of non - occluding near - field speakers is configured to output audio to the user's left ear, and the second speaker of the pair of non - occluding near - field speakers is configured to output audio to the user's right ear.

[0019] In some cases, i) the at least one far-field speaker is housed in a soundbar, or ii) the at least one far-field speaker includes a plurality of speakers configured to output at least a left channel, a right channel, and a center channel of the audio signal.

[0020] In certain embodiments, the at least one far-field speaker is configured to output a third portion of the audio signal when no output synchronized with the pair of non-blocking near-field speakers is provided, the third portion of the audio signal being different from the first portion of the audio signal. In some examples, the third portion of the audio signal is the entire audio signal.

[0021] In certain cases, the at least one far-field speaker is configured to operate differently in different modes, which include (for example): Mode 1: output audio synchronously with a pair of non-blocking near-field speakers, and Mode 2: output audio only without outputting synchronously with the pair of non-blocking near-field speakers. In additional aspects, the pair of non-blocking near-field speakers is configured to operate differently in different modes, which include (for example): Mode 1: output a subset of the frequency range of the audio signal in the case of synchronous playback with the far-field speaker, and Mode 2: output a larger frequency range of the audio signal up to the full frequency range of the audio signal in the case of outputting audio only (or not synchronously with at least one far-field speaker) than in Mode I.

[0022] In some cases, the second portion of the audio signal is user-configurable to enhance at least one of the following: externalization, spatialization, or dialogue. In certain cases, the second portion of the audio signal can be configured by the user via an interface (such as, a voice interface and / or an application interface) on a computing device (such as, a smart device). In some cases, the second portion of the audio signal can be configured to adjust tone, equalization, volume, and other parameters for different non-blocking near-field speakers. In additional embodiments, the adjustment of the audio output to the at least one far-field speaker is configured to provide the same adjustment or a proportional adjustment as the audio output to the non-blocking near-field speakers. In a further aspect, the adjustment of the audio output at a given device (such as, a non-blocking near-field speaker and / or a far-field speaker) is limited such that at least one parameter of the audio signal is limited to a certain range. In some of these aspects, the volume adjustment can be limited to a certain range, the tone adjustment can be limited to a certain range, and / or the equalization adjustment can be limited to a certain range.

[0023] In a particular implementation, user-configurable parameters (e.g., a second portion of the audio signal) can be controlled via an interface on the non-blocking near-field speaker, such as via an adjustment interface such as a button, switch, a touch interface such as a capacitive touch interface, or a voice interface via a microphone at the non-blocking near-field speaker. In some aspects, the adjustment interface at the non-blocking near-field speaker can include a scroll-through or cycle-through type adjustment mechanism, e.g., allowing the user to cycle or scroll through various modes with each interface interaction (e.g., button touch or switch flip).

[0024] In an additional implementation, the non-blocking near-field speaker can be replaced with a blocking near-field speaker (e.g., an ear-hook or in-ear headphone operating in a transparent (or, see-through) mode). In these cases, the blocking near-field speaker can operate in a shared experience (or social) mode.

[0025] In certain cases, the second portion of the audio signal is automatically customized based on the detection of the device type of the device housing at least one of the pair of non-blocking near-field speakers. In some aspects, the device type is detected using a device identifier, and the second portion of the audio signal is automatically customized based on the device identifier, e.g., to adapt the audio output to different device types (e.g., different wearable audio devices). In a particular implementation, the customized audio output is triggered based on at least one of the following: a Bluetooth (BT) connection (or a variant of the BT connection), a previous BT pairing scenario, or a coupling between previously defined devices.

[0026] In some aspects, the pair of non-blocking near-field speakers is configured to output the second portion of the audio signal synchronously with the output of the first portion of the audio signal in response to a trigger. In certain cases, the trigger can include one or more of the following: a detected connection between two devices housing different speakers (such as a wired connection, or a wireless connection such as a BT or Wi-Fi connection), a detected alignment of two devices housing different speakers (such as via BT angle of arrival (AoA) and / or angle of departure (AoD) data), a grouping request from the user (such as from an application or a voice command), a proximity or location detection (such as devices identified as being close to each other or in the same area or space via BT AoA and / or AoD or Wi-Fi RTT), or a user-initiated trigger or command to synchronize the output.

[0027] In a particular implementation, the system further includes a first device that houses the at least one far - field speaker, where the first device is configured to send the audio signal or the second portion of the audio signal to a second device that includes the pair of non - blocked near - field speakers, where the sending includes playback timing data so that the second portion of the audio signal can be output synchronously with the first portion of the audio signal. In some cases, the audio signal or the second portion of the audio signal is sent wirelessly via one or more of the following: Bluetooth (BT); BT Low Energy (LE) Audio; Broadcast (to the second device and at least one additional device), such as via synchronous unicast; Synchronous downmix audio connection via BT or other wireless connection; Multiple transmission streams such as broadcast, for example, allowing different devices with different pairs of non - blocked near - field speakers to simultaneously output different portions of the audio signal synchronously with the first portion of the audio signal, such as where one portion focuses on spatialization and / or externalization enhancement, while another portion focuses on dialogue enhancement, or where one portion includes dialogue output in a first language and another portion includes dialogue output in a different second language.

[0028] In some cases, the system further includes: a first device that includes the at least one far - field speaker; and a second device that includes the pair of non - blocked near - field speakers, where (i) the first device and the second device receive the audio signal from a common source device, or (ii) the first device receives the first portion of the audio signal from a first source device and the second device receives the second portion of the audio signal from a second source device. In some aspects, in scenario (i), the common source device may include a television or a streaming video player (such as, via wired transmission and / or wireless transmission), for example, where the first device and the second device receive the audio signal wirelessly from the television or the streaming video player, or where a soundbar receives the audio signal via a hard - wired connection to the television or the streaming video player, and the non - blocked near - field speakers receive the audio signal wirelessly from the television or the streaming video player. In some cases, in scenario (ii), the first device (such as a soundbar) receives the first portion of the audio signal from a first source device (such as a television or a streaming video player), and the second device (such as a wearable audio device) receives the second portion of the audio signal from a second source device (such as a smart phone, a tablet computer, or a computing device).

[0029] In a particular implementation, the first device and the second device receive the audio signal from a common source device, where the audio signal includes audio content that takes into account surround effects and height effects, and where outputting the audio signal at the first device and the second device excites the room housing the first device and the second device, thereby enhancing the sense of presence for the user. In some of these cases, the audio output is perceived by the user as object-based (as compared to channel-based), such that the user perceives the audio output as objects in space. In some aspects, in response to determining that the non-blocking near-field speaker(s) may be located at a minimum distance (e.g., several meters or more) from the at least one far-field speaker, the system provides an audio signal having audio content that takes into account surround effects and height effects. In some of these aspects, these surround effects and height effects are beneficial in large spaces where the distance between the far-field speaker(s) and the non-blocking near-field speaker(s), such as in religious services, concert halls, arenas, conference rooms, etc., meets a threshold.

[0030] Two or more features described in this disclosure, including those described in the Summary section, may be combined to form embodiments not specifically described herein.

[0031] Details of one or more embodiments are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a block diagram of a system including a far-field speaker and a set of non-blocking near-field speakers according to various disclosed embodiments.

[0033] Figure 2 is a schematic diagram of a space including a far-field speaker and a non-blocking near-field speaker according to various embodiments.

[0034] Figure 3 is an audio signal flow diagram according to various embodiments.

[0035] Figure 4 is an audio signal flow diagram according to various additional embodiments.

[0036] Note that the drawings of the various embodiments are not necessarily to scale. The drawings are merely intended to illustrate typical aspects of the disclosure and should not be considered as limiting the scope of the embodiments. In the drawings, like reference numerals represent like elements among the drawings. DETAILED DESCRIPTION

[0037] This disclosure is at least partially based on the recognition that coordinating audio output at different speaker systems can enhance the individual user experience as well as the group experience.

[0038] Conventional audio systems that employ dialogue enhancement rely on a mixture of spectral modification and dynamic range compression. While this hybrid approach has benefits, dialogue intelligibility is still negatively impacted by one or more speech transmission index (STI) factors such as: background noise, the quality of the audio reproduction equipment, reverberation in the space (e.g., the room), and the masking effect from other audio content in the mixture. Conventional systems fail to adequately account for these STI factors, which reduce dialogue intelligibility.

[0039] Additionally, conventional audio systems that employ binaural rendering in occluded wearable audio devices (e.g., ear-hung headphones) rely on complex room models and head position (i.e., tracking) data to externalize virtual sound sources. These conventional methods are expensive and consume a large amount of power. In addition, processing head position data can introduce latency-related drawbacks to these conventional methods.

[0040] The systems and methods disclosed according to various embodiments use at least one far-field speaker to output a first portion of an audio signal and a pair of non-occluding (e.g., open-ear) near-field speakers to output a different second portion of the audio signal synchronously with the output of the first portion. In these cases, users of the non-occluding near-field speakers (e.g., in-seat speakers or wearable audio devices that do not occlude the ear canal) can have a personalized audio experience without sacrificing the social aspects of an open-ear audio environment. For example, when compared to listening to the far-field speaker alone, users of the non-occluding near-field speakers can experience enhanced audio quality, clarity, personalization / customization, etc., without sacrificing social engagement with other users in the same space (due to the non-occluding near-field speakers). Additionally, the mixing of the audio output from the hybrid systems and techniques described herein can provide numerous audio consumption benefits such as enhanced personalization, enhanced externalization / spatialization, increased dialogue intelligibility (e.g., for the audio of video content or podcasts), and / or the ability to better experience the full spectrum of audio content (e.g., experience full-body bass compared to bass received from an open-ear wearable audio device).

[0041] For illustrative purposes, components that are commonly labeled in the figures are considered to be substantially equivalent components, and for the sake of clarity, redundant discussions of those components are omitted.

[0042] Figure 1An example of space 5 including system 10 according to various specific implementations is shown. The system includes a set of devices. In various specific implementations, the devices shown in system 10 include at least one far-field (FF) speaker 20 and a pair of non-blocking near-field (NF) speakers 30A, 30B. One or more additional devices 40 are shown, which are optional in some specific implementations. The additional device 40 can be configured to communicate with the FF speaker 20, NF speakers 30A, 30B, and / or other electronic devices in space 5 using any communication protocol or method described herein. In certain aspects, system 10 is located in or around space 5, such as a closed or partially closed room in a home, office, theater, sports or entertainment venue, religious venue, etc. In some cases, space 5 has one or more walls and a ceiling. In other cases, space 5 includes an open-air location without walls and / or a ceiling.

[0043] In various specific implementations, at least one far-field speaker 20 includes an audio device not intended to be worn by a user, e.g., a standalone loudspeaker or a set of loudspeakers such as a soundbar, portable speaker, hard-wired (i.e., semi-permanent or installed type) speaker, etc. Although speaker 20 is described as a "far-field" device, it is not necessarily located within a recognized "far-field" acoustic distance relative to any other device in system 10. That is, the far-field speaker 20 does not need to be located in the far field to function according to the various specific implementations described herein. In various specific implementations, the minimum far-field distance is defined as being approximately 0.5 meters away in some cases, approximately two (2) meters or more away in additional cases, and approximately three (3) meters or more away in certain cases. It should be understood that the minimum far-field distance can vary based on the environment, e.g., within a vehicle (approximately 0.5 meters to approximately 2 meters), in a room within a home (e.g., approximately 2 meters to approximately 5 meters), or in an entertainment venue such as a concert hall (e.g., approximately 5 meters to approximately 50 meters).

[0044] In some cases, speaker 20 includes a controller 50 and a communication (comm.) unit 60 coupled to controller 50. In some examples, communication unit 60 includes a Bluetooth module 70 (e.g., including a Bluetooth radio), enabling communication with other devices via the Bluetooth protocol. In some specific implementations, speaker 20 may further include one or more microphones (mic) 80 (e.g., a single microphone or a microphone array) and at least one electroacoustic transducer 90 for providing an audio output. Speaker 20 may also include additional electronics 100, such as a power manager and / or a power source (e.g., a battery or a power connector), a memory, sensors (e.g., an IMU, an accelerometer / gyroscope / magnetometer, an optical sensor, a voice activity detection system), etc. In some cases, the memory may include a flash memory and / or a non-volatile random access memory (NVRAM). In certain cases, the memory stores: microcode for processing and controlling the programs of controller 50 and various reference data; data generated during the execution of any of the various programs executed by controller 50; Bluetooth connection procedures; and / or various updatable data for proper custody, such as paired device data, connection data, device contact information, etc. Figure 1 Some of the above-described components shown are optional and are shown in dashed lines.

[0045] In some cases, controller 50 may include one or more microcontrollers or processors having a digital signal processor (DSP). In some cases, controller 50 is referred to as a control circuit. Controller 50 may be implemented as a chipset of a chip, which includes a plurality of independent analog processors and digital processors. Controller 50 may provide coordination of, for example, other components of speaker 20, such as control of a user interface (not shown) and application programs run by speaker 20. In various specific implementations, controller 50 includes one or more (audio) mixing and rendering control modules, and the one or more modules may include software and / or hardware for performing the audio control processes described herein. For example, controller 50 may include a mixing and rendering control module in the form of a software stack, which has instructions for controlling the output of audio to one or more speakers in system 10 according to any of the specific implementations described herein. As described herein, controller 50 and other controllers described herein are configured to control functions in the mixing and rendering audio output method according to various specific implementations.

[0046] The communication unit 60 may include a BT module 70 configured to employ a wireless communication protocol such as Bluetooth, and additional network interfaces such as those that employ one or more additional wireless communication protocols such as IEEE 802.11, Bluetooth Low Energy, or other local area network (LAN) or personal area network (PAN) protocols such as WiFi. In a particular implementation, the communication unit 60 is particularly adapted to communicate with other communication units 60 in the BT devices 30, 40 via Bluetooth. In another particular implementation, the communication unit 60 is configured to communicate with the BT devices described herein using broadcast audio via BLE or a similar connection (e.g., including a proxy connection). In a further implementation, the communication unit 60 is configured to wirelessly communicate with any other device in the system 10 via one or more of the following: Bluetooth (BT); BT Low Energy (LE) audio; broadcast (e.g., to one or two NF speakers 30A, 30B, and / or additional devices 40), such as via synchronous unicast; a synchronous mixdown audio connection (also known as SimpleSync TM , a proprietary connection protocol from Bose Corporation, Framingham, MA, USA); multiple transmission streams such as broadcast, e.g., allowing different devices with different pairs of non-blocking near-field speakers (e.g., similar to NF speakers 30A, 30B) to simultaneously output different portions of an audio signal in synchronization with a first portion of the audio signal. In a further implementation, the communication unit 60 is configured to communicate with any other device in the system 10 via a hardwired connection, e.g., between any two or more devices.

[0047] As noted herein, the controller 50 controls the general operation of the FF speaker 20. For example, the controller 50 performs processing that controls audio and data communication with additional devices (e.g., NF speakers 30A, 30B), as well as audio output, signal processing, etc. at the FF speaker 20. In addition to the general operation, the controller 50 initiates communication functions implemented in the communication module 60 when certain triggers (or events) described herein are detected. If specific conditions are met, the controller 50 initiates operations (e.g., synchronization of audio output) between the FF speaker 20 and the NF speakers 30A, 30B.

[0048] In some examples, the Bluetooth module 70 uses radio frequency (RF) communication between the FF speaker 20 and the NF speakers 30A, 30B (and additional devices 40 in certain embodiments) to achieve a wireless connection. The Bluetooth module 70 exchanges radio signals including data input / output via an antenna (not shown). For example, in the transmit mode, the Bluetooth module 70 processes data through channel coding and spreading, converts the processed data into a radio frequency (RF) signal, and transmits the RF signal. In the receive mode, the Bluetooth module 70 converts the received RF signal into a baseband signal, processes the baseband signal through despreading and channel decoding, and restores the processed signal to data. Additionally, the Bluetooth module 70 can ensure secure communication between devices and uses encryption to protect data.

[0049] As noted herein, Bluetooth-enabled devices include a Bluetooth radio or other Bluetooth-specific communication systems that implement connections via the Bluetooth protocol. In Figure 1 the example shown, the FF speaker 20 is a BT source device (also referred to as an "input device" or "host device"), and the NF speakers 30A, 30B are part of a single BT sink device (also referred to as an "output device", "destination device" or "peripheral device") or are different BT sink devices. Example Bluetooth-enabled source devices include, but are not limited to, smart phones, tablet computers, personal computers, laptop computers, notebook computers, netbook computers, radios, audio systems (e.g., portable and / or stationary), Internet Protocol (IP) phones, communication systems, entertainment systems, headsets, smart speakers, a piece of sports and / or fitness equipment, portable media players, audio storage and / or playback systems, smart watches or other smart wearable devices, and the like. Example Bluetooth-enabled sink devices include, but are not limited to, earphones, headsets, audio speakers (e.g., portable and / or stationary, with or without "smart" device functionality), entertainment systems, communication systems, smart phones, vehicle audio systems, a piece of sports and / or fitness equipment, public address (or outdoor) audio devices, wearable personal audio devices, and the like. Additional BT devices can include portable game consoles, portable media players, audio gateways, BT gateway devices (for bridging BT connections between other BT-enabled devices), audio / video (A / V) receivers as part of a home entertainment or home theater system, and the like. Bluetooth-enabled devices as described herein can change their role from source to sink or from sink to source depending on the specific application.

[0050] In various specific embodiments, a first speaker in the NF speaker (e.g., NF speaker 30A) is configured to output audio to a user's left ear, and a second speaker in the NF speaker (e.g., NF speaker 30B) is configured to output audio to a user's right ear. In a specific embodiment, the NF speakers 30A, 30B are housed in a common device (e.g., included in a common housing), or otherwise form part of a common speaker system. For example, the NF speakers 30A, 30B may include in-seat or in-headrest speakers, such as left / right speakers in the headrest and / or seatback portion of an entertainment seat, gaming seat, theater seat, automotive seat, etc. In some cases, the NF speakers 30A, 30B are positioned in the near field relative to the user's ears, e.g., up to about 30 centimeters. In some of these cases, the NF speakers 30A, 30B may include headrest speakers or body- or shoulder-worn speakers that are about 30 centimeters or less from the user's ears. In other specific cases, e.g., where the NF speakers 30A, 30B include headrest speakers, the near-field distance is about 15 centimeters or less. In further specific cases, e.g., where the NF speakers 30A, 30B include a head-mounted or near-ear wearable audio device, the near-field distance is about 10 centimeters or less. In additional specific cases, e.g., where the NF speakers 30A, 30B include an on-ear wearable audio device, the near-field distance is about 5 centimeters or less. These example near-field ranges are illustrative only, and various form factors may be considered within one or more of the example ranges of the near field as indicated herein.

[0051] In additional specific embodiments, the NF speakers 30A, 30B are part of a wearable audio device such as a wired or wireless wearable audio device. For example, the NF speakers 30A, 30B may include earpieces in a wirelessly coupled or hard-wired wearable headset. The NF speakers 30A, 30B may also be part of a wearable audio device of any form factor, e.g., a pair of audio glasses, an on-ear or near-ear audio device, or an audio device placed on or around the user's head and / or shoulder region. As noted herein according to specific embodiments, the NF speakers 30A, 30B are non-blocking near-field speakers, meaning that when worn, the speakers 30 and their enclosures do not completely obscure (or block) the user's ear canals. That is, at least some ambient acoustic signals can reach the user's ear canals without obstruction from the NF speakers 30A, 30B. In additional specific embodiments further described herein, the NF speakers 30A, 30B may include blocking devices (e.g., a pair of ear-hook headphones or earplugs with ear canal sealing features) that enable a hear-through (or "awareness") mode to pass ambient acoustic signals to the user's ears during playback.

[0052] As shown Figure 1 in FIG. 1, the NF speakers 30A, 30B may include controllers 50a, 60b and communication units 60a, 60b (e.g., having BT modules 70a, 70b), enabling communication between the FF speaker 20 and the NF speakers 30A, 30B. The additional device 40 may include one or more components described with respect to the FF speaker 20, and in certain specific embodiments, each component is shown in dashed lines as optional. The symbols "a" and "b" indicate that the components in the devices (e.g., NF speaker 30A, NF speaker 30B, additional device 40) are physically separate from the components with similar markings in the FF speaker 20, but may adopt a similar form and / or function to their corresponding marked components in the FF speaker 20. For the sake of brevity, additional descriptions of these similarly marked components are omitted. In addition, as described herein, the additional NF speakers 30A, 30B and the additional device 40 may differ from the FF speaker 20 in form factor, intended use, and / or capabilities, but in various specific embodiments, are configured to communicate with the FF device 20 according to one or more communication protocols described herein (e.g., Bluetooth, BLE, broadcast, SimpleSync, etc.).

[0053] Generally speaking, the Bluetooth modules 70, 70a, 70b include a Bluetooth radio and additional circuitry. More specifically, the Bluetooth modules 70, 70a, 70b include both a Bluetooth radio and a Bluetooth Low Energy (BLE) radio. In various specific embodiments, the presence of the BLE radio in the Bluetooth module 70 is optional. That is, as pointed out herein, various specific embodiments use only the (classic) Bluetooth radio for connection functions. In specific embodiments that include a BLE radio, the Bluetooth radio and the BLE radio are typically on the same integrated circuit (IC) and share a single antenna, while in other specific embodiments, the Bluetooth radio and the BLE radio are implemented as two separate ICs sharing a single antenna or as two separate ICs with two separate antennas. The Bluetooth specification, namely Bluetooth 5.2 (Low Energy), provides forty channels spaced 2 MHz apart to the FF speaker 20. The forty channels are labeled 0 to 39, including 3 broadcast channels and 37 data channels. The channels labeled 37, 38, and 39 are designated as broadcast channels in the Bluetooth specification, while the remaining channels 0 - 36 are designated as data channels in the Bluetooth specification. Certain example methods of Bluetooth-related pairing are described in U.S. Patent No. 9,066,327 (published on June 23, 2015), the entire content of which is incorporated by reference. In addition, methods for selecting connections between paired devices and / or prioritizing connections between paired devices are described in U.S. Patent Application No. 17 / 314,270 (filed on May 7, 2021), the entire content of which is incorporated by reference.

[0054] As noted herein, various embodiments are particularly suitable for synchronous audio output at NF speakers 30A, 30B and FF speaker 20. In some cases, the synchronous audio output is coordinated at one or more additional devices 40. In certain cases, a controller 50 at one or more of the speakers is configured to coordinate the synchronous audio output to enhance the user experience, e.g., to achieve a personalized audio experience without sacrificing the social aspects of an open-ear audio environment. For example, a user of NF speakers 30A, 30B may experience enhanced audio quality, clarity, personalization / customization, etc. compared to listening to FF speaker 20 alone, without sacrificing social engagement with other users in the same space (due to the non-blocking nature of NF speakers 30).

[0055] Figure 2Shows a specific implementation of the audio system 10 in a space 105 (e.g., a room in a home, office, or entertainment venue). This space 105 is only one example of the various spaces that can benefit from the disclosed specific implementation. In this example, a first user 110 is present at a first seating position (e.g., seat 120), and a second user 130 is present at a second seating position (e.g., seat 140). The user 110 is wearing a wearable audio device 150, which in this example includes a set of audio glasses, such as the Bose Frames audio glasses from Bose Corporation in Framingham, Massachusetts, USA. In other cases, the wearable audio device 150 may include another open-ear audio device, such as a set of on-ear or near-ear headphones. In any case, the wearable audio device 150 includes a set (e.g., two) of non-blocking NF speakers 30A, 30B. The user 130 is located in seat 140, which includes a set of non-blocking NF speakers 30A, 30B in the headrest and / or neck / backrest portion 160. An FF speaker 20 is also shown in the space, which may include a standalone speaker, such as a soundbar (e.g., one of the Bose Smart Soundbar series from Bose Corporation) or a TV speaker (e.g., the Bose TV Speaker from Bose Corporation). In additional cases, the FF speaker 20 may include home theater speakers (e.g., the Bose Surround Speaker series and / or the Bose Bass Module series) or a portable speaker such as a portable smart speaker (e.g., one of the Bose Soundlink series or the Bose Portable Smart Speaker from Bose Corporation) or a portable professional speaker such as the Bose S1 Pro Portable Speaker. In this non-limiting example, additional devices 40A and 40B are present in the space 105. For example, the additional device 40A may include a TV and / or a visual display system (e.g., a projector-based video system or a smart monitor), and the additional device 40B may include a smart device (e.g., a smart phone, a tablet computing device, a surface computing device, a laptop computer, etc.). The devices in the space 105 and the interaction of the devices are only intended to illustrate some aspects of the present disclosure.

[0056] Referring Figure 2 to the illustrative example in, according to certain specific implementations, the audio system 10 is configured to control the synchronous audio output in both the FF speaker 20 and the NF speakers 30A, 30B. In certain cases, the processes performed according to the various specific implementations are by a controller at one or more of the speakers in the audio system 10 (e.g., the controller 50 in the FF speaker 20 and / or the NF speakers 30A, 30B (Figure 1 ) are controlled by the controllers 50a and 50b). In certain specific implementations, the FF speaker 20 is configured to output a first portion of the audio signal, and the NF speakers 30A and 30B are configured to output a second portion of the audio signal synchronously with the output of the first portion of the audio signal.

[0057] Figure 3 illustrates an example audio signal flow Figure 2 in the signal flow diagram. As shown in this example, the FF speaker 20 and the NF speakers 30A and 30B can be connected to the common source device 210. In certain specific implementations, the source device 210 can include one of the additional devices (e.g., a television, an audio gateway device, a smart phone, a tablet computing device, etc.) 40 described herein. For example, the source device 210 can include a television system, a smart phone, or a tablet computer. In additional specific implementations, the source device 210 includes a network - and / or cloud - based device, such as a network - connected audio system. In further specific implementations, the FF speaker 20 and / or the NF speakers 30A and 30B act as source devices, for example, having integrated network and / or cloud communication capabilities. In this case, the FF speaker 20 and / or the NF speakers 30A and 30B receive the audio signal from a network (or cloud) - connected gateway device (e.g., a wireless or hard - wired Internet router). In Figure 3 one example shown, the source device 210 is a network - and / or cloud - connected device running a software program or software application (also referred to as an "app") that is configured to manage the audio output to the FF speaker 20 and / or the NF speakers 30A and 30B. In some examples, the source device 210 transmits the signal to both the FF speaker 20 and the NF speakers 30A and 30B. In additional examples, the source device 210 transmits signals to the FF speaker 20 or the NF speakers 30A and 30B (one or both of them), and these signals are forwarded between those speaker connections. In certain specific implementations, the NF speakers 30A and 30B forward the signal via a "snooping" - type method or otherwise synchronize the output. Although specific example scenarios are described herein, the FF speaker 20 and the NF speakers 30A and 30B can forward or otherwise send the signal in any technically feasible way, and the examples described herein (e.g., SimpleSync, broadcast, Bluetooth, etc.) should not be considered as limitations on the various specific implementations.

[0058] In a specific example, such as Figure 3 shown, the source device 210 transmits the audio signal 220 to the FF speaker 20 and the NF speakers 30A and 30B. In further specific implementations, as Figure 4As shown, two different source devices 210A and 210B respectively transmit signals to the FF speaker 20 and the NF speakers 30A, 30B. For example, using Figure 2 the space 105 in

[0059] as an example, an additional device 40A (e.g., a video system) may transmit a first portion 230 of an audio signal 220 to the FF speaker 20, and an additional device 40B (e.g., a smart phone or a tablet computer) may transmit a second portion 240 of the audio signal 220 to one or both of the NF speakers 30A, 30B. Figure 2 Continuing to refer to Figure 3 and Figure 4 and

[0060] In an example embodiment, the FF speaker 20 is configured to output a first portion of the audio signal 220 (e.g., received from the source device 210), and the NF speakers 30A, 30B are configured to output a different second portion of the audio signal 220 (e.g., received from the source device 210) synchronously with the output of the first portion from the FF speaker 20. In some cases, the synchronous output of the first portion of the audio signal 220 and the second portion of the audio signal 220 includes outputting those portions at different speakers within approximately 100 milliseconds (ms) of each other.

[0060] In some cases, the first portion of the audio signal 220 and the second portion of the audio signal 220 include different channels of common content. In certain aspects, the second portion of the audio signal 220 is binaurally encoded, e.g., to enhance the output at the NF speakers 30A, 30B. In these cases, when output synchronously with the output of the first portion of the audio signal 220 at the NF speaker 20, the second portion of the audio signal 220 increases the externalization of the content in the audio signal 220. That is, the controllers 50a, 50b at the NF speakers 30A, 30B may be configured to output the second portion of the audio signal 220 to enhance the externalization of the audio signal 220. This enhanced externalization may create or increase the user's perception of the output of the audio signal at one or more positions in three-dimensional space that are farther from the user's head than the NF speakers 30A, 30B.

[0061] According to a particular implementation, a second portion of the audio signal 220 enhances the speech content within the audio signal 220. For example, the second portion of the audio signal 220 may increase the intelligibility of the speech content within the audio signal 220. In some examples, the speech content is removed from the audio track and then added back into the audio track in the second portion of the audio signal 220. As noted herein, the signal processing (e.g., removing the speech content and adding it back) may be performed at one or more devices in the space (e.g., space 105), such as at the FF speaker 20, NF speakers 30A, 30B, and / or additional device 40. In a further implementation, one or more portions of the signal processing are performed in a distributed computing system and / or a cloud computing system. In a specific case such as Figure 2 shown, the near-field speakers 30A, 30B (e.g., wearable audio devices) add a level of audio detail (i.e., clarity) that is not easily achieved by far-field audio devices such as the FF speaker 20 (e.g., a loudspeaker several meters or more away), thus improving the impulse response of the audio output. In some aspects, the far-field speaker 20 may supplement the bass output at the non-blocking NF speakers 30A, 30B (e.g., open-ear wearable audio devices), which may strive to provide the desired bass without blocking. In such examples, the NF speakers 30A, 30B and the far-field speaker 20 have complementary audio output benefits (e.g., for Figure 2 users 110, 130).

[0062] In a further implementation, a first portion of the audio signal 220 (output at the FF speaker 20) does not include speech content, and a second portion of the audio signal 220 (output at the NF speakers 30A, 30B) includes speech content. In an example, the language of the speech content in the second portion of the audio signal 220 is selectable. For example, users 110 and 130 in the same space 105 may have different language preferences for audio playback (e.g., a user whose primary language is English and a user whose primary language is Spanish). In such cases, the system 10 may enable one or both of the users 110, 130 to select the language of the speech content output to the NF speakers 30A, 30B. This language selection may be performed via a device interface (e.g., via an interface on the additional device 40 or via any user interface on an audio device such as the wearable audio device 150). In these cases, since the first portion of the audio signal 220 output to the FF speaker 20 does not include speech, the language selection at the NF speakers 30A, 30B may be personalized without affecting other users in the space 105.

[0063] In a particular implementation, the first portion 230 of the audio signal 220 and the second portion 240 of the audio signal 220 include frequency ranges different from the audio signal 220. In some examples, the first portion 230 outputs in a frequency range where the sound field is diffuse (or non-modal) (e.g., about 200 Hz or higher). In additional examples, the frequency limitations at each device (e.g., FF speaker 20 and NF speakers 30A, 30B) can be adjusted to enhance the output. In some cases, the adjustability of the frequency limitations is based on the device type and / or is user-selectable (or user-adjustable).

[0064] In a further implementation, the frequencies of the first portion 230 and the second portion 240 differ in at least one aspect. In some examples, the first portion 230 of the audio signal has a larger frequency range than the frequency range of the second portion 240 of the audio signal. In some such embodiments, a portion of the frequency range of the first portion 230 overlaps with a portion of the frequency range of the second portion 240. In other examples, the entire frequency range of the second portion 240 is within the frequency range of the first portion 230. In an additional implementation, the second portion 240 of the audio signal excludes frequencies below a predetermined threshold.

[0065] According to certain example implementations, such as Figure 2 the implementations shown in, the FF speaker 20 is housed in a soundbar (such as one of the Bose Soundbars described herein). In a further implementation, the FF speaker 20 includes a plurality of speakers configured to output at least the left channel, right channel, and center channel of the audio signal 220 into a space (e.g., space 105). For example, the FF speaker 20 can include a set of stereo-paired portable speakers, such as portable speakers configured to operate individually and in a stereo pair. In additional examples, the FF speaker 20 can include multiple speakers in a stereo and / or surround sound speaker group, such as two, three, four, or more speakers arranged in a space (e.g., space 105).

[0066] Although certain examples of coordinated audio output between the FF speaker 20 and the NF speakers 30A, 30B are described herein, in some cases, the FF speaker 20 is configured to operate in a manner asynchronous with the NF speakers 30A, 30B. Using Figure 2As an example, in some cases, the FF speaker 20 is configured to output a third portion of the audio signal 220 when not providing an output synchronized with the NF speakers 30A, 30B. In various embodiments, the third portion of the audio signal 220 is different from the first portion of the audio signal. For example, the third portion of the audio signal 220 may include the entire audio signal. In such cases, the NF speakers 30A, 30B may be deactivated (e.g., in a sleep state and / or powered off), or may be outputting audio different from the audio signal 220.

[0067] In a particular example, the FF speaker 20 is configured to operate in different modes (e.g., in two or more modes). For example, in a first mode (Mode 1), the FF speaker 20 is configured to output audio synchronously with the pair of NF speakers 30A, 30B, such as in the scenario described in Figures 2 to 4 the reference. In a second mode (Mode 2), the FF speaker 20 is configured to output audio only, without outputting the audio synchronously with the pair of NF speakers 30A, 30B. In some of these cases, the NF speakers 30A, 30B are configured to operate differently when the FF speaker 20 operates in different modes. For example, when the FF speaker 20 operates in Mode 1, the NF speakers 30A, 30B are configured to output a subset of the frequency range of the audio signal (e.g., the audio signal 220) when outputting audio synchronously with the FF speaker 20. In an additional example, when the FF speaker 20 operates in Mode 2, the NF speakers 30A, 30B are configured to output a larger frequency range of the audio signal (e.g., the audio signal 220) than when the FF speaker 20 operates in Mode 1, up to the full frequency range of the audio signal 220. That is, when the NF speakers 30A, 30B do not output audio synchronously with the FF speaker 20, such as when outputting audio only, the NF speakers 30A, 30B may output up to the full range of the audio signal 220, or at least a larger frequency range of the audio signal than when outputting synchronously with the FF speaker 20.

[0068] As noted herein, the user experience of coordinated output at the FF speaker 20 and the NF speakers 30A, 30B can be adjustable or otherwise modifiable, for example, in response to a particular trigger and / or a user command such as a user interface command. For example, see Figures 2 to 4, the second portion 240 of the audio signal 220 is user-configurable to enhance externalization, spatialization, and / or dialogue. That is, the second portion 240 of the audio signal can be adjustable (e.g., via a user interface command) to enhance the perception of sound external to the NF speakers 30A, 30B (e.g., in space 105), enhance the perception of sound located from positions around the user (e.g., in space 105), and / or enhance the audio output of dialogue (such as dialogue in an entertainment program like a television show). In some examples, the second portion 240 of the audio signal can be configurable to enhance specific sound effects in audio playback. For example, the second portion 240 of the audio signal can be configured to enhance a portion of a sound track to provide a dramatic effect (e.g., enhance the playback of an instrument during a horror movie, enhance the voice of an actor during a tense scene, improve the clarity of dialogue in the audio signal 240, and / or improve the spatialization of the audio output).

[0069] As described herein, the second portion 240 of the audio signal is configured to be user-adjustable, for example, according to a user interface command, a user profile or setting, the type of device configured to output the second audio signal 240 (e.g., NF speakers 30A, 30B), and / or the characteristics of the space (e.g., space 105). In some examples, different users (e.g., user 110 vs. user 130) may wish to experience the output of the second portion 240 of the audio signal according to specific output settings (e.g., different volume levels, different equalization settings, etc.). These settings can be adjusted during output or at any other time via user interface adjustment and / or user profile adjustment. In some examples, the NF speakers 30A, 30B are configured to adjust the audio output settings of the second portion 240 of the audio signal according to the known or otherwise detected characteristics of the space 105 and / or the characteristics of the audio signal 240. For example, when the audio playback includes a podcast or talk show and the space 105 is a vehicle or other small cockpit, enhance the spatialization and / or dialogue clarity of the second portion 240 of the audio signal. In a further example, different users in a large space such as a concert venue, arena, chapel, etc. may wish to adjust the audio output settings at the NF speakers 30A, 30B to enhance a specific portion of the audio output. For example, a user farther from the sound field may expect different equalization and / or volume settings relative to a user closer to the sound field.

[0070] In some examples, the second portion 240 of the audio signal 220 can be via a computing device (e.g., a smart device, such as Figure 2configured by a user via an interface (e.g., a voice interface and / or an application interface) on an additional device 40B). In a particular example, the voice interface may include a voice interface such as a virtual personal assistant (VPA) that receives and / or processes commands at a wearable audio device such as wearable audio device 150.

[0071] As described herein, in certain examples, NF speakers 30A, 30B are configured to output a second portion 240 of audio signal 220 synchronously with the output of a first portion 230 of audio signal 220 in response to a trigger. In a particular case, the trigger may include one or more of the following: a detected connection between two devices housing different speakers (e.g., a wired connection, or a wireless connection such as a BT or Wi-Fi connection, such as a Wi-Fi RTT connection), a detected alignment of directions between two devices housing different speakers (such as via BT angle of arrival (AoA) and / or angle of departure (AoD) data), a grouping request from a user (such as from an application or a voice command, e.g., via additional device 40B or a wearable audio device (such as wearable audio device 150)), a proximity or location detection (e.g., devices identified as being close to each other or in the same area or space, e.g., via BT AoA and / or AoD data, and / or via Wi-Fi RTT), or a user-initiated command or a user response to a prompt to follow a trigger as described herein. Any trigger pointed out herein may cause a controller 50 at a device to prompt the user to initiate a synchronous audio output mode, e.g., using a user interface prompt. The user may then take an action in response to the prompt, e.g., by accepting, rejecting, or failing to respond to the prompt via any user interface described herein. In a particular implementation, the user may accept a prompt to initiate a synchronous audio output mode by a gesture, such as a gesture detectable by a device housing NF speakers 30A, 30B.

[0072] In some cases, the second portion 240 of the audio signal 220 may be configured to adjust the tone, equalization, volume, and / or other parameters for different non-blocking near-field speakers (e.g., NF speakers 30A, 30B). In additional specific implementations, the adjustment of the audio output to the NF speakers 30A, 30B is configured to provide the same adjustment or a proportional adjustment as the audio output to the FF speaker 20. In a further example, the adjustment of the audio output at a given device (e.g., NF speakers 30A, 30B and / or far-field speaker 20) is restricted such that at least one parameter of the audio signal is limited to a certain range. In some of these examples, the volume adjustment may be limited to a certain range, the tone adjustment may be limited to a certain range, and / or the equalization adjustment may be limited to a certain range. In a particular example, the user-configurable parameter (e.g., the second portion 240 of the audio signal 220) may be controlled via an interface on the NF speakers 30A, 30B, such as via an adjustment interface such as a button, switch, a touch interface such as a capacitive touch interface, or a voice interface via a microphone at the NF speakers 30A, 30B. In some specific examples, the adjustment interface at the NF speakers 30A, 30B may include a scroll-through or cycle-through type adjustment mechanism, e.g., allowing the user to cycle or scroll through various modes with each interface interaction (e.g., button touch or switch flip). In such cases, the user-configurable parameter (e.g., volume, tone, equalization, etc.) may be adjusted within a predefined (or lock-step) range of changes such that pressing a button once initiates a first change in the parameter, pressing the button again initiates a second (e.g., incremental) change in the parameter, etc., until the extent of the adjustment range is reached.

[0073] In some cases, the second portion 240 of the audio signal 220 is automatically customized based on the detection of the type of device that houses one or more of the NF speakers 30A, 30B (e.g., whether the device includes audio glasses, on-ear headphones, near-ear headphones, or in-seat speakers). In some examples, the device type is detected using a device identifier, and the second portion 240 of the audio signal 220 is automatically customized based on the device identifier, (e.g.) to adapt the audio output to different device types (e.g., different wearable audio devices). For example, the second portion 240 of the audio signal 220 may be output with different parameters (e.g., volume or equalization) based on the type of the NF speakers 30A, 30B such that the output of the second portion 240 of the audio signal 220 is different at a pair of audio glasses compared to an in-seat speaker. In a particular implementation, the customized audio output is triggered based on at least one of the following: a Bluetooth (BT) connection (or a variant of the BT connection), a previous BT pairing scenario, or a coupling between previously defined devices. The connection trigger may be based on the detected device identifier and / or space (e.g., Figure 2connections between any devices within the communication range of the space 105). For example, detecting a BT connection, a previous BT pairing, or another previously defined coupling between the FF speaker 20 and the NF speakers 30A, 30B (e.g., via Figure 1 the communication unit 60) may trigger customized audio output based on the NF speaker 30A, 30B device types.

[0074] Returning to Figure 2 the example configuration, continuing to refer to Figure 1 , in some cases, the system 10 includes a first device that houses the FF speaker 20, such as a soundbar or a speaker. In such cases, the first device is configured to send an audio signal 220 or a second portion 240 of the audio signal 220 to a second device that includes the NF speakers 30A, 30B. For example, the second device may include a seat 140 or a headrest / neckrest portion of the seat 140, or the second device may include a wearable audio device, such as the wearable audio device 150. The sending of the audio signal 220 (or the second portion 240 of the audio signal 220) to the second device may include sending replay timing data so that the second portion 240 of the audio signal 220 can be output synchronously with the first portion 230 of the audio signal 220. In some examples, the audio signal 220 or the second portion 240 of the audio signal 220 is wirelessly sent from the first device to the second device using one or more wireless transmission protocols. In some cases, the audio signal 220 or the second portion 240 of the audio signal 220 is transmitted from the first device to the second device via Bluetooth (BT) or BT Low Energy (LE) Audio. In additional cases, the audio signal 220 or the second portion 240 of the audio signal 220 is transmitted from the first device to the second device using broadcast (to the second device and at least one additional device) (such as via synchronous unicast). In other cases, the audio signal 220 or the second portion 240 of the audio signal 220 is transmitted from the first device to the second device via a synchronous mixdown audio connection over BT or other wireless connections. In additional examples, the audio signal 220 or the second portion 240 of the audio signal 220 is transmitted from the first device to the second device via multiple transmission streams such as broadcast, for example to allow different devices with different pairs of non-blocking near-field speakers (e.g., NF speakers 30A, 30B) to simultaneously output different portions of the audio signal 220 synchronously with the first portion 230 of the audio signal 220. In some such examples, one portion focuses on spatialization and / or externalization enhancement, while another portion focuses on dialogue enhancement, or one portion includes a dialogue output in a first language, while another portion includes a dialogue output in a different second language. This example configuration enables two users 110, 130 in the same space 105 to experience different languages and / or common content with different output parameters (e.g., at different volumes, equalization levels, etc.).

[0075] Additionally, different groups of NF speakers 30 may receive the same audio signal (e.g., the same second portion 240 of the audio signal) but locally process the received audio signal in different ways to provide an audio output at different groups of NF speakers 30. For example, a first group of NF speakers 30 and a second group of NF speakers 30 (e.g., in Figure 2 the same space 105) may receive the same second portion 240 of the audio signal and locally process this second portion 240 of the audio signal in different ways to output different audio to corresponding users (e.g., the first group processes the second portion 240 to focus on enhancing dialogue intelligibility, while the second group processes the second portion 240 to focus on enhancing externalization / spatialization). Further, different groups of NF speakers 30 may receive different audio signals, such as different second portions 240 of the audio signal, or different additional portions (e.g., the first group of NF speakers 30 receives the second portion while the second group of NF speakers 30 receives the third portion, etc.). In some of these examples, the different second portions or different additional portions (e.g., the second portion and the third portion) may be processed (e.g., by FF speakers 20) before being sent to different NF speakers 30.

[0076] In some cases, as noted herein with respect to Figure 3 and Figure 4 , i) a first device (e.g., including FF speakers 20) and a second device (e.g., including NF speakers 30A, 30B) receive an audio signal 220 from a common source device 210 ( Figure 3 ), or ii) the first device receives a first portion 230 of the audio signal 220 from a first source device 210A and the second device receives a second portion 240 of the audio signal 220 from a second source device 210B ( Figure 4 ). In certain examples, in scenario (i), the common source device 210 may include a television or a streaming video player (such as via a wired transmission and / or a wireless transmission), e.g., where the first device and the second device wirelessly receive the audio signal 220 from the television or the streaming video player, or where a soundbar receives the audio signal 220 via a hardwired connection to the television or the streaming video player, while the NF speakers 30A, 30B wirelessly receive the audio signal 220 from the television or the streaming video player. In certain examples, in scenario (ii), the first device (such as a soundbar) receives the first portion 230 of the audio signal 220 from a first source device 210A (e.g., a television or a streaming video player), and the second device (such as a wearable audio device) receives this second portion of the audio signal 220 from a second source device 210B (such as a smart phone, a tablet computer, or a computing device).

[0077] In a particular implementation, for example Figure 3 as illustrated in the example of Figure 3 , the first device and the second device receive an audio signal 220 from a common source device 210, and the audio signal 220 includes audio content that takes into account surround effects and height effects. That is, the audio signal 220 includes audio content that takes into account surround sound (e.g., radial) and / or height (e.g., vertical) effects in the audio output. Certain aspects of audio content with surround and / or height effects are described in U.S. Patent Application No. 16 / 777,404 (US PG PUB 2021 / 0243544, filed on January 30, 2020), which is hereby incorporated by reference in its entirety. In some examples, outputting the audio signal 220 at the first device and the second device excites the room (e.g., Figure 2 the space 105 of Figure 2 ) that houses the first device and the second device, thereby enhancing the sense of presence for the user. In some of these cases, the audio output is perceived by the user (e.g., user 110 and / or user 130) as object-based (as compared to channel-based), such that the user perceives the audio output as objects in space. In some examples, in response to determining that the NF speakers 30A, 30B may be located at a minimum distance (e.g., several meters or more) from at least one FF speaker 20, the system 10 provides the audio signal 220 with audio content that takes into account surround effects and height effects. In some of these examples, the surround effects and height effects are beneficial in large spaces where the distance between the FF speaker 20 and the NF speakers 30A, 30B meets a threshold, such as in a religious service, a concert hall, an arena, a conference room, etc.

[0078] Although various implementations include descriptions of non-blocking variants of the NF speakers 30A, 30B, in additional implementations, the NF speakers 30A, 30B may include blocking near-field speakers that operate in a transparent (or hear-through) mode, such as ear-hung or in-ear headphones. For example, a pair of headphones with passive and / or active noise cancellation capabilities may replace the non-blocking variants of the NF speakers 30A, 30B described herein. In these cases, the blocking near-field speakers may operate in a shared experience (or social) mode, which may be enabled via a user interface command and / or any of the triggers described herein. In a particular example, the transparent (or hear-through) mode enables the user to experience the ambient audio output by the FF speaker 20 while also experiencing the coordinated playback of the audio from the NF speakers 30A, 30B.

[0079] Additionally, while various specific implementations are described as beneficially enhancing the user audio experience without knowledge of the user's head position (e.g., via user head tracking capabilities), these specific implementations may be used in combination with a system configured to track the user's head position. In such cases, data regarding the user's head position (e.g., as indicated by an IMU, an optical tracking system, and / or a proximity detection system) may be used as input to one or more processing components (e.g., at controller 50) to further enhance the user audio experience, e.g., by adjusting the output of the second portion 240 of the audio signal (e.g., in terms of spatialization, externalization, etc.). However, data regarding the user's head position is not required for the beneficial deployment of the methods and systems according to the various specific implementations.

[0080] In any case, the methods described according to the various specific implementations have the technical effect of enhancing audio playback for a user in an environment by leveraging both near-field (non-blocking) speakers and far-field speakers. For example, the methods described according to the various specific implementations coordinate the audio output at different speaker systems to enhance the individual user experience as well as the group experience. The systems and methods described according to the various specific implementations allow users of non-blocking near-field speakers (e.g., in-seat speakers or wearable audio devices that do not block the ear canal) to have a personalized audio experience without sacrificing the social aspects of an open-ear audio environment. For example, when compared to listening to far-field speakers alone, users of non-blocking near-field speakers may experience enhanced audio quality, clarity, personalization / customization, etc., without sacrificing social engagement with other users in the same space (due to the non-blocking nature of the near-field speakers). Additionally, the systems and methods described herein allow users in the same space to share a common audio experience, i.e., the audio content output via the far-field speakers, while still enabling customization of the audio content output at the near-field speakers. Additionally, the far-field speakers may enhance the spatialization and / or externalization of the audio output for users in a non-blocking scenario, thereby providing a greater three-dimensional sound impression within the space.

[0081] Various wireless connection scenarios are described herein. It should be understood that any number of wireless connections and / or communication protocols may be used to couple devices in a space (e.g., space 105( Figure 2 ))). Examples of wireless connection scenarios and triggers for connecting wireless devices are described in more detail in U.S. Patent Application No. 17 / 714,253, filed on April 4, 2022, and U.S. Patent Application No. 17 / 314,270, filed on May 7, 2021, each of which is incorporated herein by reference in its entirety.

[0082] It should also be understood that any RF protocol can be used to communicate between devices according to a particular implementation, including Bluetooth, Wi-Fi, or other proprietary or non-proprietary protocols. In a particular implementation where the NF speakers 30A, 30B are incorporated in a wearable audio device (e.g., Figure 2 ), such an implementation can advantageously use the wireless protocol otherwise used by the wearable audio device to receive audio data other than the techniques described herein (e.g., Bluetooth), thereby eliminating the need for the wearable audio device to include additional components and costs.

[0083] In an implementation that utilizes Bluetooth LE Audio, a unicast topology can be used for one-to-one connections between the FF speaker 20 and the NF speakers 30A, 30B. In some implementations, an LE Audio broadcast topology (such as Broadcast Audio) can be used to send one or more sets of audio data to multiple sets of NF speakers 30 (although the broadcast topology can still be used for only one set of NF speakers 30A, 30B). For example, in some such implementations, the broadcast audio data is the same for all sets within range of the sets of NF speakers 30, such that all sets within the sets of NF speakers 30 receive the same audio content. However, in other such implementations, different audio data is broadcast to the sets of NF speakers 30 within range, such that some of the NF speakers 30 can select a first audio data, and other NF speakers 30 can select a second audio data different from the first audio data. The different audio data can allow for differences in audio personalization (e.g., EQ settings), dialogue language selection, dialogue intelligibility enhancement, externalization / spatialization enhancement, and / or other differences understandable based on the present disclosure. Additionally, using the LE audio broadcast topology (and other implementations differently described herein) allows for local adjustment of the volume level of the received audio content at each set of NF speakers 30 (as opposed to a single global volume level from the FF speaker 20 in the system 10).

[0084] The above description provides implementations that are compatible with the Bluetooth specification version 5.2 [Vol 0] as of December 31, 2019, and any prior versions (e.g., version 4.x and 5.x devices). Additionally, the connection techniques described herein can be used for Bluetooth LE Audio, such as to assist in establishing a unicast connection. Further, it should be understood that the method is equally applicable to other wireless protocols (e.g., non-Bluetooth, future versions of Bluetooth, etc.) where communication channels are selectively established between paired stations. Additionally, although some implementations were described above as not requiring manual intervention to initiate pairing, in some implementations, manual intervention may be required to complete the pairing (e.g., presenting "Are you sure?" to the user of the source / host device), for example, to provide further security aspects to the method.

[0085] In some specific implementations, the host-based elements of the method are implemented in software modules (e.g., "App") that are downloaded and installed on a source / host (e.g., "smartphone") to provide coordinated gap output in accordance with the above method.

[0086] Although the specific order of operations performed by certain specific implementations of the present invention is described above, it should be understood that such an order is illustrative, as alternative implementations may perform these operations in a different order, combine certain operations, overlap certain operations, and so on. References to a given implementation in the specification indicate that the implementation may include a specific feature, structure, or characteristic, but each implementation may not necessarily include that specific feature, structure, or characteristic.

[0087] The functions described herein, or portions thereof, and various modifications thereof (hereinafter referred to as "functions") may be implemented at least in part via a computer program product, such as a computer program tangibly embodied in an information carrier, such as one or more non-transitory machine-readable media, for executing, or controlling the operation of, one or more data processing devices, such as programmable processors, computers, multiple computers, and / or programmable logic components.

[0088] The computer program may be written in any form of programming language, including compiled or interpreted languages, and the computer program may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program may be deployed on one computer or distributed across one site or multiple sites and executed on multiple computers interconnected by a network.

[0089] Actions associated with implementing all or part of the functions may be performed by one or more programmable processors executing one or more computer programs to perform the functions of the calibration process. All or part of the functions may be implemented as dedicated purpose logic circuits, such as FPGAs and / or ASICs (application specific integrated circuits). Processors suitable for executing computer programs include, for example, both general and special purpose microprocessors, and any one or more processors of any type of digital computer. In general, a processor will receive instructions and data from a read-only memory or a random access memory or both. Components of a computer include a processor for executing instructions and one or more memory devices for storing instructions and data.

[0090] In various specific implementations, unless otherwise stated, electronic components described as "coupled" may be linked via conventional hardwired and / or wireless means such that the electronic components can transfer data to each other. Additionally, sub-components within a given component may be considered to be linked via conventional paths, which may not necessarily be shown.

[0091] Multiple specific implementations have been described. However, it should be understood that additional modifications can be made without departing from the scope of the inventive concept described herein, and thus, other embodiments are within the scope of the following claims.

Claims

1. An audio system, the audio system comprising: at least one far - field speaker configured to output a first portion of an audio signal; and a pair of non - obstructive near - field speakers configured to output a second portion of the audio signal synchronously with the output of the first portion of the audio signal, wherein the first portion of the audio signal is different from the second portion of the audio signal.

2. The system according to claim 1, wherein the first portion of the audio signal and the second portion of the audio signal comprise different channels of common content.

3. The system according to claim 1, wherein the second portion of the audio signal is binaurally encoded and, when output synchronously with the output of the first portion of the audio signal, the second portion of the audio signal increases the externalization of the content in the audio signal.

4. The system according to claim 1, wherein the second portion of the audio signal enhances the speech content within the audio signal.

5. The system according to claim 4, wherein the second portion of the audio signal increases the intelligibility of the speech content within the audio signal.

6. The system according to claim 1, wherein the first portion of the audio signal does not comprise speech content and wherein the second portion of the audio signal comprises speech content.

7. The system according to claim 1, wherein the first portion of the audio signal and the second portion of the audio signal comprise different frequency ranges from the audio signal.

8. The system according to claim 7, wherein at least one of the following exists: i) the first portion of the audio signal has a larger frequency range than the frequency range of the second portion of the audio signal, or ii) the second portion of the audio signal excludes frequencies below a predetermined threshold.

9. The system according to claim 1, wherein the pair of non - obstructive near - field speakers are housed in a wearable audio device.

10. The system according to claim 1, wherein a first speaker of the pair of non - obstructive near - field speakers is configured to output audio to a user's left ear and wherein a second speaker of the pair of non - obstructive near - field speakers is configured to output audio to the user's right ear.

11. The system according to claim 1, wherein i) the at least one far - field speaker is housed in a soundbar, or ii) the at least one far - field speaker comprises a plurality of speakers configured to output at least a left channel, a right channel, and a center channel of the audio signal.

12. The system according to claim 1, wherein the at least one far - field speaker is configured to output a third portion of the audio signal when not providing an output synchronous with the pair of non - obstructive near - field speakers, the third portion of the audio signal being different from the first portion of the audio signal.

13. The system according to claim 1, wherein the second portion of the audio signal is user-configurable to enhance at least one of: externalization, spatialization, or dialogue.

14. The system according to claim 1, wherein the second portion of the audio signal is automatically customized based on detection of the type of device housing at least one of the pair of non-blocking near-field speakers.

15. The system according to claim 1, wherein the pair of non-blocking near-field speakers is configured to output the second portion of the audio signal synchronously with the output of the first portion of the audio signal in response to a trigger.

16. The system according to claim 1, the system further comprising a first device housing the at least one far-field speaker, wherein the first device is configured to send the audio signal or the second portion of the audio signal to a second device including the pair of non-blocking near-field speakers, wherein the sending includes playback timing data such that the second portion of the audio signal can be output synchronously with the first portion of the audio signal.

17. The system according to claim 1, wherein the system further comprises: A first device, the first device including the at least one far-field speaker; and a second device, the second device including the pair of non-blocking near-field speakers, wherein the first device and the second device receive the audio signal from a common source device, or the first device receives the first portion of the audio signal from a first source device, and the second device receives the second portion of the audio signal from a second source device.

18. The system according to claim 1, wherein the first device and the second device receive the audio signal from a common source device, wherein the audio signal includes audio content considering surround effects and height effects, and wherein outputting the audio signal at the first device and the second device excites the room housing the first device and the second device, thereby enhancing externalization for the user.

19. A method of controlling an audio system, the method comprising: outputting a first portion of an audio signal to at least one far-field speaker; and outputting a second portion of the audio signal to a pair of non-blocking near-field speakers synchronously with the output of the first portion of the audio signal, wherein the first portion of the audio signal is different from the second portion of the audio signal.

20. The method according to claim 19, wherein the first portion of the audio signal and the second portion of the audio signal include different channels of common content, and wherein the second portion of the audio signal is user-configurable to enhance at least one of: externalization, spatialization, or dialogue.

21. The method according to claim 19, wherein the second portion of the audio signal is binaurally encoded, and wherein when output synchronously with the output of the first portion of the audio signal, the second portion of the audio signal increases the externalization of the content in the audio signal.

22. The method according to claim 19, wherein the second portion of the audio signal enhances the speech content within the audio signal, and wherein the second portion of the audio signal increases the intelligibility of the speech content within the audio signal.

23. The method according to claim 19, wherein the first portion of the audio signal does not include speech content, and wherein the second portion of the audio signal includes speech content.

24. The method according to claim 19, wherein the first portion of the audio signal and the second portion of the audio signal include different frequency ranges of the audio signal, and wherein at least one of the following exists: i) the first portion of the audio signal has a larger frequency range than the frequency range of the second portion of the audio signal, or ii) the second portion of the audio signal excludes frequencies below a predetermined threshold.

Images