System and method for managing in-vehicle audio latency

Abstract

Disclosed herein is an integrated wireless audio unit (WAU) for an in-vehicle audio system that may take multiple audio streams as inputs and output the audio streams to multiple output devices, where the output transmissions have low latency (e.g., end-to-end less than 15 ms) and are time synchronized (e.g., the latency does not float or drift). The input streams may be received via various protocols in the digital or analog domain, and the WAU may mix, fuse, and process the input streams to generate an output stream, and broadcast the output stream to an audio device within the vehicle. The WAU may employ various policies to synchronize different types of audio streams based on correlation factors. Accordingly, audio playback at the vehicle may be performed through closer integration with the vehicle speaker and the haptic transducer, and flexibility of routing the audio stream may be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the subject matter disclosed herein generally relate to audio communication within a vehicle cabin. BACKGROUND

[0002] Communicating in a vehicle while wearing a wearable audio device, such as earbuds or headphones, can be an undesirable experience. Wireless audio is generally slow, with Bluetooth® having a latency of about 30-100 ms, and having highly variable end-to-end latency. Thus, the direct sound (e.g., sound emitted directly from the speaker’s vocal cords) can be heard first, followed by the reproduced speech, creating an unnatural “echo” feel.

[0003] Additionally, wireless transmission relies on low-latency differences in the integration of audio with other system components. For example, haptic transducers or “base shakers” vibrate a seat in accordance with music playing on the seat to enhance the listening experience for an occupant on the seat. However, haptic transducers generally need to be tuned and tightly integrated with the vehicle audio system. Involvement of variable latency can cause an unnatural “lag” feel due to the haptic transducers and wearable audio devices not being in sync. Wi-Fi audio can be too power-hungry and expensive to be a viable alternative. Additionally, when multiple audio sources and audio devices are used simultaneously, the latency at each audio device can be different, and creating a natural listening experience can require individual synchronization of the wireless transmission to each audio device. SUMMARY

[0004] The present disclosure addresses at least one or more of the above concerns, in part, by a vehicle audio system including a wireless audio unit (WAU); a processor; and a non-transitory memory storing instructions that, when executed, cause the processor to: receive an audio input stream at the WAU; process the audio input stream at the WAU to generate a first audio output stream; process the audio input stream at the WAU to generate a second audio output stream, the second audio output stream being different from the first audio output stream; simultaneously transmit the first audio output stream to a first audio output device of the vehicle and the second audio output stream to a second audio output device of the vehicle; estimate a latency of the first audio output stream at the first audio output device relative to the second audio output stream at the second audio output device; and in response to the latency being greater than a threshold latency, insert a delay in the second audio output stream to synchronize the first audio output stream and the second audio output stream.

[0005] The advantages and other advantages and features described herein will become apparent from the following detailed description when considered alone or in conjunction with the accompanying drawings. It should be understood that the above summary is provided to introduce, in a simplified form, a selection of concepts further described in the detailed description. This does not imply confirmation of the key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims appended to the detailed description. Furthermore, the claimed subject matter is not limited to implementations that address any of the disadvantages pointed out above or in any part of this disclosure. Attached Figure Description

[0006] A better understanding of various aspects of this disclosure can be achieved by reading the following detailed description and referring to the accompanying drawings, wherein:

[0007] Figure 1 It is a schematic diagram of a vehicle according to one or more embodiments of this disclosure;

[0008] Figure 2 A block diagram of an exemplary embodiment of an audio system for a vehicle according to one or more embodiments of the present disclosure is shown;

[0009] Figure 3 A block diagram is shown illustrating the data flow through a vehicle's audio system when synchronizing audio streams according to one or more embodiments of the present disclosure;

[0010] Figure 4 This is a flowchart illustrating an advanced method for transmitting multiple time-synchronized audio streams to different output devices of a vehicle according to one or more embodiments of the present disclosure;

[0011] Figure 5 This is a flowchart illustrating a first method for measuring the delay of an audio signal output by a wireless device according to one or more embodiments of the present disclosure;

[0012] Figure 6 This is a flowchart illustrating a second method for measuring the delay of an audio signal output by a wireless device according to one or more embodiments of the present disclosure; and

[0013] Figure 7 This is a block diagram illustrating an exemplary audio device according to one or more embodiments of the present disclosure, capable of detecting and measuring the time delay between different audio streams.

[0014] The accompanying drawings illustrate specific aspects of the described systems and methods. Together with the following description, the drawings demonstrate and explain the structures, methods, and principles described herein. In the drawings, the dimensions of components may be enlarged or otherwise modified for clarity. Well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the described components, systems, and methods. Detailed Implementation

[0015] This document discloses systems and methods for synchronizing audio streams played at different speakers in a vehicle, including wall-mounted, roof-mounted, or seat-integrated speakers, speakers for personal audio devices (such as earbuds or headphones), subwoofers integrated into vehicle seats, haptic transducers integrated into vehicle seats, and / or other types of audio playback devices. When an audio stream is played at a set of wired conventional vehicle speakers, the latency of the audio stream may be imperceptible or easily manageable. However, in the modern vehicle cabin environment, multiple users may be listening to different audio signals or different combinations of audio signals on different speakers and devices, making the latency between signals more noticeable and more difficult to manage.

[0016] For example, since some speakers can be coupled to the vehicle's audio system via wired connections and others via wireless connections, different time delays can be observed and / or measured between different speakers. Furthermore, depending on the wireless communication protocol selected for a given audio signal, different time delays may be observed for different wireless speakers. For example, for the same audio signal, a first speaker wirelessly connected to the audio system via the Bluetooth® protocol may have a first time delay, while a second speaker wirelessly connected via the Ultra Wideband (UWB) protocol may have a second time delay. A third communication protocol may produce a third time delay, and so on.

[0017] Different latency can also depend on the type of audio signal being transmitted. For example, the same communication protocol can cause speech audio signals to play with a first latency, while music can play with a different second latency. Speech latency can occur when a speech signal spoken by the first occupant of the vehicle is received at a microphone in a personal audio device or a microphone integrated into the vehicle seat, roof, etc., and that speech is rebroadcast to other occupants of the vehicle via multiple audio devices. The latency of the rebroadcast speech relative to the direct hearing of the first occupant's voice can produce an unnatural or undesirable listening experience.

[0018] The listening experience of an audio system user can depend on the different audio signals synchronized for each listener. That is, for the first occupant of a vehicle, the playback of a first audio signal may need to be synchronized between the first occupant's first audio device and the first subwoofer; for the second occupant, the playback of a second audio signal may need to be synchronized between the second occupant's second audio device and the second subwoofer; for the third occupant, the playback of a third audio signal may need to be synchronized between the third occupant's third audio device and the third subwoofer; and so on. Different synchronizations can each depend on various factors, such as the seat the respective occupant is sitting in, the type of device the respective occupant is listening to, whether the device is connected to the audio system via a wired or wireless connection, the type of audio signal being played, and so on. Another issue is that some latency can vary, where the measurement of latency can change over time in a predictable or unpredictable way (e.g., drift).

[0019] To address this issue, this paper discloses an integrated wireless audio unit (WAU) for in-vehicle audio systems that can take multiple audio streams as input and output (e.g., broadcast) these audio streams to multiple output devices, wherein the output transmissions have low latency (e.g., less than 15 ms end-to-end) and are time-synchronized (e.g., latency has no fluctuation or drift). The input streams can be received via various protocols in the digital or analog domains, and the WAU can mix, fuse, and process the input streams to generate an output stream, which is then broadcast to audio devices within the vehicle. As described herein, the WAU can employ various strategies based on relevant factors to synchronize different types of audio streams. Therefore, audio playback at the vehicle can be performed through tighter integration with vehicle speakers and haptic transducers, increasing the flexibility of routing audio streams and providing more precise user control to manage and control the audio streams. Specifically, the voice reproduced on the audio devices can be synchronized with the actual voice of the occupants, creating a more natural listening experience for other occupants of the vehicle.

[0020] Now turn to the attached image. Figure 1 An exemplary vehicle 100 is schematically shown. Vehicle 100 includes an instrument panel 102, a driver's seat 104, a first passenger seat 106, a second passenger seat 108, and a third passenger seat 110. In other examples, vehicle 100 may include more or fewer passenger seats. The driver's seat 104 and the first passenger seat 106 are located at the front of the vehicle, close to the instrument panel 102, and can therefore be referred to as front seats. The second passenger seat 108 and the third passenger seat 110 are located at the rear of the vehicle and can be referred to as rear (or rear-seat) seats.

[0021] Vehicle 100 includes a plurality of integrated speakers 114 which may be arranged around the perimeter of vehicle 100 and / or at the seats (e.g., headrests) of vehicle 100. In some embodiments, the integrated speakers 114 are electrically coupled to an electronic control system of the vehicle, such as computing system 120, via wired connections. In other embodiments, the integrated speakers 114 may communicate wirelessly with computing system 120. As an example, an audio file may be generated by computing system 120 or selected by an occupant of vehicle 100, and the selected audio file may be played on one or more of the integrated speakers 114. In some examples, an audio alarm may be generated by computing system 120 and may also be played at the integrated speakers 114. In some embodiments, audio files, signals, and / or alarms may be selected by or generated for the occupants of vehicle 100 and may be played at integrated speakers 114 associated with and / or near the occupant's seat. In addition, in some implementations, audio files, signals and / or alarms selected or generated for the occupants may be played at an integrated speaker 114 associated with and / or near the seats of one or more other occupants of the vehicle 100.

[0022] Vehicle 100 may also include a driver seat sensor 124 coupled to or located within the driver seat 104 and a passenger seat sensor 126 coupled to or located within the first passenger seat 106. The rear seats may also include seat sensors, such as a passenger seat sensor 128 coupled to the second passenger seat 108 and a passenger seat sensor 130 coupled to the third passenger seat 110. Driver seat sensor 124 and passenger seat sensor 126 may each include one or more sensors, such as weight sensors, pressure sensors, and one or more seat position sensors that output measurement signals to computing system 120. For example, computing system 120 may use the output of a weight sensor or pressure sensor to determine whether the corresponding seat is occupied, and if occupied, to determine the weight of the person occupying the seat. As another example, computing system 120 may use the output of one or more seat position sensors to determine one or more of the following: occupant of vehicle 100, seat height, longitudinal position relative to dashboard 102 and rear seats, and the angle (e.g., tilt) of the seat back of the corresponding seat.

[0023] The computing system 120 may include a user interface (UI) 116. The computing system 120 may receive input and output information to the UI 116. For example, the user interface 116 may be included in a digital cockpit and may include a display and one or more input devices. The one or more input devices may include one or more touchscreens, knobs, dials, hard buttons, and soft buttons for receiving user input from vehicle occupants. The UI 116 may include a display screen on which information, images, videos, etc., may be displayed. The display screen may be a touchscreen, and vehicle occupants may interact with the UI 116 via control elements displayed on the display screen.

[0024] The computing system 120 may further include: an audio system 148 configured to play audio content to one or more passengers of the vehicle 100; and an in-vehicle infotainment (IVI) system 115 configured to display video content and / or receive commands for selecting and interacting with the video content on a UI 116. The video content may be displayed in conjunction with the audio content output by the audio system 148. The audio system 148 may include a transmitter that can transmit an audio stream from the audio system 148 to one or more audio devices and / or speakers located within the vehicle, such as speaker 114, as described in more detail herein.

[0025] The computing system 120 may include one or more additional UIs 117, which may be located in a position accessible to rear seat passengers of the vehicle 100. In various embodiments, the one or more additional UIs 117 may include or be constituted by an RSE system. In other words, a first UI 117 may be located behind the front seat 106, allowing a first rear passenger sitting in the rear seat 108 to interact with the first UI 117. A second UI 117 may be located behind the driver's seat 104, allowing a second rear passenger sitting in the rear seat 110 to interact with the second UI 117. In vehicles that include additional rear seats, the additional UIs 117 may be similarly arranged so that each occupant of the vehicle 100 can interact with the UI 117.

[0026] Each additional UI 117 may include the same or similar features included in UI 116, such as physical and / or virtual control elements, one or more displays, etc. Additionally, each additional UI 117 may be configured to receive the same or different video and / or audio content from computing system 120 and display that content independently of other UIs 117. For example, computing system 120 may generate a first set of visual content on a first display associated with a first UI 117; generate a different second set of visual content on a second display associated with a second UI 117; generate a different third set of visual content on a third display associated with a third UI 117; and so on. Similarly, a first occupant of vehicle 100 can interact with a first UI 117 to generate a first set of visual content on a first display screen associated with the first UI 117; a second occupant of vehicle 100 can interact with a second UI 117 to generate a different second set of visual content on a second display screen associated with the second UI 117; a third occupant of vehicle 100 can interact with a third UI 117 to generate a different third set of visual content on a third display screen associated with the third UI 117; and so on. The set of visual content may include associated audio content, which can be played at a corresponding speaker 114.

[0027] Alternatively or concurrently, audio content may be played on the personal audio devices of the occupants of vehicle 100, such as personal audio device 150 for the occupants of front passenger seat 106; personal audio device 151 for the occupants of rear seat 108; and personal audio device 152 for the occupants of rear seat 110. Personal audio devices 150-152 may include, for example, earphones, wired or wireless headphones, or other audio devices. Furthermore, personal audio devices 150-152 may include corresponding microphones configured to receive voice and / or speech data from the respective occupants. That is, personal audio device 150 may be configured to receive voice data from the front passenger occupying seat 106; personal audio device 151 may be configured to receive voice data from the first rear passenger occupying seat 108; and personal audio device 152 may be configured to receive voice data from the second rear passenger occupying seat 110. Using appropriate microphones, front-row passengers, first rear-row passengers, and second rear-row passengers can communicate with each other, wherein the voice generated by the first passenger can be received at the second passenger's personal audio device 150-152.

[0028] The computing system 120 includes a processor 142 configured to execute machine-readable instructions stored in memory 144. The processor 142 may be single-core or multi-core, and the program executed by the processor 142 may be configured for parallel or distributed processing. In some embodiments, the processor 142 is a microcontroller. The processor 142 may optionally include individual components distributed across two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 142 may be virtualized and executed by a remotely accessible, networked computing device configured in a cloud computing configuration. For example, the computing system 120 may be communicatively coupled to a wireless network.

[0029] The computing system 120 can communicate with networked computing devices via short-range communication protocols such as Bluetooth®. In some embodiments, the computing system 120 may include other electronic components capable of implementing processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphics board. In some embodiments, the processor 142 may include multiple electronic components capable of implementing processing functions. For example, the processor 142 may include two or more electronic components selected from a plurality of possible electronic components, including a central processing unit, a digital signal processor, a field-programmable gate array, and a graphics board. In yet another embodiment, the processor 142 may be configured as a graphics processing unit (GPU), including a parallel computing architecture and parallel processing capabilities.

[0030] Furthermore, memory 144 may include any non-transitory tangible computer-readable medium in which programming instructions are stored. As used herein, the term "tangible computer-readable medium" is explicitly defined to include any type of computer-readable storage device. The example methods described herein can be implemented using encoded instructions (e.g., computer-readable instructions) stored on a non-transitory computer-readable medium such as flash memory, read-only memory (ROM), random access memory (RAM), cache, or any other storage medium in which information is stored for any duration (e.g., a prolonged period of time, permanently, transiently, or while information is temporarily buffered and / or cached).

[0031] Computer memory, as referenced herein, may include volatile and non-volatile or removable and non-removable media for storing information in electronic formats such as computer-readable program instructions or computer-readable program instruction modules, data, etc., which may be standalone or part of a computing device. Examples of computer memory may include any other media that can be used to store information in a desired electronic format and can be accessed by one or more processors or at least a portion of a computing device. In various embodiments, memory 144 may include an SD memory card, an internal and / or external hard disk, a USB storage device, or similar modular memory.

[0032] Audio system 148 can be configured to play audio content to one or more of speakers 114 and / or personal audio devices 150-152. Audio system 148 can provide an in-vehicle audio experience where each user of the vehicle has their own sound zone. Passengers can use personal audio devices 150-152 and / or built-in speakers 114 to listen to audio content in their respective sound zones. Built-in speakers may include subwoofers. For example, front-seat passengers of vehicle 100 can listen to music via personal audio device 150 while simultaneously listening to the output of a subwoofer integrated into speakers 114 in seat 106. Each user can listen to their own content (e.g., music played via a personal device such as a smartphone). Each user can also choose to join or leave communication channels with other users in the vehicle.

[0033] Each user's voice area can have a dedicated microphone 180, which is configured as part of the vehicle, such as being mounted in the rear-facing side of the roof or headrest. For the communication channel, users who have opted into the communication channel can automatically hear what other users in the vehicle are saying (e.g., received via the corresponding microphone 180), and can pause or play audio content played via their wearable devices at a lower volume during the transmission of spoken content. The driver can increase control over the communication channel to issue announcements to all passengers, even if one or more passengers have opted out of the communication channel.

[0034] Each seat in vehicle 100 may also include a wireless reverse channel speaker 176. Each wireless reverse channel speaker 176 may receive and output a first audio output signal from audio system 148, which may be different from a second audio output signal transmitted from audio system 148 to the corresponding personal audio devices 150, 151, and 152 of the passenger. The first audio output signal transmitted to the wireless reverse channel speaker 176 may be or include a time delay test audio signal, which may be simultaneously output to speaker 114. The time delay between the time delay test audio signals output by wireless reverse channel speakers 176 and speaker 114 may be measured, and the measured time delay may be used to estimate the time delay between the audio output signal transmitted from audio system 148 to the corresponding personal audio device 150, 151, or 152 and the audio output signal transmitted from audio system 148 to speaker 114, so as to synchronize the audio output signal transmitted from audio system 148 to the corresponding personal audio device 150, 151, or 152 and the audio output signal transmitted from audio system 148 to speaker 114. The following will refer to Figure 5 and Figure 6 A more detailed description of the time delay measurement using the wireless reverse channel speaker 176.

[0035] Additional features of the audio system 148 include transmitting external vehicle sounds to one or more users, transmitting vehicle sounds (e.g., engine speed, signal flashing, infotainment notifications, etc.) to one or more users, and manually sending audio content from one user to another via a user interface. See below for reference. Figure 2 The configuration of the audio system 148 is described in more detail.

[0036] In some examples, computing system 120 may include multiple subsystems or module tasks that perform specific functions related to image acquisition and analysis. As used herein, the terms "system," "unit," or "module" may include hardware and / or software systems for performing one or more functions. For example, a module, unit, or system may include a computer processor, controller, or other logic-based means that performs operations based on instructions stored on a tangible and non-transitory computer-readable storage medium, such as computer memory. Alternatively, a module, unit, or system may include a hardwired means that performs operations based on hardwired logic of the means. The various modules or units shown in the figures may represent hardware that operates based on software or hardwired instructions, software that instructs the hardware to perform operations, or a combination thereof.

[0037] Now for reference Figure 2 A schematic diagram of an exemplary audio system 200 for a vehicle is shown, which may be referenced above. Figure 1The audio system 148 described in vehicle 100 is a non-limiting example. Audio system 200 includes a wireless audio unit (WAU) 202 communicatively coupled to one or more Bluetooth® interfaces 203 of one or more personal devices 204 (e.g., smartphones). WAU 202 is also communicatively coupled to one or more UWB interfaces 205 of one or more personal audio devices 206 (e.g., personal audio devices 150, 151, and 152) (such as headphones, earbuds, etc.). Although... Figure 1 Personal device 204 and personal audio device 206 are shown connected in series, but it should be understood that each device can be individually communicatively coupled to WAU 202.

[0038] The WAU 202 is also coupled to a smart amplifier 210. In addition to amplification, the smart amplifier 210 may also have digital signal processing (DSP) capabilities, which allows it to process audio signals, determine and adjust time delays, and / or add effects that can improve or mask the perceived time delay. The smart amplifier 210 may interface with multiple audio output devices (e.g., speakers 114), which may include a first plurality of headrest (seat) speakers 212, a second plurality of cabin subwoofers or subwoofers 214, and a third plurality of haptic transducers 216. The smart amplifier 210 also interfaces via an audio bus with one or more microphones 222 (e.g., microphone 180). The smart amplifier 210 is also coupled to a domain controller 220, which may also be referred to herein as a host unit. The domain controller 220 may be located on the vehicle's dashboard.

[0039] In some implementations, the domain controller 220 may interface with one or more microphones 222. In such implementations, the smart amplifier 210 may not interface with one or more microphones 222. The advantage of having one or more microphones interface with the domain controller 220 (instead of the smart amplifier 210) is that it is easier to facilitate separate communication channels between the speaker 212, the woofer 214, and the haptic transducer 216 and the microphone 222.

[0040] The interfaces of the smart amplifier 210 coupled to speakers 212, 214, and 216, and the interface of the smart amplifier 210 coupled to microphone 222, are separate interfaces. The WAU 202 can be configured to establish a UWB channel, which each of the personal audio devices 206 can optionally join (e.g., via a user selection in an app presented on the connected personal device 204, such as a smartphone). Audio can be transmitted from the WAU 202 to each of the selected headphones via the UWB channel through the first interface. Audio can be transmitted individually from one or more microphones 222 to the WAU 202 via the second interface. The personal audio devices 206 can be configured without built-in microphones, so that audio is not transmitted from the personal audio devices 206 to the WAU 202. Furthermore, each of the selected headphones is not part of the same unit as one or more of the microphones 222, nor is it coupled to one or more of the microphones 222. Therefore, audio transmission and communication in the audio system 200 are unidirectional.

[0041] Users of the audio system 200 (such as drivers of vehicles in which the audio system 200 is installed) can control the audio system 200 via the IVI screen 230. The IVI screen 230 can be... Figure 1 A non-limiting example of UI 116. Alternatively, a user can control the audio system 200 via a companion application (app) installed on a personal computing device 234, such as a driver's smartphone coupled to the domain controller 220 via a wireless or wired connection. The personal computing device 234 may be connected to the cloud 240. Thus, in some embodiments, one or more components of the audio system 200 may be connected to one or more cloud-based services hosted on the cloud 240 via the companion app and the personal computing device 234.

[0042] Figure 3 A time delay compensation diagram 300 is shown, illustrating the time delay compensation process for vehicles (such as...) Figure 1 During the playback of audio content by the occupants of vehicle 100, data flows through the audio system (such as audio system 200). Specifically, Figure 3 This demonstrates how the latency issues involved in outputting different audio streams to different target devices used by occupants can be overcome according to the systems and methods described herein. The audio system includes various components mounted on an in-vehicle computing device (such as computing system 120). These various components may include a WAU, intelligent amplifiers, and domain controllers, as referenced above. Figure 2 The WAU202, smart amplifier 210, and domain controller 220 are described.

[0043] Audio content output by the audio system and directed to the occupants may be displayed at one or more of various audio sources. The audio content may include media or music 304; communications from other occupants of the vehicle, such as 305; navigation, such as directions or alarms, 306; and / or audio content from one or more other audio sources 307.

[0044] Audio content can be mixed at mixer 308, which can select and adjust audio content from various audio sources according to setting preferences. After being mixed, the audio content can be sent to separate equalizers and tuners for different target devices. For example, the audio content may include music, wherein a first portion of the audio signal of the music is transmitted to a first equalizer and tuner 310 for wireless transmission to a user's personal audio device 320 (e.g., personal audio devices 150, 151, 152), such as a set of headphones. Simultaneously, a second portion of the audio signal of the music can be transmitted to a second equalizer and tuner 312 for wired transmission to a subwoofer 322 integrated into the user's seat. For example, the first portion may include audio data above a threshold frequency, while the second portion may include audio data below a threshold frequency.

[0045] A first amount of processing can be performed on a first portion of the audio signal at the first equalization and tuning block 310. A second amount of processing can be performed on a second portion of the audio signal at the second equalization and tuning block 312, wherein the second amount is different from the first amount. The processing performed at the first equalization and tuning block 310 and the second equalization and tuning block 312 can be specific to the corresponding output device (e.g., a subwoofer and a personal audio device). In some examples, the processing performed at the first equalization and tuning block 310 and the second equalization and tuning block 312 can be performed by an amplifier (such as...) Figure 2 The amplifier 210) performs the processing. In other examples, the processing performed at the first equalization and tuning module 310 and at the second equalization and tuning module 312 may be performed by a WAU (such as WAU 202).

[0046] A first portion and a second portion of an audio signal can be played simultaneously on a personal audio device 320 and a subwoofer 322, respectively. However, the first portion of the audio signal can be played at the personal audio device 320 with a first time delay, and the second portion of the audio signal can be played at the subwoofer 322 with a second time delay, wherein the first time delay may differ from the second time delay. In other words, since the subwoofer 322 can be connected to the audio system via a wired connection 323, and the personal audio device 320 can be connected to the audio system via a wireless connection 321, the first time delay may be longer than the second time delay. To synchronize the first and second portions of the audio signal, the time delay difference 330 between the first and second time delays can be measured (or estimated). Then, a time delay compensation 332 can be calculated, which can be used at the equalizer and tuning block 312 to synchronize the first and second portions of the audio signal. In other words, portion 350 of the time delay compensation diagram 300 represents a control loop consisting of time delay detection and time delay adjustment. The latency is compensated by inserting additional delays into lower latency paths (e.g., the second equalizer and tuning block 312 and the subwoofer 322). This can be achieved through intelligent amplifiers (such as...) Figure 2 The intelligent amplifier 210 inserts the delay into the lower delay path.

[0047] It should be understood that Figure 3 A single time delay difference 330 is depicted between audio signals played at two different playback devices. In other examples, various groups of audio signals can be played on various playback devices, and time delay differences between multiple speakers can be observed, with each time delay difference being compensated for.

[0048] Figure 4 This illustrates a method for synchronizing multiple time-synchronized audio streams from an audio system (such as...). Figure 2 The audio system 200 transmits the audio to the vehicle (such as...) Figure 1 Advanced method 400 for different output devices of a vehicle 100. Method 400 can be executed by a processor (such as processor 142) of the vehicle's computing system based on instructions stored in the vehicle's memory (such as memory 144). For example, method 400 can be executed by... Figure 2 The amplifier 210 is executed by the processor.

[0049] Method 400 begins at 402, wherein method 400 includes receiving an audio input stream. The audio input stream can be received from one or more of a variety of sources, such as a vehicle's audio system configured to output music or other entertainment, news, or media content, communications or notifications issued by the vehicle's controller or navigation system, or other sources. The audio input stream can be selected by the vehicle's IVI system (e.g., IVI system 115), accessible by the vehicle's user, for example, via controls on the vehicle's dashboard (e.g., UI 116) or controls integrated into the vehicle's seats or cabin (e.g., one or more additional UIs 117).

[0050] The audio input stream may include the voice of a vehicle occupant captured via a vehicle microphone (e.g., microphone 180). For example, a first occupant of the vehicle may wish to communicate with a second occupant of the vehicle who is listening to audio content via headphones. The first occupant may choose to join a communication channel of the audio system. The audio system may prompt the second occupant to join the communication channel. If the second occupant chooses to join the communication channel, the audio system may output an audio signal including the first occupant's voice to the second occupant's headphones via a WAU. The second occupant may respond to the first occupant's voice via a corresponding microphone within the second occupant's sound area. The voices of the first and second occupants may not be directed to the audio output device of a third occupant of the vehicle who has not chosen to join the communication channel. Alternatively, in some embodiments, the first occupant's voice output may be directed directly to the second occupant's headphones without receiving a prompt from the first occupant or providing an input to receive an audio signal. For example, a parent of a child listening to music may wish to speak to their child.

[0051] At 404, method 400 includes determining one or more target audio output devices to which an audio input stream is directed. The one or more target audio output devices may include personal audio devices for one or more passengers of the vehicle, haptic speakers or subwoofers of the vehicle that may be integrated into the vehicle's seats, one or more speakers integrated into the seats or other parts of the vehicle cabin, etc. The audio input stream can be configured (e.g., via...) Figure 3 The mixer 308 has settings preferences for output to one or more target audio output devices.

[0052] An audio input stream can be one of multiple audio input streams simultaneously output to multiple passengers in the vehicle. For example, the vehicle driver may wish to listen to a first audio content, such as a first music selection. The driver can select the first audio content via the vehicle's IVI system's dashboard UI. The first audio content can be output from one or more speakers in the vehicle.

[0053] A first passenger seated in the front passenger seat of the vehicle (e.g., seat 106) may wish to listen to second audio content, such as a second music selection. The first passenger can select the second audio content via the instrument panel UI of the vehicle's IVI system. The first passenger can choose to listen to the second audio content through headphones via the UI. For example, the IVI system may prompt the first passenger to select an audio output device from a list of candidate audio output devices displayed in the instrument panel UI, and the first passenger can select a set of headphones from that list, through which the second audio content can be output. The list of candidate audio output devices may include multiple devices detected by the vehicle's IVI system, audio system, or computing system. For example, the list of candidate audio output devices may include devices paired with the vehicle via Bluetooth®.

[0054] A second passenger sitting in the first rear seat of the vehicle (e.g., seat 108) may also wish to listen to second audio content. The second passenger can select the second audio content via a second UI of an IVI system integrated into the seat in front of the second passenger (e.g., seat 106). The IVI system can display a list of candidate audio output devices in the second UI and prompt the second passenger to select one from the list. The second passenger can select a first set of earbuds from the list, and the second audio content can be output to those earbuds.

[0055] A second passenger sitting in the first rear seat of the vehicle (e.g., seat 110) may also wish to listen to second audio content. The second passenger can select the second audio content via a second UI of an IVI system integrated into the seat in front of the second passenger (e.g., seat 104). The IVI system can display a list of candidate audio output devices in a third UI and prompt the third passenger to select an audio output device from the list. The third passenger can select a second set of earphones from the list, and the third audio content can be output to the second set of earphones.

[0056] When the first audio content is output to the driver at the vehicle's speakers, the first audio content can also be output at a first set of haptic transducers located in the driver's seat (e.g., seat 104), which can vibrate the seat according to the first music selection. Therefore, two audio output streams can be generated from the audio input stream selected by the driver: a first audio output stream to one or more speakers in the vehicle, and a second audio output stream to the first set of haptic transducers. Since the one or more speakers and the first set of haptic transducers are connected to the audio system via wired connections, the first and second audio output streams can be output with similarly low latency.

[0057] Similarly, when the second audio content is output to the first passenger's headphones and the second passenger's earpieces, the second audio content can also be output to a second set of haptic transducers positioned on the first passenger's seat and a third set of haptic transducers positioned on the second passenger's seat. These transducers can vibrate the corresponding seats according to the second music selection. Therefore, four audio output streams can be generated from the audio input stream selected by the first passenger: a third audio output stream to the first passenger's headphones, a fourth audio output stream to the second set of haptic transducers, a fifth audio output stream to the second passenger's earpieces, and a sixth audio output stream to the third set of haptic transducers. The third and fourth audio output streams may be referred to herein as paired output streams, meaning output streams associated with the same audio input stream. The fifth and sixth audio output streams are also considered paired output streams.

[0058] However, unlike the driver's first and second audio output streams, the third and fourth audio output streams may be output with different detectable delays because the headphones are connected to the audio system wirelessly and the second set of haptic transducers are connected to the audio system wiredly. Similarly, the fifth and sixth audio output streams may be output with different delays because the earbuds are connected to the audio system wirelessly and the third set of haptic transducers are connected to the audio system wiredly. That is, the third audio output stream to the headphones may be delayed relative to the fourth audio output stream to the second set of haptic transducers according to a first delay difference, and the fifth audio output stream to the earbuds may be delayed relative to the sixth audio output stream to the third set of haptic transducers according to a second delay difference. Furthermore, the first delay difference may be smaller than the second delay difference, wherein the first delay of the third audio output stream perceived by the first passenger may be smaller than the second delay of the fifth audio output stream perceived by the second passenger because the second passenger's earbuds are farther from the transmitter of the audio system than the first passenger's headphones. Therefore, each of the audio output streams can be configured individually and differently to reduce perceived latency, as described below in steps 406 to 422.

[0059] At 406, method 400 includes, for each output device, selecting a corresponding transmission method for the respective audio output stream to be directed to the output device. The transmission method may depend on the connection between the respective output device and the audio system. If the connection is wired, a first transmission method may be indicated, such as via an analog line signal or a PDM signal transmitted over a shielded cable. Alternatively, if the connection is wireless, a codec and a communication protocol may be selected. For example, the communication protocol used for a wireless connection may be a UWB communication protocol. By selecting a UWB communication protocol, the latency difference between audio output streams transmitted via wired and wireless connections can be reduced.

[0060] The codec and / or communication protocol selected for the output device may depend on the type of audio content transmitted in the audio output stream. For example, if the audio content includes music, a first codec and a first communication protocol (e.g., Bluetooth®) may be selected; if the audio content includes speech, a second codec and a second communication protocol (e.g., UWB) may be selected. In other words, the UWB codec and protocol can provide low latency but not music fidelity; therefore, UWB is a technology suitable for voice communication. On the other hand, the Bluetooth® codec and protocol increases latency at (high) fidelity in music and may therefore be more suitable for music but not for voice communication.

[0061] At 408, method 400 includes embedding a time-delay test audio signal into the audio output stream transmitted to each of the audio output devices. In various embodiments, the time-delay test audio signal may be an audio signal outside the audible frequency range, where a listener of the audio output stream may not be able to hear or perceive the time-delay test audio signal. The time-delay test audio signal may include, for example, a characteristic pattern repeated at regular intervals, which can facilitate comparison of a first audio output stream and a second audio output stream. The specific properties of the test signal may depend on the requirements of the detection algorithm. For example, autocorrelation methods work best when the test signal can provide a high correlation when the two signals are perfectly matched.

[0062] At 410, method 400 includes transmitting an audio output stream to a corresponding output device using the selected codec and communication protocol selected at 406. Additionally, at 412, method 400 includes transmitting a delay test audio signal via a wired speaker of the vehicle, such that the delay test audio signal can be captured via one or more microphones of the vehicle (e.g., microphone 180). The captured delay test audio signal can be used to determine a baseline delay of the wired transmission of the delay test audio signal, which can be compared with a variable delay of the wireless transmission of the delay test audio signal, as described below.

[0063] At 414, method 400 includes determining whether one or more of the target audio output devices are wireless devices. If it is determined at 414 that no target audio output device is wireless, meaning that the target audio output device has a wired connection to the audio system, it can be inferred that the time delay difference between the first audio output stream at the relevant target audio output device and the second audio output stream of the same content at a different wired speaker (e.g., a haptic transducer) is negligible, and method 400 proceeds to 422. At 422, method 400 includes continuing to transmit the audio output stream to the target audio output device, and method 400 terminates.

[0064] Alternatively, if one or more target audio output devices are determined to be wireless devices at 414, the time delay difference between the audio output streams played at the relevant target audio output devices and the wired speakers of the vehicle can be quite large, thus method 400 proceeds to 416.

[0065] At 416, for each detected wireless target audio output device, method 400 includes estimating the delay of the audio output stream at the target audio output device relative to the paired audio output stream. Technically, the delay can be the difference between a first delay of the (high-latency) audio output stream transmitted via the wireless connection and a second (negligible) delay of the paired (low-latency) audio output stream transmitted via the wired connection. Estimating the delay of the audio stream can be performed in various ways; see below for reference. Figure 5 and Figure 6 Two exemplary methods for estimating time delay are described.

[0066] Once the latency of the audio output stream has been estimated, at 418, method 400 includes determining whether the estimated latency at the target audio output device is greater than a threshold latency. The threshold latency can be a latency that is detectable to a listener or is considered to disorient the listener, wherein if the latency is less than the threshold latency, the listener may not detect the latency or be bothered by it, and if the latency is greater than the threshold latency, the listener may perceive the latency and / or be bothered by it. For example, the threshold latency could be 15 ms. If the estimated latency is greater than the threshold latency, method 400 proceeds to 420.

[0067] At 420, method 400 includes compensating for the estimated latency by adding a delay to the audio output stream to the wireless target audio output device (e.g., a high-latency audio output stream), as described above regarding... Figure 3 The delay compensation 332 describes a delay based on an estimated delay. By adding a delay, the delay difference between high-latency (wireless) audio streams and low-latency (wired) audio streams can be reduced.

[0068] In various implementations, delay compensation can be performed iteratively as a control loop, where after compensating for the delay, method 400 returns to 418 for the next iteration. That is, a first delay can be inserted into the audio output stream based on a first estimated delay; a second delay can be estimated and inserted into the audio output stream based on the second estimated delay; a third delay can be estimated and inserted into the audio output stream based on the third estimated delay; and so on, until the estimated delay is below a threshold delay. If the delay changes over time, iterative delay compensation can also keep the estimated delay below the threshold delay.

[0069] Figure 5A first method 500 is shown for estimating the difference in time delay between a first audio output stream output at a first audio output device and a second audio output stream output at a second audio output device, wherein the first and second audio output streams are simultaneously transmitted from a vehicle's audio system based on a selected audio input stream. Method 500 can be used as... Figure 4 A portion of method 400 is executed by a processor of the vehicle's computing system (such as processor 142 of vehicle 100) based on instructions stored in the vehicle's memory (such as memory 144). In various examples, the first audio output device may be a wireless device, such as a set of headphones or earpieces for a vehicle occupant, while the second audio output device may be a wired device, such as a set of haptic transducers integrated into the occupant's seat. The first audio output stream and the second audio output stream can be transmitted simultaneously to the first audio output device and the second audio output device, respectively. However, due to the time delay difference between wired and wireless connections, the first audio output stream may be perceived as delayed relative to the second audio output stream by the listener.

[0070] Method 500 begins at 502, wherein method 500 includes transmitting a first audio output stream to a wireless reverse channel speaker positioned at the occupant's location. The first audio output stream can be transmitted to the wireless reverse channel speaker while a first audio output stream is transmitted to a first audio output device and a second audio output stream is transmitted to a second audio output device. In various embodiments, the wireless reverse channel speaker can be concealed within the occupant's seat and / or integrated into the passenger's seat, such as... Figure 1 The wireless reverse channel speaker 176 can receive and output a first output audio signal. Due to the similarity between the wireless reverse channel speaker and the first audio output device (e.g., headphones, earphones, etc.), and because the first distance between the wireless reverse channel speaker and the transmitter of the audio system can be approximately equal to the second distance between the first audio output device and the transmitter, the first time delay of the first output audio stream at the wireless reverse channel speaker can be equal to the second time delay of the first output audio stream at the first audio output device relative to the second output audio stream at the second (wired) audio output device.

[0071] At 504, method 500 includes measuring the time delay between a first time-delay test audio signal embedded in the first audio output stream and the same second time-delay test audio signal output at a wired speaker in the vehicle, as described at step 412 of method 400. The first and second time-delay test audio signals may be in a frequency range inaudible to human listeners, so that vehicle occupants cannot detect them and will not interfere with the listening experience of the respective audio streams. The time delay of the first time-delay test audio signal relative to the second time-delay test audio signal can be measured at a wireless reverse channel speaker (e.g., referencing...). Figure 3 Described as a time delay difference 330), the wireless reverse channel speaker can compare a first time delay test audio signal output by the wireless reverse channel speaker with a second time delay test audio signal output by a wired speaker. In some examples, the wired speaker can be positioned at the location of the wireless reverse channel speaker, with multiple wired speakers positioned next to corresponding multiple wireless reverse channel speakers on each seat of the vehicle. In other examples, the second time delay test audio signal can be output by a single wired speaker of the vehicle and can be captured by a microphone (e.g., microphone 180) positioned at the corresponding seat of the vehicle, and the captured second time delay test audio signal can be compared with the first time delay test audio signal. The time delay or offset between the signals can then be determined by statistical methods (such as autocorrelation), machine learning (ML) methods, or other methods (such as those provided by artificial intelligence (AI) methods).

[0072] In some examples, the WAU of the audio system (e.g., Figure 2 The delay is measured at the WAU 202. In other examples, the delay can be measured at the associated wireless reverse channel speaker. For example, the wireless reverse channel speaker can be a device including a processor and memory, configured to compare a first delay test audio signal and a second delay test audio signal and output the delay.

[0073] In some examples, a delay test audio signal may not be used, and a third audio output stream identical to the first audio output stream may be transmitted to a wired speaker at the location of the wireless reverse channel speaker. The output of the third audio output stream at the wired speaker may be recorded and compared with the first audio output stream to determine the delay from the first audio output stream to the third audio output stream. However, by embedding the delay test audio signal into the first audio output stream and comparing the embedded delay test audio signal with the same delay test audio signal output by the wired speaker, the amount of bandwidth consumed in wireless transmission to the occupants of the vehicle can be advantageously reduced.

[0074] At 506, method 500 includes outputting a measured delay of a first delay test audio signal relative to a second delay test audio signal to calculate a delay compensation (e.g., delay compensation 332) to be applied to the second audio output stream, as described in method 400. In other words, the measured delay can be considered to be nearly identical to the estimated delay of the first audio output stream relative to the second audio output stream; therefore, the measured delay can be used to estimate the delay of the first audio output stream. Method 500 ends.

[0075] Figure 6 A second method 600 is shown for estimating the difference in time delay between a first audio output stream output at a first audio output device and a second audio output stream output at a second audio output device, wherein the first and second audio output streams are simultaneously transmitted from the vehicle's audio system based on an audio input stream selected as described above in method 500. Method 600 can also be used as... Figure 4 Method 400 is executed as a part of it. However, method 600 can be executed by a processor integrated into the personal wireless audio device based on instructions stored in the memory of the personal wireless audio device.

[0076] Brief Reference Figure 7 An exemplary wireless audio device 700 capable of performing method 600 is shown. The wireless audio device 700 includes a processor 704, a memory 706, a transceiver 708, a set of speakers 710, a microphone 712, and a time delay measurement component 714. The wireless audio device 700 is wirelessly connected to a vehicle's audio system 720, such as... Figure 1 The vehicle 100 has an audio system 200. A user 750 can listen to an audio stream received from the audio system 720 at a wireless audio device 700 via a speaker 710. For example, the speaker 710 can be headphones, and the wireless audio device 700 can be worn on the head of the user 750. The wireless audio device 700 can be similar to commercially available wireless headphones, earbuds, etc. However, the wireless audio device 700 can be configured to measure the time delay of a first audio stream wirelessly received at a transceiver 708 from the audio system 720 (via a wireless network 724) relative to a second audio stream received at a microphone 712 via a wired speaker 722 coupled to the audio system 720.

[0077] In other words, the audio system 720 can simultaneously transmit audio content via the wireless network 724 and the wired speaker 722. A first output stream of audio content wirelessly transmitted by the audio system 720 can be received by the transceiver 708, and a second output stream of audio content transmitted via the wired speaker 722 can be captured by the microphone 712. However, due to the lower data transmission rate of the wireless connection between the wireless audio device 700 and the audio system 720 compared to the wired speaker 722, and / or the distance between the wireless audio device 700 and the audio system 720, the first output stream may have a delay relative to the second output stream. To compensate for the delay, the delay measurement unit 714 can compare the first output stream with the second output stream and measure the delay of the first output stream relative to the second output stream. The measured delay can then be transmitted back to the audio system 720 via the transceiver 708. Then, the audio system 720 can compensate for the delay by introducing a delay in the second output stream, so that the first audio stream received at the transceiver 708 is time-synchronized with the second audio stream received at the microphone 712 via the speaker 722.

[0078] Returning to method 600, which begins at 602, method 600 includes receiving a first audio output stream at the wireless audio device via a transceiver (e.g., transceiver 708). At 604, a delay test audio signal embedded in the first audio output stream can be extracted from the first audio output stream, as referenced above. Figure 5 As described.

[0079] At 606, method 600 includes receiving a second time-delay test audio signal via a microphone (e.g., microphone 712) of a wireless audio device. The second time-delay test audio signal may be transmitted via a wired speaker of the vehicle, as described in step 412 of method 400.

[0080] At 608, method 600 includes measuring the time delay between a first time-delay test audio signal embedded in a first audio output stream and the same second time-delay test audio signal received at a microphone. The first and second time-delay test audio signals may be within a frequency range inaudible to human listeners, thereby making them undetectable to vehicle occupants and not interfering with the listening experience of the user of the wireless audio device. The time delay of the first time-delay test audio signal relative to the second time-delay test audio signal can be measured at the wireless audio device (e.g., at time-delay measurement component 714) (e.g., by reference to...). Figure 3 The described time delay difference is 330.

[0081] At 610, method 600 includes transmitting a measured delay of a first delay test audio signal relative to a second delay test audio signal to an audio system to calculate a delay compensation (e.g., delay compensation 332) to be applied to a second audio output stream, as described in method 400, and method 600 ends.

[0082] Therefore, methods 600 and 700 represent two different methods for measuring the time delay between a first audio output stream transmitted from the vehicle's audio system to a wireless device via a wireless network (e.g., using a communication protocol such as Bluetooth® or UWB) and the same second audio output stream transmitted to a speaker or audio output device coupled to the audio system via a wired connection. For a suitably configured wireless device (such as one associated with the audio system), the time delay can be measured at the device by advantageously using a microphone of a custom wireless audio device to capture the second audio output stream and then comparing it to the first audio output stream. Alternatively, a wireless reverse-channel audio output device can be positioned approximately at the location of the vehicle occupant's audio device, and the output of the wireless reverse-channel audio output device can be used as a proxy for the audio device to determine the time delay. In other words, the measured time delay can be used to estimate the time delay of the first audio output stream relative to the second audio output stream, which can be identical or nearly identical. The estimated time delay can then be used to introduce a delay into the second audio output stream to synchronize the second and first audio output streams.

[0083] The method can be executed simultaneously for multiple occupants in a vehicle, each located at a different position within the vehicle and potentially experiencing different latency levels than other occupants. In this way, audio playback at different sets of audio output devices can be synchronized individually for each occupant, in a synchronization manner that may not be detected by the occupant and does not interfere with the listening experience of other occupants in the vehicle. Furthermore, to reduce the amount of information transmitted wirelessly within the vehicle and conserve wireless communication bandwidth, low-data test signals embedded in the audio streams output from various audio output devices can be used for latency measurement. The low-data test signals may include characteristic patterns that repeat at regular intervals, which can facilitate comparison between the first and second audio output streams, allowing for faster and less computationally intensive latency measurement compared to measuring the latency between more complex audio streams (such as music).

[0084] The advantage of estimating the delay between a first audio output stream wirelessly transmitted to a personal audio device in a vehicle and a second audio output stream transmitted via a wired audio output device in the vehicle by measuring the delay between a first test signal embedded in a first audio output stream and a second test signal output in a second audio output stream is that it can reduce the amount of computation used to measure the delay and can reduce the amount of bandwidth consumed in measuring the delay.

[0085] This disclosure also provides support for a vehicle audio system including a wireless audio unit (WAU); a processor; and a non-transitory memory storing instructions that, when executed, cause the processor to: receive an audio input stream at the WAU; process the audio input stream at the WAU to generate a first audio output stream; process the audio input stream at the WAU to generate a second audio output stream, the second audio output stream being different from the first audio output stream; simultaneously transmit the first audio output stream to a first audio output device of the vehicle and transmit the second audio output stream to a second audio output device of the vehicle; estimate a time delay of the first audio output stream at the first audio output device relative to the second audio output stream at the second audio output device; and, in response to the time delay being greater than a threshold time delay, insert a delay into the second audio output stream to synchronize the first audio output stream and the second audio output stream. In a first example of the system, the first audio output device is a personal audio device of a vehicle occupant, which receives the first audio output stream via a wireless connection, and the second audio output device is coupled to the audio system via a wired connection. In a second example of the system (optionally including the first example), further instructions are stored in memory that, when executed, cause the processor to generate a first audio output stream using a first communication protocol and to generate a second audio output stream using a second communication protocol, which is different from the first communication protocol. In a third example of the system (optionally including one or both of the first and second examples), the first communication protocol is an ultra-wideband (UWB) protocol. In a fourth example of the system (optionally including one or more of the first to third examples), further instructions are stored in memory that, when executed, cause the processor to: detect the type of audio content included in the first audio output stream; select the UWB protocol as the first communication protocol in response to the type of audio content including a threshold amount of speech; and select the Bluetooth® protocol as the first communication protocol in response to the type of audio content including a threshold amount of music. In a fifth example of the system (optionally including one or more of the first to fourth examples), the personal audio device is one of earbuds or headphones, and the second audio output device is integrated into the occupant's seat and includes one of a haptic transducer and a subwoofer.In a sixth example of the system (optionally including one or more of the first to fifth examples), further instructions are stored in memory that, when executed, cause the processor to: embed a delay test audio signal into a first audio output stream; simultaneously transmit the first audio output stream along with a second audio output stream to a wireless reverse channel speaker positioned near the personal audio device; simultaneously transmit the delay test audio signal along with the first and second audio output streams to a wired speaker in the vehicle, the wired speaker being coupled to the audio system via a wired connection; and measure the delay of the delay test audio signal at the personal audio device relative to the delay test audio signal at the wired speaker. In a seventh example of the system (optionally including one or more of the first to sixth examples), the wireless reverse channel speaker is integrated into the occupant's seat. In an eighth example of the system (optionally including one or more of the first to seventh examples), the delay test audio signal is located in a frequency range inaudible to the vehicle occupants. In a ninth example of the system (optionally including one or more of the first to eighth examples), a first unidirectional communication channel allows audio to be sent from the WAU to either the first audio output device or the second audio output device, and a second unidirectional channel allows audio to be sent from one or more microphones in the audio system to the WAU, wherein the first and second audio output devices are not part of the same unit of the microphone or coupled to the microphone, and neither the first nor the second audio output device transmits audio signals back to the WAU, and the WAU does not send audio signals to the microphone. In a tenth example of the system (optionally including one or more of the first to ninth examples), a first interface couples the intelligent amplifier of the audio system to the first and second audio output devices, and a second interface couples the intelligent amplifier to one or more microphones in the vehicle. In an eleventh example of the system (optionally including one or more of the first to tenth examples), a first interface couples the intelligent amplifier of the audio system to the first and second audio output devices, and a second interface couples the domain controller of the audio system to one or more microphones in the vehicle.

[0086] This disclosure also provides support for a method for a vehicle audio system, the method comprising: receiving an audio input stream; processing the audio input stream at a wireless audio unit (WAU) of the audio system to generate a first audio output stream; processing the audio input stream at the WAU to generate a second audio output stream, the second audio output stream being different from the first audio output stream; simultaneously transmitting the first audio output stream to a first audio output device of the vehicle and transmitting the second audio output stream to a second audio output device of the vehicle; and, in response to detecting a time delay of the first audio output stream at the first audio output device relative to the second audio output stream at the second audio output device, inserting a delay in the second audio output stream to synchronize the first audio output stream and the second audio output stream, wherein the first audio output device is a wireless personal audio device for a vehicle occupant, and the second audio output device is integrated into an occupant's seat and includes one of a haptic transducer and a subwoofer, the second audio output device being coupled to the audio system via a wired connection. In a first example of the method, processing the audio input stream at the WAU to generate a first audio output stream and a second audio output stream further includes: detecting that the first audio output device is a wireless device; and, in response, generating the first audio output stream using a first transmission method and / or communication protocol, and generating the second audio output stream using a second transmission method and / or communication protocol, which is different from the first transmission method and / or communication protocol. In a second example of the method (optionally including the first example), the first transmission method and / or communication protocol includes an ultra-wideband (UWB) protocol. In a third example of the method (optionally including one or both of the first and second examples), processing the audio input stream at the WAU to generate the first audio output stream further includes: embedding a delay test audio signal into the first audio output stream; simultaneously transmitting the first audio output stream along with the first and second audio output streams to a wireless device positioned at an occupant's seat; simultaneously transmitting the delay test audio signal along with the first and second audio output streams to a wired speaker in the vehicle, the wired speaker being coupled to the audio system via a wired connection; and measuring the delay of the delay test audio signal at the wireless device relative to the delay test audio signal on the wired speaker. In a fourth example of the method (optionally including one or more of the first to third examples), the delay test audio signal is in a frequency range inaudible to the occupants of the vehicle. In a fifth example of the method (optionally including one or more of the first to fourth examples), the wireless device is integrated into the occupant's seat. In a sixth example of the method (optionally including one or more of the first to fifth examples), the wireless device is a personal audio device configured to capture a time-delay test audio signal at a wired speaker via a microphone of the personal audio device, and to measure the time delay of the embedded time-delay test audio signal relative to the captured time-delay test audio signal.

[0087] This disclosure also provides support for a vehicle audio system comprising: a wireless personal audio device (WAPD) including a processor and a non-transitory memory storing instructions that, when executed, cause the processor to: receive a first audio output stream transmitted from a wireless audio unit (WAU) of the audio system; extract a delay test audio signal from the first audio output stream having a frequency range inaudible to vehicle occupants; capture a second audio output stream transmitted simultaneously from the WAU to a speaker in the vehicle via a microphone of the WAPD, the second audio output stream including the delay test audio signal; measure a delay of the first audio output stream relative to the second audio output stream; and transmit the measured delay to the audio system.

[0088] In another representation, a vehicle audio system includes: a wireless audio unit (WAU); a smart amplifier; a domain controller; a processor; and a non-transitory memory storing instructions that, when executed, cause the processor to: receive an audio input stream at the WAU; process the audio input stream at the WAU to generate a first audio output stream; process the audio input stream at the WAU to generate a second audio output stream, the second audio output stream being different from the first audio output stream; simultaneously transmit the first audio output stream to a first audio output device of the vehicle and the second audio output stream to a second audio output device of the vehicle via a first interface with the smart amplifier; estimate a time delay of the first audio output stream at the first audio output device relative to the second audio output stream at the second audio output device based on a time delay test audio signal received at a microphone of the vehicle via a second interface with the domain controller; and, in response to the time delay being greater than a threshold time delay, insert a delay into the second audio output stream to synchronize the first audio output stream and the second audio output stream.

[0089] When describing the elements of various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to indicate the presence of one or more elements. The terms “first,” “second,” etc., do not indicate any order, quantity, or importance, but are used to distinguish one element from another. The terms “comprising,” “including,” and “having” are intended to be inclusive and indicate that additional elements may exist in addition to the listed elements. When the terms “connected to,” “coupled to,” etc., are used herein, an object (e.g., a material, element, structure, component, etc.) may be connected to or coupled to another object, regardless of whether the one object is directly connected to or coupled to the other object, or whether one or more intermediate objects exist between the one object and the other object. Furthermore, references to “an embodiment” or “an embodiment” in this disclosure are not intended to exclude the existence of additional embodiments that also include the listed features.

[0090] In addition to any modifications previously noted, those skilled in the art can devise many other variations and alternative arrangements without departing from the spirit and scope of this specification, and the appended claims are intended to cover such modifications and arrangements. Therefore, although the information enhancement has been described in particular and detail above in conjunction with aspects currently considered most practical and preferred, it will be apparent to those skilled in the art that various modifications can be made without departing from the principles and concepts set forth herein, including but not limited to changes in form, function, operation, and manner of use. Furthermore, as used herein, the examples and embodiments are intended in all respects to be illustrative only and should not be construed as limiting in any way.

Claims

1. A vehicle audio system, comprising: Wireless Audio Unit (WAU); processor; as well as Non-transitory memory, which stores instructions that, when executed, cause the processor to: Receive the audio input stream at the WAU; The audio input stream is processed at the WAU to generate a first audio output stream; The audio input stream is processed at the WAU to generate a second audio output stream, which is different from the first audio output stream. Simultaneously, the first audio output stream is transmitted to the first audio output device of the vehicle, and the second audio output stream is transmitted to the second audio output device of the vehicle; Estimate the time delay of the first audio output stream at the first audio output device relative to the second audio output stream at the second audio output device; and In response to the delay being greater than a threshold delay, a delay is inserted into the second audio output stream to synchronize the first audio output stream and the second audio output stream.

2. The audio system of claim 1, wherein the first audio output device is a personal audio device of an occupant of the vehicle, the personal audio device receiving the first audio output stream via a wireless connection, and the second audio output device is coupled to the audio system via a wired connection.

3. The audio system of claim 2, wherein further instructions are stored in the memory, which, when executed, cause the processor to generate the first audio output stream using a first communication protocol and to generate the second audio output stream using a second communication protocol, the second communication protocol being different from the first communication protocol.

4. The audio system of claim 3, wherein the first communication protocol is an ultra-wideband (UWB) protocol.

5. The audio system of claim 3, wherein further instructions are stored in the memory, which, when executed, cause the processor to: Detect the type of audio content included in the first audio output stream; In response to the audio content of the aforementioned type including a threshold amount of speech, the UWB protocol is selected as the first communication protocol; and In response to the type of audio content including a threshold amount of music, the Bluetooth® protocol is selected as the first communication protocol.

6. The audio system of claim 2, wherein the personal audio device is one of an earbud or a headphone, and the second audio output device is integrated into the occupant's seat and includes one of a haptic transducer and a subwoofer.

7. The audio system of claim 2, wherein further instructions are stored in the memory, and when executed, the instructions cause the processor to: Embed a time delay test audio signal in the first audio output stream; The first audio output stream is simultaneously transmitted to a wireless reverse channel speaker located near the personal audio device, along with the first audio output stream and the second audio output stream. The time-delay test audio signal, along with the first audio output stream and the second audio output stream, is simultaneously transmitted to the vehicle's wired speaker, which is coupled to the audio system via a wired connection; and The delay of the delay test audio signal on the personal audio device is measured relative to the delay test audio signal at the wired speaker.

8. The audio system of claim 7, wherein the wireless reverse channel speaker is integrated into the occupant's seat.

9. The audio system of claim 7, wherein the time delay test audio signal is in a frequency range inaudible to the occupants of the vehicle.

10. The audio system of claim 1, wherein a first unidirectional communication channel allows audio to be transmitted from the WAU to either the first audio output device or the second audio output device, and a second unidirectional channel allows audio to be transmitted from a microphone in one or more microphones of the audio system to the WAU, and the first audio output device and the second audio output device are not part of the same unit of the microphone or coupled to the microphone; and The first audio output device and the second audio output device do not transmit audio signals back to the WAU, and the WAU does not send audio signals to the microphone.

11. The audio system of claim 10, wherein the first interface couples the intelligent amplifier of the audio system to the first audio output device and the second audio output device, and the second interface couples the intelligent amplifier to the one or more microphones of the vehicle.

12. The audio system of claim 10, wherein the first interface couples the intelligent amplifier of the audio system to the first audio output device and the second audio output device, and the second interface couples the domain controller of the audio system to the one or more microphones of the vehicle.

13. A method for an audio system for a vehicle, the method comprising: Receive audio input stream; The audio input stream is processed at the wireless audio unit (WAU) of the audio system to generate a first audio output stream; The audio input stream is processed at the WAU to generate a second audio output stream, which is different from the first audio output stream. Simultaneously, the first audio output stream is transmitted to the first audio output device of the vehicle, and the second audio output stream is transmitted to the second audio output device of the vehicle; as well as In response to detecting a time delay between the first audio output stream at the first audio output device and the second audio output stream at the second audio output device, a delay is inserted into the second audio output stream to synchronize the first audio output stream and the second audio output stream; The first audio output device is a wireless personal audio device for the occupant of the vehicle, and the second audio output device is integrated into the occupant's seat and includes one of a haptic transducer and a subwoofer, and the second audio output device is coupled to the audio system via a wired connection.

14. The method of claim 13, wherein processing the audio input stream at the WAU to generate the first audio output stream and the second audio output stream further comprises: The system detects that the first audio output device is a wireless device, and in response, uses a first transmission method and / or communication protocol to generate the first audio output stream. as well as The second audio output stream is generated using a second transmission method and / or communication protocol, which is different from the first transmission method and / or communication protocol.

15. The method of claim 14, wherein the first transmission method and / or communication protocol includes an ultra-wideband (UWB) communication protocol.

Images