System and method for providing enhanced audio

Abstract

A system for providing spatialized audio in a vehicle, the system comprising: a vehicle orientation sensor that outputs a vehicle orientation signal and is disposed on the vehicle; and a controller configured to receive a user orientation signal output from a user orientation sensor on a wearable device that moves with the head of the first user during use, wherein the controller is further configured to determine an orientation of the head of the user relative to the vehicle based at least on a difference between the vehicle orientation signal and the user orientation signal, the controller is further configured to output a first spatial audio signal to a first binaural device according to the orientation of the user's head relative to the vehicle such that the first binaural device generates a first spatial acoustic signal perceived by the user as originating from a first virtual source location within the cabin of the vehicle.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 366,294, filed on June 13, 2022, and entitled “Systems and Methods for Providing Augmented Audio,” which is incorporated herein by reference in its entirety. Background Art

[0003] The present disclosure relates generally to systems and methods for providing enhanced audio in a vehicle cabin, and more particularly to methods of enhancing the bass response of at least one binaural device disposed in a vehicle cabin. Summary of the invention

[0004] All examples and features mentioned below can be combined in any technically possible way.

[0005] A system for providing spatialized audio in a vehicle, the system comprising: a vehicle orientation sensor that outputs a vehicle orientation signal and is disposed on the vehicle; and a controller configured to receive a user orientation signal output from a user orientation sensor disposed on a wearable device that moves with a first user's head during use, wherein the controller is further configured to determine an orientation of the user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the user orientation signal, and the controller is further configured to output a first spatial audio signal to a first binaural device based on the orientation of the user's head relative to the vehicle, so that the first binaural device generates a first spatial acoustic signal that is perceived by the user as originating from a first virtual source localization within a cabin of the vehicle.

[0006] In an example, the system also includes an error sensor configured to detect an orientation of a user's head relative to a vehicle and output an error sensor signal, wherein the controller is further configured to correct a drift between the user orientation signal and the vehicle orientation signal based on the orientation of the user's head detected by the error sensor.

[0007] In an example, the error sensor includes at least one of: a time-of-flight sensor, a lidar device, a camera, or a microphone disposed on the user's head.

[0008] In an example, the controller samples the error sensor signal at a slower rate than the vehicle orientation signal and the user orientation signal.

[0009] In an example, the vehicle orientation sensor includes a first plurality of sensors, wherein the user orientation sensor includes a second plurality of sensors, wherein the controller is further configured to correct a drift between the user orientation signal and the vehicle orientation signal based on a measure of similarity between at least one sensor of the first plurality of sensors and one sensor of the second plurality of sensors.

[0010] In an example, the controller includes a first controller and a second controller, the first controller receiving a vehicle orientation signal and a user orientation signal and determining the orientation of the user's head relative to the vehicle and outputting a position signal to the second controller, the second controller receiving the position signal and outputting a first spatial acoustic signal to the first binaural device.

[0011] In an example, the wearable device is a first binaural device.

[0012] In an example, the system also includes a plurality of speakers disposed in a periphery of a cabin of the vehicle, wherein the first spatial audio signal includes at least a high frequency range of a first content signal, wherein the controller is further configured to drive the plurality of speakers with a drive signal so as to generate first bass content of the first content signal in the cabin.

[0013] In an example, a controller is configured to receive a second user orientation signal output from a second user orientation sensor provided on a second wearable device, which moves with the second user's head during use, wherein the controller is further configured to determine the orientation of the second user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the second user orientation signal, and the controller is further configured to output a second spatial audio signal to a second binaural device based on the orientation of the second user's head relative to the vehicle, so that the second binaural device generates a second spatial acoustic signal that is perceived by the second user as originating from a first virtual source localization or a second virtual source localization within a cabin of the vehicle.

[0014] In an example, the second spatial audio signal includes at least a high frequency range of a second content signal, wherein the controller is further configured to drive the plurality of speakers according to a first array configuration so as to produce a first bass content in a first listening zone in the cabin, and to drive the plurality of speakers according to a second array configuration so as to produce a second bass content of the second content signal in a second listening zone in the cabin, wherein in the first listening zone, an amplitude of the first bass content is greater than an amplitude of the second bass content, and in the second listening zone, an amplitude of the second bass content is greater than an amplitude of the first bass content.

[0015] In an example, a vehicle orientation sensor: is either integrated with the vehicle or brought in by a user and fixed to the vehicle.

[0016] A method for providing spatialized audio in a vehicle, the method comprising: receiving a user orientation signal output from a user orientation sensor disposed on a wearable device, the wearable device moving with a first user's head during use; receiving a vehicle orientation signal from a vehicle orientation sensor disposed on the vehicle; determining an orientation of the user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the user orientation signal; and outputting a first spatial audio signal to a first binaural device based on the orientation of the user's head relative to the vehicle, so that the first binaural device generates a first spatial acoustic signal perceived by the user as originating from a first virtual source localization within a cabin of the vehicle.

[0017] In an example, the method also includes receiving an error sensor signal output from an error sensor, the error sensor being configured to detect an orientation of a user's head relative to a vehicle; and correcting a drift between the user orientation signal and the vehicle orientation signal based on the orientation of the user's head detected by the error sensor.

[0018] In an example, the error sensor includes at least one of: a time-of-flight sensor, a lidar device, a camera, a microphone disposed on the user's head.

[0019] In an example, the error sensor signal is sampled at a slower rate than the rate at which the vehicle orientation signal and the user orientation signal are sampled.

[0020] In an example, the method further includes the step of correcting drift between the user orientation signal and the vehicle orientation signal, wherein the vehicle orientation sensor includes a first plurality of sensors, wherein the user orientation sensor includes a second plurality of sensors, wherein the drift is corrected based on a measure of similarity between at least one sensor of the first plurality of sensors and one sensor of the second plurality of sensors.

[0021] In an example, the wearable device is a first binaural device.

[0022] In an example, the method further comprises driving the plurality of speakers with the drive signal such that a first bass content of the spatial audio signal is generated in the cabin.

[0023] In the example, the method also includes receiving a second user orientation signal output from a second user orientation sensor disposed on a second wearable device, which moves with the second user's head during use; determining the orientation of the second user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the second user orientation signal; and outputting a second spatial audio signal to a second binaural device based on the orientation of the second user's head relative to the vehicle, so that the second binaural device generates a second spatial acoustic signal that is perceived by the second user as originating from a first virtual source localization or a second virtual source localization within a cabin of the vehicle.

[0024] In an example, the method further includes driving the plurality of speakers according to a first array configuration so as to generate a first bass content in a first listening zone in the cabin, and driving the plurality of speakers according to a second array configuration so as to generate a second bass content of a second spatial audio signal in a second listening zone in the cabin, wherein in the first listening zone, the amplitude of the first bass content is greater than the amplitude of the second bass content, and in the second listening zone, the amplitude is less than the amplitude of the second bass content.

[0025] The details of one or more implementations are discussed in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent in the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the various aspects.

[0027] Figure 1A An audio system for providing enhanced audio in a vehicle cabin according to an example is depicted.

[0028] Figure 1B An audio system for providing enhanced audio in a vehicle cabin according to an example is depicted.

[0029] Figure 2 An open-ear wearable device according to an example is depicted.

[0030] Figure 3 An open-ear wearable device according to an example is depicted.

[0031] Figure 4 A flow chart of a method for providing enhanced audio in a vehicle cabin according to an example is depicted.

[0032] Figure 5 An audio system for providing enhanced spatialized audio in a vehicle cabin according to an example is depicted.

[0033] Figure 6 Depicted is a flow chart of a method for providing enhanced spatialized audio in a vehicle cabin according to an example.

[0034] Fig. 7A Depicted is a cross-plot according to an example.

[0035] Figure 7B Depicted is a cross-plot according to an example.

[0036] Fig. 8A An audio system for providing enhanced spatialized audio in a vehicle cabin according to an example is depicted.

[0037] Figure 8B An audio system for providing enhanced spatialized audio in a vehicle cabin according to an example is depicted.

[0038] Figure 8C An audio system for providing enhanced spatialized audio in a vehicle cabin according to an example is depicted.

[0039] Fig.8D An audio system for providing enhanced spatialized audio in a vehicle cabin according to an example is depicted.

[0040] Fig. 9A Representations of user orientation and vehicle orientation compared to measured user orientation and vehicle orientation are depicted.

[0041] Fig. 9B Representations of user orientation and vehicle orientation compared to measured user orientation and vehicle orientation are depicted.

[0042] Fig. 9C Representations of user orientation and vehicle orientation compared to measured user orientation and vehicle orientation are depicted.

[0043] Fig. 10A Depicted is a representation of a wearable device having microphones disposed on opposing sides, such that a delay between receipt of audio signals may be detected.

[0044] Fig. 10B Depicted is a representation of a wearable device having microphones disposed on opposing sides, such that a delay between receipt of audio signals may be detected.

[0045] Fig.11A Depicted is a representation of the acceleration detected by a separate accelerometer of a user orientation sensor.

[0046] Fig. 11B Depicted is a representation of the acceleration detected by a separate accelerometer of a user orientation sensor.

[0047] Fig.12 A flow chart of a method for providing spatialized audio to binaural devices within a vehicle based on the orientation of a user's head relative to the vehicle is depicted. DETAILED DESCRIPTION

[0048] Vehicle audio systems that include only ambient speakers are limited in their ability to provide different audio content to different passengers. While a vehicle audio system can be arranged to provide separate bass content zones with satisfactory isolation, the same cannot be said for high-range content, where the wavelengths are too short to adequately create separate listening zones with separate content using ambient speakers alone.

[0049] Leakage of high-range content between listening zones can be addressed by providing each user with a wearable device such as headphones. If each user wears a pair of headphones, an independent audio signal can be provided to each user with minimal sound leakage. However, minimal leakage comes at the expense of isolating each passenger from the environment, which is undesirable in a vehicle environment. This is particularly true for drivers who need to be able to hear sounds in the environment (such as sounds generated by emergency vehicles or the voices of passengers), but it is also true for other passengers who generally want to be able to carry on a conversation and communicate with each other.

[0050] This can be addressed by providing each user with a binaural device such as an open-ear wearable device or near-field speakers (such as headrest speakers) that provide independent high-range audio content to each passenger while maintaining an open path to the user's ears, allowing the user to interact with the environment. However, in a moving vehicle, open-ear wearable devices and near-field speakers generally do not provide sufficient bass response because road noise tends to mask the same frequency bands.

[0051] Now turn to Figure 1A, which shows a schematic diagram representing an audio system for providing enhanced audio in a vehicle cabin 100. As shown, the vehicle cabin 100 includes a group of ambient speakers 102. (For the purposes of this disclosure, a speaker is any device that receives an electrical signal and converts it into an acoustic signal.) A controller 104 disposed in the vehicle is configured to receive a first content signal u1 and a second content signal u2. The first content signal u1 and the second content signal u2 are audio signals (and may be received as analog or digital signals according to any suitable protocol) that each include bass content (i.e., content below 250 Hz ± 150 Hz) and high-range content (i.e., content above 250 Hz ± 150 Hz). The controller 104 is configured to drive the ambient speakers 102 using drive signals d1-d4 to form at least a first array configuration and a second array configuration. The first array configuration formed by at least a subset of the ambient speakers 102 constructively combines the acoustic energy generated by the ambient speakers 102 to produce the bass content of the first content signal u1 in a first listening zone 106 disposed at a first seating position P1. Similarly, the second array configuration formed by at least a subset of the ambient speakers 102 constructively combines the acoustic energy generated by the ambient speakers 102 to produce the bass content of the second content signal u2 in the second listening zone 108 arranged at the second seating position P2. In addition, the first array configuration can destructively combine the acoustic energy generated by the ambient speakers 102 to form a substantial null at the second listening zone 108 (and any other seating position in the vehicle cabin), and the second array configuration can destructively combine the acoustic energy generated by the ambient speakers 102 to form a substantial null at the first listening zone (and any other seating position in the vehicle cabin).

[0052] It should be understood that in various examples, there may be some or all overlap between a subset of ambient speakers 102 arranged to produce bass content for a first content signal u1 in a first listening zone 106 and a subset of ambient speakers 102 arranged to produce bass content for a second content signal u2 in a second listening zone.

[0053] Given substantially the same amplitude of the bass content in the first content signal and the second content signal, the arrangement of the ambient speakers 102 means that in the first listening zone 106 the amplitude of the bass content of the first content signal u1 is greater than the amplitude of the bass content of the second content signal u2. Similarly, the amplitude of the bass content of the second content signal u2 is greater than the amplitude of the bass content of the first content signal u1. The net effect is that a user sitting at position P1 perceives the bass content of the first content signal u1 primarily as greater than the bass content of the second content signal u2, which may not be perceived in some cases. Similarly, a user sitting at position P2 perceives the bass content of the second content signal u2 primarily as greater than the bass content of the first content signal u1. In one example, in the first listening zone, the amplitude of the bass content of the first content signal u1 is at least 3 dB greater than the amplitude of the bass content of the second content signal u2, and similarly, in the second listening zone, the amplitude of the bass content of the second content signal u2 is at least 3 dB greater than the amplitude of the bass content of the first content signal u1.

[0054] Although only four ambient speakers 102 are shown, it should be understood that any number of ambient speakers 102 greater than one may be used. Furthermore, for purposes of this disclosure, the ambient speakers 102 may be disposed in or on vehicle doors, pillars, ceilings, floors, dashboards, rear decks, trunks, may be disposed under seats, integrated within seats, or disposed in a center console in the cabin 100, or in or on any other driven point in the cabin structure that creates acoustic bass energy in the cabin.

[0055] In various examples, the first content signal u1 and the second content signal u2 (as well as any other received content signals) may be received from one or more of a mobile device (e.g., via a Bluetooth connection), a radio signal, a satellite radio signal, or a cellular signal, although other sources are also contemplated. Furthermore, each content signal need not be received simultaneously, but may be previously received and stored in a memory for later playback. Furthermore, as described above, the first content signal u1 and the second content signal u2 may be received as analog or digital signals according to any suitable communication protocol. Furthermore, because the first content signal u1 and the second content signal u2, which are composed of a set of binary values, may be transmitted digitally, the bass content and high-range content of these content signals refer to constituent signals of the corresponding frequency ranges of the bass content and the high-range content when the content signals are converted into analog signals before being converted by a speaker or other device.

[0056] like Figure 1A As shown, binaural devices 110 and 112 are positioned to generate a stereo first acoustic signal 114 in the first listening zone 106 and a stereo second acoustic signal 116 in the second listening zone, respectively. Figure 1AAs shown, the binaural devices 110 and 112 include speakers 118, 120 disposed in respective headrests disposed near the listening zones 106, 108. For example, the binaural device 110 includes a left speaker 118L and a right speaker 118R, the left speaker being disposed in the headrest to transmit a left first acoustic signal 114L to a left ear of a user sitting in a first seated position P1, and the right speaker transmitting a right first acoustic signal 114R to a right ear of the user. In the same manner, the binaural device 112 includes a left speaker 120L and a right speaker 120R, the left speaker being disposed in the headrest to transmit a left second acoustic signal 116L to a left ear of a user sitting in a second seated position P2, and the right speaker transmitting a right second acoustic signal 116R to a right ear of the user. Although the acoustic signals 114, 116 are shown as including left and right stereo components, it should be understood that in some examples, one or both acoustic signals 114, 116 may be mono signals, in which the left and right sides are identical. Each of the binaural devices 110, 112 may also employ a set of cross-cancellation filters that eliminate the audio produced by the opposite side on each respective side. Thus, for example, the binaural device 110 may employ a set of cross-cancellation filters to eliminate the audio produced for the user's right ear at the user's left ear, and vice versa. In examples where the binaural device is a wearable device (e.g., an open-ear headphone) and has a driving point close to the ear, crosstalk cancellation is generally not required. However, in the case of a more distant headrest speaker or wearable device (e.g., Bose Sound Wear), the binaural device will typically employ some measured crosstalk cancellation to achieve binaural control.

[0057] Although the first binaural device 110 and the second binaural device 112 are shown as speakers disposed in the headrest, it should be understood that the binaural device described in the present disclosure may be any device suitable for transmitting independent left-ear acoustic signals and right-ear acoustic signals (i.e., stereo signals) to a user sitting at a corresponding position. Therefore, in an alternative example, the first binaural device 110 and / or the second binaural device 112 may be composed of speakers located in other areas of the vehicle cabin 100 suitable for transmitting independent left-ear acoustic signals and right-ear acoustic signals to the user, such as upper seat backs, roof linings, or any other places disposed near the user's ears. In yet another alternative example, the first binaural device 110 and / or the second binaural device 112 may be an open-ear wearable device worn by a user sitting at a corresponding seated position. For the purposes of the present disclosure, an open-ear wearable device is any device designed to be worn by a user and capable of transmitting independent left-ear acoustic signals and right-ear acoustic signals while maintaining an open path to the user's ears. Figure 2 and Figure 3Two examples of such open-ear wearable devices are shown. The first open-ear wearable device is a pair of frames 200 featuring left and right speakers 202L and 202R located in left and right temples 204L and 204R, respectively. The second open-ear wearable device is a pair of open-ear headphones 300 featuring left and right speakers 302L and 302R. Both the frame 200 and the open-ear headphones 300 retain an open path to the user's ears while being able to provide independent acoustic signals to the user's left and right ears.

[0058] The controller 104 may provide at least the high-range content of the first content signal u1 to the first binaural device 110 via the binaural signal b1, and provide at least the high-range content of the second content signal u2 to the second binaural device 112 via the binaural signal b2. (In the example, the entire range of the first content signal u1 and the second content signal u2, including the bass content, is transmitted to the first binaural device 110 and the second binaural device 112, respectively.) Therefore, the first acoustic signal 114 includes at least the high-range content of the first content signal u1, and the second acoustic signal 116 includes at least the high-range content of the second signal u2. The generation of the bass content of the first content signal u1 in the first listening zone 106 by the ambient speaker 102 enhances the generation of the high-range content of the first signal u1 generated by the first binaural device 110, and the generation of the bass content of the second content signal u2 in the second listening zone 108 by the ambient speaker 102 enhances the generation of the high-range content of the second content signal u2 generated by the second binaural device.

[0059] The user sitting at the seating position P1 thus perceives the first content signal u1 played in the first listening zone 106 from the combined output of the first array configuration of the ambient speakers 102 and the first binaural device 110. Likewise, the user sitting at the seating position P2 perceives the second content signal u2 played in the second listening zone 108 from the combined output of the second array configuration of the ambient speakers 102 and the second binaural device 112.

[0060] Fig. 7A and Figure 7B An example graph depicting the frequency crossover between bass content and high-range content of an example content signal (e.g., first content signal u1) at 100 Hz and 200 Hz, respectively. As described above, the crossover between bass content and high-range content may occur at, for example, 250 Hz ± 150 Hz, so a crossover of 100 Hz or 200 Hz is an example of this range. As shown, the combined total response at the listening area is perceived as a flat response. (Of course, a flat response is only one example of a frequency response, and other examples may, for example, boost the bass, midrange, and / or treble, depending on the desired equalization.)

[0061] The binaural signals b1, b2 (and any other binaural signals generated for additional binaural devices) are typically N-channel signals, where N ≥ 2 (since there is at least one channel for each ear). N may be related to the number of speakers in the rendering system (e.g., if the headrest has four speakers, the associated binaural signal typically has four channels). In the case where the binaural device employs crosstalk cancellation, there may be some overlap between the content in the channels for cancellation purposes. However, typically, the mixing of the signals is performed by a crosstalk cancellation filter provided within the binaural device, rather than in the binaural signal received by the binaural device.

[0062] The controller 104 may provide the binaural signals b1, b2 in a wired or wireless manner. For example, when the binaural device 110 or 112 is an open-ear wearable device, the corresponding binaural signals b1, b2 may be sent via Bluetooth, WiFi, or any other suitable wireless protocol.

[0063] In addition, the controller 104 may also be configured to time-align the generation of bass content in the first listening zone 106 with the generation of high-range content by the first binaural device 110 to account for wireless, acoustic, or other transmission delays inherent in the generation of these signals. Similarly, the controller 104 may also be configured to time-align the generation of bass content in the second listening zone 108 with the generation of high-range content by the second binaural device 112. There will be some inherent delay between the output of the drive signals d1-d4 and the point in time when the bass content converted by the ambient speakers 102 arrives at the respective listening zones 106, 108. This delay includes the time required for the drive signals d1-d4 to be converted into acoustic signals by the respective speakers 102 and to travel from the respective speakers 102 to the first listening zone 106 or the second listening zone 108. (But it is contemplated that other factors may also affect the delay.) Because each ambient speaker 102 may be located at some unique distance from the first listening zone 106 and the second listening zone 108, the delay may be calculated separately for each ambient speaker 102. In addition, there will be some delay between the output of the binaural signals b1, b2 and the corresponding generation of the acoustic signals 114, 116 in the first listening zone 106 and the second listening zone 108. This delay will be a function of the time to process the received binaural signals b1, b2 (in the case where the binaural signals are encoded in a communication protocol such as a wireless protocol, and / or in the case where the binaural devices perform some additional signal processing) and convert the binaural signals b1, b2 into the acoustic signals 114, 116, as well as the time for the acoustic signals 114, 116 to travel to the user sitting at the position P1, P2 (but because each binaural device is located relatively close to the user, this may be negligible). (Again, other factors may affect the delays.) Therefore, taking into account these delays, the controller 104 can time the generation of the drive signals d1-d4 and the binaural signals b1, b2 so that the generation of the bass content of the first content signal u1 through the ambient speakers 102 in the first listening zone 106 is time-aligned with the generation of the high-range content of the first content signal u1 through the first binaural device 110, and the generation of the bass content of the second content signal u2 through the ambient speakers 102 in the second listening zone 108 is time-aligned with the generation of the high-range content of the second content signal u2 through the second binaural device 112.

[0064] For the purposes of this disclosure, "time alignment" refers to the alignment of the generation time of the bass content and the high-range content of a given content signal at a given point in space (e.g., a listening area) so that the content is accurately reproduced at that given point in space. It should be understood that the bass content and the high-range content only need to be time-aligned to a sufficient degree for the user to perceive that the content signal is accurately reproduced. Typically, in a time-aligned acoustic signal, a 90° offset at the crossover frequency between the bass content and the high-range content is acceptable. To provide several examples at several different crossover frequencies, an acceptable offset may be + / -2.5ms for 100Hz, + / -1.25ms for 200Hz, + / -1ms for 250Hz, and + / -0.625ms for 400Hz. However, it should be understood that for the purposes of this disclosure, any offset up to 180° at the crossover frequency is considered to be time-aligned.

[0065] like Fig. 7A and Figure 7B As shown, there is additional overlap between the bass content and the high-range content across the crossover frequency. The phases of these frequencies within the overlap can be individually shifted to align the high-range content and the bass content in time; as will be appreciated, the phase shift applied will depend on the frequency. For example, one or more all-pass filters may be included that are designed to introduce a phase shift to at least the overlapping frequencies of the high-range content and the bass content in order to achieve the desired time alignment across frequencies.

[0066] Time alignment may be pre-established for a given binaural device. In the example of a headrest speaker, the delays between receiving the binaural signal and generating the acoustic signal will always be the same, and therefore these delays may be set as factory settings. However, in the case where the binaural devices 110, 112 are wearable devices, based on the different times required to process the corresponding binaural signals b1, b2 and generate the acoustic signals 114, 116, the delays will typically vary from one wearable device to another (this is particularly true in the case of wireless protocols with well-known variable delays). Therefore, in one example, the controller 104 may store multiple delay presets for aligning the generation of bass content with the generation of acoustic signals 114, 116 for various wearable devices or various types of wearable devices. Therefore, when the controller 104 is connected to a particular wearable device, it may identify the wearable device (e.g., a pair of Bose Frames) and retrieve from a storage device a specific pre-stored delay for aligning the bass content with the acoustic signals 114, 116 generated by the identified wearable device. In an alternative example, the pre-stored delays may be associated with a specific device type. For example, if the delays associated with a wearable device operating a specific communication protocol (e.g., Bluetooth) or protocol version (e.g., Bluetooth version) are generally the same, the controller 104 may select the delay based on the detected communication protocol or communication protocol version. These pre-stored delays for a given device or device type may be determined by the following method: using a microphone in a given listening area and calibrating the delay manually or by an automated method until the bass content of a given content signal at the listening area is aligned with the acoustic signal time of a given binaural device. In yet another example, the delays may be calibrated based on user input. For example, a user wearing an open-ear wearable device may sit in a seated position P1 or P2 and adjust the generation of drive signals d1-d4 and / or binaural signals b1, b2 until the bass content is correctly aligned with the high range of acoustic signals 114, 116. In another example, the device may report the delays necessary for time alignment to the controller 104.

[0067] In an alternative example, time alignment can be determined automatically during runtime rather than by a set of pre-stored delays. In an example, a microphone can be provided on or near the binaural device (e.g., on a headrest or on a wearable device) and used to generate a signal to a controller to determine the delay for time alignment. A method for automatically determining time alignment is described in US2020 / 0252678, entitled “Latency Negotiation in a Heterogeneous Network of Synchronized Speakers,” the entire contents of which are incorporated herein by reference, but any other suitable method for determining delays may be used.

[0068] As described above, time alignment can be achieved over a certain frequency range using an all-pass filter. To account for different delays of various binaural devices, the specific filter implemented can be selected from a set of stored filters, or the phase change achieved by the all-pass filter can be adjusted. As described above, the selected filter or phase change can be based on different devices or device types, can be input by a user, can be based on the delay detected by a microphone on the wearable device, can be based on the delay reported by the wearable device, etc.

[0069] exist Figure 1A In the example of , the controller 104 generates both the drive signals d1-d4 and the binaural signals b1, b2. However, in an alternative example, one or more mobile devices may provide the binaural signals b1, b2. For example, Figure 1BAs shown, the mobile device 122 provides a binaural signal b1 to the binaural device 110 (e.g., where the binaural device 110 is an open-ear wearable device) via a wired connection or a wireless (e.g., Bluetooth) connection. For example, a user can wear the open-ear wearable binaural device 110 to enter the vehicle cabin 100 and listen to music via the Bluetooth connection (binaural signal b1) paired with the mobile device 122. After entering the vehicle cabin 100, the controller 104 can begin to provide the bass content of the first content signal u1 while the mobile device 122 continues to provide the binaural signal b1 to the open-ear wearable binaural device 110. In this example, the controller 104 can receive the first content signal u1 from the mobile device 122 so as to generate the bass content of the first content signal u1 in the first listening zone 106. Therefore, the mobile device 122 can be paired with both the binaural device 110 and the controller 104 (or otherwise connected to both) to provide the binaural signal b1 and the first content signal u1. In an alternative example, the mobile device 122 may broadcast a single signal received by both the controller 104 and the binaural device 110 (in this example, each device may apply a corresponding high pass / low pass for crossover). For example, the Bluetooth 5.0 standard provides such isochronous channels for broadcasting signals locally to nearby devices. In an alternative example, the mobile device 122 may send metadata of the content sent by the first binaural signal b1 to the first binaural device 110 to the controller 104 instead of sending the first content signal u1, thereby allowing the controller 104 to obtain the correct first content signal u1 (i.e., the same content) from an external source such as a streaming service.

[0070] Although in Figure 1B Only one mobile device 122 is shown in FIG. 1 , but it should be understood that any number of mobile devices may provide binaural signals to any number of binaural devices (eg, binaural devices 110 , 112 ) disposed in the vehicle cabin 100 .

[0071] Of course, if combined Figure 1B As described, the controller 104 may receive the first content signal u1 from the mobile device. Therefore, in one example, the user may wear the open-ear wearable first binaural device 110 when entering the vehicle, at which point the mobile device 122 stops sending content to the first binaural device and instead provides the first content signal u1 to the controller 104, which assumes that the binaural signal b1 is sent, for example, via a wireless connection such as Bluetooth. Similarly, for multiple binaural devices (e.g., binaural devices 110, 112) that receive signals from multiple mobile devices, the controller 104 may assume that the corresponding binaural signals (e.g., binaural signals b1, b2) are sent to the binaural devices instead of the mobile devices.

[0072] The controller 104 may include a processor 124 (e.g., a digital signal processor) and a non-transitory storage medium 126 storing program code that, when executed by the processor 124, implements the various functions and methods described in the present disclosure. However, it should be understood that in some examples, the controller 104 may be implemented as hardware only (e.g., as an application specific integrated circuit or a field programmable gate array) or as some combination of hardware, firmware, and software.

[0073] To arrange the ambient speakers 102 to provide bass content to the first listening zone 106 and the second listening zone 108, the controller 104 may implement a plurality of filters, each of which adjusts the acoustic output of the ambient speakers 102 such that the bass content of the first content signal u1 is constructively combined at the first listening zone 106 and the bass content of the second signal u2 is constructively combined at the second listening zone 108. While such filters are typically implemented as digital filters, the filters may alternatively be implemented as analog filters.

[0074] In addition, despite the Figure 1A and Figure 1B Only two listening zones 106 and 108 are shown in FIG. 1 , but it should be understood that the controller 104 can receive any number of content signals and create any number of listening zones (including only one listening zone) by filtering the content signals to arrange the surrounding speakers, each listening zone receiving the bass content of a unique content signal. For example, in a five-seat car, the surrounding speakers can be arranged to produce five independent listening zones, each listening zone producing the bass content of a unique content signal (i.e., where the amplitude of the bass content for the corresponding content signal is the loudest, assuming that the bass content of each content signal is played at substantially equal amplitudes in other listening zones). In addition, independent binaural devices can be provided at each listening zone and receive independent binaural signals that are enhanced by and time-aligned with the bass content produced in the corresponding listening zone.

[0075] In the above example, binaural devices 110, 112 (or any other binaural devices) can transmit the same content to two users. In this example, controller 104 can utilize the bass content produced by surrounding loudspeakers 102 to enhance the acoustic signal produced by binaural devices, without creating an independent listening area for playing independent content. The bass content can be time-aligned with the high-range content played from two binaural devices 110, 112, so that both users perceive the content signal played, including the high-range signal transmitted by binaural devices 110, 112 and the bass content played by surrounding loudspeakers 102. Although each device receives the same program content signal, it is conceivable that the user will select the same content of different volume levels. In this case, controller 104 can adopt the first array configuration and the second array configuration to create an independent volume zone, instead of creating an independent listening zone, in which each user perceives the same program content with different volumes.

[0076] In the example, it is not necessary that each user has the same binaural device with associated, on the contrary, some users can only listen to the content produced by the surrounding speakers 102. For this example, the surrounding speakers 102 will not only produce bass content, but also produce the high-range content of the program content signal (e.g., program content signal u1). For users using binaural devices, the program content signal is perceived as a stereo signal, as provided by binaural signals (e.g., binaural signals b1) and by means of the left and right speakers of the binaural devices. In fact, it should be understood that in each of the examples described in the present disclosure, some or complete overlaps may exist in the frequency spectrum between the signals produced by the surrounding speakers 102 and the binaural devices (e.g., binaural devices 110, 112). Those who use binaural devices with overlaps with the surrounding speakers 102 in the frequency spectrum receive an enhanced experience with improved stereo, audio grading and perceived spatial sense.

[0077] It should be understood that navigation prompts and phone calls are program content signals that can be directed to specific users in the listening zone. Thus, the driver can hear navigation prompts generated by a binaural device (e.g., binaural device 110) with bass enhanced by surrounding speakers while passengers listen to music in different listening zones.

[0078] Additionally, the microphones on the wearable binaural devices can be used for voice pickup for traditional uses such as phone calls, vehicle-based or mobile device-based speech recognition, digital assistants, etc.

[0079] In addition, the controller 104 may implement multiple filters rather than a single set of filters, depending on the configuration of the vehicle cabin 100. For example, various parameters within the cabin will change the acoustics of the vehicle cabin 100, including the number of passengers in the vehicle, whether the windows are rolled up or down, the position of the seats in the vehicle (e.g., whether the seats are upright or tilted or moved forward or backward in the vehicle cabin), etc. These parameters may be detected by the controller 104 (e.g., by receiving a signal from the vehicle's onboard computer) and the correct set of filters may be implemented to provide the first array configuration, the second array configuration, and any additional array configurations. For example, various filter sets may be stored in the memory 126 and retrieved based on the detected cabin configuration.

[0080] In an alternative example, the filter can be a set of adaptive filters that are adjusted based on a signal received from an error microphone (e.g., disposed on a binaural device or otherwise disposed within a corresponding listening zone) so as to adjust the filter coefficients to align the first listening zone at the corresponding seating position (the first seating position P1 or the second seating position P2), or to adjust for varying cabin configurations (such as whether the window is rolled up or down).

[0081] Figure 4 A flow chart of a method 400 for providing enhanced audio to a user in a vehicle cabin is depicted. The steps of the method 400 may be performed by a controller, such as the controller 104, in communication with a set of ambient speakers disposed in the vehicle, such as the ambient speakers 102, and further in communication with a set of binaural devices, such as the binaural devices 110, 112, disposed at respective seating locations within the vehicle.

[0082] At step 402, a first content signal and a second content signal are received. These content signals may be received from a number of potential sources such as a mobile device, radio, satellite radio, cellular connection, etc. These content signals each represent audio that may include bass content and high range content.

[0083] At step 404 and step 406, the plurality of ambient speakers are driven according to a first array configuration (step 404) and a second array configuration (step 406) such that the bass content of the first content signal is produced in a first listening zone in the cabin and the bass content of the second content signal is produced in a second listening zone. The nature of the arrangement produces the listening zones such that when the bass content of the first content signal is played in the first listening zone at the same amplitude as the bass content of the second signal is played in the second listening zone, in the first listening zone, the amplitude of the bass content of the first content signal will be greater (e.g., at least 3 dB greater) than the amplitude of the bass content of the second content signal, and in the second listening zone, the amplitude of the bass content of the second signal will be greater (e.g., at least 3 dB greater) than the amplitude of the bass content of the first content signal. In this manner, a user seated at the first seating position perceives the amplitude of the first bass content as being greater than the amplitude of the second bass content. Likewise, a user seated at the second seating position perceives the amplitude of the second bass content as being greater than the amplitude of the first bass content.

[0084] At steps 408 and 410, the high-range content of the first content signal is provided to a first binaural device positioned to produce the high-range content in the first listening zone (step 408), and the high-range content of the second content signal is provided to a second binaural device positioned to produce the high-range content in the second listening zone (step 410). The end result is that a user sitting at the first seated position perceives the first content signal from a combination of the outputs of the first binaural device and the ambient speakers, and a user sitting at the second seated position perceives the second content signal from a combination of the outputs of the second binaural device and the ambient speakers. In other words, the ambient speakers utilize the bass of the first content signal in the first listening zone to enhance the high-range of the first content signal as produced by the first binaural device, and utilize the bass of the second content signal in the second listening zone to enhance the high-range of the second content signal as produced by the second binaural signal. In various alternative examples, the first binaural device is an open-ear wearable device or speaker disposed in a headrest.

[0085] In addition, the generation of the bass content of the first content signal in the first listening zone can be time-aligned with the generation of the high range of the first content signal by the first binaural device in the first listening zone, and the generation of the second bass content in the second listening zone can be time-aligned with the generation of the high range of the second content signal by the second binaural device. In an alternative example, the first high range content or the second high range content can be provided by the mobile device to the first binaural device or the second binaural device, and the generation of the bass content is time-aligned with the provision.

[0086] Although the method 400 is described for two independent listening zones and two binaural devices, it should be understood that the method 400 can be extended to any number of listening zones (including only one listening zone) disposed within a vehicle and at which corresponding binaural devices are disposed. In the case of a single binaural device and listening zone, isolation from other seats is no longer important, and the multiple ambient speaker filters can be different from the multi-zone case to optimize bass presentation. (The case of a single user can be determined, for example, by a user interface or by sensors disposed in the seat.)

[0087] Now go to Figure 5 , which shows an alternative schematic diagram of a vehicle audio system arranged in a vehicle cabin 100, in which a peripheral speaker 102 is used to enhance the bass content of at least one binaural device that produces spatialized audio. In this example, the controller 504 (an alternative example of the controller 104) is configured to generate binaural signals b1, b2 as spatial audio signals, which cause the binaural devices 110 and 112 to generate acoustic signals 114, 116 as spatial acoustic signals, which are perceived by the user as originating from virtual audio sources SP1 and SP2, respectively. The binaural signal b1 is generated as a spatial audio signal based on the position of the head of a user sitting at position P1. Similarly, the binaural signal b2 is generated as a spatial audio signal based on the position of the head of the user sitting at position P2. Similar to Figure 1A and Figure 1B For example, these spatialized acoustic signals produced by the binaural devices 110 , 112 may be enhanced by bass content produced by the ambient speakers 102 and driven by the controller 504 .

[0088] like Figure 5As shown, the first head tracking device 506 and the second head tracking device 508 are configured to detect the position of the head of the user sitting at the seated position P1 and the user sitting at the seated position P2, respectively. In various examples, the first head tracking device 506 and the second head tracking device 508 may include a time-of-flight sensor configured to detect the position of the user's head within the vehicle cabin 100. However, the time-of-flight sensor is only a possible example. Alternatively, multiple 2D cameras can be used, which use epipolar geometry (such as an eight-point algorithm) to triangulate the distance from one of the camera focuses. Alternatively, each head tracking device may include a laser radar device that produces a black and white image, where the ranging data for each pixel is used as a data set. In an alternative example in which each user wears an open-ear wearable device, head tracking can be completed or enhanced by tracking the corresponding position of the open-ear wearable device on the user, because head tracking will generally be related to the position of the user's head. In other alternative examples, capacitive sensing, inductive sensing, and inertial measurement unit tracking combined with imaging can be used. It should be appreciated that the above-described specific implementation of a head tracking device is intended to convey that a range of possible devices and combinations of devices may be used to track the position of a user's head.

[0089] For purposes of this disclosure, detecting the position of a user's head may include detecting any part of the user, or any part of a wearable device worn by the user, from which the center position of the user's skull may be derived. For example, the positioning of the user's ears may be detected, from which a line may be drawn between the tragus to find the middle in a manner that approximates finding the center. Detecting the position of the user's head may also include detecting the orientation of the user's head, which may be derived based on any method used to find the pitch, yaw, and roll angles. Yaw is particularly important because it typically affects the ear distance to each binaural speaker the most.

[0090] The first head tracking device 506 and the second head tracking device 508 may communicate with a head tracking controller 510, which receives the respective outputs h1, h2 of the first head tracking device 506 and the second head tracking device 508, and determines the position of the head of the user sitting at the position P1 or the position P2 based on these outputs, and generates output signals to the controller 504 accordingly. For example, the head tracking controller 510 may receive the raw output data h1 from the first head tracking device 506, interpret the position of the head of the user sitting at the position P1, and output a position signal e1 representing the detected position to the controller 504. Similarly, the head tracking controller 510 may receive the output data h2 from the second head tracking device 508, and interpret the position of the head of the user sitting at the seated position P2, and output a position signal e2 representing the detected position to the controller 504. The position signals e1 and e2 may be transmitted in real time as coordinates representing the position of the user's head (e.g., including orientation as determined by pitch, yaw, and roll).

[0091] The controller 510 may include a processor 512 and a non-transitory storage medium 514 storing program code, which, when executed by the processor 512, performs the various functions and methods disclosed herein for generating position signals (including receiving the output signal of each head tracking device 506, 508) and for generating position signals e1, e2 to the controller 104. In an example, the controller 510 may determine the position of the user's head through stored software or using a neural network that has been trained to detect the position of the user's head based on the output of the head tracking device. In an alternative example, each head tracking device 506, 130 may include its own controller for implementing the functions of the controller 510. In yet another example, the controller 504 may directly receive the output of the head tracking devices 506, 508 and perform the processing of the controller 510.

[0092] The controller 504 receiving the position signals e1 and / or e2 may generate binaural signals b1 and / or b2 such that at least one of the binaural devices 110, 112 generates an acoustic signal that is perceived by the user as originating from a virtual point in the space within the vehicle cabin 100 rather than from the actual location of the speakers (e.g., speakers 118, 120) that generate the acoustic signal. For example, the controller 504 may generate the binaural signal b1 such that the binaural device 110 generates an acoustic signal that is perceived by the user sitting at the seated position P1 as originating from the spatial point SP1 (at Figure 5, because this is a virtual sound source). Similarly, the controller 504 can generate a binaural signal b2 so that the binaural device 112 generates an acoustic signal 116 that is perceived by the user sitting at the seated position P2 as originating from the spatial point SP2. This can be accomplished by filtering and / or attenuating the binaural signals b1, b2 according to a plurality of head-related transfer functions (HRTFs) that adjust the acoustic signals 114, 116 to determine the sound from the virtual spatial point (e.g., spatial points SP1, SP2). Since the signals are binaural, i.e., related to the two ears of the listener, the system can utilize one or more HRTFs to simulate sounds specific to various locations around the listener. It should be understood that the specific left and right HRTFs used by the controller 504 can be selected based on a given combination of azimuth and elevation angles detected between the relative positions of the left and right ears of the user and the corresponding spatial positions SP1, SP2. More specifically, a plurality of HRTFs may be stored in a memory and retrieved and implemented based on the detected positions of the left and right ears of the user and the selected spatial positions SP1, SP2. However, it should be understood that in the case where the binaural devices 110, 112 are open-ear wearable devices, the positioning of the user's ears may be replaced by or determined from the positioning of the open-ear wearable devices.

[0093] Despite Figure 5Two different spatial points SP1, SP2 are shown in , but it should be understood that the same spatial point can be used for two binaural devices 110, 112. In addition, for a given binaural device, any point in space can be selected as the spatial point from which the acoustic signal generated is virtualized. (The selected point in space can be a moving point in space, for example, to simulate an audio generating object in motion.) For example, the left channel audio signal, the right channel audio signal, or the center channel audio signal can be simulated as if they were generated at a location close to the peripheral speaker 102. In addition, the realism of the simulated sound can be enhanced by adding additional virtual sound sources at locations within the environment (i.e., the vehicle cabin 100) to simulate the effect that the sound generated at the virtual sound source location is reflected by an acoustic reflective surface and returned to the listener. Specifically, for each virtual sound source generated within the environment, additional virtual sound sources can be generated and placed at different locations to simulate first-order and second-order reflections of the sound, which correspond to the sound propagated from the first virtual sound source and acoustically reflected by the surface and propagated back to the listener's ears (first-order reflections), and the sound propagated from the first virtual sound source and acoustically reflected by the first and second surfaces and propagated back to the listener's ears (second-order reflections). The method of implementing HRTF and virtual reflections to create spatialized audio is discussed in more detail in US Patent Publication US2020 / 0037097A1 entitled "Systems and methods for sound source virtualization", the entire contents of which are incorporated herein by reference. In the example, the virtual sound source can be located outside the vehicle. Similarly, the first-order and second-order reflections do not need to be calculated for the actual surfaces within the vehicle, but can be calculated for the virtual surfaces outside the vehicle, for example, to create the impression that the user is in a larger area than the cabin, or at least to optimize the reverberation effect and quality of the sound for an environment better than the cabin of the vehicle.

[0094] The controller 504 combines Figure 1A and Figure 1BThe controller 104 is described in a manner that is further configured, that is, the bass content produced by the ambient speakers 102 can be used to enhance the spatialized acoustic signals 114, 116 (e.g., in a time-aligned manner). For example, the ambient speakers 102 can be used to generate the bass content of the first content signal u1, and the binaural device 110 generates the high-range content of the first content signal as the spatialized acoustic signal, which is perceived by the user at the seated position P1 as originating from the spatial position SP1. Although the bass content produced by the ambient speakers 102 in the first listening zone 106 may not be a stereo signal, the user sitting at the seated position P1 can still perceive the first content signal u1 as originating from the spatial position SP1. Similarly, the ambient speakers can enhance the bass content of the second content signal u2 in the second listening zone - the binaural device 112 generates the high-range of the second content signal as the spatialized acoustic signal. The user at the seating position P2 perceives the second content signal u2 as originating from the spatial position SP2 at the second listening zone, where the bass content is provided as a monophonic acoustic signal from the ambient loudspeaker 102 .

[0095] Despite Figure 5 Two binaural devices 110, 112 are shown in , but it should be understood that only a single spatialized binaural signal (e.g., binaural signal b1) can be provided to one binaural device. In addition, it is not necessary for each binaural device to provide a spatialized acoustic signal; instead, one binaural device (e.g., binaural device 110) can provide a spatialized acoustic signal, while another binaural device (e.g., binaural device 112) can provide a non-spatialized acoustic signal. In addition, as described above, each binaural device can receive the same binaural signal so that each user hears the same content, with the bass content of the content enhanced by the surrounding speakers 102 (which does not necessarily have to be produced in an independent listening zone). In addition, Figure 5 The example can be extended to any number of listening zones and any number of binaural devices.

[0096] The controller 504 may also implement an upmixer that receives, for example, a left program content signal and a right program content signal and generates left, right, center, etc. channels within the vehicle. The spatialized audio presented by the binaural device (e.g., binaural devices 110, 112) may be utilized to enhance the user's perception of the sources of these channels. Thus, in effect, multiple virtual sound sources may be selected to accurately create the impression of left, right, center, etc. audio channels.

[0097] Figure 6A flow chart of a method 600 for providing enhanced audio to a user in a vehicle cabin is depicted. The steps of the method 600 may be performed by a controller, such as the controller 504, in communication with a set of ambient speakers disposed in the vehicle, such as the ambient speakers 102, and further in communication with a set of binaural devices, such as the binaural devices 110, 112, disposed at respective seating locations within the vehicle.

[0098] At step 602, a content signal is received. The content signal may be received from a number of potential sources such as a mobile device, radio, satellite radio, cellular connection, etc. The content signal is an audio signal including bass content and high range content.

[0099] At step 604, a spatial audio signal is output to a binaural device according to a position signal indicating the position of the user's head in the vehicle, so that the binaural device generates a spatial acoustic signal perceived by the user as originating from a virtual source. The virtual source may be a selected position in the vehicle cabin, such as, in the example, a peripheral speaker close to the vehicle. This may be accomplished by filtering and / or attenuating the audio signal output to the binaural device according to a plurality of head-related transfer functions (HRTFs), which adjust the acoustic signal to simulate the sound from the virtual source (e.g., spatial points SP1, SP2). Since the signal is binaural, i.e., related to the two ears of the listener, the system may utilize one or more HRTFs to simulate the sounds specific to each location around the listener. It should be understood that the specific left HRTF and right HRTF used may be selected based on a given combination of azimuth and elevation angles detected between the relative positions of the left and right ears of the user and the corresponding spatial positions. More specifically, a plurality of HRTFs may be stored in a memory and retrieved and implemented based on the detected positions of the left and right ears of the user and the selected spatial positions.

[0100] The position of the user's head can be determined based on the output of a head tracking device (such as head tracking devices 506, 508), which may include, for example, a time-of-flight sensor, a lidar device, multiple two-dimensional cameras, a wearable inertial motion unit, a proximity sensor, or a combination of these components. In addition, other suitable devices are contemplated. The output of the head tracking device can be processed by a dedicated controller (e.g., controller 510), which can implement software or a neural network that is trained to detect the position of the user's head. hereinafter, in conjunction with Figures 8 to 9, Fig.12 Examples in which an inertial measurement unit is used are described in more detail.

[0101] At step 606, the ambient speakers are driven so that the bass content of the content signal is produced in the cabin. In this way, the spatial acoustic signal produced by the binaural device is enhanced by the ambient speakers in the vehicle cabin. Detecting the position of the user's head may include detecting any part of the user or any part of a wearable device worn by the user (from which the corresponding position of the user's ears or the position of the wearable device worn by the user can be derived), including directly detecting the position of the user's ears or directly detecting the position of the wearable device.

[0102] Although method 600 describes a method for enhancing a spatial acoustic signal provided by a single binaural device, method 600 can be extended to enhance multiple content signals provided by multiple binaural devices by arranging ambient speakers to produce bass content of corresponding content signals in different listening zones throughout the cabin. Figure 1A and Figure 1B The steps of this method are described.

[0103] Fig. 8A An example is depicted in which a user orientation sensor is used for head tracking. The user orientation sensor 802 is disposed on a wearable device worn on the user's head (the user orientation sensor may be included in the wearable device) and outputs a user orientation signal m1. As used in the present disclosure, the orientation sensor is composed of a sensor or multiple sensors suitable for detecting the orientation of the body. The output signal of the orientation sensor may directly represent the orientation (e.g., as a change in the pitch, roll, and yaw of the body) or may contain other data from which the orientation can be derived, such as acceleration, specific force, or angular rate in one or more directions (further contemplated to be other suitable types of data). In one example, the orientation sensor may be an inertial measurement unit. The inertial measurement unit may include, for example, an accelerometer, a gyroscope, and / or a magnetometer, and may output a signal directly representing the orientation or representing other measurements (such as specific force and angular rate). Alternatively, in addition to the inertial measurement unit, sensors such as accelerometers, gyroscopes, and / or magnetometers may be used to determine the orientation of the user. The user orientation signal m1 may be used by the controllers 510, 504 to provide a spatial audio signal b1 to the binaural device. (As described above, in various alternative examples, a single controller, such as controller 504, may be used to process position signals from user orientation sensor 802 and output spatial signals to the binaural device. Other architectures and combinations of controllers are contemplated herein.)

[0104] The wearable device on which the user orientation sensor 802 is disposed may be the binaural device 110 (ie, when the binaural device 110 is a wearable device, such as, for example, Figure 2 and Figure 3However, in alternative examples, the binaural device 110 may be disposed elsewhere, such as in a headrest, and the user orientation sensor 802 may be worn on the user's head in other ways.

[0105] However, while the user orientation sensor 802 converts the motion from the user's head into an orientation signal, the motion introduced by the vehicle (e.g., motion caused by turning or bumping) will likewise be picked up by the user orientation sensor 802. In order to distinguish between the motion of the vehicle and the motion of the user's head, a separate signal representing the vehicle orientation (vehicle orientation signal m2) can be used to isolate the orientation change introduced by the motion of the user's head from the orientation change introduced by the motion of the vehicle. In other words, the orientation change common to the vehicle orientation signal m2 and the user orientation signal m2 can be attributed to the vehicle, and thus the motion of the user's head can be isolated by finding the difference between the user orientation signal m1 and the vehicle orientation signal m2. The controller 510 can therefore determine the orientation of the user's head relative to the vehicle (i.e., the vehicle acts as a reference frame for the head motion) by finding the difference between the user orientation signal m1 and the vehicle orientation signal m2. As described in the present disclosure, the difference can include a three-dimensional difference and can be found by subtraction (including vector subtraction or its equivalent), but other methods of finding the difference can be used, such as by a machine learning algorithm.

[0106] Any suitable input representing the orientation (or change in orientation) of the vehicle may be used. Figure 8B As shown, the vehicle orientation sensor 804 can be set on the vehicle (the vehicle orientation sensor can be included in the vehicle) and output the vehicle orientation signal m2. For optimal performance, the vehicle orientation sensor 804 must be fixed to the vehicle so that changes in the vehicle orientation are picked up by the vehicle orientation sensor 804. In one example, the vehicle orientation sensor 804 can be attached to a location inside the vehicle or to an outer surface of the vehicle. The vehicle orientation sensor 804 can be fixed to the vehicle during manufacturing, or alternatively, can be retrofitted to the vehicle by the user. For example, the vehicle orientation sensor 804 can be set in a mouse-shaped locator or mobile device that, for example, the user brings into the vehicle and attaches to a fixed location (such as a dashboard or a center console).

[0107] Other suitable sources of vehicle orientation signal m2 include inputs representing vehicle parameters such as speed, acceleration, steering angle, etc., which can also be used to determine changes in vehicle orientation. Figure 8CAs shown, these parameters may be received by controller 510 from vehicle control unit 808 and used to calculate a change in the orientation of the vehicle, which may then be subtracted (or otherwise removed, such as by a machine learning algorithm) from the user orientation signal m1 to isolate the orientation of the user's head. Other potentially suitable inputs include navigation data (e.g., as determined from a GPS or cellular signal), or camera data (e.g., as used in an autonomous vehicle) may be used as input to determine the orientation of the vehicle.

[0108] In the case where two orientation sensors are used to track the movement of the user's head and the vehicle (e.g., Figure 8B As shown in FIG. 1 , a small error in the measured orientation may generally appear as a drift between the measured orientation of the user's head and the vehicle. This drift results in an error in the measured orientation of the user's head, which means that the user will perceive the virtual sound source (e.g., SP1) as being incorrectly positioned and drifting relative to the user's head.

[0109] For the purpose of explanation, 9A to 9C A simplified example of the relative drift between the measured yaw of the user's head and the measured yaw of the vehicle is depicted. Fig. 9A Depicted are the properly calibrated and measured orientations of the user and vehicle. In this example, the two measured orientations (m1, m2) point in the same direction as the orientations of the vehicle and user (depicted as dashed lines). Fig. 9B In , the orientations represented by the user orientation signal m1 and the vehicle orientation signal m2 both deviate from the correct orientation by approximately 45°. Although the measured orientation does not match the actual orientation, the spatial audio signal will be correctly rendered because the two measured orientations have the same error. This is because the vehicle is used as a reference frame for spatialized audio. However, in Fig. 9C In the example, the orientations represented by the user orientation signal m1 and the vehicle orientation signal m2 have drifted relative to each other. In this example, since the virtual sound source is positioned at a point relative to the vehicle, the user will perceive the virtual sound source as being incorrectly positioned in space (i.e., in a wrong position in the vehicle).

[0110] To correct for the relative drift between the user orientation signal m1 and the vehicle orientation signal m2, the individual error sensors may be periodically sampled to locate the orientation of the user's head relative to the vehicle. For example, if the controller 510 determines from the orientation sensors 802, 804 that the orientation of the user's head relative to the vehicle is angled (e.g., as if the user is looking out a window), but the error sensors determine that the orientation of the user's head relative to the vehicle is not angled (e.g., the user is looking straight ahead), the controller 510 may correct the drift measured by the orientation sensors 802, 804 by removing the drift (e.g., by subtraction or by other methods such as machine learning).

[0111] Because the relative drift between the user orientation signal m1 and the vehicle orientation signal m2 may take some time to accumulate, the error sensor does not need to be sampled as fast as the user orientation signal m1 and the vehicle orientation signal m2 are sampled. For example, if the user orientation signal m1 and the vehicle orientation signal m2 are sampled every millisecond, the error sensor can be sampled every second. (These sampling rates are provided as examples only and other suitable sampling rates can be used.) For example, if Figure 8B As shown, a head tracking device 506 such as a time-of-flight sensor, a lidar device, or one or more cameras may be used as an error sensor. Using these types of sensors to determine the orientation of the user's head is computationally expensive, and therefore by only using these sensors to correct for drift, they can be sampled at a slower rate, thereby saving computational resources.

[0112] In another example, the error sensor may be at least one microphone located on the wearable device. In one example, the error sensor may be two or more microphones disposed on opposite sides of the user's head. The orientation of the wearable device may be calculated by measuring the delay between receiving an acoustic signal at each of the microphones. FIG. 10A to FIG. 10B An example of this is shown in FIG. As shown, a wearable device (here, a pair of Bose Frames 200) includes microphones 1002a and 1002b disposed on opposite temples, respectively. The acoustic signal is emitted from a speaker (such as a peripheral speaker 102a) (although any suitable speaker may be used). Fig. 10A In the orientation of the frame 200 depicted in FIG, the microphone 1002a is approximately a distance d1 from the speaker 102a, and the microphone 1002b is approximately a distance d2 from the speaker 102a. Therefore, the microphone 1002a will receive the acoustic signal generated by the speaker 102a before the microphone 1002b receives the acoustic signal from the speaker 102a. In other words, there will be some delay between the microphone 1002a and the microphone 1002b receiving the same acoustic signal generated by the speaker 102a.

[0113] If the user's head turns, such as Fig. 10B As shown, the distance between microphones 1002a, 1002b and speaker 102a changes. In this example, the distance from each microphone 1002a, 1002b to speaker 102a becomes approximately the same d3, and therefore the same signal will be received at approximately the same time. The lack of delay indicates that the user is away from (or toward the periphery) speaker 102a. The delay between the reception of the acoustic signal thus becomes a representative for measuring the distance between each microphone 1002a, 1002b and speaker 102a.

[0114] By monitoring the delay (including the lack of delay) of the common acoustic signal received from each microphone disposed on the opposite side of the user's head, the orientation of the user's head relative to the speaker can be determined. The controller 510 can, for example, therefore receive the output of microphones 1002a, 1002b and implement a lookup table or perform calculations to convert the delay into the orientation of the user's head relative to the vehicle (because the speaker is in a known position in the vehicle). For example, the time when the same signal is received can be determined by performing a similarity metric calculation (e.g., cross-correlation) between the samples of microphones 1002a and 1002b. This method for detecting orientation is particularly useful for determining the yaw of the user's head relative to the vehicle. Similar to other examples of detecting the orientation of the user's head, this example can be used to correct the drift of the relative orientation detected by orientation sensors 802, 804.

[0115] In an alternative example, a single microphone (rather than multiple microphones) can be used on the wearable device to detect the orientation of the user's head. By comparing the arrival times of acoustic signals from two or more sound sources (e.g., peripheral speakers 102) set in a known position, the orientation of the microphone relative to the known speaker can be determined. If the arrival time of the first acoustic signal is earlier than the arrival time of the second acoustic signal, it can be determined that the source generating the first acoustic signal is closer to the microphone setting than the source generating the second acoustic signal, provided that the acoustic signals are generated simultaneously. (Alternatively, if the time when the acoustic signals are generated or the delay between the generation of the acoustic signals is known, the acoustic signals do not need to be generated simultaneously.)

[0116] These and additional examples of using one or more microphones to detect the position or orientation of a wearable device are further described in US2022 / 0082688, entitled “Methods and systems for determining position and orientation of a device using acoustic beacons,” which is hereby incorporated by reference in its entirety.

[0117] In another example, rather than using an error sensor, the difference in the output of a discrete sensor located within each user orientation sensor 802, 804 may be used to determine the drift between orientation sensors. Fig.11A and Fig. 11B An example of this is described in the figures which depict simplified representations of the individual output signals of a plurality of accelerometers arranged orthogonally to one another within each orientation sensor. (Such an arrangement is common in inertial measurement units, which, as described above, are examples of suitable orientation sensors.) More specifically, Fig.11AThe outputs of three accelerometers within the user orientation sensor 802 are depicted, with each accelerometer being positioned to detect acceleration in a particular direction, as represented by the three-dimensional axes. Thus, curve 1102 depicts the output of an accelerometer that is configured to detect acceleration on the z-axis, curve 1104 depicts the output of an accelerometer that is configured to detect acceleration on the x-axis, and curve 1106 depicts the output of an accelerometer that is configured to detect acceleration on the y-axis. Similarly, Fig. 11B 100 and 101 depict outputs of accelerometers disposed within vehicle orientation sensor 804. Thus, curve 1108 depicts the output of an accelerometer disposed to detect acceleration on the z-axis, curve 1110 depicts the output of an accelerometer disposed to detect acceleration on the x-axis, and curve 1112 depicts the output of an accelerometer disposed to detect acceleration on the y-axis.

[0118] Comparing which accelerometer of each orientation sensor detects a common signal (e.g., acceleration resulting from motion) can reveal the relative orientation of the user orientation sensor 802 relative to the vehicle orientation sensor 804. For example, Fig.11A and Fig. 11B As shown, the same measured acceleration, such as caused by a vehicle hitting a bump in the road, is detected by the z-axis accelerometer in the user orientation sensor 802 (curve 1102), but by the x-axis accelerometer of the vehicle orientation sensor 804 (curve 1110). It can therefore be determined that the z-axis accelerometer of the user orientation sensor 802 is pointing in the same direction as the x-axis accelerometer of the vehicle orientation sensor 804. Therefore, the controller 510 can determine the relative orientation of each user orientation sensor 802 and 804 by comparing the similarities in the output of each accelerometer. If two accelerometers pick up the same acceleration, it can be assumed that they are pointing in the same direction.

[0119] The similarity between the outputs of each accelerometer can be determined in any suitable manner. For example, a cross-correlation can be found between each accelerometer. Therefore, a cross-correlation can be found between the output of the x-axis accelerometer of the user orientation sensor 802 and each of the outputs of the x-axis, y-axis, and z-axis accelerometers of the vehicle orientation sensor 804. This can be repeated for the y-axis accelerometer and the z-axis accelerometer of the user orientation sensor 802 to find a complete mapping of the measure of similarity between each accelerometer. In most cases, it is impossible for one accelerometer to be completely aligned with another accelerometer, and therefore the relative orientation of the user orientation sensor 802 relative to the vehicle orientation sensor 804 can be determined by comparing the degree of similarity that each accelerometer output has with each other accelerometer output. By finding the degree of similarity between each accelerometer output, the controller 510 can find the relative orientation of the user orientation sensor 802 relative to the vehicle orientation sensor 804. In practice, a lookup table can be used to compare the similarity measure between each accelerometer to find the relative orientation.

[0120] Although the above examples use accelerometer outputs, it should be understood that any type of sensor output can be compared in this manner. For example, an inertial measurement unit typically also includes three orthogonally positioned gyroscopes and three orthogonally positioned magnetometers. The similarity of the outputs of these sensors can also be compared to determine the relative orientation of the user orientation sensor 802 and the vehicle orientation sensor 804. In addition, the outputs of other types of sensors other than those within the inertial measurement unit or the outputs of the same type of sensors outside the inertial measurement unit can be used in a similar manner to determine the relative orientation of the user orientation sensor 802 and the vehicle orientation sensor 804.

[0121] Because the similarities in the outputs of the comparison sensors are not subject to previous measurements, this method of finding the relative orientation of the user orientation sensor 802 with respect to the vehicle orientation sensor 804 is not affected by drift. Therefore, this method can be used to correct for relative drift between the orientation sensors 802 and 804. Alternatively, the relative orientation can be found exclusively in this manner, rather than relying on the orientation output from the orientation sensors 802, 804, but because this is more computationally intensive, it is more suitable for correcting errors.

[0122] It should be understood that multiple orientation sensors (e.g., Figure 8B The additional user orientation sensor 806 shown in FIG. 1 can be used to track the orientation of additional passengers (e.g., passengers sitting at position P2) within the vehicle. In addition, as described above, the surrounding speakers 102 can be used to create a separate bass zone so that users sitting in separate seats can experience spatial audio using a binaural device with bass enhanced by the surrounding speakers.

[0123] Fig.12 A flow chart of a method 1200 for providing spatialized audio to binaural devices within a vehicle based on the orientation of a user's head relative to the vehicle is depicted. The steps of method 1200 may be performed by a controller (such as controller 504) or a combination of controllers (such as controllers 510 and 504) that communicates with a set of ambient speakers (such as ambient speakers 102) disposed in the vehicle and further communicates with a set of binaural devices (such as binaural devices 110, 112) disposed at corresponding seating locations within the vehicle.

[0124] At step 1202, a user orientation signal output from a user orientation sensor disposed on a wearable device is received, the wearable device moving with the head of a first user during use. In various examples, the wearable device may be a binaural device worn by the user (e.g., Figure 2 and Figure 3 ). However, in an alternative example, the wearable device may be separate from the binaural device (which may be located away from the user, such as in a headrest).

[0125] At step 1204, a vehicle orientation signal is received. The vehicle orientation may be received from a second orientation sensor. The vehicle orientation sensor may be provided within the vehicle (e.g., during manufacturing) or may be brought into the vehicle, such as in a mobile device or mouse-like locator, and secured to the vehicle. Alternatively, a separate source of vehicle orientation may be used, such as an input received from a vehicle control unit having data representing vehicle parameters such as speed, acceleration, and steering angle. However, it is contemplated that other inputs such as navigation data or camera data for an autonomous vehicle may be used alone or in combination with other methods to determine the orientation of the vehicle.

[0126] At step 1206, the orientation of the user's head relative to the vehicle is based at least on the difference between the vehicle orientation signal and the user orientation signal. To isolate the movement of the user's head relative to the vehicle, a difference may be found between the user orientation signal and the vehicle orientation signal. The difference may be found by, for example, subtraction (including subtraction of vectors), but other methods of finding the difference may be used, such as using a machine learning algorithm.

[0127] At step 1208, a first spatial audio signal is output to a first binaural device based on the orientation of the user's head relative to the vehicle, such that the first binaural device generates a first spatial acoustic signal perceived by the first user as originating from a first virtual source localized within the vehicle cabin. The position signal representing the orientation of the user's head relative to the vehicle provides a basis for localizing the spatial audio signal within the vehicle such that the user perceives it as fixed relative to the vehicle. (This method may be repeated for any number of users wearing orientation sensors. As described above, various head orientations of individual users may be used to provide spatialized audio to different users located in separate bass zones throughout the cabin, such as created by arrayed ambient speakers.)

[0128] At step 1210, the drift between the user orientation signal and the vehicle orientation signal is corrected based on the detected orientation of the user's head. This can be achieved by periodically sampling an error sensor (such as a time-of-flight sensor, a lidar device, or one or more cameras) to determine the position of the user's head relative to the vehicle. Alternatively, the error sensor can be one or more microphones disposed on one or opposite sides of the user's head for detecting the reception delay of the same signal or different acoustic signals generated at a known time by a sound source disposed in a known position.

[0129] In another example, analysis of common signals between sensors may reveal the orientation of one orientation signal relative to another orientation signal. This may be accomplished by comparing a similarity (e.g., cross-correlation) measure between sensors of a user orientation sensor and sensors of a vehicle orientation sensor. The correlation between signals of different sensors generally corresponds to the degree to which each sensor is set in a common orientation. Thus, the orientation of two orientation sensors may be determined by analysis of the similarity of signals of different sensors.

[0130] The orientation as determined by the error sensor or by analysis of the signals common to the orientation sensors may be used to correct for drift in the orientation of the user's head detected by the user orientation sensor (e.g., by subtraction or by another other method, such as machine learning). Because drift accumulation takes some time, the error sensor may be sampled at a slower rate than the rate at which the user orientation sensor (and the vehicle orientation sensor) disposed on the user's head is typically sampled, or the orientation may be determined separately in other ways.

[0131] The functionality or parts thereof described herein, as well as various modifications thereof (hereinafter referred to as "functionality") may be implemented at least in part via a computer program product, such as a computer program tangibly embodied in an information carrier, such as one or more non-transitory machine-readable media or storage devices, for execution by one or more data processing devices, such as programmable processors, computers, multiple computers and / or programmable logic components, or for controlling the operation of one or more data processing devices, such as programmable processors, computers, multiple computers and / or programmable logic components.

[0132] A computer program may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may be deployed on one computer or executed on multiple computers at one site or distributed across multiple sites and interconnected by a network.

[0133] The actions associated with implementing all or part of the functions may be performed by one or more programmable processors executing one or more computer programs to perform the functions of the calibration process. All or part of the functions may be implemented as dedicated logic circuits, such as FPGAs and / or ASICs (application-specific integrated circuits).

[0134] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Generally speaking, a processor will receive instructions and data from a read-only memory or a random access memory, or both. The components of a computer include a processor for executing instructions and one or more memory devices for storing instructions and data.

[0135] Although several invention embodiments have been described and shown herein, a person of ordinary skill in the art will readily envision a variety of other components and / or structures for performing the functions described herein and / or obtaining one or more of the results and / or advantages described herein, and each of such variations and / or modifications is considered to be within the scope of the invention embodiments described herein. More generally, a person of ordinary skill in the art will readily understand that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications using the teachings of the present invention. A person of ordinary skill in the art will recognize or be able to determine many equivalents of the specific invention embodiments described herein using only routine experiments. Therefore, it should be understood that the above embodiments are presented only by way of example, and within the scope of the appended claims and their equivalents, the invention embodiments may be practiced in a manner different from that specifically described and claimed. The invention embodiments disclosed herein relate to each individual feature, system, article, material, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials and / or methods is included within the inventive scope of the present disclosure if such features, systems, articles, materials and / or methods are not mutually inconsistent.

Claims

1. A system for providing spatialized audio in a vehicle, the system comprising: a vehicle orientation sensor, the vehicle orientation sensor outputting a vehicle orientation signal and being disposed on the vehicle; and A controller, the controller being configured to receive a user orientation signal output from a user orientation sensor provided on a wearable device, the wearable device moving with the head of a first user during use, wherein the controller is further configured to determine the orientation of the user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the user orientation signal, the controller being further configured to output a first spatial audio signal to a first binaural device based on the orientation of the user's head relative to the vehicle, so that the first binaural device generates a first spatial acoustic signal perceived by the user as originating from a first virtual source localization within a cabin of the vehicle.

2. The system according to claim 1 further comprises an error sensor configured to detect the orientation of the user's head relative to the vehicle and output an error sensor signal, wherein the controller is further configured to correct a drift between the user orientation signal and the vehicle orientation signal according to the orientation of the user's head detected by the error sensor.

3. The system of claim 2, wherein the error sensor comprises at least one of: a time-of-flight sensor, a lidar device, a camera, or a microphone disposed on the user's head. 4 . The system of claim 2 , wherein the controller samples the error sensor signal at a rate that is slower than a rate at which the vehicle orientation signal and the user orientation signal are sampled.

5. The system of claim 1 , wherein the vehicle orientation sensor comprises a first plurality of sensors, wherein the user orientation sensor comprises a second plurality of sensors, wherein the controller is further configured to correct a drift between the user orientation signal and the vehicle orientation signal based on a measure of similarity between at least one sensor of the first plurality of sensors and one sensor of the second plurality of sensors.

6. The system according to claim 1, wherein the controller comprises a first controller and a second controller, the first controller receives the vehicle orientation signal and the user orientation signal and determines the orientation of the user's head relative to the vehicle and outputs a position signal to the second controller, and the second controller receives the position signal and outputs the first spatial acoustic signal to the first binaural device.

7. The system of claim 1, wherein the wearable device is the first binaural device.

8. The system according to claim 1, further comprising: a plurality of speakers disposed in a periphery of a cabin of the vehicle, The first spatial audio signal at least includes a high frequency range of the first content signal. The controller is further configured to drive the plurality of speakers using a driving signal such that a first bass content of the first content signal is generated in the cabin.

9. A system according to claim 8, wherein the controller is configured to receive a second user orientation signal output from a second user orientation sensor provided on a second wearable device, the second wearable device moving with the head of the second user during use, wherein the controller is further configured to determine the orientation of the second user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the second user orientation signal, the controller is further configured to output a second spatial audio signal to a second binaural device based on the orientation of the second user's head relative to the vehicle, so that the second binaural device generates a second spatial acoustic signal perceived by the second user as originating from the first virtual source localization or the second virtual source localization within the cabin of the vehicle.

10. A system according to claim 9, wherein the second spatial audio signal includes at least a high frequency range of a second content signal, wherein the controller is further configured to drive the plurality of speakers according to a first array configuration so as to produce the first bass content in a first listening zone in the cabin, and to drive the plurality of speakers according to a second array configuration so as to produce second bass content of the second content signal in a second listening zone in the cabin, wherein in the first listening zone, the amplitude of the first bass content is greater than the amplitude of the second bass content, and in the second listening zone, the amplitude of the second bass content is greater than the amplitude of the first bass content.

11. The system of claim 1, wherein the vehicle orientation sensor is: integrated with the vehicle or brought into and secured to the vehicle by a user.

12. A method for providing spatialized audio in a vehicle, the method comprising: receiving a user orientation signal output from a user orientation sensor disposed on a wearable device, the wearable device moving with a first user's head during use; receiving a vehicle orientation signal from a vehicle orientation sensor disposed on the vehicle; determining an orientation of the user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the user orientation signal; as well as Outputting a first spatial audio signal to a first binaural device according to the orientation of the user's head relative to the vehicle causes the first binaural device to produce a first spatial acoustic signal perceived by the user as originating from a first virtual source localization within a cabin of the vehicle.

13. The method according to claim 12, further comprising: receiving an error sensor signal output from an error sensor configured to detect the orientation of the user's head relative to the vehicle; as well as A drift between the user orientation signal and the vehicle orientation signal is corrected based on the orientation of the user's head detected by the error sensor.

14. The method of claim 13, wherein the error sensor comprises at least one of: a time-of-flight sensor, a lidar device, a camera, or a microphone disposed on the user's head.

15. The method of claim 13, wherein the error sensor signal is sampled at a slower rate than a rate at which the vehicle orientation signal and the user orientation signal are sampled.

16. The method according to claim 12 further includes a step of correcting a drift between the user orientation signal and the vehicle orientation signal, wherein the vehicle orientation sensor includes a first plurality of sensors, wherein the user orientation sensor includes a second plurality of sensors, wherein the drift is corrected based on a measure of similarity between at least one sensor of the first plurality of sensors and one sensor of the second plurality of sensors. The method of claim 12 , wherein the wearable device is the first binaural device.

18. The method of claim 12, further comprising driving a plurality of speakers with a drive signal such that a first bass content of the first spatial audio signal is produced in the chamber.

19. The method according to claim 18, further comprising receiving a second user orientation signal output from a second user orientation sensor disposed on a second wearable device, the second wearable device moving with the head of the second user during use, determining an orientation of the second user's head relative to the vehicle based at least on a difference between the vehicle orientation signal and the second user orientation signal, A second spatial audio signal is output to a second binaural device based on the orientation of the second user's head relative to the vehicle, so that the second binaural device generates a second spatial acoustic signal perceived by the second user as originating from the first virtual source localization or the second virtual source localization within the cabin of the vehicle.

20. The method according to claim 19, further comprising: The multiple speakers are driven according to a first array configuration so as to generate the first bass content in a first listening zone in the cabin, and the multiple speakers are driven according to a second array configuration so as to generate second bass content of the second spatial audio signal in a second listening zone in the cabin, wherein in the first listening zone, the amplitude of the first bass content is greater than the amplitude of the second bass content, and in the second listening zone, the amplitude of the second bass content is greater than the amplitude of the first bass content.

Images