How to process audio for immersive audio playback

Abstract

1. A method (1000) of generating multiple audio channels from audio including height and non-height audio channels for reproduction on an immersive loudspeaker system having at least one height loudspeaker (5) in a listening environment, the method comprising: applying (1500) a virtual height filter (1300) to a portion of each height channel (1010) to attenuate spectral components of the height channel that emanate directly from the loudspeaker (1;2;3;4) and amplify spectral components of the height channel that are reflected from a roof or an area near the roof in the listening environment when reproducing the height channel by one of the loudspeakers to generate multiple virtual height filtered audio signals that are added to corresponding non-height audio channels for reproduction by a corresponding loudspeaker; and reproducing the remaining portion of each height audio channel on the at least one height loudspeaker (5).

Description

[Technical Field]

[0001] Cross-references to related applications This application claims priority to the following priority applications: U.S. Provisional Application No. 63 / 291,598 (reference number: D21147AUSP1) filed on 20 December 2021, U.S. Provisional Application No. 63 / 353,778 (reference number: D21147AUSP2) filed on 20 June 2022, and European Patent Application No. 22179943.0 (reference number: D21147AEP) filed on 20 June 2022.

[0002] Technical field This disclosure relates to the field of audio processing. In particular, this disclosure relates to a method for generating at least two audio channels from audio of an immersive audio format, which includes multiple front and rear height audio channels and multiple front and rear non-height audio channels, for playback of multiple audio channels in an immersive loudspeaker system. This disclosure further relates to an apparatus comprising a processor configured to perform this method, a vehicle comprising said apparatus, a program, and a computer-readable storage medium. [Background technology]

[0003] Vehicles typically include a loudspeaker system for audio playback. The in-vehicle loudspeaker system may be used to play audio from sources such as tapes, CDs, audio streaming services, or applications running remotely within the vehicle's in-car entertainment system or via a device connected to the vehicle. The device may be a portable device connected to the vehicle wirelessly or via cable. More recently, streaming services such as Spotify and Tidal are integrated into the in-car entertainment system, either directly to the vehicle's hardware (commonly known as the "head unit") or via a smartphone using Bluetooth®, Apple CarPlay, or Android Auto. The in-vehicle loudspeaker system may also be used to play terrestrial and / or satellite radio. A conventional loudspeaker system for vehicles is a stereo loudspeaker system. A stereo loudspeaker system may include a total of four loudspeakers: a pair of front loudspeakers and a pair of rear loudspeakers for the front and rear passengers, respectively. More recently, however, with the introduction of DVD players in vehicles, surround loudspeaker systems have been introduced to vehicles to support playback of the DVD audio format.

[0004] Immersive audio is becoming mainstream in movie theaters and home listening environments. Given its prevalence in these settings, it's natural to assume that immersive audio will also be played in vehicles. Dolby Atmos Music is already available through various streaming services. Immersive audio is often distinguished from surround audio formats by including one or more overhead or height audio channels. Therefore, overhead or height loudspeakers are used to play immersive audio.

[0005] Figure 1 shows an interior view of vehicle 100. Vehicle 100 includes a loudspeaker system including loudspeakers 10, 11, 30, 31, 41, 42, 43, and 44. The loudspeakers are shown only on the left side of vehicle 100. Except for loudspeakers 10 and 11, corresponding loudspeakers may be symmetrically arranged on the right side of vehicle 100. In particular, the loudspeaker system in Figure 1 includes tweeter loudspeakers 41, 42, and 43, full-range front loudspeakers 30 and rear loudspeakers 31, a center loudspeaker 10, a low-frequency loudspeaker or subwoofer 11, and a height loudspeaker 44. The tweeter loudspeaker 41 is located near the vehicle's dashboard. The tweeter loudspeaker 42 is located low on the front pillar of vehicle 100. However, not only the tweeter loudspeakers 41, 42, and 43, but also the full-range front loudspeakers 30 and rear loudspeakers 31 can be placed in any position suitable for the particular implementation. The height loudspeaker 44 is placed on the roof of the vehicle, in this example, above the driver's seat.

[0006] High-end vehicles may include multiple overhead or height-of-speaker loudspeakers, but some vehicles still use a limited number of height-of-speaker stereo loudspeakers. In fact, height-of-speaker loudspeakers dramatically increase the complexity of the loudspeaker system within a vehicle. Height-of-speaker loudspeakers typically need to be placed in the roof of a vehicle that is not suited to this purpose. For example, vehicles typically have low roofs that limit the available height for placing height-of-speaker loudspeakers. Furthermore, vehicles are often sold with the option of installing a sunroof to expose windows in the roof, and incorporating or placing many height-of-speaker loudspeakers in the roof presents a difficult industrial design challenge. Additional audio cables may also be required for such height-of-speaker loudspeakers. The placement of multiple overhead or height-of-speaker loudspeakers also raises concerns when considering vehicle safety and the placement of, for example, airbags in the same or nearby location as the overhead or height-of-speaker loudspeakers. For all these reasons, integrating multiple height-of-speaker loudspeakers within a vehicle can be costly due to space and industrial design constraints. On the other hand, using only a limited number of height loudspeakers dramatically limits the spatial resolution of sound in the vehicle's height plane. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] It would be advantageous to play immersive audio content in an immersive loudspeaker system having at least one height loudspeaker. In the context of this disclosure, “immersive loudspeaker system” is a loudspeaker / speaker system comprising at least one height loudspeaker (for example, located above the listener’s head, i.e., in the so-called height plane) and a plurality of front and rear non-height loudspeakers (for example, located below the listener’s head, i.e., in the so-called base plane).

[0008] To improve the user's audio experience without using more than two overhead loudspeakers, it is advantageous to create a perception of pitch with satisfactory spatial resolution by playing immersive audio content on an immersive loudspeaker system that includes at least one height loudspeaker. [Means for solving the problem]

[0009] Aspects of this disclosure provide a method for generating multiple audio channels from audio in an immersive format, which includes multiple front and rear height audio channels and multiple front and rear non-height audio channels, in order to reproduce multiple audio channels in an immersive loudspeaker system including at least one height loudspeaker and multiple front and rear non-height loudspeakers. The method includes applying a virtual height filter to a portion of each of the multiple front and rear height audio channels. The virtual height filter is configured to at least partially attenuate the spectral components of the at least one height channel that emanates directly from the loudspeaker from which the height channel is reproduced when at least one of the front and rear height audio channels is reproduced by the corresponding loudspeaker among the front and rear non-height loudspeakers. The virtual height filter is also configured to at least partially amplify the spectral components of the height channels reflected from the roof or an area near the roof in the listening environment in order to generate multiple front and rear virtual height filtered audio signals. The method further includes generating an aggregated audio signal for playback by a corresponding front or rear non-height loudspeaker, by adding each of the front and rear virtual height filtered audio signals of a plurality of front and rear non-height filtered audio signals to the corresponding front and rear non-height audio channels of the plurality of front and rear non-height audio channels. The method further includes playing the remaining portions of each of the front and rear height audio channels of the plurality of front and rear height channels using at least one height loudspeaker.

[0010] In the context of this disclosure, the term “channel” means an audio signal to which metadata is optionally added, in which position is encoded as a channel identifier, e.g., front left or upper right surround. “Channel-based audio” is audio formatted for playback through a given set of loudspeaker zones having associated nominal positions, e.g., 5.1, 7.1, etc. The term “object” or “object-based audio” means one or more audio channels having a parametric source description, such as apparent source position (e.g., 3D coordinates) or apparent source width.

[0011] When the aforementioned portion of the height channel is reproduced from one of the non-height loudspeakers without filtering, the sound can radiate along various paths. Some sound can radiate along a direct path from the non-height loudspeaker to the listening position (for example, to the ears of a passenger or driver). Other parts of the sound can radiate along a reflected path from the on-height loudspeaker to the listening position. For example, some sound can be reflected from the roof or an area near the roof inside the listening area and therefore radiated from the roof or an area near the roof to the listening position. Sound radiated along a direct path is undesirable when the height channel is reproduced by a non-height loudspeaker. By applying a virtual height filter to the portions of the height channel, the spectral components of the portions of the height channel reflected from the roof or near the roof are amplified, while the spectral components of the portions of the height channel directly emitted towards the loudspeaker are attenuated. Configured as described above, this method compensates for undesirable direct sound and introduces perceptual height cues to the audio signal supplied to the non-height loudspeaker, thereby improving the positioning and perceived quality of the virtual height signal. For example, a directional auditory model has been developed to create a virtual height filter that improves the perceived quality of playback when used to process audio played by a non-height loudspeaker. In addition, by directly playing the rest of the height audio channel using at least one height loudspeaker, a desired balance of front and rear height audio channels between the non-height loudspeaker and at least one height loudspeaker can be achieved for improved spatial resolution in the height plane, providing an improved immersive listening experience. Improved spatial resolution in the height plane can be achieved by using a single height loudspeaker or only two height loudspeakers.

[0012] In one embodiment, a portion of each of the multiple front and rear height channels may be a function of one or more of the positions of the at least one height loudspeaker along the length of the listening environment, or a desired balanced distribution of multiple audio channels between the loudspeakers of an immersive loudspeaker system.

[0013] In one embodiment, applying a virtual height filter to a portion of each of a plurality of front and rear height audio channels may include applying the virtual height filter to a portion of the front height audio channels that are proportional to the position of the at least one height loudspeaker that moves from the front of the listening environment towards the rear.

[0014] In one embodiment, applying a virtual height filter to a portion of each of a plurality of front and rear height audio channels may include applying the virtual height filter to a portion of the rear height audio channels that are proportional to the position of the at least one height loudspeaker that moves from the rear to the front of the listening environment.

[0015] In one embodiment, using the at least one height loudspeaker to reproduce the remaining portions of each of the multiple front and rear height audio channels may include reproducing the remaining portions of each of the multiple front height audio channels in proportion to the position of the at least one height loudspeaker, which moves from the rear of the listening environment towards the front.

[0016] In one embodiment, using the at least one height loudspeaker to reproduce the remaining portions of each of the multiple front and rear height channels may include reproducing the remaining portions of each of the multiple rear channels in proportion to the position of the at least one height loudspeaker, which moves from the front of the listening environment towards the rear.

[0017] In one embodiment, the audio of the immersive audio format may further include front-right and front-left non-height audio channels, rear-right and rear-left non-height audio channels, upper-front-right and upper-front-left height audio channels, and upper-rear-right and upper-rear-left height audio channels. Front virtual height filtered audio signals may be added to the corresponding front-right and front-left non-height audio channels. Rear virtual height filtered audio signals may be added to the corresponding rear-right and rear-left non-height audio channels. These additions produce four added audio signals for playback by the corresponding front-right and front-left, as well as rear-right and rear-left non-height loudspeakers.

[0018] In one embodiment, the immersive loudspeaker system may include a single overhead loudspeaker.

[0019] In one embodiment, the upper front right height audio channel, the upper rear right height audio channel, the upper front left height audio channel, and the remaining portion of the upper rear left height audio channel may be combined and played by a single overhead loudspeaker.

[0020] In one embodiment, the immersive loudspeaker system may include two overhead loudspeakers, for example, an overhead right height loudspeaker and an overhead left height loudspeaker.

[0021] In one embodiment, the upper front right height audio channel and the remaining portion of the upper rear right height audio channel may be combined and played by the upper right height loudspeaker. Similarly, the upper front left height audio channel and the remaining portion of the upper rear left height audio channel may be combined and played by the upper left height loudspeaker.

[0022] In one embodiment, the virtual height filter may have a filter transfer function, and the method may further include determining the filter transfer function of the virtual height filter from one or more parameters that identify the filter transfer function.

[0023] In one embodiment, the method may further include storing the one or more parameters in a processor as a look-up table or as an analytical function.

[0024] In one embodiment, the virtual height filter may have a filter transfer function that has a peak at a first frequency and a notch at a second frequency higher than the first frequency.

[0025] In one embodiment, at least two of the front and rear non-height loudspeakers may be laterally spaced apart from the listening position. The method may further include determining a filter transfer function for the virtual height filter based on the relative distances of the at least two loudspeakers from the listening position and the elevation of the roof or an area near the roof relative to the listening position.

[0026] In one embodiment, at least two of the front and rear non-height loudspeakers may be laterally spaced apart from the listening position. The method may further include obtaining a plurality of filter transfer functions for a plurality of virtual height filters based on a range of relative distances of the two non-height loudspeakers from the listening position and a range of elevations of the roof or an area near the roof relative to the listening position, and selecting one filter transfer function from the plurality of filter transfer functions.

[0027] In one embodiment, the selected filter transfer function may be the average of the plurality of filter transfer functions.

[0028] In one embodiment, selecting one filter transfer function from the plurality of filter transfer functions may involve selecting one or more parameters that identify the selected filter transfer function based on the average distance of the at least two loudspeakers from the listening position and the average elevation of the roof or an area close to the roof relative to the listening position.

[0029] In one embodiment, the acquisition, selection, application, addition, and playback steps of the method described above may be applied sequentially and iteratively to each selected filter transfer function until the filter transfer function provides a high degree of maximum perception of sound for the playback of the at least two channels.

[0030] In some embodiments, the method may further include applying a gain to a virtual height filter. In some embodiments, the gain may be user-configurable.

[0031] In one embodiment, the listening environment is a vehicle.

[0032] Another aspect of this disclosure provides an apparatus comprising a processor and memory coupled to the processor, wherein the processor is configured to perform any of the methods described herein.

[0033] Another aspect of this disclosure is providing a vehicle equipped with such a device.

[0034] Other aspects of this disclosure provide a program that, when executed by a processor, includes instructions causing the processor to perform a method of processing audio, and further provides a computer-readable storage medium storing such a program. [Brief explanation of the drawing]

[0035] Embodiments of this disclosure are shown in the accompanying drawings as examples, not limitations. Here, similar reference numerals refer to similar elements. [Figure 1]A schematic left-side interior view of a vehicle having a loudspeaker system arranged according to one embodiment of the present disclosure is shown. [Figure 2] This flowchart shows an example of a method for generating multiple audio channels from audio in an immersive audio format according to one embodiment of the present disclosure. [Figure 3] An example of a method for generating multiple audio channels from audio in an immersive audio format, according to one embodiment of the present disclosure, is schematically shown. [Figure 4] A schematic top view of a vehicle with a single height loudspeaker is shown. [Figure 5] A schematic top view of a vehicle having a loudspeaker system arranged according to one embodiment of the present disclosure is shown. [Figure 6] A schematic diagram illustrating the typical sound pathways within a vehicle is shown. [Figure 7] Several examples of virtual height filters according to some embodiments of this disclosure are schematically shown. [Figure 8] A schematic example of a method for generating multiple audio channels from audio in an immersive audio format according to one embodiment of the present disclosure is provided below. [Figure 9] A schematic top view of a vehicle with two height loudspeakers is shown. [Figure 10] An example of the energy distribution of an immersive loudspeaker system according to one embodiment of the present disclosure is schematically shown. [Figure 11] An example of the energy distribution of an immersive loudspeaker system according to one embodiment of the present disclosure is schematically shown. [Figure 12] This is a schematic diagram of an example of an apparatus for carrying out the methods according to the embodiments of this disclosure. [Modes for carrying out the invention]

[0036] Numerous specific details are provided below to provide a full understanding of this disclosure. However, this disclosure may be implemented without these specific details. Also, well-known parts may be described with less comprehensive detail. Drawings are schematic and include parts relevant to understanding this disclosure, but other parts may be omitted or merely suggested.

[0037] Figure 2 shows a flowchart illustrating an example of Method 1000 for generating multiple audio channels from audio in an immersive audio format (e.g., an immersive bitstream) according to one embodiment of the present disclosure. The audio in the immersive audio format includes multiple front and rear height audio channels and multiple front and rear non-height audio channels. Method 1000 may be used to play the generated multiple audio channels in a listening environment using an immersive loudspeaker system. The listening environment may be a room, an audio studio, a vehicle, or any other enclosed or partially enclosed environment suitable for listening to audio. In this disclosure, the listening environment is assumed to be a vehicle. The vehicle may be any type of passenger or non-passenger vehicle used, for example, for commercial purposes or for transporting goods. However, other types of listening environments other than vehicles may also be envisioned. The immersive loudspeaker system is a loudspeaker / speaker system having at least one height loudspeaker and multiple front and rear non-height loudspeakers, as described above. The aforementioned at least one height loudspeaker is positioned above the listener's head (or above the passengers and / or driver's head in the case of a vehicle). The non-height loudspeaker is positioned below the listener's head (or below the passengers and / or driver's head in the case of a vehicle).

[0038] For example, referring to Figure 4, a schematic top view of a vehicle 4000, in this example a four-seater passenger car, is shown. The passenger car 4000 is an example of a listening environment. The passenger car 4000 has four seats 4100, 4200, 4300, and 4400. The vehicle 4000 is equipped with an immersive loudspeaker system having five loudspeakers 1-5. The immersive loudspeaker of the vehicle 4000 has one height loudspeaker 5, two front loudspeakers 1 and 2, and two rear loudspeakers 3 and 4. The loudspeaker on the left side of the vehicle 4000 and its counterpart on the right side of the vehicle 4000 are positioned to reflect symmetrically with respect to a central axis 4500 that crosses the center of the vehicle 4000 along its length. However, other arrangements of the loudspeakers 1-4 are also possible. The height loudspeaker 5 is positioned approximately in the middle of the vehicle 4000 along the longitudinal direction of the vehicle. This is schematically illustrated by dashed arrows 105 showing the value of factor c, which increases from 0 at the rear of vehicle 4000 to 1 at the front of vehicle 4000. In the middle of vehicle 4000, the value of factor c is 0.5. Factor c can be used to calculate the portion of the height channel that is filtered by applying a so-called "virtual" filter based on the position of the height loudspeaker 5, in the manner described with reference to Figure 3, and to calculate the remaining portion of the same height channel that is reproduced by the height loudspeaker 5. It will be understood that the potential listeners located in seats 4100, 4200, 4300, and 4400, respectively, may be symmetrically off-center with respect to any pair of front loudspeakers 1 and 2 as well as rear loudspeakers 3 and 4. For example, a driver seated in driver's seat 4100 is symmetrically off-center with respect to front loudspeakers 1 and 2. The driver is on the right side of vehicle 4000, closer to loudspeaker 1 than to the corresponding loudspeaker 2. In Figures 1 and 4, the driver's seat is shown on the left side of the vehicle (left side when facing forward). However, it should be understood that the location of the driver's seat within a vehicle can vary in different regions. For example, in the United Kingdom, Australia, and Japan, the driver's seat is located on the right side of the vehicle relative to the direction of travel.

[0039] An immersive loudspeaker system may be a surround loudspeaker system having two additional height loudspeakers, for example, as shown with reference to Figure 1. The loudspeaker system shown in Figure 1 is a loudspeaker system having five front or surround loudspeakers, two left audio loudspeakers (e.g., left and left surround), two right audio loudspeakers (e.g., right and right surround), a center loudspeaker, one LFE loudspeaker, and two overhead loudspeakers (left and right overhead loudspeakers). The two left audio loudspeakers correspond to loudspeakers 30, 31 (for mid-range or full-range frequencies), 41, 42, and 43 (for high-range frequencies). The center loudspeaker corresponds to loudspeaker 10. The LFE loudspeaker corresponds to loudspeaker 11. The left overhead loudspeaker corresponds to loudspeaker 44.

[0040] Alternatively, an immersive loudspeaker system may be a surround loudspeaker system having a single-height loudspeaker, as shown in Figure 4, for example.

[0041] Alternatively, in another example, an immersive loudspeaker system may be a surround loudspeaker system having two height loudspeakers, for example, as shown in Figure 9.

[0042] Referring to Figure 5, a schematic top view of another exemplary vehicle 5000 is shown. Vehicle 5000 may be a 6- or 7-seater vehicle with seats distributed in three different rows. Vehicle 5000 may be, for example, a sports utility vehicle (SUV) or a minibus. Vehicle 5000 has six seats 5110, 5120, 5130, 5140, 5150, and 5160. A typical 7.1.2 loudspeaker system may be implemented for vehicle 5000. The loudspeaker system shown in Figure 5 comprises three left loudspeakers 5210, 5230, and 5250 (for example, left and two left surround), three right loudspeakers 5220, 5240, and 5260 (for example, right and two right surround), a center loudspeaker 5270, an LFE loudspeaker 5280, a left-center overhead loudspeaker 5235, and a right-center overhead loudspeaker 5125.

[0043] In one embodiment, the audio of the immersive audio format may be audio rendered in the immersive audio format.

[0044] An immersive audio format (for example, a rendered) audio may include at least one height channel. In one embodiment, the immersive audio format may be an object-based audio format that supports elevation, such as the Dolby Atmos format. In another embodiment, the immersive audio format may be a channel-based audio format that supports elevation, such as the XYZ audio format, where X≧2 is the number of front or surround audio channels, Y≧0 is a low-frequency or subwoofer audio channel, if present, and Z≧1 is the at least one height audio channel. In one embodiment, an object-based audio format (for example, one that supports elevation) may be rendered or pre-rendered to a corresponding channel-based audio format in order to generate loudspeaker feeds corresponding to the channels of the channel-based audio format.

[0045] The method schematically shown in Figure 2 will be explained with reference to Figure 3. Figure 3 schematically shows an example of a method for generating multiple audio channels from audio in an immersive audio format according to one embodiment of the present disclosure. The immersive loudspeaker system schematically shown in Figure 3 has the same number and type of loudspeakers as shown in vehicle 4000 shown with reference to Figure 4.

[0046] Referring to Figure 3, the audio in the immersive audio format may include eight audio channels: non-height front channels 1050 and 1100 (e.g., left and right front channels), non-height rear channels 1125 and 1150 (e.g., left and right rear channels or left and right surround channels), and four height channels 1020, 1010, 1030, and 1040 (e.g., upper front left channel TFL, 1020, upper front right channel TFR, 1010, upper rear left channel TRL, 1030, upper rear right channel TRR, 1040). The example loudspeaker system in Figure 3 is a surround loudspeaker system with loudspeakers 1, 2, 3, and 4 and a single overhead loudspeaker 5 (Voice of God). The same loudspeaker system is shown inside vehicle 4000 with reference to Figure 4. However, similar loudspeaker systems may be used in a room or in a different listening environment. In the example in Figure 3, loudspeaker 1 is used for playback of audio in an immersive audio format for channel 1050 and a portion of the virtual filtered height channel 1200. In the example in Figure 3, loudspeaker 2 is used for playback of audio in an immersive audio format for channel 1100 and a portion of the virtual filtered height channel 1175. Method 1000 generates multiple audio channels 1011, 1018, 1033, 1063, and 1065 from the audio in an immersive audio format, as described below herein. Since the multiple audio channels 1011, 1018, 1033, 1063, and 1065 are generated from the eight channels 1050, 1100, 1125, 1150, 1020, 1010, 1030, and 1040 of the immersive audio format, it can be said that the eight channels of the immersive audio format are downmixed to the five audio channels 1011, 1018, 1033, and 1063 for playback by the corresponding loudspeakers 1, 2, 3, 4, and 5.

[0047] Referring to Figures 2 and 3, Method 1000 includes applying virtual height filters 1400, 1300, 2500, and 2600 to portions of each of a plurality of front and rear height channels 1020, 1010, 1030, and 1040. This is to at least partially attenuate the spectral components of the portion of the height channel directly emanating from the non-height loudspeakers when one of the front and rear height audio channels (any of channels 1020, 1010, 1030, or 1040) is reproduced by one of the front non-height loudspeakers 1 and 2 or the rear non-height loudspeakers 3 and 4, and at least partially amplify the spectral components of the portion of the height channel reflected from the roof or a roof-adjacent area in the listening environment, which in this example is a vehicle, in order to generate a plurality of front virtual height filtered audio signals 1200 and 1175 and rear virtual height filtered audio signals 1225 and 1250. Referring to Figure 2, Method 1000 further includes adding each of the multiple forward virtual height filtered audio signals 1200 and 1175 to the corresponding forward non-height audio channels 1050 and 1100 to generate added audio signals 1011 and 1018 for playback by the corresponding forward loudspeakers 1 and 2. Method 1000 further includes adding the rear virtual height filtered audio signals 1225 and 1250 to the non-height audio channels 1125 and 1150 to generate added audio signals 1033 and 1063 for playback by the corresponding loudspeakers 3 and 4. Method 1000 further includes playing the remaining portions of each of the multiple forward and rear height channels 1020, 1010, 1030 and 1040 using the at least one height loudspeaker 5. From Figure 3, it can be seen that the remaining portions of the front and rear height channels 1020, 1010, 1030, and 1040 are summed up to form signal 1065, which is supplied to the height loudspeaker 5.

[0048] In one embodiment, the portions of the front and rear height channels to which the virtual height filter is applied are a function of the position of the at least one height loudspeaker along the length of the vehicle. In another embodiment, the portions of the front and rear height channels to which the virtual height filter is applied are a linear function of the position of the at least one height loudspeaker along the length of the vehicle. For example, as shown in Figure 4, the height (or overhead) loudspeaker 5 may be located midway along the length of the vehicle 4000 and along the axis 4500. As described above, in the example of Figure 4, the height loudspeaker 5 is located midway along the vehicle 4000. The position of the height loudspeaker 5 is schematically indicated in Figure 4 by the value of the multiplier factor. When the height loudspeaker 5 is located midway along the length of the vehicle 4000, c = 0.5. In this example, referring to Figure 3, when the height (or overhead) loudspeaker 5 is located midway along the length of the vehicle, 50% of the energy of each height channel 1020, 1010, 1030, and 1040 may be "virtually" filtered by the corresponding virtual filters 1400, 1300, 2500, and 2600, and the remaining 50% of the energy may be reproduced by the height loudspeaker 5. This example is schematically represented with reference to Figure 10. Figure 10 schematically shows an example of the energy distribution of the audio system in the loudspeaker system of vehicle 4000 of Figure 4, using the exemplary method of Figure 3. When c = 0.5 when the height loudspeaker 5 is located midway along vehicle 4000, the audio signal supplied to the height loudspeaker 5 is 0.5TFL + 0.5TFR + 0.5TRL + 0.5TRR. The audio signal supplied to loudspeaker 1 is L + (0.5TFL) filter Therefore, it is equal to the energy of the non-height front left channel plus half the energy of the upper front left channel filtered by the virtual filter 1400. The audio signal supplied to loudspeaker 2 is R+(0.5TFR) filterTherefore, it is equal to the energy of the non-height front right channel plus half the energy of the upper front right channel filtered by the virtual filter 1300. The audio signal supplied to the loudspeaker 3 is L S +(0.5TRL) filter Therefore, it is equal to the energy of the non-height rear (or surround) left channel plus half the energy of the upper rear left channel filtered by the virtual filter 2500. The audio signal supplied to loudspeaker 4 is R S +(0.5TRR) filter This means that the energy of the non-height rear (or surround) right channel is equal to half the energy of the upper rear right channel filtered by the virtual filter 2600. The energy distribution is such that energy is conserved between the loudspeakers of the loudspeaker system; that is, the distribution is energy neutral. This effect is schematically shown in Figure 10 by the loudspeakers enclosed by dashed lines. Sound in the height plane is distributed among the “virtual” loudspeakers enclosed by dashed lines, and the listener has the perception that sound in the height plane, i.e., sound associated with the height channels of the immersive audio format, is emitted by such virtual loudspeakers. Sound in the height plane has improved spatial resolution compared to conventional methods using a single height loudspeaker. The listener’s perception of sound in the height plane is the same as the perception of sound produced by a helicopter; that is, the perception that audio is being emitted from all directions surrounding the listener, even by using a single height loudspeaker. In conventional solutions, such as those in immersive loudspeaker systems that do not redistribute height channels between height and non-height loudspeakers and do not "virtually" filter the portion of the height channel supplied to the non-height loudspeakers, the helicopter effect is not produced, but the sound is perceived as simply being emitted by the physical position of a single height loudspeaker.

[0049] In one embodiment, applying a virtual height filter to a portion of each of a plurality of front and rear height channels involves applying the virtual height filter to a portion of the front height channel that is proportional to the position of the at least one height loudspeaker as it moves away from the front of the listening environment and towards the rear. For example, referring to Figure 4, the value of c may decrease as the position of height loudspeaker 5 approaches the rear of the vehicle 4000. In the example of Figure 3, the portions of the upper front left and right channels (TFL and TFR) supplying loudspeakers 1 and 2 increase proportionally as the value of c decreases. That is, in this particular example, such portions increase by factor 1-c.

[0050] In a further embodiment, applying a virtual height filter to a portion of each of the multiple front and rear height channels includes applying the virtual height filter to a portion of the rear height channel that is proportional to the position of the at least one height loudspeaker as it moves away from the rear and closer to the front of the listening environment. For example, referring to Figure 4, the value of c may increase as the position of the height loudspeaker 5 moves closer to the front and further away from the rear of the vehicle 4000. In the example of Figure 3, the portions of the upper rear left and right channels (TRL and TRR) supplying loudspeakers 3 and 4 increase proportionally as the value of c increases. That is, in this particular example, such portions increase by factor c.

[0051] Similarly, in one embodiment, playing the remainder of each of a plurality of front and rear height audio channels using the at least one height loudspeaker includes playing the remainder of each of a plurality of front height audio channels in proportion to the position of the at least one height loudspeaker as it moves from the rear to the front of the listening environment. Referring to Figure 4, for example, the value of c may increase as the position of the height loudspeaker 5 moves closer to the front of the vehicle 4000 and further away from the rear. In the example of Figure 3, the remainder of the upper front left and right channels (TFL and TFR) supplied to the height loudspeaker 5 increases proportionally as the value of c increases; that is, in this particular example, such remainder increases by factor c.

[0052] Similarly, in one embodiment, playing the remaining portions of each of the multiple front and rear height audio channels using the at least one height loudspeaker includes playing the remaining portions of each of the multiple rear height audio channels in proportion to the position of the at least one height loudspeaker as it moves away from the front and towards the rear of the listening environment. Referring to Figure 4, for example, the value of c may decrease as the position of the height loudspeaker 5 moves closer to the rear of the vehicle 4000 and away from the front. In the example of Figure 3, the remaining portions of the upper rear left and right channels (TRL and TRR) that supply the height loudspeaker 5 increase proportionally as the value of c decreases; that is, in this particular example, such portions increase by factor 1-c.

[0053] The embodiments described with reference to Figures 3, 4, and 10 are understood to be merely examples of how energy distribution from the height channel to the non-height loudspeaker may be carried out. In actual implementations, the portion(s) of the height channel filtered by the virtual filter supplying the non-height loudspeaker may not depend on the position of the height loudspeaker. In actual implementations, the portion(s) of the height channel filtered by the virtual filter supplying the non-height loudspeaker may be determined based on the best performance of the loudspeaker system, for example, based on the best immersive audio experience for the listener. Similarly, in embodiments of this disclosure, factor c may be adjusted for the best immersive audio experience for the listener, instead of, for example, based on the position of the at least one height loudspeaker. It is also understood that embodiments of this disclosure are not limited to the use of factor c as shown in Figure 3. Other linear or nonlinear functions of the position of the height loudspeaker, or other linear or nonlinear functions of other parameters of the height and / or non-height audio channels, such as volume level, equalization level, dialogue enhancement, etc., may be assumed. Other ratios between the portion of the height channel that is virtually filtered and the remaining portion of the height channel that is reproduced by the height loudspeaker can be selected depending on the appropriate implementation.

[0054] In one embodiment, the portions of the front and rear height channels to which the virtual height filter is applied are a function of a desired balance distribution of multiple audio channels between the loudspeakers of an immersive loudspeaker system. The desired balance distribution can provide the best immersive audio experience for the listener. For example, during the calibration or testing phase, it may be found that a specific ratio of the portion of the height channel that is virtual filtered to the rest of the height channel reproduced by the height loudspeaker provides a desired height spatial resolution, i.e., spatial resolution in the height plane.

[0055] In another embodiment, the portions of the front and rear height channels to which the virtual height filter is applied are a function of both the desired balance distribution of multiple audio channels between the loudspeakers of the immersive loudspeaker system and the position of the at least one height loudspeaker along the length of the vehicle. The desired spatial resolution in the height plane may be a function of the position of the height (or overhead) loudspeaker.

[0056] Generally, as mentioned above, the selection of the forward and rear height channel portions filtered by the virtual height filter is based on empirical data or experimentation to provide the best performance for the immersive loudspeaker system.

[0057] In one embodiment, the portion of the front or rear height channel to which the virtual height filter is applied may be a function of the listener's (driver's or passenger's) position relative to the position of the height (or overhead) loudspeaker. In another embodiment, the portion of the front or rear height channel to which the virtual height filter is applied may be dynamically adjusted / triggered in response to a change in the listener's position relative to the position of the height (or overhead) loudspeaker. For example, the listener's position may be adjusted in the height plane or base plane, and the portion of the front or rear height channel to which the virtual height filter is applied may be modified based on this adjustment. For example, the adjustment of the portion may be triggered by adjusting the seat in any direction of the vehicle.

[0058] The virtual height filter 1300 is configured to at least partially attenuate the spectral components of a portion of the height channel 1010 that are directly emitted from the loudspeaker 2 when a portion of the height channel 1010 is reproduced by the loudspeaker 2. The virtual height filter 1300 is further configured to at least partially amplify the spectral components of a portion of the height channel 1010 that are reflected from the roof or an area close to the roof inside the vehicle to generate a virtual height filtered audio signal 1175. The virtual height filter 1400 is configured to at least partially attenuate the spectral components of a portion of the height channel 1020 that are directly emitted from the loudspeaker 1 when a portion of the height channel 1020 is reproduced by the loudspeaker 1. The virtual height filter 1400 is further configured to at least partially amplify the spectral components of a portion of the height channel 1020 that are reflected from the roof or an area close to the roof inside the vehicle to generate a virtual height filtered audio signal 1200. The virtual height filter 2500 is configured to at least partially attenuate a portion of the spectral components of the height channel 1030 that are directly emitted from the loudspeaker 3 when a portion of the height channel 1030 is reproduced by the loudspeaker 3. The virtual height filter 2500 is further configured to at least partially amplify a portion of the spectral components of the height channel 1030 that are reflected from the roof or an area close to the roof inside the vehicle to generate a virtual height filtered audio signal 1225. The virtual height filter 2600 is configured to at least partially attenuate a portion of the spectral components of the height channel 1030 that are directly emitted from the loudspeaker 4 when a portion of the height channel 1040 is reproduced by the loudspeaker 4. The virtual height filter 2600 is further configured to at least partially amplify a portion of the spectral components of the height channel 1040 that are reflected from the roof or an area close to the roof inside the vehicle to generate a virtual height filtered audio signal 1250.

[0059] Refer to Figure 6 for further explanation. Figure 6 schematically shows exemplary paths 6300 and 6400 in which sound reproduced by the loudspeaker 6000 may travel from the loudspeaker 6000 to the listening position 6100 (e.g., inside the vehicle). The loudspeaker 6000 may be any of the non-height loudspeakers shown, for example, with reference to the loudspeaker systems in Figures 1 and 3. In particular, the loudspeaker 6000 may be any of the front or rear (or surround) left or right loudspeakers shown. Preferably, since height cues are typically more dominant in high-frequency signals than in low-frequency signals, the loudspeaker 6000 may be any high-frequency loudspeaker associated with any of the left, right, or surround loudspeakers, such as the loudspeakers (e.g., tweeters) 41, 42, and 43 shown in Figure 1. The listening position 6100 may be at the ear / head of a passenger or driver in the vehicle. The sound reproduced by the loudspeaker 6000 may radiate along the reflection path 6300, shown by the dashed line in Figure 6, and along the direct path 6400, shown by the solid line in Figure 6. The reflection path 6300 is an indirect path from the loudspeaker 6000 to the listening position 6100, formed by sound reflected from a surface 6500 located above the listening position 6100. Inside the listening environment, the surface 6500 may be the ceiling of the room or an area close to the ceiling of the room. Inside a vehicle, the surface 6500 may be the roof of the vehicle or an area close to the roof of the vehicle. An area close to the roof may be the upper inner portion of the front or rear windshield of the vehicle, or the upper inner portion of the side windows of the vehicle. In general, the surface 6500 may be any part inside the vehicle that is located at a higher altitude (e.g., above) than the listening position during sound reproduction. To enhance the perception of the altitude of the sound, it is desirable for the sound to radiate along the reflection path 6300. However, some sound from the loudspeaker 6000 will travel along the direct path 6400, reducing the perception of sound coming from the position on the surface 6500 where the sound is reflected toward the listening position 6100.The amount of this undesirable direct sound compared to the desired reflected sound can be a function of the directional pattern of the loudspeaker 6000. Non-height loudspeakers located at approximately half the total height inside a vehicle (for example, roughly halfway up the door) have been shown to improve the perception of sound height.

[0060] To compensate for undesirable direct sound, incorporating signal processing to introduce perceptual height cues into the audio signal supplied to the loudspeaker 6000 has been shown to improve the positioning and perceived quality of the virtual height signal. For example, a directional auditory model has been developed to create a virtual height filter, which, when used to process audio played by the loudspeaker, improves the perceived quality of the playback. In one embodiment, the virtual height filter is derived from both the physical loudspeaker position relative to the listening position and the virtual loudspeaker position (above the listening position). For the physical loudspeaker position, the first directional filter is determined based on a model of sound traveling directly from the loudspeaker position to the listener's ear at the listening position. Such filters may be derived from a database of HRTF (Head-Related Transfer Function) measurements or a model of directional auditory perception, such as a parametric binaural auditory model, apinna model, or other similar transfer function models that utilize cues to help perceive height. Models that take the auricle model into account are generally useful because they help define how height is perceived, but the filter function is not intended to isolate the auricle effect, but rather to handle the ratio of sound levels from one direction to another, and the auricle model is just one example of such a model among the binaural auditory models that can be used. However, other models can also be used.

[0061] The inverse of this filter is then determined and used to remove directional cues for audio traveling along a direct path from the physical loudspeaker location to the listening position. Next, for a virtual loudspeaker location, a second directional filter is determined using the same directional auditory model, based on a model of sound traveling directly from the virtual loudspeaker location to the ears of a listener at the same listening position. This filter is applied directly and provides the directional cues that the ear would receive if the sound were emitted from a virtual loudspeaker location above the listening position. In practice, the first and second directional filters may be combined to allow for a single filter that at least partially removes (attenuates) directional cues from the physical loudspeaker location and at least partially inserts (amplifies) directional cues from the virtual loudspeaker location. Such a single filter provides a frequency response curve referred herein to as a “height filter transfer function,” “virtual height filter response curve,” “desired frequency transfer function,” “height cue response curve,” or similar terms, to describe a filter or filter response curve that filters, for example, attenuates, a direct sound component from a height component in an audio loudspeaker system.

[0062] Regarding the filter model, if P1 represents the frequency response in dB of a first filter that models sound transmission from a physical loudspeaker location, and P2 represents the frequency response in dB of a second filter that models sound transmission from a virtual loudspeaker location, then the total response PT of the virtual height filter in dB can be expressed as PT = α(P2 - P1), where α is a scaling factor or gain that controls the strength of the filter. At α = 1, the filter is applied to the maximum extent, and at α = 0, the filter does nothing (0 dB response). In practice, α can be set somewhere between 0 and 1 based on the relative balance between reflected and direct sound (e.g., α = 0.5). As the level of direct sound increases compared to reflected sound, α should also increase to give more complete directional cues to the virtual loudspeaker location through this undesirable direct sound path. However, α should not be so large as to impair the perceived timbre of the audio traveling along the reflected path, which already contains adequate directional cues. Generally, the exact values ​​of filters P1 and P2 are functions of the azimuth angle of the physical loudspeaker position relative to the listening position and the elevation of the reflected loudspeaker position. This elevation is a function of the distance from the listening position to the physical loudspeaker position and the difference between the height of the roof or an area close to the roof (surface 6500 in Figure 6) and the height of the speaker.

[0063] Figure 7 shows exemplary curves 7200, 7300, and 7400 of virtual height filters according to several embodiments of the present disclosure. Curves 7200, 7300, and 7400 are represented in a figure where the amplitude of the virtual height filter is shown in decibels (dB) on the vertical axis, with frequency in Hertz (Hz) on the horizontal axis.

[0064] Curves 7200, 7300, and 7400 represent the filter transfer functions of three different virtual height filters. Figure 7 shows that the filter transfer functions 7200, 7300, and 7400 of the three different filters have a peak at a first frequency of approximately 8000 Hz and a notch at a second frequency higher than the first frequency of approximately 12000 Hz. However, the peak and notch may be at different frequencies. The three different transfer functions can be obtained by applying different scaling factors / different gains to the virtual height filters, as described above. In some embodiments, the gain may be user-configurable so that the "strength" of the virtual height filter can be adjusted by the user according to a particular implementation.

[0065] In one embodiment, as shown with reference to Figure 2, the method of the present disclosure may further include determining the filter transfer function of a virtual height filter from one or more parameters that identify the filter transfer function.1800 For example, one or more parameters may indicate at least one value of the peak, peak frequency, notch, and notch frequency of the filter transfer function representing the virtual height filter. For example, the parameters may be stored in memory, or in a processor containing memory, for example, as a lookup table or analysis function. These parameters may be retrieved from memory by a processing unit from which the virtual height filter can be reconstructed. The reconstructed virtual height filter may thus be used and applied to the height channel. By using one or more parameters to identify the filter transfer function, the processing of the height channel is simplified because the virtual height filter is described by a small number of parameters rather than being locally generated.

[0066] Two front and / or rear non-height loudspeakers are spaced laterally apart with respect to the listening position (for example, in the case of a vehicle, the driver or passenger position). In one embodiment, as shown with reference to Figure 2, the method of the present disclosure may further include determining a filter transfer function for a virtual height filter based on the relative distance of the two non-height front and / or rear loudspeakers from the listening position and the elevation of the roof or an area close to the roof relative to the listening position.

[0067] For example, in one embodiment, one or more sensors may be positioned at or near the listening position to measure the relative distance of the at least two front or rear loudspeakers from the listening position, and the elevation of the roof or an area close to the roof relative to the listening position. For example, in one embodiment, such sensors may be embedded in the headrest of each seat of the vehicle at approximately the same height as the listener's head. The measurements may be performed in the initial calibration stage of the method, or alternatively, in substantially real time along with audio playback.

[0068] Alternatively, additionally, or optionally, the filter transfer function of a virtual height filter may be based on a predetermined absolute distance between one or more listening positions and each of the non-height front and / or rear loudspeakers, and a predetermined altitude of the roof relative to the listening positions. For example, the distance between one or more listening positions (e.g., any of the positions in seats 4100, 4200, 4300, or 4400 in Figure 4) and pairs of non-height front loudspeakers 1 and 2 and / or non-height rear loudspeakers 3 and 4, as well as the altitude of the roof, may be determined / predetermined by environmental characteristics, such as the vehicle's interior design and the placement of the loudspeakers. The method of this disclosure may use this predetermined information to obtain the filter transfer function of a virtual height filter. For example, in one embodiment, step 1800 of determining the filter transfer function of a virtual height filter from one or more parameters may involve accessing predetermined parameters. For example, the parameters may be obtained / measured for one vehicle of a certain type and then stored in the memory of the in-vehicle computing system of the same type of vehicle. Such offline calibration has the advantage that the vehicle does not need to have sensors to measure and acquire the filter transfer function online.

[0069] Alternatively, additionally, or optionally, in embodiments such as those shown with reference to Figure 2, Method 1000 may further include obtaining a plurality of filter transfer functions for a plurality of virtual height filters, typically prior to step 1500. The plurality of virtual height filters may be obtained based on a range of relative distances between a plurality of height and non-height loudspeakers from the listening position and a range of elevations of the roof or an area close to the roof relative to the listening position. For example, the range of distances between a non-height or height loudspeaker and the listening position (one or more) may be measured for a plurality of different listening positions and / or a plurality of non-height or height loudspeaker positions, for example, during the calibration stage. Similarly, the range of elevations of the roof (or its virtual loudspeaker positions) may be measured for a plurality of different listening positions, for example, during the calibration stage. Method 1000 further includes selecting one filter transfer function from the plurality of filter transfer functions. For example, in one embodiment, the selected filter transfer function may be based on the average distance from the listening position to the height loudspeaker and on the average elevation of the roof or an area close to the roof (or virtual loudspeaker position) relative to the listening position. In another embodiment, the selected virtual height filter (or its filter transfer function) is the average of a plurality of filter transfer functions. For example, the selected transfer function may be determined by interpolation between a plurality of filter transfer functions. In yet another embodiment, method 1000, including steps 1900 and 2000, may be applied sequentially and iteratively, for example, during the calibration phase, as shown in Figure 2. The application is performed for each selected filter transfer function in each iteration until the selected filter transfer function provides the optimal (e.g., maximum) perception of sound height with a desired spatial resolution in the height plane at one or more listening positions. In other words, method 1000, including steps 1900 and 2000, may be applied iteratively until the (selected) filter transfer function provides reproduction of the plurality of audio channels with maximum spatial resolution in the height plane.Generally, for simpler and more effective audio processing in certain types of vehicles, a single filter transfer function may be selected that performs well on average for most of the listening position / loudspeaker position and the elevation of the roof or area near the roof (or the elevation of the virtual loudspeaker position). However, the filter transfer function may also be determined adaptively in virtually real time by sensors, for example, as described above. Adaptively determining the filter transfer function can provide a more accurate determination of the filter transfer function and, therefore, an improved spatial resolution in the height plane.

[0070] In one embodiment, with further reference to Figure 2, each filter transfer function of the multiple transfer functions acquired in step 1900 may be determined from one or more parameters stored in memory, for example, as a LUT or analysis function, as described above. This method allows for the active / adaptive selection of the filter transfer function parameters for a particular vehicle type or when the sensor is used.

[0071] In one embodiment, with further reference to Figure 2, step 1800, which determines the filter transfer function of a virtual height filter from one or more parameters (either based on predetermined distance / altitude information or based on actual measurements), may be triggered when movement of a listener located at one or more listening positions is detected. For example, one or more sensors can be used to detect the movement of a listener. When used inside a vehicle, such sensors may be located, for example, at each seat in the vehicle. The one or more sensors may be configured to detect the presence of passengers or a driver in the vehicle, thus enabling the use of correct distance information to be used by the processing method to obtain the filter transfer function.

[0072] In one embodiment, one or more seat sensors or different sets of sensors may be used to detect a new listening position, for example, a new position of the listener's head (or the position of the listener's ears). For example, a driver or passenger may adjust their seat horizontally and / or vertically for a more comfortable seating position in the vehicle. In this embodiment, the method may look up / get a virtual height filter (or its filter transfer function) according to the newly detected listening position. Thus, the correct information may be used according to the new listening position based on a correct set of distance information from a given listener to a loudspeaker and a given set of roof elevation information, or based on actual measurements. For example, when a predetermined set of one or more parameters that identify a virtual height filter (or its filter transfer function) are stored as an analysis function or lookup table (LUT), different analysis functions or different LUTs may correspond to different (e.g., detected) seats or listening positions. These embodiments, in which step 1800 may be triggered when listener movement is detected, may be combined with the embodiments described above, in which the portion of the front or rear height channel to which the virtual height filter is applied may be dynamically adjusted / triggered in response to a change in the listener's position relative to the position of the height (or overhead) loudspeaker. For example, one or more parameters that identify a particular filter transfer function and the portion of the front or rear height to which the particular filter transfer function is applied may be determined when a new listener position is detected and / or based on the new listener position. The new listener position may be determined using any of the sensors or sets of sensors described above. Thus, a more accurate and targeted determination of the filter transfer function and the ratio of the portion of the height channel to be filtered to the rest of the height channel to be reproduced by the height loudspeaker can be achieved.

[0073] As mentioned above, an immersive loudspeaker system may include two height loudspeakers.

[0074] For example, referring to Figure 8, the immersive loudspeaker system includes two height loudspeakers 6 and 7 and four non-height loudspeakers 1, 2, 3, and 4. The two height loudspeakers 6 and 7 may be the upper right height loudspeaker and the upper left height loudspeaker, respectively, as shown in vehicle 8000 in Figure 9. Figure 9 schematically shows vehicle 9000. Vehicle 9000 is similar to vehicle 4000 shown with reference to Figure 4, but includes an immersive loudspeaker system with two height loudspeakers 6 and 7. Similar to vehicle 4000 in Figure 4, the height loudspeakers 6 and 7 in the example in Figure 9 are located midway along the length of vehicle 9000. The midway position of the height loudspeakers 6 and 7 is indicated by a value of 0.5 for factor c. The (input) immersive audio format used with reference to the example shown in Figure 8 has eight audio channels, which is the same as that described with reference to Figure 3 and is not repeated here.

[0075] The virtual height filter 1300 is applied to a portion of the height channel 1010 to generate the virtual height filtered signal 1175. The virtual height filtered signal 1175 is added to the non-height channel 1100 to generate signal 1018. The virtual height filter 1400 is applied to a portion of the height channel 1020 to generate the virtual height filtered signal 1200. The virtual height filtered signal 1200 is added to the non-height channel 1050 to generate signal 1011. The virtual height filter 2500 is applied to a portion of the height channel 1030 to generate the virtual height filtered signal 1225. The virtual height filtered signal 1225 is added to the non-height channel 1125 to generate signal 1033. The virtual height filter 2600 is applied to a portion of the height channel 1040 to generate the virtual height filtered signal 1250. The virtual height filtered signal 1250 is added to the non-height channel 1150 to generate signal 1063. Channel signals 1011, 1018, 1033, and 1063 are supplied to loudspeakers 1, 2, 3, and 4, respectively, for playback. The remainder of height channel 1020 is supplied to height loudspeaker 7 along with the remainder of height channel 1030. The remainder of height channel 1010 is supplied to height loudspeaker 6 along with the remainder of height channel 1040. By using two height loudspeakers 6 and 7 and redistributing the height channels between the height loudspeakers and non-height loudspeakers of the immersive loudspeaker system, the spatial resolution in the height plane is further improved. This is better illustrated with reference to Figure 11.

[0076] FIG. 11 schematically shows an example of the energy distribution of the audio system in the loudspeaker system of vehicle 9000 of FIG. 9 by using the exemplary method of FIG. 8. When c = 0.5 when the height loudspeakers 6 and 7 are located in the middle of the vehicle 9000, the audio signal supplied to the height loudspeaker 6 is 0.5TFR + 0.5TRR, and the audio signal supplied to the height loudspeaker 7 is 0.5TFL + 0.5TRL. The audio signal supplied to the loudspeaker 1 is L+(0.5TFL) filter That is, it is equal to the energy of the non-height front left channel plus half of the energy of the upper front left channel filtered by the virtual filter 1400. The audio signal supplied to the loudspeaker 2 is R+(0.5TFR) filter That is, it is equal to the energy of the non-height front right channel plus half of the energy of the upper front right channel filtered by the virtual filter 1300. The audio signal supplied to the loudspeaker 3 is L S +(0.5TRL) filter That is, it is equal to the energy of the non-height rear (or surround) left channel plus half of the energy of the upper rear left channel filtered by the virtual filter 2500. The audio signal supplied to the loudspeaker 4 is R S +(0.5TRR) filterThis means that the energy of the non-height rear (or surround) right channel is equal to half the energy of the upper rear right channel filtered by the virtual filter 2600. The energy distribution is such that energy is conserved between the loudspeakers of the loudspeaker system, i.e., the distribution is energy-neutral. This effect is schematically shown in Figure 11 by the loudspeakers enclosed by dashed lines. Sound in the height plane is distributed between the “virtual” loudspeakers enclosed by dashed lines, and the listener has the perception that sound in the height plane, i.e., sound associated with the height channels of the immersive audio format, is emitted by such virtual loudspeakers. Sound in the height plane has improved spatial resolution compared to the conventional method using two height loudspeakers. The listener’s perception of sound in the height plane can be said to be the perception of sound produced by a helicopter, i.e., the perception that audio is being emitted from all directions surrounding the listener, even by using two height loudspeakers. In conventional solutions, such as those in immersive loudspeaker systems that do not redistribute the height channel between the height and non-height loudspeakers and do not "virtually" filter the portion of the height channel supplied to the non-height loudspeaker, the helicopter effect is not produced, but the sound is perceived as simply being emitted by the physical positions of the two height loudspeakers.

[0077] It will be understood that the various ways of determining the height channel portion based on height loudspeakers, which are virtually filtered and described with reference to embodiments in Figures 3, 4, and 10, are also applicable to embodiments in Figures 8, 9, and 11, which have two height loudspeakers. Therefore, these various methods for determining the height channel portion will not be repeated here.

[0078] Similar to the embodiments described with reference to Figures 3, 4, and 10, the embodiments described with reference to Figures 8, 9, and 11 are understood to be merely examples of how energy distribution from the height channel to the non-height loudspeaker may be carried out. In actual implementations, the portion of the height channel to be filtered by the virtual filter supplied to the non-height loudspeaker may be independent of the height loudspeaker's position. In actual implementations, the portion of the height channel filtered by the virtual filter supplied to the non-height loudspeaker may be determined based on the best performance of the loudspeaker system, for example, based on the best immersive audio experience for the listener. Similarly, in embodiments of this disclosure, factor c may be adjusted for the best immersive audio experience for the listener, instead of based on, for example, the positions of two height loudspeakers. It is also understood that embodiments of this disclosure are not limited to the use of factor c as shown in Figure 9. Other linear or nonlinear functions of the height loudspeaker's position, or other linear or nonlinear functions of other parameters of the height and / or non-height audio channels, such as volume level, equalization level, dialogue enhancement, etc., may be conceived. Other ratios between the portion of the height channel that is virtually filtered and the remaining portion of the height channel reproduced by the height loudspeakers may be selected according to the appropriate implementation. For example, if the height loudspeakers 6 and 7 are located between the rear and middle of the vehicle 9000, c may be equal to 0.25. For example, if the height loudspeakers 6 and 7 are located between the middle and front of the vehicle 9000, c may be 0.75. Alternatively, c may further be equal to 0.25 or 0.75 (or any other value appropriate for a particular implementation), regardless of the position of the loudspeakers 6 and 7.

[0079] In one embodiment, the portions of the front and rear height channels to which the virtual height filter is applied are a function of a desired balance distribution of multiple audio channels between the loudspeakers of an immersive loudspeaker system. The desired balance distribution can provide the best immersive audio experience for the listener. For example, during the calibration or testing phase, a specific ratio of the portion of the height channel that is virtual filtered to the rest of the height channel reproduced by the height loudspeaker may be found to provide a desired height spatial resolution, i.e., a desired spatial resolution in the height plane.

[0080] In another embodiment, the portions of the front and rear height channels to which the virtual height filter is applied are a function of both the desired balance distribution of multiple audio channels between the loudspeakers of the immersive loudspeaker system and the position of the at least one height loudspeaker along the length of the vehicle. The desired spatial resolution in the height plane may be a function of the position of the height (or overhead) loudspeaker.

[0081] Generally, as mentioned above, the selection of the forward and rear height channel portions filtered by the virtual height filter is based on empirical data or experimentation to provide the best performance for the immersive loudspeaker system.

[0082] In one embodiment, the portion of the front or rear height channel to which the virtual height filter is applied may be a function of the listener's (driver's or passenger's) position relative to the position of the height (or overhead) loudspeaker. In another embodiment, the portion of the front or rear height channel to which the virtual height filter is applied may be dynamically adjusted / triggered in response to a change in the listener's position relative to the position of the height (or overhead) loudspeaker. For example, the listener's position may be adjusted in the height plane or base plane, and the portion of the front or rear height audio channel to which the virtual height filter is applied may be modified based on this adjustment. For example, the adjustment of the portion may be triggered by adjusting the seat in any direction of the vehicle.

[0083] Any other suitable immersive audio format and / or loudspeaker configuration can be conceived to suit a particular implementation.

[0084] For example, in addition to the (input) channels in the examples shown with reference to Figures 3 and 8, audio in an immersive audio format may also include a center (C) channel and / or a low-frequency effect (LFE) channel (not shown in either Figure 3 or 8). As explained above, since height cues are typically more dominant in high-frequency signals than in low-frequency signals, when present, the center channel and / or LFE channel are typically not mixed / added with the virtual filtered height channel but are fed directly to the corresponding loudspeakers (e.g., center and LFE loudspeakers).

[0085] In some embodiments (not shown), when a center channel is present in the input immersive audio format, the center channel may be mixed / added together with the non-height front left and front right channels. In such embodiments, adding a virtual filtered height channel(s) to the non-height audio channels(s) (i.e., the front left and / or front right channels) may be performed after the front left and front right channels have been mixed with the center channel. In other words, in such embodiments, the non-height channel signals to be added to the virtual filtered signal are already mixed with the center channel.

[0086] Similar considerations are applicable to loudspeaker configurations such as those shown in Figure 5, which also include middle left (ML) and middle right (MR) loudspeakers 5230 and 5240.

[0087] In some embodiments, non-height channels, such as the front left and front right channels and / or the rear left and rear right channels, are processed before being mixed with the corresponding virtual filtered portions of the height channels. For example, the front left and front right channels and / or the rear left and rear right channels may be processed to compensate for off-center listening positions of passengers / drivers in a vehicle. Off-center listening position compensation may be performed using the algorithm described in European Patent No. 1994795B1, which is incorporated herein by reference in its entirety. European Patent No. 1994795B1 demonstrates that it is possible to simultaneously make two symmetrically off-center listening positions from the same pair of (stereo) loudspeakers into "virtual centers." This follows the same principle as reducing the phase difference of the interaural phase difference (IDP) of a single listening position. In the case of two listening positions, the phase difference of the IDPs obtained for each of the two listening positions is simultaneously reduced so that each IDP at each listening position has a value between -90 degrees and 90 degrees over the desired frequency range. By compensating for off-center listening positions and mixing the filtered portion of the height channel with the corresponding compensated front and / or rear non-height channels, panning of the content of the height channel portion across the front and / or rear loudspeakers can be prevented.

[0088] Exemplary computing device A method for generating multiple audio channels from audio in an immersive audio format to reproduce multiple audio channels in an immersive loudspeaker system comprising at least one height loudspeaker and multiple front and rear non-height loudspeakers has been described. Furthermore, the disclosure also relates to apparatus for performing these methods. Furthermore, the disclosure relates to a vehicle that may be equipped with an apparatus for performing these methods. An example of such an apparatus 1440 is schematically shown in Figure 12. The apparatus 1440 may comprise a processor 1410 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application-specific integrated circuits (ASICs), one or more radio frequency integrated circuits (RFICs), or any combination thereof) and a memory 1420 coupled to the processor 1410. The memory 1420 may store an analysis function (or a set thereof) or a lookup table (or a set thereof) representing one or more parameters that identify the filter transfer function of a virtual height filter for, for example, different listening positions and / or roof height and / or different vehicles. The processor may be configured to perform some or all of the steps of the method described throughout this disclosure, for example, by retrieving a set of analysis functions and / or LTUs from memory 1420. To perform a method for generating multiple audio channels, the apparatus 1440 may receive as input channels of (e.g., rendered) audio in an immersive audio format, such as a height channel and one or more front or surround audio channels 1425. In this case, the apparatus 1440 may output two or more channel signals 1430 for playback of the channel signals in an immersive loudspeaker system.

[0089] Device 1440 may be a server computer, client computer, personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, smartphone, web appliance, network router, switch or bridge, or any machine capable of executing instructions (sequentially or otherwise) that specify actions to be performed by such device. Furthermore, although only a single device 1440 is shown in Figure 12, this disclosure relates to any collection of devices that individually or collectively execute instructions to perform any one or more of the methods described herein.

[0090] This disclosure further relates to a program (e.g., a computer program) that, when executed by a processor, includes instructions causing the processor to perform some or all of the steps of the method described herein.

[0091] Furthermore, this disclosure relates to a computer-readable (or machine-readable) storage medium on which the above program is recorded. The term “computer-readable storage medium” includes, but is not limited to, data repositories in the form of solid-state memory, optical media, and magnetic media.

[0092] The embodiments described herein may be implemented in hardware, software, firmware, or a combination thereof. For example, the embodiments may be implemented on a system having electronic circuits and components, such as a computer system. Examples of computer systems include desktop computer systems, portable computer systems (e.g., laptops), handheld devices (e.g., smartphones or tablets), and networking devices. A system for implementing an embodiment may comprise, for example, at least one of integrated circuits (ICs), programmable logic devices (PLDs), such as field-programmable gate arrays (FPGAs), digital signal processors (DSPs), application-specific ICs (ASICs), central processing units (CPUs), and graphics processing units (GPUs).

[0093] Certain implementations of the embodiments described herein may include a computer program product that, when executed by a data processing system, causes the data processing system to perform any of the methods of the embodiments described herein. The computer program product may include non-temporary media for storing the instructions, such as magnetic data storage media including floppy disks and hard disk drives, optical data storage media including CD-ROMs and DVDs, and physical media such as ROMs, flash memory, such as flash RAM or USB flash drives. In another example, the computer program product may include a data stream containing the instructions, or a file containing the instructions stored in a distributed computing system, such as one or more data centers.

[0094] This disclosure is not limited to the embodiments and examples described above. Numerous modifications and variations can be made without departing from the scope of this disclosure as defined by the appended claims.

[0095] Various aspects of the present invention can be understood from the following enumerated example embodiments (A-EEE and B-EEE).

[0096] A-EEE1. A method for generating discrete channels from an immersive bitstream, A step of identifying one or more height channels and one or more non-height channels of the immersive bitstream; A step of processing one or more height channels using a virtual height filter and a non-standard mixed technique; The step includes mixing one or more processed height channels with one or more non-height channels, method.

[0097] B-EEE1. A method (1000) for generating at least two audio channels from audio of an immersive audio format including at least one height audio channel (1010) and at least two non-height audio channels (1050, 1100), wherein the at least two audio channels are to be played back in a non-immersive loudspeaker system of at least two audio loudspeakers (1, 2) in a vehicle, the method being: - When the at least one audio height channel is reproduced by one of the at least two loudspeakers, the step (1500) of at least partially attenuating the spectral components of the at least one height channel (1010) directly emitted from the loudspeaker (1, 2) and at least partially amplifying the spectral components of the at least one height channel reflected from the roof or an area near the roof inside the vehicle to generate at least one virtual height-filtered audio signal (1175); The process includes the step (1700) of mixing the at least one virtual height-filtered audio signal (1175) with at least one of the two non-height audio channels to generate the at least two audio channels (1008, 1016), method. B-EEE2. The method according to B-EEE1 (1000), wherein the audio of the immersive audio format further includes at least two additional non-height audio channels (1125, 1150), and the virtual height-filtered audio signal (1175) is mixed with each of the non-height audio channels (1050, 1100, 1125, 1150) to produce four audio channels (1008, 1016, 1032, 1064). B-EEE3. The method according to any one of B-EEE1 to 3, wherein the audio of the immersive audio format includes at least two height audio channels (1010, 1020), the virtual height filter (1300, 1400) is applied to each of the at least two height audio channels (1010, 1020) to produce at least two virtual height filtered audio signals (1175, 1200), and each of the virtual height filtered audio signals (1175, 1200) is mixed with one of the at least two non-height channels (1100, 1050). B-EEE4. The method according to any one of B-EEE1 to 3, wherein the audio of the immersive audio format includes four height audio channels (1010, 1020, 1030, 1040) and four non-height audio channels (1050, 1100, 1125, 1150), the virtual height filters (1300, 1400, 2500, 2600) are applied to each of the four height audio channels (1010, 1020, 1030, 1040) to generate four virtual height filtered audio signals (1175, 1200, 1225, 1250), and each of the virtual height filtered audio signals (1175, 1200, 1225, 1250) is mixed with one of the four non-height channels (1100, 1050, 1125, 1150). B-EEE5. The method according to any one of B-EEE1 to B-4, wherein the non-immersive loudspeaker system is a stereo or surround loudspeaker system. B-EEE6. The method according to any one of B-EEE1 to 5, wherein the virtual height filter has a filter transfer function, and the method further comprises determining the filter transfer function of the virtual height filter from one or more parameters that identify the filter transfer function. B-EEE7. The method according to any one of B-EEE1 to 6, wherein the virtual height filter has a filter transfer function having a peak at a first frequency and a notch at a second frequency higher than the first frequency. B-EEE8. Methods of B-EEE 6 and 7, wherein one or more parameters indicate at least one value among the peak, first frequency, notch, and second frequency of the filter transfer function. B-EEE 9. The method according to any one of B-EEE1 to 8, wherein the at least two audio loudspeakers (1, 2) are spaced laterally apart with respect to the listening position. B-EEE 10. The method according to B-EEE 9, further comprising the step (1800) of determining a filter transfer function for the virtual height filter based on the relative distance of the at least two loudspeakers from the listening position and the height of the roof or an area close to the roof relative to the listening position. B-EEE 11. The method of B-EEE 9, (1900) A step of obtaining a plurality of filter transfer functions for a plurality of virtual height filters based on a range of relative distances between the at least two loudspeakers from the listening position and a range of altitudes of the roof or an area close to the roof relative to the listening position; The process further includes the step of selecting one filter transfer function from the aforementioned plurality of filter transfer functions (2000), method. B-EEE 12. The method according to claim 11, wherein the selected filter transfer function is the average of the plurality of filter transfer functions. B-EEE 13. The method of claim 11, to the extent that it is dependent on any one of claims 6 to 8, wherein selecting one filter transfer function from the plurality of filter transfer functions includes selecting one or more parameters that identify the selected filter transfer function based on the average distance of the at least two loudspeakers from the listening position and based on the average elevation of the roof or an area close to the roof relative to the listening position. B-EEE 14. The method according to any one of B-EEE 11 to 13, wherein the acquisition (1900), selection (2000), application (1500), and mixing (1700) steps are applied sequentially and iteratively to each selected filter transfer function until the filter transfer function provides reproduction of the at least two channels having the highest degree of perception of sound. B-EEE 15. The method according to any one of B-EEE 6 to 14, further comprising storing one or more of the parameters in a processor as a lookup table or as an analysis function. B-EEE 16. The method according to any one of B-EEE1 to 15, further comprising applying gain to the virtual height filter. B-EEE 17. A method of B-EEE 16, wherein the gain is user-configurable. B-EEE 18. The method according to any one of B-EEE 1 to 17, wherein the audio of the immersive audio format is audio rendered in the immersive audio format, and / or the immersive audio format is Dolby Atmos or any XYZ audio format, where X≧2 is the number of front or surround audio channels, Y≧0 is a low-frequency effect or subwoofer audio channel, if present, and Z≧1 is the at least one height audio channel. B-EEE 19. An apparatus configured to perform the method described in any one of B-EEE 1 to 18. B-EEE 20. A vehicle comprising a loudspeaker system of at least two audio loudspeakers (1, 2), further comprising the equipment of B-EEE 19. B-EEE 21. A program that, when executed by a processor, includes instructions that cause the processor to perform the method described in any one of B-EEE 1 to 18. B-EEE 22. A computer-readable storage medium containing the program described in B-EEE 21.

Claims

1. A method for generating multiple audio channels from audio in an immersive audio format, which includes multiple front and rear height audio channels and multiple front and rear non-height audio channels, in order to reproduce multiple audio channels in a listening environment using an immersive loudspeaker system which includes at least one height loudspeaker and multiple front and rear non-height loudspeakers, the method being: - A step of applying a virtual height filter to a portion of each of the multiple front and rear height audio channels to generate multiple front and rear virtual height filtered audio signals, such that when one of the front and rear height audio channels is reproduced by the corresponding loudspeaker among the multiple front and rear non-height loudspeakers, the spectral components of the one height audio channel directly emitted from the non-height loudspeaker are at least partially attenuated, and the spectral components of the one height audio channel reflected from the roof or an area near the roof in the listening environment are at least partially amplified; - A step of generating an aggregated audio signal for playback by the corresponding front and rear non-height loudspeakers, in addition to the corresponding front and rear non-height audio channels of the plurality of front and rear non-height audio channels, by taking each of the plurality of front and rear virtual height filtered audio signals; The process includes the step of using at least one of the height loudspeakers to reproduce the remaining portions of each of the front and rear height audio channels of the plurality of front and rear height channels, method.

2. The method according to claim 1, wherein the portion of each of the plurality of front and rear height audio channels is a function of one or more of the positions of the at least one height loudspeaker along the length of the listening environment, or a specified balance distribution of the plurality of front and rear height audio channels between the loudspeakers of the immersive loudspeaker system.

3. The method according to claim 2, wherein the portion of each front and rear height audio channel is a linear or nonlinear function of the position of the at least one height loudspeaker along the length of the listening environment.

4. The method according to claim 1, wherein applying the virtual height filter to a portion of each of the plurality of front and rear height audio channels includes applying the virtual height filter to a portion of the front height audio channels in proportion to the position of the at least one height loudspeaker that moves from the front to the rear of the listening environment.

5. The method according to claim 1, wherein applying the virtual height filter to a portion of each of the plurality of front and rear height audio channels includes applying the virtual height filter to a portion of the rear height audio channels in proportion to the position of the at least one height loudspeaker that moves from the rear to the front of the listening environment.

6. The method according to claim 1, wherein playing the remainder of each of the plurality of front and rear height audio channels using the at least one height loudspeaker includes playing the remainder of each front height audio channel in a proportionate manner to the position of the at least one height loudspeaker moving from the rear to the front of the listening environment.

7. The method according to claim 1, wherein playing the remaining portions of each of the plurality of front and rear height audio channels using the at least one height loudspeaker includes playing the remaining portions of each rear height audio channel in proportion to the position of the at least one height loudspeaker, which is further away from the front and closer to the rear of the listening environment.

8. The method according to claim 1, wherein the audio of the immersive audio format further includes front-right and front-left non-height audio channels, rear-right and rear-left non-height audio channels, upper-front-right and upper-front-left height audio channels, and upper-rear-right and upper-rear-left height audio channels, wherein the front virtual height filtered audio signal is added to the corresponding front-right and front-left non-height audio channels, and the rear virtual height filtered audio signal is added to the corresponding rear-right and rear-left non-height audio channels to generate four added audio signals for playback by the corresponding front-right and front-left, and rear-right and rear-left non-height loudspeakers.

9. The method according to claim 1, wherein the immersive loudspeaker system includes a single upper loudspeaker.

10. The aforementioned immersive loudspeaker system includes a single upper loudspeaker, The method according to claim 8, wherein the upper front right height audio channel, the upper rear right height audio channel, the upper front left height audio channel, and the remaining portion of the upper rear left height audio channel are combined and reproduced by the single upper loudspeaker.

11. The method according to claim 1, wherein the immersive loudspeaker system includes an upper right height loudspeaker and an upper left height loudspeaker.

12. The method according to claim 8, wherein the remaining portions of the upper front right height audio channel and the upper rear right height audio channel are combined and played by the upper right height loudspeaker, and the remaining portions of the upper front left height audio channel and the upper rear left height audio channel are combined and played by the upper left height loudspeaker.

13. The method according to claim 1, wherein the virtual height filter has a filter transfer function, and the method further comprises the step of determining the filter transfer function of the virtual height filter from one or more parameters that identify the filter transfer function.

14. The method according to claim 1, wherein the virtual height filter has a filter transfer function having a peak at a first frequency and a notch at a second frequency higher than the first frequency.

15. The method according to claim 13, wherein the one or more parameters indicate at least one value among the peak, first frequency, notch, and second frequency of the filter transfer function.

16. The method according to claim 1, wherein at least two of the front and / or rear non-height loudspeakers are spaced laterally apart with respect to the listening position, and the method further comprises determining a filter transfer function for the virtual height filter based on the relative distance of the at least two loudspeakers from the listening position and the elevation of the roof or an area close to the roof with respect to the listening position.

17. At least two of the aforementioned front and rear non-height loudspeakers are spaced laterally apart from the listening position, and the method is: The steps include obtaining a plurality of filter transfer functions for a plurality of virtual height filters based on the range of relative distances between the two loudspeakers from the listening position and the range of altitudes of the roof or an area close to the roof relative to the listening position; The further step includes selecting one filter transfer function from the plurality of filter transfer functions. The method according to claim 1.

18. The method according to claim 17, The selected filter transfer function is the average of the multiple filter transfer functions, or Selecting one filter transfer function from the plurality of filter transfer functions includes selecting one or more parameters that identify the selected filter transfer function based on the average distance of the two loudspeakers from the listening position and the average altitude of the roof or an area close to the roof relative to the listening position. method.

19. The method according to claim 17, wherein the acquisition, selection, application, addition, and playback stages are applied sequentially and iteratively to each selected filter transfer function until the filter transfer function provides the highest degree of sonic perception for the playback of the multiple audio channels to be generated.

20. The method according to claim 13, further comprising storing one or more of the parameters in the processor as a lookup table or as an analysis function.

21. The method according to claim 1, further comprising applying a gain or a user-configurable gain to the virtual height filter.

22. The method according to claim 1, wherein the audio of the immersive audio format is audio rendered in the immersive audio format, and / or the immersive audio format is Dolby Atmos or any XYZ audio format, where X≧2 is the number of front or surround audio channels, Y≧0 is a low-frequency effect or subwoofer audio channel, if present, and Z>1 is the plurality of front and rear height audio channels.

23. The aforementioned listening environment is a vehicle, and the method further: The steps include processing the non-height audio channels to compensate for listening positions that are off-center from the passenger or driver in the vehicle, The method according to any one of claims 1 to 22.

24. An apparatus configured to perform the method described in any one of claims 1 to 22.

25. A vehicle comprising an immersive loudspeaker system having at least one height loudspeaker and a plurality of front and rear non-height loudspeakers, further comprising the device according to claim 24.

26. A program that, when executed by a processor, includes instructions that cause the processor to perform the method according to any one of claims 1 to 22.

27. A computer-readable storage medium storing the program described in claim 26.

Images