Multi-channel crosstalk processing

Abstract

The audio system processes a multi-channel input audio signal into a stereo signal for a left speaker and a right speaker while retaining the spatial sense of the sound field of the input audio signal. The multi-channel input audio signal includes a first left and right channel pair and a second left and right channel pair, the first left and right channel pair including a left input channel and a right input channel, and the second left and right channel pair including a left peripheral input channel and a right peripheral input channel. Subband spatial processing can be applied to the first and second left and right channel pairs. A first crosstalk processing is applied to the first left and right channel pair to generate a first crosstalk processed channel. A second crosstalk processing is applied to the second left and right channel pair to generate a second crosstalk processed channel. A left output channel and a right output channel are generated from the first and second crosstalk processed channels. The crosstalk processing can include crosstalk cancellation or crosstalk simulation.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to the field of audio signal processing, and more particularly to spatially enhanced multi-channel audio. Background Art

[0002] Surround sound refers to the reproduction of the sound of an audio signal including multiple channels using speakers located around the listener. For example, 5.1 surround sound uses six channels for front speakers, left and right speakers, a subwoofer, and rear (or "surround") left and rear right speakers. In another example, 7.1 surround sound uses eight channels by dividing the rear left and rear right speakers of the 5.1 surround sound configuration into four independent speakers, such as a left surround speaker, a right surround speaker, a left rear surround speaker, and a right rear surround speaker. The audio channels of a multi-channel audio signal can be associated with an angular position corresponding to the position of the speakers outputting the audio channels. Therefore, when an audio signal is output to speakers at different positions, the multi-channel audio signal allows the listener to perceive the sense of space in the sound field. However, when a multi-channel audio signal for surround sound is output to a stereo (e.g., left and right) speaker or a head-mounted speaker, the sense of space may be lost. Summary of the invention

[0003] Embodiments are directed to processing a (e.g., surround sound) multi-channel input audio signal into a stereo output signal for a left speaker and a right speaker while preserving or enhancing the spatial sense of the sound field of the multi-channel input audio signal. Among other things, the processing results in a listening experience whereby each channel of the audio signal is perceived as originating from the same or similar direction as would occur if the audio signal were rendered on a surround sound system (e.g., 5.1, 7.1, etc.).

[0004] In some example embodiments, a multi-channel input audio signal including a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel is received. Sub-band spatial processing is performed on the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel to create a spatial enhancement channel. The sub-band spatial processing may include gain adjustment of a mid component and a side sub-band component of the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel. Crosstalk processing is performed on the spatial enhancement channel to create a crosstalk-processed left channel and a crosstalk-processed right channel. A left output channel is generated from the left crosstalk-processed channel, and a right output channel is generated from the right crosstalk-processed channel. The crosstalk processing may include crosstalk cancellation or crosstalk simulation.

[0005] The left and right peripheral channels may include left and right surround input channels, and / or left and right surround back input channels.The multi-channel input audio signal may also include a center channel and a low frequency channel which may be combined with the output of the crosstalk processing.

[0006] In some embodiments, sub-band spatial processing is performed on each of the corresponding left and right channel pairs. For example, the sub-band spatial processing may be performed by gain-adjusting the middle and side sub-band components of the left and right input channels, gain-adjusting the middle and side sub-band components of the left and right peripheral input channels, and combining the gain-adjusted middle and side sub-band components of the left, right, left and right peripheral input channels into left and right combined channels. Crosstalk processing is performed on the left and right combined channels to generate output channels.

[0007] In some embodiments, sub-band spatial processing is performed on the combined left and right channels. For example, the sub-band spatial processing may include: combining the left input channel and the left peripheral input channel into a left combined channel, combining the right input channel and the right peripheral input channel into a right combined channel, and gain adjusting the middle sub-band components and the side sub-band components of the left combined channel and the right combined channel to create a left spatial enhancement channel and a right spatial enhancement channel. Crosstalk processing is performed on the left spatial enhancement channel and the right spatial enhancement channel to generate output channels.

[0008] In some embodiments, a binaural filter is applied to at least a portion of the input channels. For example, a binaural filter is applied to peripheral input channels to adjust the angular position associated with the peripheral input channels. In some embodiments, a binaural filter is applied to any input channel suitable for adjusting the angular position associated with the input channels, including the left or right input channels.

[0009] Some embodiments may include a system for processing a multi-channel input audio signal. The system includes a circuit device configured to: receive a multi-channel input audio signal including a plurality of left and right channel pairs, a first left and right channel pair of the plurality of left and right channel pairs including a left input channel and a right input channel, a second left and right channel pair of the plurality of left and right channel pairs including a left peripheral input channel and a right peripheral input channel; apply a first crosstalk process to the first left and right channel pair to generate a first crosstalk processed channel; apply a second crosstalk process to the second left and right channel pair to generate a second crosstalk processed channel; and generate a left output channel and a right output channel from the first and second crosstalk processed channels.

[0010] In some embodiments, the circuit device is further configured to: apply a first sub-band spatial processing to a first left and right channel pair, the first sub-band spatial processing comprising gain adjustment of a middle component and a side component of a left input channel and a right input channel; apply a second sub-band spatial processing to a second left and right channel pair, the second sub-band spatial processing comprising gain adjustment of a middle component and a side component of a left peripheral input channel and a right peripheral input channel.

[0011] Some embodiments may include a non-transitory computer-readable medium storing program code that, when executed by a processor, causes the processor to: receive a multi-channel input audio signal comprising a plurality of left and right channel pairs, a first left and right channel pair of the plurality of left and right channel pairs comprising a left input channel and a right input channel, a second left and right channel pair of the plurality of left and right channel pairs comprising a left peripheral input channel and a right peripheral input channel; apply a first crosstalk process to the first left and right channel pair to generate a first crosstalk processed channel; apply a second crosstalk process to the second left and right channel pair to generate a second crosstalk processed channel; and generate a left output channel and a right output channel from the first and second crosstalk processed channels.

[0012] In some embodiments, the computer-readable medium further includes program code that causes the processor to: apply a first sub-band spatial processing to a first left and right channel pair, the first sub-band spatial processing comprising gain adjusting mid and side components of the left and right input channels; and apply a second sub-band spatial processing to a second left and right channel pair, the second sub-band spatial processing comprising gain adjusting mid and side components of the left and right peripheral input channels.

[0013] Some embodiments may include a method for processing a multi-channel input audio signal. The method may include, by a circuit device: receiving a multi-channel input audio signal including a plurality of left and right channel pairs, a first left and right channel pair of the plurality of left and right channel pairs including a left input channel and a right input channel, a second left and right channel pair of the plurality of left and right channel pairs including a left peripheral input channel and a right peripheral input channel; applying a first crosstalk process to the first left and right channel pair to generate a first crosstalk processed channel; applying a second crosstalk process to the second left and right channel pair to generate a second crosstalk processed channel; and generating a left output channel and a right output channel from the first and second crosstalk processed channels.

[0014] In some embodiments, the method further includes, by the circuit device: applying a first sub-band spatial processing to a first left and right channel pair, the first sub-band spatial processing comprising gain adjustment of a mid component and a side component of a left input channel and a right input channel; applying a second sub-band spatial processing to a second left and right channel pair, the second sub-band spatial processing comprising gain adjustment of a mid component and a side component of a left peripheral input channel and a right peripheral input channel. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 An example of a surround sound stereo audio reproduction system according to one embodiment is illustrated.

[0016] Figure 2 An example of an audio system according to one embodiment is illustrated.

[0017] Figure 3 An example of a sub-band spatial processor according to one embodiment is illustrated.

[0018] Figure 4 An example of a crosstalk cancellation processor according to one embodiment is illustrated.

[0019] Figure 5 An example embodiment of a method for utilizing Figure 2 An example of a method for enhancing an audio signal in an audio system is shown in FIG.

[0020] Figure 6 An example of an audio system according to one embodiment is illustrated.

[0021] Figure 7 An example embodiment of a method for utilizing Figure 6 An example of a method for enhancing an audio signal in an audio system is shown in FIG.

[0022] Figure 8 An example of a computer system according to one embodiment is illustrated.

[0023] Fig. 9 An example of an audio system according to one embodiment is illustrated.

[0024] Fig.10 An example of an audio system according to one embodiment is illustrated.

[0025] Fig.11 An example embodiment of a method for utilizing Fig. 9 or Fig.10 An example of a method for enhancing an audio signal in an audio system is shown in FIG.

[0026] Fig.12 An example of a crosstalk simulation processor according to one embodiment is illustrated. DETAILED DESCRIPTION

[0027] The features and advantages described in the specification are not all inclusive, and in particular, many additional features and advantages will be apparent to those of ordinary skill in the art in view of the drawings, the specification, and the claims. In addition, it should be noted that the language used in the specification is primarily selected for readability and instructional purposes, and may not be selected to outline or limit the inventive subject matter.

[0028] The accompanying drawings (Figures) and the following description relate to the preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that can be employed without departing from the principles of the present invention.

[0029] Reference will now be made in detail to several embodiments of the present invention(s), examples of which are illustrated in the accompanying drawings. Note that similar or identical reference numerals may be used in the drawings wherever feasible and may indicate similar or identical functionality. The drawings depict the embodiments for illustrative purposes only. Those skilled in the art will readily appreciate from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.

[0030] Example surround sound stereo and example audio system

[0031] The audio system discussed herein provides crosstalk processing and spatial enhancement for a multi-channel surround sound audio signal output to stereo (e.g., left and right) speakers. The signal processing results in the preservation or enhancement of the spatial sense of the sound field encoded in the multi-channel surround sound audio signal. Among other things, the spatial sense achieved using a multi-speaker surround sound system is achieved using stereo speakers.

[0032] Figure 1An example of a surround sound stereophonic audio reproduction system 100 according to one embodiment is illustrated. The system 100 is an example of a 7.1 surround sound system that provides audio signal reproduction to a listener 140. The system 100 includes a left speaker 110L, a right speaker 110R, a center speaker 115, a subwoofer 125, a left surround speaker 120L, a right surround speaker 120R, a left surround back speaker 130L, and a right surround back speaker 130R. The center speaker 115 and the subwoofer 125 can be positioned in front of the listener 140, which defines a forward axis of 0°. The left speaker 110L can be positioned at an angle between -20° and -30° relative to the forward axis, and the right speaker 110R can be positioned at an angle between 20° and 30° relative to the forward axis. The left surround speaker 120L can be positioned at an angle between -90° and -110° relative to the forward axis, and the right surround speaker 120R can be positioned at an angle between 90° and 110° relative to the forward axis. The left surround back speaker 130L can be positioned at an angle between -135° and -150° relative to the forward axis, and the right surround back speaker 130R can be positioned at an angle between 135° and 150° relative to the forward axis. The system 100 can be configured to receive an audio signal including channels for each speaker 110, 115, 120 and 130 and the subwoofer 125. The multiple speakers and their positioning arrangement provide a sense of space that can be perceived by the listener 140 in the sound field. As discussed in more detail below, the audio system can be configured to process a multi-channel input audio signal for the surround sound system 100 into an enhanced stereo signal for the left and right speakers (e.g., speakers 110L and 110R) that reproduces or simulates the sense of spatiality in the sound field generated by the surround sound system 100 using the multi-channel audio signal.

[0033] Figure 2 An example of an audio system 200 according to one embodiment is illustrated. The audio system 200 receives input audio signals including a left input channel 201A, a right input channel 210B, a center input channel 210C, a low frequency input channel 210D, a left surround input channel 210E, a right surround input channel 210F, a left surround back input channel 210G, and a right surround back input channel 210H.

[0034] Channels 210E, 210F, 210G, and 210H are examples of peripheral channels for surround speakers. Peripheral channels may include channels in addition to left and right input channels. Peripheral channels may include channel pairs, such as left and right pairs, or front and back pairs, or other pairs of arrangements. For example, when the input audio signal is output by the surround sound audio reproduction system 100, the left surround speaker 120L receives the left surround input channel 210E, the right surround speaker 120R receives the right surround input channel 210F, the left surround rear speaker 130L receives the left surround rear input channel 210G, and the right surround rear speaker 130R receives the right surround rear input channel 210H. In some embodiments, the input audio signal has fewer or more peripheral channels. For example, the audio input signal for a 5.1 surround sound system may include only two peripheral channels, such as left and right surround input channels that may be output to left and right surround speakers. Similarly, left speaker 110L may receive left input channel 210A, right speaker 110R may receive right input channel 210B, center speaker 115 may receive center input channel 210C, and subwoofer 125 may receive low frequency input channel 210D. The input audio signals provide a sense of spatiality of the sound field when output by surround sound stereophonic audio reproduction system 100.

[0035] The audio system 200 receives an input audio signal and generates an output signal including a left output channel 290L and a right output channel 290R. The audio system 200 can combine the input channels of the input audio signal and can further provide enhancements such as sub-band spatial processing and crosstalk cancellation to generate the output audio signal. The left output channel 290L can be provided to the left speaker, and the right output channel 290R can be output to the right speaker. The output audio signal uses the left speaker and the right speaker (e.g., the left speaker 110L and the right speaker 110R) to provide a spatial sense of the sound field, which is usually achieved by outputting the input audio signal using a surround sound system including multiple (e.g., peripheral) speakers.

[0036] The audio system 200 includes gains 215A, 215B, 215C, 215D, 215E, 215F, 215G and 215H, sub-band spatial processors 230A, 230B and 230C, an overhead filter 220, a divider 240, binaural filters 250A, 250B, 250C and 250D, a left channel combiner 260A, a right channel combiner 260B, a crosstalk cancellation processor 270, a left channel combiner 260C, a right channel combiner 260D and an output gain 280.

[0037] Each of the gains 215A to 215H may receive a corresponding input channel 210A to 210H and may apply a gain to the input channel 210A to 210H. The gains 215A to 215H may be different to adjust the gains of the input channels relative to each other, or may be the same. In some embodiments, positive gains are applied to the left and right peripheral input channels 210E, 210F, 210G, and 210H, while negative gains are applied to the center input channel 210C. For example, gain 215A may apply a 0dB gain, gain 215B may apply a 0dB gain, gain 215C may apply a -3dB gain, gain 215D may apply a 0db gain, gain 215E may apply a 3dB gain, gain 215F may apply a 3dB gain, gain 215G may apply a 3dB gain, and gain 215H may apply a 3dB gain.

[0038] Gain 215A and gain 215B are coupled to sub-band spatial processor 230. Similarly, gains 215E and 215F are coupled to sub-band spatial processor 230B, and gains 215G and 215H are coupled to sub-band spatial processor 230C. Sub-band spatial processors 230A, 230B, and 230C each apply sub-band spatial processing to corresponding left and right channel pairs.

[0039] Each sub-band spatial processor 230 performs sub-band spatial processing on the left and right input channels by gain adjusting the mid and side sub-band components of the left and right input channels to generate left and right spatial enhancement channels. The sub-band spatial processor 230A performs sub-band spatial processing on the left and right input channels, while the other sub-band spatial processors 230B and 230C each perform sub-band spatial processing on the corresponding left and right peripheral channels. Depending on the number of peripheral channels in the input audio signal, the audio system 200 may include more or fewer sub-band spatial processors. In some embodiments, channels that do not have left / right counterparts (such as the center input channel 210C, the low-frequency input channel 210D, or other types of channels such as rear center, overhead center, etc.) can bypass SBS processing.

[0040] Subband spatial processor 230B is coupled to binaural filters 250A and 250B. Subband spatial processor 230B provides a left spatial enhancement channel to binaural filter 250A and provides a right spatial enhancement channel to binaural filter 250B. Similarly, subband spatial processor 230C is coupled to binaural filters 250C and 250D. Subband spatial processor 230C provides a left spatial enhancement channel to binaural filter 250C and provides a right spatial enhancement channel to binaural filter 250D. Additional details about subband spatial processor 230 are provided in Figure 3 is shown in and discussed below.

[0041] Each of the binaural filters 250A, 250B, 250C, and 250D applies a head-related transfer function (HRTF) that describes the target source position from which the listener should perceive the sound of the input channel. Each binaural filter receives the input channel and generates a left output channel and a right output channel by applying the HRTF that adjusts the angular position associated with the input channel. The angular position may include an angle defined relative to the listener 140 in an XY "azimuth" plane, such as Figure 1, and may also include angles defined in the Z axis, such as for ambisonic signals or channel-based formats containing signals intended to be rendered above or below the XY plane relative to the listener 140. For example, the binaural filter 250A may be configured to apply a filter based on a left surround input channel 210E associated with an angle (defined in the XY plane) between -90° and -110° relative to the forward axis of the left surround speaker 120L. The binaural filter 250B may be configured to apply a filter based on a right surround input channel 210F associated with an angle between 90° and 110° relative to the forward axis of the right surround speaker 120L. The binaural filter 250C may be configured to apply a filter based on a left surround back input channel 210G associated with an angle between -135° and -150° relative to the forward axis of the left surround back speaker 130L. Binaural filter 250D can be configured to apply filter based on right surround rear input channel 210H associated with the angle between 135 ° and 150 ° relative to the forward axis of right surround rear speaker 130R. In some embodiments, binaural processing can be completely bypassed to retain the spectral uniformity between channels. One or more of binaural filters 250A, 250B, 250C and 250D can be omitted from audio system 200. However, binaural filters 250A, 250B, 250C and 250D can be used to enhance spatial imaging. In some embodiments, binaural filtering can be applied to channels other than peripheral input channels. For example, binaural filters can be applied to each of the left spatial enhancement channel and the right spatial enhancement channel output from subband spatial processor 230A to adjust for different left output speaker positions and right output speaker positions. In another example, if the input audio signal includes channels associated with other speaker positions (i.e., overhead, rear center, etc.), binaural processing can be applied to other input channels. In this sense, binaural processing may be applied to one or more of the left input channel 210A, the right input channel 210B, the center input channel 210C, or the low frequency input channel 210D. In some embodiments, HRTFs are not applied, and one or more of the binaural filters 250A, 250B, 250C, and 250D may be bypassed or omitted from the system 200.

[0042] An example binaural filter can be defined by Equation 1:

[0043] S o (z) = H (θ, z) S i (z) Equation (1)

[0044] Where S o and S i are the output and input signals respectively.i and S o The angle of each channel in is encoded. The z value is an arbitrary complex number, and our solution for it is a function of the encoded frequency. Therefore, H(θ,z) is a function of the angle θ and z, returning a transfer function, which is itself a function of z, which can be selected or interpolated from a set of transfer functions, which may be derived from an anthropometric database. In this representation, if multi-channel processing is desired, the angle θ and S and H(θ) as a function of z can be evaluated as vectors. In this case, each coefficient in S(z) and H(θ,z) corresponds to a different channel, while each coefficient in θ associates an angle with each channel.

[0045] In some embodiments, the input audio signal is an ambient stereo audio signal that defines a speaker-independent representation of a sound field. The ambient sound audio signal can be decoded into a multi-channel audio signal for a surround sound system. The channels can be associated with speaker positions at different locations, including positions above or below the listener. A binaural filter can be applied to each decoded input channel of the ambient sound audio signal to adjust for the associated position of the decoded input audio channel.

[0046] In some embodiments, binaural filtering is performed before sub-band spatial processing. For example, binaural filters can be applied to one or more input channels that are suitable for adjusting the angular position associated with the channels. For each left input channel pair and right input channel pair, the left output channel of the binaural filter can be combined, and the right output channel of the binaural filter can be combined, and the sub-band spatial processing can be applied to the combined left and right channels. In some embodiments, binaural filters are applied to central input channel 210C or low-frequency input channel 210D. In some embodiments, binaural filters are applied to each input channel except low-frequency input channel 210D.

[0047] The left channel combiner 260A is coupled to the sub-band spatial processor 230A and the binaural filters 250A, 250B, 250C, and 250D. The left channel combiner 260A receives the left output channel of the sub-band spatial processor 230A and the binaural filters 250A, 250B, 250C, and 250D, and combines these channels into a left combined channel. The right channel combiner 260B is also coupled to the sub-band spatial processor 230A and the binaural filters 250A, 250B, 250C, and 250D. The right channel combiner 260B receives the right output channel of the sub-band spatial processor 230A and the binaural filters 250A, 250B, 250C, and 250D, and combines these channels into a right combined channel.

[0048] The crosstalk cancellation processor 270 receives the left input channel and the right input channel and performs crosstalk cancellation to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel. The crosstalk cancellation processor is coupled to the left channel combiner 260A to receive the left combined channel, and is coupled to the right channel combiner 260B to receive the right combined channel. Here, the left combined channel and the right combined channel processed by the crosstalk cancellation processor 270 represent the mixed left corresponding input channel and the right corresponding input channel. Additional details about the crosstalk cancellation processor 270 are described in Figure 4 is shown in and discussed below.

[0049] The elevated filter 220 receives the center input channel 210C and applies a high frequency shelf or peak filter. The elevated filter 220 provides a "voice boost" on the center input channel 210C. In some embodiments, the elevated filter 220 is bypassed or omitted from the audio system 200. The elevated filter 220 can attenuate or amplify frequencies above the corner frequency. The elevated filter 220 is coupled to the left channel combiner 260C and the right channel combiner 260D. In some embodiments, the elevated filter 220 is defined by a 750Hz corner frequency, a +3dB gain, and a 0.8Q factor. The elevated filter 220 generates a left center channel and a right center channel as outputs, such as by separating the center input channel into two separate left and right center channels.

[0050] The distributor 240 receives the low frequency input channel 210D and separates the low frequency input channel 210D into a left low frequency channel and a right low frequency channel. The distributor 240 is coupled to the left channel combiner 260C and the right channel combiner 260D and provides the left low frequency channel to the left channel combiner 260C and the right low frequency channel to the right channel combiner 260D.

[0051] The left channel combiner 260C is coupled to the crosstalk cancellation processor 270, the high shelf filter 220 and the distributor 240. The left channel combiner 260C receives the left crosstalk channel from the crosstalk cancellation processor 270, the left center channel from the high shelf filter 220 and the left low frequency channel from the distributor 240 and combines these channels into a left output channel.

[0052] The right channel combiner 260D is coupled to the crosstalk cancellation processor 270, the high shelf filter 220 and the distributor 240. The right channel combiner 260D receives the right crosstalk channel from the crosstalk cancellation processor 270, the right output channel from the high shelf filter 220 and the right low frequency channel from the distributor 240 and combines these channels into a right output channel.

[0053] In some embodiments, the left center channel from the high shelf filter 220 and the left low frequency channel from the distributor 240 are combined by the left channel combiner 260A with the left spatial enhancement channel from the subband spatial processor 230A and the left output channel from the binaural filters 250A, 250B, 250C and 250D to generate the left combined channel. Similarly, the right output channel from the high shelf filter 220 and the right low frequency channel from the distributor 240 are combined by the right channel combiner 260B with the right spatial enhancement channel from the subband spatial processor 230A and the right output channel from the binaural filters 250A, 250B, 250C and 250D to generate the right combined channel. The left combined channel and the right combined channel are input into the crosstalk cancellation processor 270. Here, the center and low frequency channels receive the crosstalk cancellation operation. The left channel combiner 260C and the right channel combiner 260D can be omitted. In some embodiments, one of the center or low frequency channels receives a crosstalk cancellation operation.

[0054] Output gain 280 is coupled to left channel combiner 260C and right channel combiner 260D. Output gain 280 applies gain to the left output channel from left channel combiner 260C and applies gain to the right output channel from right channel combiner 260D. Output gain 280 may apply the same gain to the left and right output channels, or may apply different gains. Output gain 280 outputs left output channel 290L and right output channel 290R representing the channels of the output signal of audio system 200.

[0055] Example Subband Spatial Processor

[0056] Figure 3 An example of a sub-band spatial processor 230 according to one embodiment is illustrated. The sub-band spatial processor 230 is an example of a sub-band spatial processor 230A, 230B, or 230C of the audio system 200. The sub-band spatial processor 230 includes a spatial band allocator 340, a spatial band processor 345, and a spatial band combiner 350. The spatial band allocator 340 is coupled to the spatial band processor 345, and the spatial band processor 345 is coupled to the spatial band combiner 350.

[0057] The spatial frequency band distributor 340 includes an L / R to M / S converter 312 which receives the left input channel X L and right input channel X R , and convert these inputs into spatial components X m and the non-spatial component X s . Spatial component X s You can use the left input channel X L and right input channel X R The non-spatial component X is generated by subtractingm You can use the left input channel X L and right input channel X R Add to generate.

[0058] The spatial frequency band processor 345 receives the non-spatial components X m And apply a set of subband filters to generate the enhanced non-spatial subband components E m The spatial band processor 345 also receives the spatial sub-band components X s And apply a set of subband filters to generate the enhanced non-spatial subband components E m The subband filters may include various combinations of peak filters, notch filters, low pass filters, high pass filters, low shelf filters, high shelf filters, band pass filters, band stop filters, and / or all pass filters.

[0059] In some embodiments, the spatial frequency band processor 345 includes a processor for the non-spatial components X m The subband filter for each of the n frequency subbands and for the spatial component X s For example, for n=4 subbands, the spatial frequency band processor 345 includes a subband filter for each of the n frequency subbands of the non-spatial component X. m The series of subband filters includes: an intermediate equalization (EQ) filter 362 (1) for subband (1), an intermediate EQ filter 362 (2) for subband (2), an intermediate EQ filter 362 (3) for subband (3), and an intermediate EQ filter 362 (4) for subband (4). Each intermediate EQ filter 362 applies a filter to the non-spatial component X m The frequency subband part of the enhanced non-spatial component E m .

[0060] The spatial frequency band processor 345 also includes a processor for the spatial components X s A series of subband filters for frequency subbands of X include a side equalization (EQ) filter 364 (1) for subband (1), a side EQ filter 364 (2) for subband (2), a side EQ filter 364 (3) for subband (3), and a side EQ filter 364 (4) for subband (4). Each side EQ filter 364 applies a filter to the spatial component X s The frequency subband part of the enhanced spatial component E s .

[0061] Non-spatial component X m and the spatial component X sEach of the n frequency subbands may correspond to a frequency range. For example, subband (1) may correspond to 0 to 300 Hz, subband (2) may correspond to 300 to 510 Hz, subband (3) may correspond to 510 to 2700 Hz, and subband (4) may correspond to 2700 Hz to the Nyquist frequency. In some embodiments, the n frequency subbands are a set of merged critical bands. A corpus of audio samples from a variety of music genres may be used to determine the critical bands. The long-term average energy ratio from the middle to the side components on the 24 Bark scale critical bands is determined from the samples. Continuous frequency bands with similar long-term average ratios are then grouped together to form a set of critical bands. The range of the frequency subbands and the number of frequency subbands may be adjustable.

[0062] In some embodiments, middle EQ filter 362 or side EQ filter 364 may include a biquad filter having a transfer function defined by Equation 2:

[0063]

[0064] where z is a complex variable. The filter can be implemented using a direct type I topology defined by Equation 3:

[0065]

[0066] Where X is the input vector and Y is the output. Other topologies may be good for certain processors, depending on their maximum word length and saturation behavior.

[0067] The biquad can then be used to implement any second-order filter with real-valued inputs and outputs. To design a discrete-time filter, a continuous-time filter is designed and transformed to discrete time via a bilinear transform. Additionally, frequency warping can be used to compensate for any resulting shifts in center frequency and bandwidth.

[0068] For example, a peak filter may include an S-plane transfer function defined by Equation 4:

[0069]

[0070] where s is a complex variable, A is the amplitude of the peak, and Q is the filter "quality" (the standard derivation is: ). The digital filter coefficients are:

[0071] b 0 =1+αA

[0072] b 1 =-2*cos(ω 0 ) 1=-2cos(ω 0 )

[0073] b 2 =1-αA where ω 0 is the center frequency of the filter in radians, and

[0074] The spatial band combiner 350 receives the mid component and the side component, applies a gain to each component, and converts the mid component and the side component into a left channel and a right channel. For example, the spatial band combiner 350 receives the enhanced non-spatial component E m and the enhanced spatial component E s , and in the enhanced non-spatial component E m and the enhanced spatial component E s Convert to left spatial enhancement channel E L and right spatial enhancement channel E R Global mid gain and side gain were performed before.

[0075] More specifically, the spatial band combiner 350 includes a global middle gain 322, a global side gain 324, and an M / S to L / R converter 326 coupled to the global middle gain 322 and the global side gain 324. The global middle gain 322 receives the enhanced non-spatial component E m and applies the gain, and the global side gain 324 receives the enhanced spatial component E s and applies the gain. The M / S to L / R converter 326 receives the enhanced non-spatial component E from the global intermediate gain 322 m and the enhanced spatial component E from the global side gain 324 s , and convert these inputs into the left spatial enhancement channel E L and right spatial enhancement channel E R .

[0076] Example Crosstalk Cancellation Processor

[0077] Figure 4 The crosstalk cancellation processor 270 is illustrated according to an example embodiment. The crosstalk cancellation processor 270 receives the left channel (eg, the left spatial enhancement channel E) from the left channel combiner 260A. L ) as input and receives the right channel (eg, the right spatial enhancement channel E) from the right channel combiner 260B. R ) as input, and performs crosstalk cancellation on the left and right channels to generate a left output channel O L and right output channel O R .

[0078] The crosstalk cancellation processor 270 includes an in-band divider 410, inverters 420 and 422, opposite side estimators 430 and 440, combiners 450 and 452, and an in-band and out-of-band combiner 460. These components operate together to divide the input channels T L , T R The output channel O is divided into an in-band component and an out-of-band component, and crosstalk cancellation is performed on the in-band component to generate an output channel O L , O R .

[0079] By dividing the input audio signal E into different frequency band components and by performing crosstalk cancellation on selective components (e.g., in-band components), crosstalk cancellation can be performed for specific frequency bands while avoiding degradation in other frequency bands. If crosstalk cancellation is performed without dividing the input audio signal E into different frequency bands, the audio signal after such crosstalk cancellation may exhibit significant attenuation or amplification in non-spatial and spatial components in low frequencies (e.g., below 350 Hz), high frequencies (e.g., above 12000 Hz), or both. By selectively performing crosstalk cancellation in the in-band where the vast majority of influential spatial cues are located (e.g., between 250 Hz and 14000 Hz), balanced overall energy can be maintained across the entire frequency spectrum of the mixture, especially in the non-spatial components.

[0080] The in-band and out-of-band distributor 410 converts the input channel E L 、E R Separate into in-band channels E L,In 、E R,In and out-of-band channel E L,Out 、E R,Out Specifically, the in-band and out-of-band distributor 410 divides the left enhancement compensation channel E L Divided into left in-band channel E L,In and left out-band channel E L,Out Similarly, the in-band and out-of-band distributor 410 divides the right enhancement compensation channel E R Separated into right in-band channel E R,In and right out-band channel E R,Out Each in-band channel may contain a portion of the respective input channel corresponding to a frequency range comprising, for example, 250 Hz to 14 kHz. The frequency band range may be adjustable, for example according to loudspeaker parameters.

[0081] The inverter 420 and the contralateral estimator 430 operate together to generate the left contralateral cancellation component S L , to compensate for the left in-band channel E L,In Similarly, the inverter 422 and the opposite side estimator 440 operate together to generate the right opposite side cancellation component S R, to compensate for the right in-band channel E R,In The contralateral sound component caused by

[0082] In one approach, the inverter 420 receives the in-band channel E L,In , and the received in-band channel E L,In The polarity of the channel E is reversed to generate the reverse phase in-band channel E L,In' The contralateral estimator 430 receives the inverted in-band channel E L,In' , and extract the inverse-phase in-band channel E corresponding to the contralateral sound component by filtering L,In' Because the filtering is in the anti-phase band, channel E L,In' Therefore, the part extracted by the opposite side estimator 430 becomes the in-band channel E L,In Therefore, the portion extracted by the contralateral estimator 430 becomes the left contralateral cancellation component S L , which can be added to the corresponding in-band channel E R,In To reduce the in-band sound channel E L,In In some embodiments, the inverter 420 and the contra-side estimator 430 are implemented in a different order.

[0083] The inverter 422 and the opposite side estimator 440 are related to the in-band channel E R,In A similar operation is performed to generate the right contralateral cancellation component S R Therefore, for the sake of brevity, its detailed description is omitted here.

[0084] In one example implementation, the contralateral estimator 430 includes a filter 432, an amplifier 434, and a delay unit 436. The filter 432 receives an inverted input channel E L,In' , and through the filter function, extract the inverse phase in-band channel E corresponding to the contralateral sound component L,In' An example filter implementation is a notch or shelf filter with a center frequency selected between 5000 and 10000 Hz and a Q selected between 0.5 and 1.0. The gain (G) in decibels is dB ) can be derived from Equation 5:

[0085] G dB = -3.0 - log 1.333 (D) Equation (5)

[0086] Where D is the delay amount of the delay unit 1556A / B in samples, for example at a sampling rate of 48 kHz. Another implementation is a low pass filter with a corner frequency selected between 5000 and 10000 Hz and a Q selected between 0.5 and 1.0. In addition, the amplifier 434 amplifies the extracted portion by a corresponding gain factor G L,In , and the delay unit 436 delays the amplified output from the amplifier 434 according to the delay function D to generate the left contralateral cancellation component S L The opposite side estimator 440 includes a filter 442, an amplifier 444, and a delay unit 446. The delay unit 446 is used to invert the in-band channel E. R,In' A similar operation is performed to generate the right contralateral cancellation component S R In one example, the contralateral estimators 430, 440 generate the left contralateral cancellation component S according to the following equation: L , S R :

[0087] S L =D[G L,In *F[E L,In ']] Equation (6)

[0088] S R =D[G R,In *F[E R,In ']] Equation (7)

[0089] Where F[] is the filter function and D[] is the delay function.

[0090] The configuration of the crosstalk cancellation can be determined by the speaker parameters. In one example, the filter center frequency, delay amount, amplifier gain, and filter gain can be determined based on the angle formed between the two output speakers of the output signal relative to the listener, or other characteristics of the speakers such as relative position, power, etc. In some embodiments, values ​​between the speaker angles are used to interpolate other values.

[0091] The combiner 450 cancels the right-side component S R Combined to left in-band channel E L,In To generate the left in-band compensation channel U L , and the combiner 452 combines the left contralateral cancellation component SL into the right in-band channel E R,In Generate right in-band compensation channel U R The in-band and out-of-band combiner 460 combines the left in-band compensation channel U L With out-of-band channel E L,Out Combined to generate the left output channel O L , and the right in-band compensation channel U R With out-of-band channel E R,OutCombined to generate the right output channel O R .

[0092] Therefore, the left output channel O L Includes in-band channel T R,In The right contralateral cancellation component S corresponding to the anti-phase of a part of the contralateral sound R , and the right output channel O R Includes in-band channel T L,In The left contralateral cancellation component S corresponding to the anti-phase of a part of the contralateral sound L In this configuration, the right output channel O R The wave front of the same-side sound component output by the right speaker (eg, speaker 110R) arriving at the right ear can cancel the sound component according to the left output channel O. L The wavefront of the contralateral sound component output by the right speaker (e.g., speaker 110L). Similarly, according to the left output channel O L The wavefront of the same-side sound component output by the left speaker reaches the left ear and can be eliminated according to the right output channel O R The wavefront of the contralateral sound component output by the right speaker. Therefore, the contralateral sound component can be reduced to enhance spatial detectability.

[0093] Example of audio signal enhancement process

[0094] Figure 5 An example embodiment of a method for utilizing Figure 2 An example of a method 500 for enhancing an audio signal by the audio system 200 shown in FIG. In some embodiments, the method 500 may include different and / or additional steps, or some steps may be in a different order.

[0095] The audio system 200 receives 505 a multi-channel input audio signal. The multi-channel audio signal may be a surround sound audio signal including a left input channel, a right input channel, at least one left peripheral input channel, and at least one right peripheral input channel. The multi-channel audio signal may also include a center input channel 210C and a low frequency input channel 210D. For example, the input audio signal may be for a 7.1 surround sound system including a left input channel 210A and a right input channel 210B, and peripheral channels including a left surround input channel 210E and a right surround input channel 210F, and a left surround back input channel 210G and a right surround back input channel 210H. In another example of an input audio signal for a 5.1 surround sound system, the peripheral channels may include a single left peripheral channel and a single right peripheral channel.

[0096] The audio system 200 (e.g., gains 215A through 215H) applies 510 gains to channels of a multi-channel input audio signal. The gains 215A through 215H may vary to control the contribution of particular input channels to the output signals generated by the audio system 200. In some embodiments, the center input channel 210C receives a negative gain while the peripheral input channels receive a positive gain.

[0097] The audio system 200 (e.g., the subband spatial processor 230A) generates 515 left and right spatial enhancement channels by performing subband spatial processing on the left and right input channels. For example, the subband spatial processor 230A generates the spatial enhancement channels by adjusting the gains of n subbands of the mid and side components of the left and right input channels 210A and 210B.

[0098] The audio system 200 (e.g., the sub-band spatial processor 230B and / or 230C) generates 520 left spatially enhanced peripheral channels and right spatially enhanced peripheral channels by performing sub-band spatial processing on the left peripheral input channel and the right peripheral input channel. For example, the sub-band spatial processor 230B adjusts the gains of n sub-bands of the mid components and the side components of the left surround input channel 210E and the right surround input channel 210F to generate left spatially enhanced peripheral channels and right spatially enhanced peripheral channels. The sub-band spatial processor 230C adjusts the gains of n sub-bands of the mid components and the side components of the left surround back input channel 210G and the right surround back input channel 210H to generate left spatially enhanced peripheral channels and right spatially enhanced peripheral channels.

[0099] The audio system 200 (e.g., binaural filters 250A to 250D) applies 525 binaural filters to each of the peripheral channels of left spatial enhancement and right spatial enhancement. For example, binaural filter 250A generates a left output channel and a right output channel from the peripheral channels of the left spatial enhancement output from the sub-band spatial processor 230B by applying a head-related transfer function (HRTF). Binaural filter 250B generates a left output channel and a right output channel from the right channel of the spatial enhancement output from the sub-band spatial processor 230B by applying HRTF. Binaural filter 250C generates a left output channel and a right output channel from the left channel of the spatial enhancement output from the sub-band spatial processor 230C by applying HRTF. Binaural filter 250D generates a left output channel and a right output channel from the right channel of the spatial enhancement output from the sub-band spatial processor 230C by applying HRTF. In some embodiments, binaural filtering is bypassed.

[0100] The audio system 200 (eg, the high shelf filter 220) applies 530 the high shelf filter to the center input channel 210C. In some embodiments, a gain is applied to the center input channel 210C. Additionally, the high shelf filter 220 separates the center input channel 210C into a left center channel and a right center channel.

[0101] The audio system 200 (eg, the distributor 240 ) splits 535 the low frequency input channel into a left low frequency channel and a right low frequency channel.

[0102] The audio system 200 (eg, the left channel combiner 260A) combines 540 the left spatial enhancement channel from the subband spatial processor 230A and the left output channels of the binaural filters 250A, 250B, 250C, and 250D to generate a left combined channel. For example, the left spatial enhancement channel may be summed with the left output channel.

[0103] The audio system 200 (eg, the right channel combiner 260B) combines 545 the right spatial enhancement channel from the subband spatial processor 230A and the right output channels of the binaural filters 250A, 250B, 250C, and 250D to generate a right combined channel. For example, the right spatial enhancement channel may be summed with the right output channel.

[0104] The audio system 200 (eg, the crosstalk cancellation processor 270 ) performs 550 crosstalk cancellation on the left and right combined channels to generate left and right crosstalk cancellation channels.

[0105] The audio system 200 (e.g., left channel combiner 260C and right channel combiner 260D) combines 555 the left crosstalk cancellation channel from the crosstalk cancellation processor 270 with the left low frequency channel from the divider 240 and the left center channel from the overhead filter 220 to generate a left output channel, and combines the right crosstalk cancellation channel from the crosstalk cancellation processor 270 with the right low frequency channel from the divider 240 and the right center channel from the overhead filter 220 to generate a right output channel. In addition, the audio system 200 (e.g., output gain 280) may apply a gain to each of the left and right output channels. The audio system 200 outputs an output audio signal including left and right output channels 290L and 290R. Example Audio System and Example Audio Processing Process

[0106] Figure 6An example of an audio system 600 according to one embodiment is illustrated. The audio system 600 may be similar to the audio system 200, but may differ from the audio system 200 at least in that the left and right input channels are combined with the left and right peripheral channels prior to sub-band spatial processing of the audio system 600. Here, a single sub-band spatial processor and corresponding sub-band spatial processing steps may be used, rather than separate sub-band spatial processors for left and right channel pairs as shown for the audio system 200.

[0107] The audio system 600 receives an input audio signal. The input audio signal may include a left input channel 610A, a right input channel 610B, a center input channel 610C, a low frequency input channel 610D, a left surround input channel 610E, a right surround input channel 610F, a left surround back input channel 610G, and a right surround back input channel 610H. Channels 610E, 610F, 610G, and 610H are examples of peripheral channels that may be provided to surround speakers. In some embodiments, the audio system 600 may receive and process input audio signals having fewer or more channels.

[0108] The audio system 600 generates an output signal including a left output channel 690L and a right output channel 690R using enhancements such as sub-band spatial processing and crosstalk cancellation of the input audio signal. The left output channel 690L may be provided to a left speaker, and the right output channel 690R may be output to a right speaker. The output audio signal provides a sense of space of a sound field associated with a surround sound input audio signal using a left speaker and a right speaker (e.g., a left speaker 110L and a right speaker 110R).

[0109] The audio system 600 includes gains 615A, 615B, 615C, 615D, 615E, 615F, 615G and 615H, an overhead filter 620, a divider 640, binaural filters 650A, 650B, 650C and 650D, a left channel combiner 660A, a right channel combiner 660B, a sub-band spatial processor 630, a crosstalk cancellation processor 670, a left channel combiner 660C, a right channel combiner 660D and an output gain 680.

[0110] Each of the gains 615A to 615H may receive a corresponding input channel 610A to 610H and may apply a gain to the input channels 610A to 610H. The gains 615A to 615H may be different to adjust the gains of the input channels relative to each other, or may be the same. In some embodiments, positive gains are applied to the left and right peripheral input channels 610E, 610F, 610G, and 610H, while negative gains are applied to the center input channel 610C. For example, gain 615A may apply a 0dB gain, gain 615B may apply a 0dB gain, gain 615C may apply a -3dB gain, gain 615D may apply a 0dB gain, gain 615E may apply a 3dB gain, gain 615F may apply a 3dB gain, gain 615G may apply a 3dB gain, and gain 615H may apply a 3dB gain.

[0111] 6. Gain 615A for left input channel 610A is coupled to left channel combiner 660A. Gain 615B for right input channel 610B is coupled to right channel combiner 660B. Gain 615C is coupled to high shelf filter 620. Gain 615D is coupled to divider 640. Gains 615E, 615F, 615G, and 615H for peripheral input channels are each coupled to binaural filter 650. In particular, gain 615E is coupled to binaural filter 650A, gain 615F is coupled to binaural filter 650B, gain 615G is coupled to binaural filter 650C, and gain 615H is coupled to binaural filter 650D.

[0112] Each of binaural filters 650A, 650B, 650C and 650D applies a head-related transfer function (HRTF) describing a target source position, from which the listener should perceive the sound of the input channel. Each binaural filter receives the input channel and generates a left output channel and a right output channel by applying the HRTF. The discussion of binaural filters 250A, 250B, 250C and 250D of audio system 200 may be applicable to binaural filters 650A, 650B, 650C and 650D. For example, each of binaural filters 650A to 650D may be adjusted for the angular position application associated with their corresponding input channels. In certain embodiments, one or more of binaural filters 650A to 650D may be bypassed, or omitted from audio system 600.

[0113] The left channel combiner 660A is coupled to the gain 615A and the binaural filters 650A to 650D. The left channel combiner 660A receives the left output channel of the binaural filters 650A to 650D and combines the left output channel with the output of the gain 615A. The right channel combiner 660B is coupled to the gain 615B and the binaural filters 650A to 650D. The right channel combiner 660B receives the right output channel of the binaural filters 650A to 650D and combines the right output channel with the output of the gain 615B.

[0114] In some embodiments, binaural filtering is performed after sub-band spatial processing. For example, binaural filters can be applied to the left and right outputs of the sub-band spatial processor 630 adjusted for the angular positions associated with the channels. In some embodiments, binaural filters are applied to peripheral input channels, such as Figure 6 In some embodiments, the binaural filter is applied to the center input channel 610C or the low frequency input channel 610D. In some embodiments, the binaural filter is applied to each input channel except the low frequency input channel 610D.

[0115] The subband spatial processor 630 performs subband spatial processing on the left input channel and the right input channel by gain adjusting the middle component and the side subband component of the left input channel and the right input channel to generate a left spatial enhancement channel and a right spatial enhancement channel as output. The subband spatial processor 630 is coupled to the left channel combiner 660A to receive the left combined channel from the left channel combiner 660A and is coupled to the right channel combiner 660B to receive the right combined channel from the right channel combiner 660B. Unlike the subband spatial processors 230A, 230B, and 230C of the audio system 200, which each process the corresponding left input channel and right input channel, the subband spatial processor 630 processes the left channel and the right channel after combining them into the left combined channel and the right combined channel. Therefore, the audio system 600 may include only a single subband spatial processor 630. In some embodiments, Figure 3 The sub-band spatial processor 230 shown in FIG. 6 is an example of a single sub-band spatial processor 630 .

[0116] The crosstalk cancellation processor 670 performs crosstalk cancellation on the output of the sub-band spatial processor 630, which may represent a down-mixed stereo signal of the input audio signal. The crosstalk cancellation processor 670 receives the left input channel and the right input channel from the sub-band spatial processor 630 and performs crosstalk cancellation to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel. The crosstalk cancellation processor 670 is coupled to the left channel combiner 260A and the right channel combiner 260B. In some embodiments, Figure 4 The crosstalk cancellation processor 270 shown in FIG. 6 is an example of the crosstalk cancellation processor 670 .

[0117] The elevated filter 620 receives the center input channel 610C and applies a high frequency shelf or peak filter. The elevated filter 620 provides a "voice boost" on the center input channel 610C. In some embodiments, the elevated filter 620 is bypassed or omitted from the audio system 600. The elevated filter 620 can attenuate frequencies above the corner frequency. The elevated filter 620 is coupled to the left channel combiner 660C and the right channel combiner 660D. In some embodiments, the elevated filter 620 is defined by a 750Hz corner frequency, a +3dB gain, and a 0.8Q factor. The elevated filter 620 generates a left center channel and a right center channel as outputs.

[0118] The distributor 640 receives the low frequency input channel 610D and separates the low frequency input channel 610D into a left low frequency channel and a right low frequency channel. The distributor 640 is coupled to the left channel combiner 660C and the right channel combiner 660D and provides the left low frequency channel to the left channel combiner 660C and the right low frequency channel to the right channel combiner 660D.

[0119] The left channel combiner 660C is coupled to the crosstalk cancellation processor 670, the high shelf filter 620 and the distributor 640. The left channel combiner 660C receives the left crosstalk channel from the crosstalk cancellation processor 670, the left center channel from the high shelf filter 620 and the left low frequency channel from the distributor 640 and combines these channels into a left output channel.

[0120] The right channel combiner 660D is coupled to the crosstalk cancellation processor 670, the high shelf filter 620 and the distributor 640. The right channel combiner 660D receives the right crosstalk channel from the crosstalk cancellation processor 670, the right center channel from the high shelf filter 620 and the right low frequency channel from the distributor 640 and combines these channels into a right output channel.

[0121] In some embodiments, the left center channel from the elevated filter 620 and the left low-frequency channel from the distributor 640 are combined by the output of the left channel combiner 660A with the left output channel of the binaural filter 650A to 650D and the gain 615A to generate the left combined channel. The right center channel from the elevated filter 620 and the right low-frequency channel from the distributor 640 are combined by the output of the right channel combiner 660B with the right output channel of the binaural filter 650A to 650D and the gain 615B to generate the right combined channel. The left combined channel and the right combined channel are input to the sub-band spatial processor 630 and the crosstalk cancellation processor 670. Here, the center and the low-frequency channel receive the sub-band spatial processing and crosstalk cancellation operation. The left channel combiner 660C and the right channel combiner 660D can be omitted. In some embodiments, one of the center or the low-frequency channel receives the sub-band spatial processing and the crosstalk cancellation operation.

[0122] Output gain 680 is coupled to left channel combiner 660C and right channel combiner 660D. Output gain 680 applies gain to the left output channel from left channel combiner 660C and applies gain to the right output channel from right channel combiner 660D. Output gain 680 can apply the same gain to the left and right output channels, or can apply different gains. Output gain 680 outputs left output channel 690L and right output channel 690R representing the channels of the output signal of audio system 600.

[0123] Figure 7 An example embodiment of a method for utilizing Figure 6 An example of a method 700 for enhancing an audio signal with the audio system 600 shown in FIG. In some embodiments, the method 700 may include different and / or additional steps, or some steps may be in a different order.

[0124] The audio system 600 receives 705 a multi-channel input audio signal. The input audio signal may include a left input channel 610A, a right input channel 610B, at least one left peripheral input channel, and at least one right peripheral input channel. The multi-channel audio signal may also include a center input channel 610C and a low frequency input channel 610D.

[0125] The audio system 600 (eg, gains 615A through 615H) applies 710 gains to the channels of the multi-channel input audio signal. The gains 615A through 615H may vary to control the contribution of a particular input channel to the output signal generated by the audio system 600.

[0126] The audio system 600 (e.g., binaural filters 650A to 650D) applies 715 binaural filters to each of the left and right peripheral channels. For example, binaural filter 650A generates a left output channel and a right output channel from the left surround input channel 610E by applying a head-related transfer function (HRTF). Binaural filter 650B generates a left output channel and a right output channel from the right surround input channel 610F by applying the HRTF. Binaural filter 650C generates a left output channel and a right output channel from the left surround back input channel 610G by applying the HRTF. Binaural filter 650D generates a left output channel and a right output channel from the right surround back input channel 610H by applying the HRTF.

[0127] The audio system 600 (eg, the high shelf filter 620) applies 720 the high shelf filter to the center input channel 610C. In some embodiments, a gain is applied to the center input channel 610C. Additionally, the high shelf filter 620 separates the center input channel 610C into a left center channel and a right center channel.

[0128] The audio system 600 (eg, the splitter 640 ) splits 725 the low frequency input channel into a left low frequency channel and a right low frequency channel.

[0129] The audio system 600 (eg, the left channel combiner 660A) combines 730 the left input channel 610A and the left output channels of the binaural filters 650A, 650B, 650C, and 650D to generate a left combined channel.

[0130] The audio system 600 (eg, the right channel combiner 660B) combines 735 the right input channel 610B and the right output channels of the binaural filters 650A, 650B, 650C, and 650D to generate a right combined channel.

[0131] The audio system 600 (e.g., the sub-band spatial processor 630) generates 740 left and right spatial enhancement channels by performing sub-band spatial processing on the left and right combined channels. For example, the sub-band spatial processor 630 receives the left and right combined channels from the left and right channel combiners 660A and 660B, and generates the spatial enhancement channels by adjusting the gains of n sub-bands of the mid and side components of the left and right combined channels.

[0132] The audio system 600 (eg, the crosstalk cancellation processor 670) performs 745 crosstalk cancellation on the left and right spatial enhancement channels from the sub-band spatial processor 630 to generate left and right crosstalk cancellation channels.

[0133] The audio system 600 (e.g., the left channel combiner 660C and the right channel combiner 660D) combines 750 the left crosstalk cancellation channel from the crosstalk cancellation processor 670 with the left low frequency channel from the divider 640 and the left center channel from the overhead filter 620 to generate a left output channel, and combines the right crosstalk cancellation channel from the crosstalk cancellation processor 670 with the right low frequency channel from the divider 640 and the right center channel from the overhead filter 620 to generate a right output channel. In addition, the audio system 600 (e.g., the output gain 680) may apply a gain to each of the left and right output channels. The audio system 600 outputs an output audio signal including left and right output channels 690L and 690R.

[0134] Note that the systems and processes described herein may be embodied in embedded electronic circuits or electronic systems. The systems and processes may also be embodied in a computing system that includes one or more processing systems (e.g., digital signal processors) and memory (e.g., programmed read-only memory or programmable solid-state memory), or in some other circuit devices, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) circuits.

[0135] Figure 8 An example of a computer system 800 according to one embodiment is illustrated. Computer system 800 is an example of a circuit device that implements an audio system. At least one processor 802 coupled to a chipset 804 is illustrated. Chipset 804 includes a memory controller hub 820 and an input / output (I / O) controller hub 822. Memory 806 and a graphics adapter 812 are coupled to the memory controller hub 820, and a display device 818 is coupled to the graphics adapter 812. Storage device 808, keyboard 810, pointing device 814, and network adapter 816 are coupled to the I / O controller hub 822. Other embodiments of computer 800 have different architectures. For example, in some embodiments, memory 806 is directly coupled to processor 802.

[0136] The storage device 808 includes one or more non-transitory computer-readable storage media, such as a hard drive, a compact disk read-only memory (CD-ROM), a DVD, or a solid-state memory device. The memory 806 stores instructions and data used by the processor 802. For example, the memory 806 may store instructions that, when executed by the processor 802, cause the processor 802 or configure the processor 802 to perform a method discussed herein, such as method 500 or 700. The pointing device 814 is used in conjunction with the keyboard 810 to enter data into the computer system 800. The graphics adapter 812 displays images and other information on the display device 818. In some embodiments, the display device 818 includes a touch screen capability for receiving user input and selections. The network adapter 816 couples the computer system 800 to a network. Some embodiments of the computer 800 have a computer system 800 that is compatible with the computer system 800. Figure 8 Components may be different from those shown in and / or other components. For example, computer system 800 may be a server lacking a display device, a keyboard, and other components.

[0137] The computer 800 is adapted to execute computer program modules to provide the functionality described herein. As used herein, the term "module" refers to computer program instructions and / or other logic that are used to provide a specified functionality. Therefore, a module can be implemented in hardware, firmware, and / or software. In one embodiment, a program module formed by executable computer program instructions is stored on a storage device 808, loaded into a memory 806, and executed by a processor 802.

[0138] Other examples of circuit devices that may implement an audio system may include an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), and the like.

[0139] Example audio system and example audio processing

[0140] Fig. 9 An example of an audio system 900 according to one embodiment is illustrated. The audio system 900 is similar to the audio system 200, except that crosstalk processing is performed on each left and right channel pair before being combined into the left output channel 990L and the right output channel 990R. Applying crosstalk processing and sub-band spatial processing to each left and right channel pair separately provides the opportunity for unique sub-band spatial processing and crosstalk processing configurations for each "virtual" speaker pair. For example, the sub-band spatial processing for a given left and right channel pair can be configured to apply more or less per-band emphasis to the spatial components in the signal, resulting in an increase or decrease in perceived spatial "intensity" compared to other channel pairs. Similarly, for a given left and right channel pair, the crosstalk processing filters and delay parameters can be uniquely configured based on the binaural filtering applied to that channel pair to achieve maximum perceived effect.

[0141] The audio system 900 receives an input audio signal, which includes a left input channel 910A, a right input channel 910B, a center input channel 910C, a low frequency input channel 910D, a left surround input channel 910E, a right surround input channel 910F, a left surround rear input channel 910G, and a right surround rear input channel 910H. The left input channel 910A and the right input channel 910B form a left and right channel pair for the front speaker. The left surround input channel 910E and the right surround input channel 910F form another left and right channel pair, and the left surround rear input channel 910G and the right surround rear input channel 910H form another left and right channel pair. These other left and right channel pairs are peripheral left and right channel pairs. The audio system 900 performs one or more of subband spatial processing and crosstalk cancellation on each of the left and right channel pairs, and combines the output into a left output channel 990L and a right output channel 990R.

[0142] The audio system 900 includes gains 915A, 915B, 915C, 915D, 915E, 915F, 915G and 915H, binaural filters 950A, 950B, 950C, 950D, 950E and 950F, sub-band spatial processors 930A, 930B and 930C, crosstalk cancellation processors 970A, 970B and 970C, an overhead filter 920, a distributor 940, a left channel combiner 960A, a right channel combiner 960B and an output gain 980.

[0143] Each of the gains 915A through 915H may receive a corresponding input channel 910A through 910H and may apply a gain to the input channel 910A through 910H. The gains 915A through 915H may be different to adjust the gains of the input channels relative to each other, or may be the same.

[0144] Binaural filters are applied to the channels of the left and right channel pairs. Gain 915A is coupled to binaural filter 950A, gain 915B is coupled to binaural filter 950B, gain 915E is coupled to binaural filter 950C, gain 915F is coupled to binaural filter 950D, gain 915G is coupled to binaural filter 950E, and gain 915H is coupled to binaural filter 950F. Each of binaural filters 950A, 950B, 950C, 950D, 950E, and 950F applies a head-related transfer function (HRTF) that describes a target source position from which a listener should perceive the sound of the input channel. Each binaural filter receives an input channel and generates a left output channel and a right output channel by applying an HRTF adjusted for an angular position associated with the input channel. The angular position may include angles defined in an XY “azimuth” plane relative to the listener 140, such as Figure 1 , and may also include angles defined in the Z axis, such as for an ambient sound signal or a channel-based format containing signals intended to be rendered above or below the XY plane relative to the listener 140 .

[0145] For example, binaural filter 950A may apply filters based on left input channel 910A associated with an angle between -30° and -45° relative to the forward axis of left speaker 110L. Binaural filter 950B may apply filters based on right input channel 910B associated with an angle between 30° and 45° relative to the forward axis of right speaker 110R. Binaural filter 950C may apply filters based on left surround input channel 910E associated with an angle between -90° and -110° relative to the forward axis of left surround speaker 120L. Binaural filter 950D may apply filters based on right surround input channel 910F associated with an angle between 90° and 110° relative to the forward axis of right surround speaker 120R. Binaural filter 950E may apply filters based on left surround back input channel 910G associated with -135° to -150° relative to the forward axis of left surround back speaker 130L. Binaural filter 950F may apply a filter based on the right surround back input channel 910H being associated with an angle between 135° and 150° relative to the forward axis of the right surround back speaker 130R. Each of binaural filters 950A through 950F generates a left channel and a right channel.

[0146] In some embodiments, binaural processing on the left and right input channels 910A and 910B can be bypassed. Here, binaural filters 950A and 950B can be omitted from the audio system 900. In some embodiments, binaural processing can be completely bypassed in order to preserve inter-channel spectral uniformity. One or more of binaural filters 950A, 950B, 950C, 950D, 950E, or 950F can be omitted from the audio system 900.

[0147] In some embodiments, the input audio signal is an ambient stereo audio signal that defines a speaker-independent representation of a sound field. The ambient stereo audio signal can be decoded into a multi-channel audio signal for a surround sound system. The channels can be associated with speaker positions at different locations, including positions above or below the listener. A binaural filter can be applied to each decoded input channel of the ambient sound audio signal to adjust for the associated position of the decoded input audio channels.

[0148] Each of the sub-band spatial processors 930 applies sub-band spatial processing to different left and right channel pairs. The sub-band spatial processor 930A is coupled to each of the binaural filters 950A and 950B. The sub-band spatial processor 930A receives a left channel from each of the binaural filters 950A and 950B, combines the left channels into a combined left channel, and applies sub-band spatial processing to the combined left channel. The sub-band spatial processor 930A receives a right channel from each of the binaural filters 950A and 950B, combines the right channels into a combined right channel, and applies sub-band spatial processing to the combined right input channel. The sub-band spatial processor 930A performs sub-band spatial processing on the left input channel and the right input channel by gain adjusting the middle component and the side sub-band component of the left input channel and the right input channel to generate a left spatial enhancement channel and a right spatial enhancement channel.

[0149] The subband spatial processor 930B is coupled to each of the binaural filters 950C and 950D. The subband spatial processor 930B receives a left channel from each of the binaural filters 950C and 950D, combines the left channels into a combined left channel, and applies subband spatial processing to the combined left channel. The subband spatial processor 930B receives a right channel from each of the binaural filters 950C and 950D, combines the right channels into a combined right channel, and applies subband spatial processing to the combined right channel. The subband spatial processor 930B performs subband spatial processing on the left and right input channels by gain adjusting the middle and side subband components of the left and right input channels to generate left and right spatial enhancement channels.

[0150] The subband spatial processor 930C is coupled to each of the binaural filters 950E and 950F. The subband spatial processor 930C receives the left channel from each of the binaural filters 950E and 950F, combines the left channels into a combined left channel, and applies subband spatial processing to the combined left channel. The subband spatial processor 930C receives the right channel from each of the binaural filters 950E and 950F, combines the right channels into a combined right channel, and applies subband spatial processing to the combined right channel. The subband spatial processor 930C performs subband spatial processing on the left and right input channels by gain adjusting the middle and side subband components of the left and right input channels to generate left and right spatial enhancement channels.

[0151] Each of the crosstalk cancellation processors 970 applies crosstalk cancellation to a different left and right channel pair. Crosstalk cancellation processor 970A is coupled to sub-band spatial processor 930A, crosstalk cancellation processor 970B is coupled to sub-band spatial processor 930B, and crosstalk cancellation processor 970C is coupled to sub-band spatial processor 930C.

[0152] The crosstalk cancellation processor 970A receives the left and right spatial enhancement channels from the sub-band spatial processor 930A and applies crosstalk cancellation processing to the left and right spatial enhancement channels to generate left and right output channels. These left and right output channels correspond to the left and right channel pairs formed by the left and right input channels 910A and 910B after sub-band spatial processing and crosstalk cancellation.

[0153] The crosstalk cancellation processor 970B receives the left and right spatial enhancement channels from the sub-band spatial processor 930B and applies crosstalk cancellation processing to the left and right spatial enhancement channels to generate left and right output channels. These left and right output channels correspond to the left and right channel pairs formed by the left and right surround input channels 910E and 910F after sub-band spatial processing and crosstalk cancellation.

[0154] The crosstalk cancellation processor 970C receives the left and right spatial enhancement channels from the sub-band spatial processor 930C and applies crosstalk cancellation processing to the left and right spatial enhancement channels to generate left and right output channels. These left and right output channels correspond to the left and right channel pairs formed by the left and right surround rear input channels 910G and 910H after sub-band spatial processing and crosstalk cancellation.

[0155] An elevated filter 920 is coupled to the gain 915C. The elevated filter 920 receives the center input channel 910C and applies a high frequency shelf or peak filter. The elevated filter 920 can attenuate or amplify frequencies above the corner frequency. In some embodiments, the elevated filter 920 is defined by a 750Hz corner frequency, a +3dB gain, and a 0.8Q factor. The elevated filter 920 generates a left center channel and a right center channel as outputs, such as by separating the center input channel into two separate left center channel and right center channel. In some embodiments, the elevated filter 920 is bypassed or omitted from the audio system 900.

[0156] The splitter 940 is coupled to the gain 915D. The splitter 940 receives the low frequency input channel 910D and separates the low frequency input channel 910D into a left low frequency channel and a right low frequency channel.

[0157] The left channel combiner 960A and the right channel combiner 960B are respectively coupled to the crosstalk cancellation processor 970A, the crosstalk cancellation processor 970B, the crosstalk cancellation processor 970C, the high shelf filter 920, and the distributor 940. The left channel combiner 960A receives the left channels output from each of the crosstalk cancellation processor 970A, the crosstalk cancellation processor 970B, the crosstalk cancellation processor 970C, the high shelf filter 920, and the distributor 940, and combines the left channels into a left output channel. The right channel combiner 960B receives the right channels output from each of the crosstalk cancellation processor 970A, the crosstalk cancellation processor 970B, the crosstalk cancellation processor 970C, the high shelf filter 920, and the distributor 940, and combines the right channels into a right output channel.

[0158] Output gain 980 is coupled to left channel combiner 960A and right channel combiner 960B. Output gain 980 applies gain to the left output channel from left channel combiner 960A and applies gain to the right output channel from right channel combiner 960B. Output gain 980 can apply the same gain to the left and right output channels, or can apply different gains. Output gain 980 outputs left output channel 990L and right output channel 990R representing the channels of the output signal of audio system 900.

[0159] Fig.10 An example of an audio system 1000 is illustrated according to one embodiment. Audio system 1000 is similar to audio system 900, but differs from audio system 900 at least in that binaural filters are applied after sub-band spatial processing on one or more of the left and right channel pairs and before crosstalk cancellation processing.

[0160] The audio system 1000 includes gains 915A, 915B, 915C, 915D, 915E, 915F, 915G, and 915H, subband spatial processors 930A, 930B, and 930C, crosstalk cancellation processors 970A, 970B, and 970C, as well as an overhead filter 920, a divider 940, a left channel combiner 960A, a right channel combiner 960B, and an output gain 980. The audio system 1000 also includes binaural filters 1050A, 1050B, 1050C, 1050D, 1050E, and 1050F.

[0161] Binaural filters 1050A and 1050B are coupled to the sub-band spatial processor 930A and the crosstalk cancellation processor 970A. The binaural filters 1050A and 1050B apply binaural filtering to the left and right channel pair including the left input channel 910A and the right input channel 910B after the sub-band spatial processing and before the crosstalk cancellation processing. In some embodiments, the binaural filters 1050A and 1050B can be bypassed or excluded from the audio system 1000.

[0162] The audio system 100 applies similar sub-band spatial processing, binaural filtering, and crosstalk cancellation processing to each peripheral left and right channel pair. To process the left and right channel pair including the left surround input channel 910E and the right surround input channel 910F, binaural filters 1050C and 1050D are coupled to the sub-band spatial processor 930B and the crosstalk cancellation processor 970B. To process the left and right channel pair including the left surround back input channel 910G and the right surround back input channel 910H, binaural filters 1050E and 1050F are coupled to the sub-band spatial processor 930C and the crosstalk cancellation processor 970C.

[0163] In some embodiments, the crosstalk cancellation processors 970A, 970B, and 970C may each be a crosstalk simulation processor. Instead of generating a crosstalk cancellation channel, the crosstalk simulation processor generates a crosstalk simulation channel with an additional crosstalk effect.

[0164] Fig.11 An example embodiment of a method for utilizing Fig. 9 The audio system 900 shown in Fig.10 An example of a method 1100 for enhancing an audio signal with an audio system 1000 is shown in FIG. In some embodiments, the method 1100 may include different and / or additional steps, or some steps may be in a different order. The method 1100 is discussed in more detail below with reference to the audio system 900 .

[0165] The audio system 900 receives 1105 a multi-channel input audio signal including a left and right channel pair. The multi-channel audio signal may be a surround sound audio signal including a plurality of left and right channel pairs. For example, a left input channel and a right input channel may form a first left and right channel pair, and at least one left peripheral input channel and at least one right peripheral input channel may form another left and right channel pair. The multi-channel input signal may include a plurality of left and right channel pairs for peripheral input channels. For example, left surround input channels 910E and 910F form a surround pair, and left surround rear input channels 910G and right surround rear input channels 910H form a rear surround pair. The multi-channel audio signal may also include a central input channel and a low frequency input channel.

[0166] The audio system 900 (eg, gains 915A through 915H) applies 1110 gains to channels of a multi-channel input audio signal. The gains 915A through 915H may vary to control the contribution of a particular input channel to an output signal generated by the audio system 900.

[0167] The audio system 900 (e.g., binaural filters 950A to 950F) applies 1115 a binaural filter to each of the left channel pair and the right channel pair of the multi-channel input audio signal. For each channel, the binaural filter is adjusted for the angular position associated with the channel. In some embodiments, the binaural filter is applied to the peripheral left and right channel pairs, but is not applied to the left and right channel pair that includes the left input channel and the right input channel.

[0168] The audio system 900 (e.g., the sub-band spatial processors 930A, 930B, and 930C) applies 1120 sub-band spatial processing to each left and right channel pair to generate a spatial enhancement channel. For example, the sub-band spatial processor 930A applies sub-band spatial processing to a left and right channel pair including a left input channel 910A and a right input channel 910B to generate a spatial enhancement channel. The sub-band spatial processing includes gain adjustment of the mid component and the side component of the left input channel 910A and the right input channel 910B.

[0169] Subband spatial processing is also applied to at least one of the left and right channel pairs for peripheral channels. For example, the subband spatial processor 930B applies subband spatial processing to the left and right channel pairs including the left surround input channel 910E and the right surround input channel 910F to generate spatial enhancement channels. The subband spatial processing includes gain adjustment of the middle component and the side component of the left surround input channel 910E and the right surround input channel 910F. The subband spatial processor 930C applies subband spatial processing to the left and right channel pairs including the left surround rear input channel 910G and the right surround rear input channel 910H to create spatial enhancement channels. The subband spatial processing includes gain adjustment of the middle component and the side component of the left surround rear input channel 910G and the right surround rear input channel 910H. In this way, a spatial enhancement channel is created for each of the left and right channel pairs.

[0170] In some embodiments, sub-band spatial processing is performed for each left and right channel pair before binaural filtering, such as for the audio system 1000. Fig.10 Here, each of the left spatial enhancement channel and the right spatial enhancement channel output from the sub-band spatial processors 930A, 930B, and 930C is input to the binaural filter.

[0171] The audio system 900 (e.g., crosstalk cancellation processors 970A, 970B, and 970C) applies 1125 crosstalk processing to generate crosstalk-processed channels for each left and right channel pair. The crosstalk processing may include crosstalk cancellation or crosstalk simulation. In the case of crosstalk cancellation, the crosstalk-processed channels include crosstalk cancellation channels. In the case of crosstalk simulation, the crosstalk-processed channels include crosstalk simulation channels. Crosstalk cancellation can be used for speaker outputs, and crosstalk simulation can be used for headphone outputs. For each left and right channel pair, the crosstalk processing may include applying a filter, a time delay, and a gain to at least one of the spatial enhancement channels to generate the crosstalk-processed channels. In some embodiments, crosstalk processing may be performed on each left and right channel pair before sub-band spatial processing is performed on each left and right channel pair.

[0172] The audio system 900 (e.g., the left channel combiner 960A and the right channel combiner 960B) generates 1130 left and right output channels from the crosstalk processed channels. For example, the left channel combiner 960A combines the left channel of the crosstalk processed channels from each crosstalk cancellation processor 970A, 970B, and 970C to generate the left output channel, and the right channel combiner 960B combines the right channel of the crosstalk processed channels from each crosstalk cancellation processor 970A, 970B, and 970C to generate the right output channel.

[0173] The left channel combiner 960A can also combine the left channel with the left low frequency channel and the left center channel to generate a left output channel. The right channel combiner 960B can also combine the right channel with the right low frequency channel and the right center channel to generate a right output channel. The audio system 900 (e.g., the high shelf filter 920) applies the high shelf filter to the center input channel of the multi-channel input audio signal to generate the left center channel and the right center channel. The audio system 900 (e.g., the distributor 940) applies the low frequency input channel to separate the center input channel of the multi-channel input audio signal to generate the left low frequency channel and the right low frequency channel.

[0174] Fig.12 An example of a crosstalk simulation processor 1200 according to one embodiment is illustrated. When the crosstalk processing is crosstalk simulation, the crosstalk simulation processor 1200 can be used in an audio system instead of a crosstalk cancellation processor. The crosstalk simulation processor 1200 can be used to provide a speaker-like listening experience on head-worn speakers.

[0175] The crosstalk simulation processor 1200 includes a left head shadow low pass filter 1202, a left head shadow high pass filter 1204, a left crosstalk delay 1210, and a left head shadow gain 1224 to process the left channel (eg, the left spatial enhancement channel E LThe crosstalk simulation processor 1200 further includes a right head shadow low pass filter 1206, a right head shadow high pass filter 1208, a right crosstalk delay 1212, and a right head shadow gain 1226 to process the right channel (eg, the right spatial enhancement channel E R ).

[0176] The left head shadow low pass filter 1202 and the left head shadow high pass filter 1204 each apply a modulation that models the frequency response of the signal after passing through the listener's head. The left crosstalk delay 1210 applies a time delay that represents the trans-ear distance traversed by the contralateral sound component relative to the ipsilateral sound component. The frequency response can be generated based on empirical experiments to determine the frequency-dependent characteristics of the sound wave modulation of the listener's head. In some embodiments, the left crosstalk delay 1210 can be applied before the left head shadow low pass filter 1202 and the left head shadow high pass filter 1204. The left head shadow gain 1224 applies a gain to generate a left crosstalk simulated channel. L .

[0177] The right head shadow low pass filter 1206 and the right head shadow high pass filter 1208 each apply a modulation that models the frequency response of the signal after passing through the listener's head. The right crosstalk delay 1212 applies a time delay that represents the trans-ear distance traversed by the contralateral sound component relative to the ipsilateral sound component. The frequency response can be generated based on empirical experiments to determine the frequency-dependent characteristics of the sound wave modulation of the listener's head. In some embodiments, the right crosstalk delay 1212 can be applied before the right head shadow low pass filter 1206 and the right head shadow high pass filter 1208. The right head shadow gain 1226 applies a gain to generate a right crosstalk simulated channel. L .

[0178] The application of the head shadow low pass filter, head shadow high pass filter, crosstalk delay, and head shadow gain to each of the left and right channels may be performed in a different order, and one or more of these stages may be skipped. Using both low pass and high pass filters on both the left and right channels may result in a more accurate model of the frequency response through the listener's head.

[0179] Other considerations

[0180] The disclosed configuration may include many benefits and / or advantages. For example, a multi-channel input signal may be output to stereo speakers while retaining or enhancing the spatial sense of the sound field. Such as on a mobile device, sound bar, or smart speaker, a high-quality listening experience may be obtained without an expensive multi-speaker sound system.

[0181] After reading this disclosure, those skilled in the art will appreciate additional alternative embodiments of the principles disclosed herein. Therefore, although specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the precise structures and components disclosed herein. Various modifications, changes and variations may be made to the arrangement, operation and details of the methods and devices disclosed herein without departing from the scope described herein, which will be apparent to those skilled in the art.

[0182] Any steps, operations or processes described herein may be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, the software modules are implemented using a computer program product that includes a computer-readable medium (e.g., a non-transitory computer-readable medium) containing computer program code that can be executed by a computer processor to perform any or all steps, operations or processes described.

Claims

1. A system for processing a multi-channel input audio signal, include: A circuit arrangement, the circuit arrangement being configured to: receiving a multi-channel input audio signal comprising a plurality of left and right channel pairs, a first left and right channel pair of the plurality of left and right channel pairs comprising a left input channel and a right input channel, and a second left and right channel pair of the plurality of left and right channel pairs comprising a left peripheral input channel and a right peripheral input channel; Applying a first crosstalk simulation to the first left and right channel pair to generate a first crosstalk simulation processed left channel and a first crosstalk simulation processed right channel; applying first binaural filtering and second crosstalk simulation processing to the second left and right channel pair to generate a second crosstalk simulation processed left channel and a second crosstalk simulation processed right channel, the first binaural filtering comprising applying a first binaural filter to adjust an angular position associated with the left peripheral input channel and applying a second binaural filter to adjust an angular position associated with the right peripheral input channel; generating a left output channel by combining the left channel after the first crosstalk analog processing and the left channel after the second crosstalk analog processing; as well as A right output channel is generated by combining the first crosstalk analog processed right channel and the second crosstalk analog processed right channel.

2. The system of claim 1 , wherein the circuit device is further configured to: applying a first sub-band spatial processing to the first left and right channel pair, the first sub-band spatial processing comprising gain adjustment of mid and side components of the left and right input channels; and A second sub-band spatial processing is applied to the second left and right channel pair, the second sub-band spatial processing comprising gain adjustment of mid and side components of the left and right peripheral input channels.

3. The system of claim 1 , wherein the circuit device is configured to apply the first binaural filtering to the second left and right channel pair before applying sub-band spatial processing to the second left and right channel pair, the sub-band spatial processing comprising gain adjusting mid and side components of the left and right peripheral input channels.

4. The system of claim 1 , wherein the circuit device is configured to apply the first binaural filtering to the second left and right channel pair after applying sub-band spatial processing to the second left and right channel pair and before applying the second crosstalk simulation processing to the second left and right channel pair, the sub-band spatial processing comprising gain adjustment of mid and side components of the left and right peripheral input channels.

5. The system of claim 1 , wherein the circuit arrangement is further configured to apply a second binaural filter to the first left and right channel pair by: applying a third binaural filter to adjust the angular position associated with the left input channel; and A fourth binaural filter is applied to adjust the angular position associated with the right input channel.

6. The system of claim 5 , wherein the circuit device is configured to apply the second binaural filtering to the first left and right channel pair before applying sub-band spatial processing to the first left and right channel pair, the sub-band spatial processing comprising gain adjustment of mid and side components of the left and right input channels.

7. The system of claim 5, wherein the circuit device is configured to apply the second binaural filtering to the first left and right channel pair after applying sub-band spatial processing to the first left and right channel pair and before applying the first crosstalk simulation processing to the first left and right channel pair, wherein the sub-band spatial processing includes gain adjustment of mid and side components of the left and right input channels.

8. The system of claim 1, wherein the circuit device configured to apply the first crosstalk simulation process to the first left and right channel pair include: The circuit arrangement is configured to apply a filter, a time delay, and a gain to at least one of the left input channel or the right input channel.

9. The system of claim 1, wherein the circuit device configured to apply the second crosstalk simulation process to the second left and right channel pair include: The circuit arrangement is configured to apply a filter, a time delay, and a gain to at least one of the left peripheral input channel or the right peripheral input channel.

10. The system of claim 9, wherein the circuit arrangement is further configured to: applying a high shelf filter to a center input channel of the multi-channel input audio signal to generate a left center channel and a right center channel; applying a splitter to a low frequency input channel of the multi-channel input audio signal to generate a left low frequency channel and a right low frequency channel; combining the left center channel and the left low frequency channel with the first crosstalk analog processed left channel and the second crosstalk analog processed left channel to generate the left output channel; as well as The right center channel and the right low frequency channel are combined with the first crosstalk analog processed right channel and the second crosstalk analog processed right channel to generate the right output channel.

11. The system of claim 1 , wherein the second left and right channel pair comprising the left peripheral input channel and the right peripheral input channel is one of: Surrounding pairs; or Rear surround pair.

12. A non-transitory computer readable medium storing program code which, when executed by a processor, causes the processor to: receiving a multi-channel input audio signal comprising a plurality of left and right channel pairs, a first left and right channel pair of the plurality of left and right channel pairs comprising a left input channel and a right input channel, and a second left and right channel pair of the plurality of left and right channel pairs comprising a left peripheral input channel and a right peripheral input channel; Applying a first crosstalk simulation process to the first left and right channel pair to generate a first crosstalk simulation processed left channel and a first crosstalk simulation processed right channel; applying binaural filtering and second crosstalk simulation processing to the second left and right channel pair to generate a second crosstalk simulation processed left channel and a second crosstalk simulation processed right channel, the binaural filtering comprising applying a first binaural filter to adjust an angular position associated with the left peripheral input channel and applying a second binaural filter to adjust an angular position associated with the right peripheral input channel; generating a left output channel by combining the left channel after the first crosstalk analog processing and the left channel after the second crosstalk analog processing; as well as A right output channel is generated by combining the first crosstalk analog processed right channel and the second crosstalk analog processed right channel.

13. The computer-readable medium of claim 12, further comprising program code that causes the processor to: applying a first sub-band spatial processing to the first left and right channel pair, the first sub-band spatial processing comprising gain adjustment of mid and side components of the left and right input channels; and A second sub-band spatial processing is applied to the second left and right channel pair, the second sub-band spatial processing comprising gain adjustment of mid and side components of the left and right peripheral input channels.

14. The computer-readable medium of claim 12, wherein the program code causes the processor to: apply the first binaural filtering to the second left and right channel pair before applying sub-band spatial processing to the second left and right channel pair, the sub-band spatial processing comprising gain adjusting mid and side components of the left and right peripheral input channels.

15. The computer-readable medium of claim 12, further comprising program code that causes a processor to perform the following operations: applying the first binaural filtering to the second left and right channel pair after applying sub-band spatial processing to the second left and right channel pair and before applying the second crosstalk simulation processing to the second left and right channel pair, the sub-band spatial processing comprising gain adjusting mid and side components of the left peripheral input channel and the right peripheral input channel.

16. The computer-readable medium of claim 12, further comprising program code that causes the processor to apply a second binaural filter to the first left and right channel pair by: applying a third binaural filter to adjust the angular position associated with the left input channel; and A fourth binaural filter is applied to adjust the angular position associated with the right input channel.

17. The computer-readable medium of claim 16, wherein the program code causes the processor to apply the second binaural filtering to the first left and right channel pair before applying sub-band spatial processing to the first left and right channel pair, the sub-band spatial processing comprising gain adjustment of mid and side components of the left and right input channels.

18. The computer-readable medium of claim 16, wherein the program code causes the processor to apply the second binaural filtering to the first left and right channel pair after applying sub-band spatial processing to the first left and right channel pair and before applying the first crosstalk simulation processing to the first left and right channel pair, the sub-band spatial processing comprising gain adjustment of mid and side components of the left and right input channels.

19. The computer-readable medium of claim 12, wherein the program code that causes the processor to apply the first crosstalk simulation process to the first left and right channel pair include: Program code causes the processor to apply a filter, a time delay, and a gain to at least one of the left input channel or the right input channel.

20. The computer-readable medium of claim 12, wherein the program code causing the processor to apply the second crosstalk simulation process to the second left and right channel pair include: Program code causes the processor to apply a filter, a time delay, and a gain to at least one of the left peripheral input channel or the right peripheral input channel.

21. The computer readable medium of claim 20, wherein the program code further causes the processor to: applying a high shelf filter to a center input channel of the multi-channel input audio signal to generate a left center channel and a right center channel; applying a splitter to a low frequency input channel of the multi-channel input audio signal to generate a left low frequency channel and a right low frequency channel; combining the left center channel and the left low frequency channel with the first crosstalk analog processed left channel and the second crosstalk analog processed left channel to generate the left output channel; as well as The right center channel and the right low frequency channel are combined with the first crosstalk analog processed right channel and the second crosstalk analog processed right channel to generate a right output channel.

22. The computer-readable medium of claim 12, wherein the second left and right channel pair comprising the left peripheral input channel and the right peripheral input channel is one of: Surrounding pairs; or Rear surround pair.

23. A method for processing a multi-channel input audio signal, comprising, by a circuit device: receiving a multi-channel input audio signal comprising a plurality of left and right channel pairs, a first left and right channel pair of the plurality of left and right channel pairs comprising a left input channel and a right input channel, and a second left and right channel pair of the plurality of left and right channel pairs comprising a left peripheral input channel and a right peripheral input channel; Applying a first crosstalk simulation process to the first left and right channel pair to generate a first crosstalk simulation processed left channel and a first crosstalk simulation processed right channel; applying binaural filtering and second crosstalk simulation processing to the second left and right channel pair to generate a second crosstalk simulation processed left channel and a second crosstalk simulation processed right channel, the binaural filtering comprising applying a first binaural filter to adjust an angular position associated with the left peripheral input channel and applying a second binaural filter to adjust an angular position associated with the right peripheral input channel; generating a left output channel by combining the left channel after the first crosstalk analog processing and the left channel after the second crosstalk analog processing; as well as A right output channel is generated by combining the first crosstalk analog processed right channel and the second crosstalk analog processed right channel.

24. The method according to claim 23, further comprising, by the circuit device: applying a first sub-band spatial processing to the first left and right channel pair, the first sub-band spatial processing comprising gain adjustment of mid and side components of the left and right input channels; and A second sub-band spatial processing is applied to the second left and right channel pair, the second sub-band spatial processing comprising gain adjustment of mid and side components of the left and right peripheral input channels.

25. The method of claim 23, wherein the first binaural filtering is applied to the second left and right channel pair before applying sub-band spatial processing to the second left and right channel pair, the sub-band spatial processing comprising gain adjusting mid and side components of the left and right peripheral input channels.

26. The method of claim 23, wherein the first binaural filtering is applied to the second left and right channel pair after applying sub-band spatial processing to the second left and right channel pair and before applying the second crosstalk simulation processing to the second left and right channel pair, the sub-band spatial processing comprising gain adjusting mid and side components of the left and right peripheral input channels.

27. The method according to claim 23, further comprising: include: The circuit device applies a second binaural filter to the first left and right channel pair in the following manner: applying a third binaural filter to adjust an angular position associated with the left input channel; as well as A fourth binaural filter is applied to adjust the angular position associated with the right input channel.

28. The method of claim 27, wherein the second binaural filtering is applied to the first left and right channel pair before applying sub-band spatial processing to the first left and right channel pair, the sub-band spatial processing comprising gain adjustment of mid and side components of the left and right input channels.

29. The method of claim 27, wherein the second binaural filtering is applied to the first left and right channel pair after applying sub-band spatial processing to the first left and right channel pair and before applying the first crosstalk simulation processing to the first left and right channel pair, the sub-band spatial processing comprising gain adjusting mid and side components of the left and right input channels.

30. The method of claim 23, wherein the first crosstalk simulation process applied to the first left and right channel pair comprises applying a filter, a time delay, and a gain to at least one of the left input channel or the right input channel.

31. The method of claim 23, wherein applying the second crosstalk simulation process to the second left and right channel pair comprises applying a filter, a time delay, and a gain to at least one of the left peripheral input channel or the right peripheral input channel.

32. The method according to claim 31 , further comprising, by the circuit device: applying a high shelf filter to a center input channel of the multi-channel input audio signal to generate a left center channel and a right center channel; applying a splitter to a low frequency input channel of the multi-channel input audio signal to generate a left low frequency channel and a right low frequency channel; combining the left center channel and the left low frequency channel with the first crosstalk analog processed left channel and the second crosstalk analog processed left channel to generate the left output channel; as well as The right center channel and the right low frequency channel are combined with the first crosstalk analog processed right channel and the second crosstalk analog processed right channel to generate the right output channel.

33. The method of claim 23, wherein the second left and right channel pair comprising the left peripheral input channel and the right peripheral input channel is one of: Surrounding pairs; or Rear surround pair.

Images