Multi-channel subband spatial processing for loudspeaker

JP2024161241APending Publication Date: 2024-11-15BOOMCLOUD 360 INC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2024153106
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2018-03-22
Filing Date
2024-09-05
Publication Date
2024-11-15

Smart Images

  • Figure 2024161241000001_ABST
    Figure 2024161241000001_ABST
Patent Text Reader

Abstract

To provide a multi-channel subband spatial processing for a loudspeaker.SOLUTION: An audio system processes a multi-channel surround sound input audio signal into a stereo signal for left and right speakers, and preserves the spatial sense of the sound field of the input audio signal. A subband spatial processing is performed on a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel of an input signal to create a spatially enhanced channel. A binaural filter can be applied to the peripheral input channel or the spatial enhanced channel. Crosstalk cancellation is performed on the spatially enhanced channel to create a left crosstalk cancelled channel and a right crosstalk cancelled channel.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The disclosed embodiments relate generally to the field of audio signal processing, and more specifically to spatially enhanced multi-channel audio. [Background technology]

[0002] Surround sound refers to the sound reproduction of an audio signal that includes multiple channels with speakers arranged around a listener. For example, 5.1 surround sound uses six channels: front speakers, left and right speakers, a subwoofer, and rear (or "surround") left and right speakers. In another example, 7.1 surround sound uses eight channels by separating the rear left and right speakers of the 5.1 surround sound configuration into four separate speakers, such as a left surround speaker, a right surround speaker, a left rear surround speaker, and a right rear surround speaker. The audio channels of a multichannel audio signal may be associated with angular positions that correspond to the location of the speakers at which the audio channels are output. Thus, a multichannel audio signal allows a listener to perceive a spatial sense of a sound field when the audio signals are output to speakers at different locations. However, when multi-channel audio signals for surround sound are output to stereo (eg, left and right) speakers or head-mounted speakers, the sense of space can be lost. Summary of the Invention

[0003] Exemplary embodiments relate to processing a multi-channel input audio signal (e.g., surround sound) into stereo output signals for left and right speakers while preserving or enhancing the spatial sensation of the sound field of the multi-channel input audio signal. In particular, the processing results in a listening experience in which each channel of the audio signal is perceived to originate from the same or similar direction as would occur if the audio signal were rendered in a surround sound system (e.g., 5.1, 7.1, etc.).

[0004] In an exemplary embodiment, a multi-channel input audio signal is received, including a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel. Subband spatial processing is performed on the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel to create spatial enhancement channels. The subband spatial processing may include adjusting gains of mid and side subband components of the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel. Crosstalk cancellation is performed on the spatial enhancement channels to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel. A left output channel is generated from the left crosstalk cancellation channel, and a right output channel is generated from the right crosstalk channel.

[0005] The left and right peripheral channels may include a left surround input channel and a right surround input channel, and / or a left surround rear input channel and a right surround rear input channel. The multi-channel input audio signal may further include a center channel and a low frequency channel that may be combined with the output of the crosstalk cancellation.

[0006] In one embodiment, subband spatial processing is performed on each of the corresponding left and right channel pairs. For example, the subband spatial processing may be performed by gain adjusting the mid and side subband components of the left and right input channels, gain adjusting the mid and side subband components of the left and right peripheral input channels, and combining the adjusted mid and side subband components of the left, right, left and right peripheral input channels into a left combined channel and a right combined channel. Crosstalk cancellation is performed in the left and right combined channels to generate an output channel.

[0007] In one embodiment, subband spatial processing is performed on the combined left and right channels. For example, the subband spatial processing may include combining the left input channel and the left peripheral input channel into a left combined channel, combining the right input channel and the right peripheral input channel into a right combined channel, and gain adjusting the mid and side subband components of the left and right combined channels to generate left and right spatial emphasis channels. Crosstalk cancellation is performed on the left and right spatial emphasis channels to generate the output channels.

[0008] In one embodiment, a binaural filter is applied to at least some of the input channels. For example, to adjust the angular position associated with a peripheral input channel, the binaural filter is applied to the peripheral input channel. In one embodiment, the binaural filter is applied to any input channel as appropriate to adjust the angular position associated with the input channel, including the left or right input channel.

[0009] One embodiment may include a system for processing a multi-channel input audio signal, the system including a circuit configured to receive a multi-channel input audio signal including a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel, subband spatial processing being performed on the left input channel, the right input channel, the left peripheral channel, and the right peripheral channel to generate a spatial enhancement channel, the subband spatial processing including gain adjustment of mid and side subband components of the left input channel, the right input channel, the left peripheral channel, and the right peripheral channel, and to perform crosstalk cancellation on the spatial enhancement channel to produce a left crosstalk cancellation channel and a right crosstalk cancellation channel, and to generate a left output channel from the left crosstalk cancellation channel and a right output channel from the right crosstalk cancellation channel.

[0010] An embodiment may include a non-transitory computer readable medium storing program code. The program code may be software comprised of executable instructions. The program code may be executed by one or more processors. The program code, when executed by the processor, causes the processor to receive a multi-channel input audio signal including a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel. The program code, when executed by the processor, causes the processor to perform sub-band spatial processing on the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel to generate a spatial enhancement channel. The sub-band spatial processing may include gain adjustment of mid and side sub-band components of the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel. The program code, when executed by the processor, causes the processor to perform crosstalk cancellation on the spatial enhancement channel to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel. When executed by the processor, the program code may also cause the processor to generate a left output channel from the left crosstalk canceled channel and a right output channel from the right crosstalk canceled channel. [Brief description of the drawings]

[0011] [Figure 1] 1 is an example of a surround sound stereo audio reproduction system, according to one embodiment. [Diagram 2] 1 is an example of an audio system, according to one embodiment. [Diagram 3] 1 is an example of a sub-band spatial processor, according to one embodiment. [Figure 4] 1 is an example of a crosstalk cancellation processor, according to one embodiment. [Diagram 5] 3 is an example of a method for enhancing an audio signal using the audio system shown in FIG. 2 according to one embodiment. [Figure 6] 1 is an example of an audio system, according to one embodiment. [Figure 7] 7 is an example of a method for enhancing an audio signal using the audio system shown in FIG. 6, according to one embodiment. [Figure 8] 1 is an example of a computer system, according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The features and advantages described in the specification are not all-inclusive, and many additional features and advantages will be apparent to those skilled in the art, especially upon consideration of the drawings, specification, and claims. In addition, the language used in the specification has been selected primarily for ease of reading and educational purposes, and may not have been selected to delineate or limit the subject matter of the invention.

[0013] It should be noted that the figures and the following description relate to preferred embodiments by way of example only, and that from the following description, alternative embodiments of the structures and methods disclosed herein will be readily apparent as viable alternatives that may be used without departing from the principles of the invention.

[0014] Reference will now be made in detail to one embodiment of the present invention, an example of which is illustrated in the accompanying drawings. It should be noted that wherever possible, like or similar reference numerals may be used in the drawings to indicate like or similar functionality. The figures depict embodiments for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be used without departing from the principles of the present invention as described herein.

[0015] (Example of surround sound stereo and audio system) The audio system discussed in this embodiment provides crosstalk processing and spatial enhancement for multi-channel surround sound audio signals for output to stereo (e.g., left and right) speakers. As a result of the signal processing, the spatial sensation of the sound field encoded in the multi-channel surround sound audio signal is preserved or enhanced. In particular, the spatial sensation realized using a multi-speaker surround sound system is realized using stereo speakers.

[0016] 1 is an example of a surround sound stereo audio reproduction system 100 according to one embodiment. The system 100 is an example of a 7.1 surround sound system that provides playback of audio signals to a listener 140. The system 100 includes a left speaker 110L, a right speaker 110R, a center speaker 115, a subwoofer 125, a left surround speaker 120L, a right surround speaker 120R, a left surround rear speaker 130L, and a right surround speaker 130R. The center speaker 115 and the subwoofer 125 may be positioned in front of the listener 140, which defines a front axis at 0°. The left speaker 110L may be positioned at an angle of −20° to −30° relative to the front axis, and the right speaker 110R may be positioned at an angle of 20° to 30° relative to the front axis. The left surround speaker 120L may be positioned at an angle of -90° to -110° relative to the front axis, and the right surround speaker 120R may be positioned at an angle of 90° to 110° relative to the front axis. The left surround rear speaker 130L may be positioned at an angle of -135° to -150° relative to the front axis, and the right surround speaker 130R may be positioned at an angle of 135° to 150° relative to the front axis. The system 100 may be configured to receive an audio signal that includes channels for each of the speakers 110, 115, 120, and 130, and the subwoofer 125. The multiple speakers and their arrangement provide a sense of space in the sound field that can be perceived by the listener 140. As described in more detail below, the audio system may be configured to process a multi-channel input audio signal of the surround sound system 100 into enhanced stereo signals for left and right speakers (e.g., speakers 110L and 110R) that reproduce or simulate the spatial sensation of the sound field generated by the surround sound system 100 using the multi-channel audio signal.

[0017] 2 is an example of an audio system 200, according to one embodiment. Audio system 200 receives input audio signals including a left input channel 210A, a right input channel 210B, a center input channel 210C, a low frequency input channel 210D, a left surround input channel 210E, a right surround input channel 210F, a left surround rear input channel 210G, and a right surround rear input channel 210H.

[0018] Channels 210E, 210F, 210G, and 210H are examples of surround speaker peripheral channels. The peripheral channels may include channels other than the left and right input channels. The peripheral channels may include channel pairs, such as a left-right pair, a front-back pair, or other pair arrangements. For example, when an input audio signal is output by the surround sound stereo audio reproduction system 100, the left surround speaker 120L receives the left surround input channel 210E, the right surround speaker 120R receives the right surround input channel 210F, the left surround rear speaker 130L receives the left surround rear input channel 210G, and the right surround rear speaker 130R receives the right surround rear input channel 210H. In one embodiment, the input audio signal has fewer or more peripheral channels. For example, an audio input signal for a 5.1 surround sound system may include only two peripheral channels, such as left and right surround input channels output to the left and right surround speakers. Similarly, the left speaker 110L may receive the left input channel 210A, the right speaker 110R may receive the right input channel 210B, the center speaker 115 may receive the center input channel 210C, and the subwoofer 125 may receive the low frequency input channel 210D. When output by the surround sound stereo audio reproduction system 100, the input audio signals provide a spatial sense of the sound field.

[0019] The audio system 200 receives an input audio signal and generates an output signal including a left output channel 290L and a right output channel 290R. The audio system 200 can combine the input channels of the input audio signal, and can further provide enhancements such as sub-band spatial processing and crosstalk cancellation, to generate an output audio signal. The left output channel 290L can be provided to a left speaker, and the right output channel 290R can be output to a right speaker. The output audio signal uses left and right speakers (e.g., left speaker 110L and right speaker 110R) to provide a spatial sense of a sound field, which is typically achieved by outputting the input audio signal using a surround sound system including multiple (e.g., peripheral) speakers.

[0020] Audio system 200 includes gains 215A, 215B, 215C, 215D, 215E, 215F, 215G, and 215H, subband spatial processors 230A, 230B, and 230C, high shelf filter 220, divider 240, binaural filters 250A, 250B, 250C, and 250D, left channel combiner 260A, right channel combiner 260B, crosstalk cancellation processor 270, left channel combiner 260C, right channel combiner 260D, and output gain 280.

[0021] Each of the gains 215A to 215H can receive a respective input channel 210A to 210H and can provide a gain to the input channel 210A to 210H. The gains 215A to 215H can be different or the same to adjust the gain of the input channels relative to each other. In one embodiment, a positive gain is provided to the left and right peripheral input channels 210E, 210F, 210G, and 210H, and a negative gain is applied to the center channel 210C. For example, the gain 215A can provide a 0db gain, the gain 215B can provide a 0dB gain, the gain 215C can provide a -3dB gain, the gain 215D can provide a 0db gain, the gain 215E can provide a 3dB gain, the gain 215F can provide a 3dB gain, the gain 215G can provide a 3dB gain, and the gain 215H can provide a 3dB gain.

[0022] Gain 215A and gain 215B are coupled to subband spatial processor 230A. Similarly, gains 215E and 215F are coupled to subband spatial processor 230B, and gains 215G and 215H are coupled to subband spatial processor 230C. Subband spatial processors 230A, 230B, and 230C apply subband spatial processing to the corresponding left and right channels, respectively.

[0023] Each subband spatial processor 230 performs subband spatial processing on the left and right input channels by gain adjusting the mid and side subband components of the left and right input channels to generate left and right spatial emphasis channels. Subband spatial processor 230A performs subband spatial processing on the left and right input channels, while other subband spatial processors 230B and 230C perform subband spatial processing on the corresponding left and right surround channels, respectively. Depending on the number of surround channels in the input audio signal, audio system 200 may include more or fewer subband spatial processors. In one embodiment, channels that do not have a left / right counterpart (center input channel 210C, low frequency input channel 210D, or other types of channels, rear center, overhead center, etc.) may bypass SBS processing.

[0024] Sub-band spatial processor 230B is coupled to binaural filters 250A and 250B. Sub-band spatial processor 230B provides a left spatial emphasis channel to binaural filter 250A and a right spatial emphasis channel to binaural filter 250B. Similarly, sub-band spatial processor 230C is coupled to binaural filters 250C and 250D. Sub-band spatial processor 230C provides a left spatial emphasis channel to binaural filter 250C and a right spatial emphasis channel to binaural filter 250D. Additional details regarding sub-band spatial processor 230 are shown in FIG. 3 and described below.

[0025] Each of the binaural filters 250A, 250B, 250C, and 250D applies a head-related transfer function (HRTF) that describes the location of a target source from which a listener should perceive the sound of the input channel. Each binaural filter receives an input channel and generates a left or right output channel by applying an HRTF that adjusts the angular position associated with the input channel. As shown in FIG. 1, the angular position may include an angle defined in an X-Y "azimuth" plane relative to the listener 140a, and may further include an angle defined in a Z axis, such as an Ambisonics signal, or a channel-based format that includes a signal intended to be rendered above or below the X-Y plane relative to the listener 140. For example, the binaural filter 250A may be configured to apply a filter based on the left surround input channel 210E that is associated with an angle (defined in the X-Y plane) between -90° and -110° relative to the forward axis of the left surround speaker 120L. Binaural filter 250B may be configured to apply a filter based on right surround input channel 210F associated with an angle between 90° and 110° relative to the front axis of right surround speaker 120L. Binaural filter 250C may be configured to apply a filter based on left surround rear input channel 210G associated with an angle between −135° and −150° relative to the front axis of left surround rear speaker 130L. Binaural filter 250D may be configured to apply a filter based on right surround rear input channel 210H associated with an angle between 135° and 150° relative to the front axis of rear speaker 130R. In one embodiment, binaural processing may be bypassed entirely to maintain inter-channel spectral uniformity. One or more of binaural filters 250A, 250B, 250C, and 250D may be omitted from audio system 200. However, the binaural filters 250A, 250B, 250C, and 250D may be used to enhance spatial imaging. In one embodiment, the binaural filters may be applied to channels other than the surround input channels.For example, a binaural filter may be applied to each of the left and right spatial emphasis channels output from the subband spatial processor 230A to accommodate different left and right output speaker positions. In another example, if the input audio signal includes channels associated with other speaker positions (i.e., overhead, rear center, etc.), binaural processing may be applied to the other input channels. In that sense, binaural processing may be applied to one or more of the left input channel 210A, the right input channel 210B, the center input channel 210C, or the low frequency input channel 210D. In one embodiment, no HRTFs are applied, and one or more of the binaural filters 250A, 250B, 250C, and 250D may be bypassed or omitted from the system 200.

[0026] An example of a binaural filter is defined in Equation 1.

[0027]

number

[0028] where So and Si are the output and input signals, respectively. The argument θ encodes the angle of each channel in So and Si. The value z is any complex number whose solution is a function encoding frequency. Thus, H(θ,z) is a function of both the angle θ and z, returning a transfer function that is a function of z, selected or interpolated from a set of transfer functions, possibly derived from an anthropometric base. In this notation, the angle θ may also be evaluated as a vector, as may S and H(θ) as functions of z. In this case, each coefficient of S(z) and H(θ,z) corresponds to a different channel, and each coefficient of θ relates the angle of each channel.

[0029] In one embodiment, the input audio signal is an Ambisonics audio signal that defines a speaker-independent representation of a sound field. The Ambisonics audio signal may be decoded into a multi-channel audio signal for a surround sound system. The channels may be associated with the locations of speakers at various locations, including above or below the listener. A binaural filter may be applied to each decoded input channel of the Ambisonics audio signal to adjust the position relative to the decoded input audio channel.

[0030] In one embodiment, binaural filtering is performed before the sub-band spatial processing. For example, a binaural filter may be applied to one or more input channels as appropriate to adjust the angular position associated with the channels. For each pair of left and right input channels, the left output channel of the binaural filter may be combined, and the right output channel of the binaural filter may be combined, and the sub-band spatial processing may be applied to the combined left and right channels. In one embodiment, the binaural filter is applied to the center input channel 210C or the low frequency input channel 210D. In one embodiment, the binaural filter is applied to each input channel except the low frequency input channel 210D.

[0031] The left channel combiner 260A is coupled to the sub-band spatial processor 230A and to the binaural filters 250A, 250B, 250C, and 250D. The left channel combiner 260A receives the left output channel of the sub-band spatial processor 230A and to the binaural filters 250A, 250B, 250C, and 250D and combines these channels into a left combined channel. The right channel combiner 260B is also coupled to the sub-band spatial processor 230A and to the binaural filters 250A, 250B, 250C, and 250D. The right channel combiner 260A receives the right output channel of the sub-band spatial processor 230A and to the binaural filters 250A, 250B, 250C, and 250D and combines these channels into a right combined channel.

[0032] The crosstalk cancellation processor 270 receives the left and right input channels and performs crosstalk cancellation to generate left and right crosstalk cancellation channels. The crosstalk cancellation processor is coupled to the left channel combiner 260A to receive the left combined channel and to the right channel combiner 260B to receive the right combined channel, where the left and right combined channels processed by the crosstalk cancellation processor 270 represent the mixed down left and right corresponding input channels. Additional details regarding the crosstalk cancellation processor 270 are shown in FIG. 4 and described below.

[0033] The high shelf filter 220 receives the center input channel 210C and applies a high frequency shelving or peaking filter. The high shelf filter 220 provides a "voice lift" to the center input channel 210C. In one embodiment, the high shelf filter 220 is bypassed or omitted from the audio system 200. The high shelf filter 220 can attenuate or amplify frequencies above a cutoff frequency. The high shelf filter 220 is coupled to the left channel combiner 260C and the right channel combiner 260D. In one embodiment, the high shelf filter 220 is defined by a cutoff frequency of 750 Hz, a +3 dB gain, and a Q factor of 0.8. The high shelf filter 220 produces a left center channel and a right center channel as outputs, such as by splitting the center input channel into two separate left and right center channels.

[0034] The splitter 240 receives the low frequency input channel 210D and separates the low frequency input channel 210D into left and right low frequency channels. The splitter 240 is coupled to the left channel combiner 260C and the right channel combiner 260D, and provides the left low frequency channel to the left channel combiner 260C and the right low frequency channel to the right channel combiner 260D.

[0035] The left channel combiner 260C is coupled to the crosstalk cancellation processor 270, the high shelf filter 220, and the splitter 240. The left channel combiner 260C receives the left crosstalk channel from the crosstalk cancellation processor 270, the left center channel from the high shelf filter 220, and the left low frequency channel from the splitter 240 and combines these channels into a left output channel.

[0036] The right channel combiner 260D is coupled to the crosstalk cancellation processor 270, the high shelf filter 220, and the splitter 240. The right channel combiner 260D receives the right crosstalk channel from the crosstalk cancellation processor 270, the right output channel from the high shelf filter 220, and the right low frequency channel from the splitter 240 and combines these channels into the right output channel.

[0037] In one embodiment, the left center channel from the high shelf filter 220 and the left low frequency channel from the splitter 240 are combined by the left channel combiner 260A with the left spatial enhancement channel from the subband spatial processor 230A and the left output channel of the binaural filters 250A, 250B, 250C, and 250D to generate a left combined channel. Similarly, the right output channel from the high shelf filter 220 and the right low frequency channel from the splitter 240 are combined by the right channel combiner 260B with the right spatial enhancement channel from the subband spatial processor 230A and the right output channel of the binaural filters 250A, 250B, 250C, and 250D to generate a right combined channel. The left and right combined channels are input to the crosstalk cancellation processor 270, where the center and low frequency channels receive the crosstalk cancellation operation. The left channel combiner 260C and the right channel combiner 260D may be omitted. In one embodiment, one of the center or low frequency channels receives the crosstalk cancellation operation.

[0038] An output gain 280 is coupled to the left channel combiner 260C and the right channel combiner 260D. The output gain 280 provides gain to the left output channel from the left channel combiner 260C and to the right output channel from the right channel combiner 260D. The output gain 280 may provide the same gain or different gains to the left and right output channels. The output gain 280 outputs a left output channel 290L and a right output channel 290R that represent the channels of the output signal of the audio system 200.

[0039] (Example of a subband spatial processor) 3 is an example of a sub-band spatial processor 230, according to one embodiment. The sub-band spatial processor 230 is an example of a sub-band spatial processor 230A, 230B, or 230C of the audio system 200. The sub-band spatial processor 230 includes a spatial frequency band distributor 340, a spatial frequency band processor 345, and a spatial frequency band combiner 350. The spatial frequency band distributor 340 is coupled to the spatial frequency band processor 345, and the spatial frequency band processor 345 is coupled to the spatial frequency band combiner 350.

[0040] The spatial frequency band splitter 340 splits the left input channel X L and right input channel XR and converts these inputs into spatial components X m and the non-spatial component X s The L / R to M / S converter 312 converts the spatial components X s is the left input channel X L and right input channel X R The non-spatial component X can be generated by subtracting m is the left input channel X L and right input channel X R It can be generated by adding:

[0041] The spatial frequency band processor 345 extracts the non-spatial component X m and applies a set of subband filters to obtain the non-spatial enhancement component E mThe spatial frequency band processor 345 also generates the spatial components X S and applies a set of subband filters to obtain the non-spatial enhancement component E m The subband filters may include various combinations of peak filters, notch filters, low pass filters, high pass filters, low shelf filters, high shelf filters, band pass filters, band stop filters, and / or all pass filters.

[0042] In one embodiment, the spatial frequency band processor 345 includes a non-spatial component X m and the subband filters for each of the n frequency subbands of the spatial component X s For example, in the case of n=4 subbands, the spatial frequency band processor 345 generates a non-spatial component X 1 including a mid equalization EQ filter 362(1) for subband(1), a mid EQ filter 362(2) for subband(2), a mid EQ filter 362(3) for subband(3), and a mid EQ filter 362(4) for subband(4). m Each mid EQ filter 362 includes a series of subband filters for the non-spatial components X m The non-spatial emphasis component E m Generate.

[0043] The spatial frequency band processor 345 processes the spatial components X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X12, X13, X14, X15, X16, X17, X28, X29, X30, X31, X32, X33, X44, X45, X46, X47, X48, X51, X52, X63, X74, X85, X96, X97, X108, X119, X120, X131, X141, X152, X162, X173, X184, X194, X195, X206, X210, X33, X45, X53, X64, X75, X86, X97 s Each side EQ filter 364 further includes a series of subband filters for the frequency subbands of spatial component X S The spatial emphasis component E S Generate.

[0044] non-spatial component X m and spatial component X s Each of the n frequency subbands may correspond to a range of frequencies. For example, frequency subband (1) may correspond to 0-300 Hz, frequency subband (2) may correspond to 300-510 Hz, frequency subband (3) may correspond to 510-2700 Hz, and frequency subband (4) may correspond to 2700 Hz to the Nyquist frequency. In one embodiment, the n frequency subbands are a set of integrated critical bands. The critical bands may be determined using a corpus of audio samples of various music genres. The long-term average energy ratio of the mid and side components across the 24 Bark scale critical bands is determined from the samples. Adjacent frequency bands with similar long-term average ratios are then grouped together to form a set of critical bands. The range of the frequency subbands is adjustable, as is the number of frequency subbands.

[0045] In one embodiment, the mid EQ filter 362 or the side EQ filter 364 may include a biquad filter having a transfer function defined by Equation 2:

[0046]

number

[0047] where z is a complex number. The filter can be implemented using a direct form I topology defined in Equation 3.

[0048]

number

[0049] where X is the input vector and Y is the output. Other topologies may be beneficial for a particular processor, depending on maximum word length and saturation behavior.

[0050] Biquads can then be used to implement any second-order filter with real-valued inputs and outputs. To design a discrete-time filter, a continuous-time filter is designed and transformed to a discrete-time filter via a bilinear transformation. Furthermore, compensation for the resulting shifts in center frequency and bandwidth can be achieved using frequency warping.

[0051] For example, the peaking filter may include an S-plane transfer function defined in Equation 4:

[0052]

number

[0053] where s is a complex variable, A is the amplitude of the peak, and Q is the "quality" of the filter (normally

[0054]

number

[0055] The digital filter coefficients are given by the formula:

[0056]

number

[0057]

number

[0058]

number

[0059]

number

[0060]

number

[0061]

number

[0062] where ω0 is the angle and

[0063]

number

[0064] is the center frequency of the filter, expressed as

[0065] The spatial frequency band combiner 350 receives the mid and side components, applies a gain to each component, and converts the mid and side components into left and right channels. For example, the spatial frequency band combiner 350 may convert the non-spatially enhanced components E m and the spatial enhancement component E s and receives the non-spatial emphasis component E m and the spatial enhancement component E s to the left spatial emphasis channel E L Right spatial emphasis channel E R Perform global mid and side gain before converting to . More specifically, the spatial frequency band combiner 350 includes a global mid gain 322, a global side gain 324, and an M / S to L / R converter 326 coupled to the global mid gain 322 and the global side gain 324. The global mid gain 322 is a non-spatial emphasis component E m The global mid gain 322 receives and applies a gain to the non-spatially emphasized component E s , and the global side gain 324 receives and applies a gain to the spatially emphasized component E s . The M / S to L / R converter 326 receives and applies a gain to the non-spatially emphasized component E s from the global mid gain 322 . m and the global side gain 324 to the spatial emphasis component E s and routes these inputs to the left spatial emphasis channel E L and right spatial emphasis channel E R Convert to.

[0066] (Example of a crosstalk cancellation processor) 4 illustrates a crosstalk cancellation processor 270 according to an example embodiment. The crosstalk cancellation processor 270 receives as input the left spatial enhancement channel E L , receives the right spatial emphasis channel E R as an input from the right channel combiner 260B, and receives the right spatial emphasis channel E L , E R Perform crosstalk cancellation on the left output channel O L and the right output channel O R Generate.

[0067] The crosstalk cancellation processor 270 includes an in-out band splitter 410, inverters 420 and 422, contralateral estimators 430 and 440, combiners 450 and 452, and an in-out band combiner 460. These components work together to split the input channel T L , T R into in-band and out-of-band components, perform crosstalk cancellation on the in-band component, and output channel O L , O R Generate.

[0068] By splitting the input audio signal E into different frequency band components and performing crosstalk cancellation on selective components (e.g., in-band components), crosstalk cancellation can be performed on specific frequency bands while preventing degradation in other frequency bands. If crosstalk cancellation is performed without splitting the input audio signal E into different frequency bands, the audio signal after such crosstalk cancellation may show significant attenuation or amplification in non-spatial and spatial components at low frequencies (e.g., below 350 Hz), high frequencies (e.g., above 12000 Hz), or both. By selectively performing crosstalk cancellation in-band (e.g., 250 Hz to 14000 Hz), where most of the impactful spatial cues reside, a balanced overall energy can be preserved across the mixed spectrum, especially in the non-spatial components.

[0069] The in-out band splitter 410 splits the input channel E L , E R In-band channel E L,IN , E R,IN and out-of-band channel E L,OUT , E R,OUT In particular, the in-out band splitter 410 splits the left emphasis compensation channel E L Left in-band channel E L、In and left out-of-band channel E L、Out Similarly, the in-out band splitter 410 splits the right emphasis compensation channel E R Right in-band channel E R、In and right out-of-band channel E R、Out Each in-band channel may, for example, encompass a portion of a respective input channel corresponding to a frequency range including 250 Hz to 14 kHz. The range of the frequency band may, for example, be adjustable according to the parameters of the speaker.

[0070] Left In-Band Channel E L、In To compensate for the contralateral sound component due to LSimilarly, the right inband channel E R、In To compensate for the contralateral sound component due to R Generate.

[0071] In one approach, the inverter 420 may be configured to L、In Receive the in-band channel E L、In Invert the polarity of the inverted in-band channel E L、In The contralateral estimator 430 generates the inverted in-band channel E L、In ', and through filtering, the inverted in-band channel E L、In The filtering is done by extracting a portion of the inverted in-band channel E L、In ', the portion extracted by the contralateral estimator 430 is the in-band channel E L、In Therefore, the part extracted by the contralateral estimator 430 is the inverse of the left contralateral cancellation component S L This corresponds to the in-band channel E R、In You can add an in-band channel E L、In In one embodiment, the inverter 420 and the contralateral estimator 430 are implemented in different sequences.

[0072] The inverter 422 and the contralateral estimator 440 are connected to the in-band channel E R、In Do the same for 、 Right contralateral cancellation component S R Therefore, for the sake of brevity, a detailed description thereof is omitted here.

[0073] In one exemplary implementation, the contralateral estimator 430 includes a filter 432, an amplifier 434, and a delay unit 436. The filter 432 receives an inverting input channel E L、In ' and through the filtering function 、Inverted in-band channel E corresponding to the contralateral sound component L、In An example of a filter implementation is a notch or high shelf filter with a center frequency chosen between 5000 and 10000 Hz and a Q chosen between 0.5 and 1.0. The gain in decibels (GdB) can be derived from Equation 5: GdB = -3.0-log1. 333 (D) Equation (5) where D is the amount of delay through delay unit 1556A / B in samples at a sampling rate of, for example, 48 KHz. An alternative implementation is a low pass filter with a corner frequency selected between 5000 and 10000 Hz and a Q selected between 0.5 and 1.0. Additionally, amplifier 434 has a corresponding gain factor G L、In and a delay unit 436 delays the amplified output from the amplifier 434 according to a delay function D to produce the left contralateral cancellation component S L The contralateral estimator 440 includes a filter 442, an amplifier 444, and an inverted in-band channel E R、In ’ A similar operation is performed on the right contralateral cancellation component S R In one example, the contralateral estimators 430, 440 include a delay unit 446 that generates the left contralateral cancellation component S according to the following equation: L , S R Generate. S L = D [G L、In * F [E L、In’ ]] Formula (6) S R = D [G R、In * F [E R、In’ ]] Formula (7) where F[] is the filter function and D[] is the delay function.

[0074] The crosstalk cancellation configuration can be determined by the speaker parameters. In one example, the filter center frequency, delay, amplifier gain, and filter gain can be determined according to the angle formed between the two output speakers of the output signal to the listener, or other functions of the speakers, such as relative position, power, etc. In one embodiment, the values ​​between the speaker angles are used to interpolate other values.

[0075] The combiner 450 outputs the right contralateral cancellation component S R Left in-band channel E L、In is combined with the left in-band compensation channel U L and combiner 452 generates the left contralateral cancellation component S L Right in-band channel E R、In The right in-band compensation channel U R The in-out band combiner 460 generates the left in-band compensation channel U L to the out-of-band channel E L、Out Left output channel in combination with L and generates the right in-band compensation channel U R to the out-of-band channel E R,OUT Combined with the right output channel O R Generate.

[0076] Therefore, the left output channel O L is the in-band channel T due to the contralateral sound. R、In The right contralateral cancellation component S corresponds to the inverse of a part of R Right output channel O R is the in-band channel T due to the contralateral sound. L The left contralateral cancellation component S corresponds to the inverse of a part of L In this configuration, the right output channel O R The wavefront of the same-side audio component output from the right speaker (e.g., speaker 110R) in response to L In the same manner, the wavefront of the contralateral sound component output from the right speaker (e.g., speaker 110L) can be cancelled according to the left output channel O LThe wavefront of the same-side audio output from the left speaker according to R Therefore, the wavefront of the contralateral sound component output from the right speaker can be canceled according to the contralateral sound component, thereby improving spatial detection ability by reducing the contralateral sound component.

[0077] (Example of audio signal enhancement process) Figure 5 is an example of a method 500 for enhancing an audio signal using the audio system 200 shown in Figure 2, according to one embodiment. In one embodiment, the method 500 may include different and / or additional steps, or some steps may be in a different order.

[0078] At 505, the audio system 200 receives a multi-channel input audio signal. The multi-channel audio signal may be a surround sound audio signal including a left input channel, a right input channel, at least one left peripheral input channel, and at least one right peripheral input channel. The multi-channel audio signal may further include a center input channel 210C and a low frequency input channel 210D. For example, the input audio signal may be for a 7.1 surround sound system including a left input channel 210A and a right input channel 210B and a peripheral channel including a left surround input channel 210E and a right surround input channel 210F, and a left surround rear input channel 210G and a right surround rear input channel 210H. In another example of an input audio signal for a 5.1 surround sound system, the peripheral channel may include a single left peripheral channel and a single right peripheral channel.

[0079] At 510, audio system 200 (e.g., gains 215A through 215H) provide gains to channels of a multi-channel input audio signal. The gains 215A through 215H may be varied to control the contribution of a particular input channel to the output signal generated by audio system 200. In some embodiments, center channel 210C receives a negative gain while the surround input channels receive a positive gain.

[0080] At 515, the audio system 200 (e.g., the subband spatial processor 230A) generates left and right spatial enhancement channels by performing subband spatial processing on the left and right input channels. For example, the subband spatial processor 230A generates the spatial enhancement channels by adjusting the gains of n subbands of the mid and side components of the left and right input channels 210A and 210B.

[0081] At 520, the audio system 200 (e.g., subband spatial processors 230B and / or 230C) perform subband spatial processing on the left and right ambient input channels to generate left and right spatially enhanced ambient channels. For example, the subband spatial processor 230B adjusts the gains of n subbands of the mid and side components of the left surround channel 210E and the right surround channel 210F to generate the left and right spatially enhanced ambient channels. The subband spatial processor 230C adjusts the gains of n subbands of the mid and side components of the left surround rear channel 210G and the right surround rear channel 210H to generate the left and right spatially enhanced ambient channels.

[0082] At 525, audio system 200 (e.g., binaural filters 250A-250D) apply binaural filters to each of the left and right spatially enhanced surround channels. For example, binaural filter 250A generates left and right output channels from the left spatially enhanced surround channel output from sub-band spatial processor 230B by applying a head-related transfer function (HRTF). Binaural filter 250B generates left and right output channels from the right spatially enhanced channel output by sub-band spatial processor 230B by applying a HRTF. Binaural filter 250C generates left and right output channels from the left spatially enhanced channel output by sub-band spatial processor 230C by applying a HRTF. Binaural filter 250D generates left and right output channels from the right spatially enhanced channel output by sub-band spatial processor 230C by applying a HRTF. In one embodiment, binaural filtering is bypassed.

[0083] At 530, the audio system 200 (e.g., the high shelf filter 220) applies a high shelf filter to the center input channel 210C. In one embodiment, a gain is applied to the center input channel 210C. Additionally, the high shelf filter 220 splits the center input channel 210C into a left center channel and a right center channel.

[0084] At 535, the audio system 200 (eg, splitter 240) splits the low frequency input channel into left and right low frequency channels.

[0085] At 540, audio system 200 (e.g., left channel combiner 260A) combines the left spatial emphasis channel from subband spatial processor 230A with the left output channel of binaural filters 250A, 250B, 250C, and 250D to generate a left combined channel. For example, the left spatial emphasis channel may be added to the left output channel.

[0086] At 545, the audio system 200 (e.g., right channel combiner 260B) combines the right spatial emphasis channel from the subband spatial processor 230A with the right output channel of the binaural filters 250A, 250B, 250C, and 250D to generate a right combined channel. For example, the right spatial emphasis channel may be added to the right output channel.

[0087] At 550, the audio system 200 (eg, the crosstalk cancellation processor 270) performs crosstalk cancellation on the left and right combined channels to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel.

[0088] At 555, the audio system 200 (e.g., left channel combiner 260C and right channel combiner 260D) combines the left crosstalk cancellation channel from the crosstalk cancellation processor 270 with the left low frequency channel from the splitter 240 and the left center channel from the high shelf filter 220 to generate a left peripheral channel. Also, the right crosstalk cancellation channel from the crosstalk cancellation processor 270 is combined with the right low frequency channel from the splitter 240 and the right center channel from the high shelf filter 220 to generate a right peripheral output channel. Additionally, the audio system 200 (e.g., output gain 280) can provide gain to each of the left and right output channels. The audio system 200 outputs an output audio signal including left and right output channels 290L and 290R.

[0089] (Example of an audio system and an example of an audio processing process) 6 is an example of an audio system, according to one embodiment. Audio system 600 may be similar to audio system 200, but may differ from audio system 200 in that at least the left and right input channels are combined with the left and right surround channels prior to subband spatial processing in audio system 600. Here, rather than separate subband spatial processors for the left and right speaker pairs as shown in audio system 200, a single subband spatial processor and corresponding subband spatial processing steps may be used.

[0090] The audio system 600 receives an input audio signal. The input audio signal may include a left input channel 610A, a right input channel 610B, a center input channel 610C, a low frequency input channel 610D, a left surround input channel 610E, a right surround input channel 610F, a left surround rear input channel 610G, and a right surround rear input channel 610H. The channels 610E, 610F, 610G, and 610H are examples of peripheral channels that may be provided to surround a speaker. In an embodiment, the audio system 600 may receive and process an input audio signal having fewer or more channels.

[0091] The audio system 600 uses sub-band spatial processing and enhancements such as crosstalk cancellation on the input audio signal to generate an output signal including a left output channel 690L and a right output channel 690R. The left output channel 290L may be provided to a left speaker, and the right output channel 290R may be output to a right speaker. The output audio signal uses the left and right speakers (e.g., left speaker 110L and right speaker 110R) to provide a spatial sense of a sound field associated with a surround sound input audio signal.

[0092] Audio system 600 includes gains 615A, 615B, 615C, 615D, 615E, 615F, 615G, and 615H, a high shelf filter 620, a splitter 640, binaural filters 650A, 650B, 650C, and 650D, a left channel combiner 660A, a right channel combiner 660B, a subband spatial processor 630, a crosstalk cancellation processor 670, a left channel combiner 660C, a right channel combiner 660D, and an output gain 680.

[0093] Each of the gains 615A to 615H can receive a respective input channel 610A to 610H and can provide a gain to the input channel 610A to 610H. The gains 215A to 215H can be different or the same to adjust the gain of the input channels relative to each other. In one embodiment, a positive gain is provided to the left and right peripheral input channels 210E, 210F, 210G, and 210H, and a negative gain is provided to the center channel 210C. For example, the gain 215A can provide a 0db gain, the gain 215B can provide a 0dB gain, the gain 215C can provide a -3dB gain, the gain 215D can provide a 0db gain, the gain 215E can provide a 3dB gain, the gain 215F can provide a 3dB gain, the gain 215G can provide a 3dB gain, and the gain 215H can provide a 3dB gain.

[0094] The gain 615A of the left input channel 610A is coupled to a left channel combiner 660A. The gain 615B of the right input channel 610B is coupled to a right channel combiner 660B. The gain 615C is coupled to a high shelf filter 620. The gain 615D is coupled to a splitter 640. The gains 615E, 615F, 610G, and 610H of the surround input channels are each coupled to a binaural filter 650. In particular, the gain 610E is coupled to binaural filter 650A, the gain 615F is coupled to binaural filter 650B, the gain 615G is coupled to binaural filter 650C, and the gain 615H is coupled to binaural filter 650D.

[0095] Each of the binaural filters 650A, 650B, 650C, and 650D applies a head-related transfer function (HRTF) that describes the location of the target source from which the listener should perceive the sound of the input channel. Each binaural filter receives an input channel and generates left and right output channels by applying an HRTF. The discussion of the binaural filters 250A, 250B, 250C, and 250D of the audio system 200 may be applicable to the binaural filters 650A, 650B, 650C, and 650D. For example, each of the binaural filters 650A to 650D may apply an adjustment to an angular position associated with the respective input channel. In an embodiment, one or more of the binaural filters 650A to 650D may be bypassed or omitted from the audio system 600.

[0096] The left channel combiner 660A is coupled to the gain 615A and to the binaural filters 650A through 650D. The left channel combiner 660A receives the left output channels of the binaural filters 650A through 650D and combines the left output channel with the output of the gain 615A. The right channel combiner 660B is coupled to the gain 615B and to the binaural filters 650A through 650D. The right channel combiner 660B receives the right output channels of the binaural filters 650A through 650D and combines the right output channel with the output of the gain 615B.

[0097] In one embodiment, binaural filtering is performed prior to the sub-band spatial processing. For example, binaural filters may be applied to the left and right outputs of the sub-band spatial processor 630 as appropriate to adjust the angular position associated with the channels. In one embodiment, binaural filters are applied to the peripheral input channels as shown in FIG. 6. In one embodiment, binaural filters are applied to the center input channel 610C or the low frequency input channel 610D. In one embodiment, binaural filters are applied to each input channel except the low frequency input channel 610D.

[0098] The subband spatial processor 630 performs subband spatial processing on the left and right input channels by gain adjusting the mid and side subband components of the left and right input channels to generate left and right spatial enhancement channels. The subband spatial processor 630 is coupled to the left channel combiner 660A to receive the left combined channel from the left channel combiner 660A and to the right channel combiner 660B to receive the right combined channel from the right channel combiner 660B. Unlike the subband spatial processors 230A, 230B, and 230C of the audio system 200, which process the corresponding left and right input channels, the subband spatial processor 630 processes the left and right channels after combining the left and right channels. Thus, the audio system 600 may include only a single subband spatial processor 630. In one embodiment, the subband spatial processor 230 shown in FIG. 3 is an example of the subband spatial processor 630.

[0099] The crosstalk cancellation processor 670 performs crosstalk cancellation on the output of the sub-band spatial processor 630, which may represent a mixed-down stereo signal of the input audio signal. The crosstalk cancellation processor 670 receives the left and right input channels from the sub-band spatial processor 630 and performs crosstalk cancellation to generate left and right crosstalk cancellation channels. The crosstalk cancellation processor 670 is coupled to the left channel combiner 660C and the right channel combiner 660D. In one embodiment, the crosstalk cancellation processor 270 shown in FIG. 4 is an example of the crosstalk cancellation processor 670.

[0100] The high shelf filter 620 receives the center input channel 610C and applies a high frequency shelving or peaking filter. The high shelf filter 620 provides a "voice lift" to the center input channel 610C. In one embodiment, the high shelf filter 620 is bypassed or omitted from the audio system 600. The high shelf filter 620 can attenuate or amplify frequencies above a cutoff frequency. The high shelf filter 620 is coupled to a left channel combiner 660C and a right channel combiner 660D. In one embodiment, the high shelf filter 620 is defined by a cutoff frequency of 750 Hz, a gain of +3 dB, and a Q factor of 0.8. The high shelf filter 620 produces a left center channel and a right center channel as outputs.

[0101] The splitter 640 receives the low frequency input channel 610D and separates the low frequency input channel 610D into left and right low frequency channels. The splitter 640 is coupled to the left channel combiner 660C and the right channel combiner 660D, and provides the left low frequency channel to the left channel combiner 660C and the right low frequency channel to the right channel combiner 660D.

[0102] The left channel combiner 660C is coupled to the crosstalk cancellation processor 670, the high shelf filter 620, and the splitter 640. The left channel combiner 660C receives the left crosstalk channel from the crosstalk cancellation processor 670, the left center channel from the high shelf filter 620, and the left low frequency channel from the splitter 640 and combines these channels into a left output channel.

[0103] The right channel combiner 660D is coupled to the crosstalk cancellation processor 670, the high shelf filter 620, and the splitter 640. The right channel combiner 660D receives the right crosstalk channel from the crosstalk cancellation processor 670, the right center channel from the high shelf filter 620, and the right low frequency channel from the splitter 640 and combines these channels into a right output channel.

[0104] In one embodiment, the left center channel from the high shelf filter 620 and the left low frequency channel from the splitter 640 are combined with the output of the left output channel and gain 615A of the binaural filters 650A to 650D by the left channel combiner 660A to generate the left combined channel. The right center channel from the high shelf filter 620 and the right low frequency channel from the splitter 640 are combined with the output of the right output channel and gain 615B of the binaural filters 650A to 650D by the right channel combiner 660B to generate the right combined channel. The left and right combined channels are input to the sub-band spatial processor 630 and the crosstalk cancellation processor 670, where the center frequency channel and the low frequency channel receive sub-band spatial processing and crosstalk cancellation operations. The left channel combiner 660C and the right channel combiner 660D may be omitted. In one embodiment, one of the center or low frequency channels receives sub-band spatial processing and crosstalk cancellation operations.

[0105] An output gain 680 is coupled to the left channel combiner 660C and the right channel combiner 660D. The output gain 680 provides gain to the left output channel from the left channel combiner 660C and to the right output channel from the right channel combiner 660D. The output gain 680 may provide the same gain or different gains to the left and right output channels. The output gain 680 outputs a left output channel 690L and a right output channel 690R that represent the channels of the output signal of the audio system 600.

[0106] Figure 7 shows an example of a method 700 for enhancing an audio signal using the audio system 600 shown in Figure 6, according to one embodiment. In one embodiment, the method 700 may include different and / or additional steps, or some steps may be in a different order.

[0107] At 705, the audio system 600 receives a multi-channel input audio signal. The input audio signal may include a left input channel 610A, a right input channel 610B, at least one left peripheral input channel, and at least one right peripheral input channel. The multi-channel audio signal may further include a center input channel 610C and a low frequency input channel 610D.

[0108] At 710, the audio system 600 (e.g., gains 615A through 615H) provide gain to channels of a multi-channel input audio signal. The gains 615A through 615H may be varied to control the contribution of a particular input channel to the output signal generated by the audio system 600.

[0109] At 715, audio system 600 (e.g., binaural filters 650A-650D) apply binaural filters to each of the left and right surround channels. For example, binaural filter 650A generates left and right output channels from left surround input channel 610E by applying a head-related transfer function (HRTF). Binaural filter 650B generates left and right output channels from right surround input channel 610F by applying a HRTF. Binaural filter 650C generates left and right output channels from left surround rear input channel 610G by applying a HRTF. Binaural filter 650D generates left and right output channels from right surround rear input channel 610H by applying a HRTF.

[0110] At 720, the audio system 600 (e.g., high shelf filter 620) applies a high shelf filter to the center input channel 610C. In one embodiment, gain is applied to the center input channel 610C. Additionally, the high shelf filter 620 splits the center input channel 610C into a left center channel and a right center channel.

[0111] At 725, the audio system 600 (eg, splitter 640) splits the low frequency input channel into left and right low frequency channels.

[0112] At 730, audio system 600 (eg, left channel combiner 660A) combines the left input channel 610A with the left output channel of binaural filters 650A, 650B, 650C, and 650D to generate a left combined channel.

[0113] At 735, the audio system 600 (eg, right channel combiner 660B) combines the right input channel 610B and the right output channels of the binaural filters 650A, 650B, 650C, and 650D to generate a right combined channel.

[0114] At 740, the audio system 600 (e.g., the subband spatial processor 630) generates a left spatial enhancement channel and a right spatial enhancement channel by performing subband spatial processing on the left and right combined channels. For example, the subband spatial processor 630 receives the left and right combined channels from the left channel combiner 660A and the right channel combiner 660B, and generates the spatial enhancement channel by adjusting the gains of the n subbands of the mid and side components of the left and right combined channels.

[0115] At 745, the audio system 600 (eg, the crosstalk cancellation processor 670) performs crosstalk cancellation on the left and right spatial enhancement channels from the subband spatial processor 630 to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel.

[0116] At 750, the audio system 600 (e.g., left channel combiner 660C and right channel combiner 660D) combine the left crosstalk cancellation channel from the crosstalk cancellation processor 670 with the left low frequency channel from the splitter 640 and the left center channel from the high shelf filter 620 to generate a left output channel, and combine the right crosstalk cancellation channel from the crosstalk cancellation processor 670 with the right low frequency channel from the splitter 640 and the right center channel from the high shelf filter 620 to generate a right output channel. Additionally, the audio system 600 (e.g., output gain 680) can provide gain to each of the left and right output channels. The audio system 600 outputs an output audio signal including left and right output channels 690L and 690R.

[0117] It should be noted that the systems and processes described herein may be implemented in embedded electronic circuits or systems. The systems and processes may also be embodied in a computing system that includes one or more processing systems (e.g., digital signal processors) and memory (e.g., programmed read-only memory or programmable solid-state memory), or some other circuitry, such as an application specific integrated circuit (ASIC) or field programmable gate array (FPGA) circuitry.

[0118] 8 illustrates an example of a computer system 800, according to one embodiment. Audio systems 200 and 600 may be implemented on system 800. Illustrated is at least one processor 802 coupled to a chipset 804. Chipset 804 includes a memory controller hub 820 and an input / output (I / O) controller hub 822. Memory 806 and graphics adapter 812 are coupled to memory controller hub 820, and display device 818 is coupled to graphics adapter 812. Storage device 808, keyboard 810, pointing device 814, and network adapter 816 are coupled to I / O controller hub 822. Other embodiments of computer 800 have different architectures. For example, in one embodiment, memory 806 is directly coupled to processor 802.

[0119] The storage device 808 includes one or more non-transitory computer-readable storage media, such as a hard drive, a compact disc read-only memory (CD-ROM), a DVD, or a solid-state memory device. The memory 806 holds instructions and data used by the processor 802. For example, the memory 806 can store instructions that, when executed by the processor 802, cause the processor 802 to perform or configure a method discussed herein, such as method 500 or 700. The pointing device 814 is used in combination with the keyboard 810 to input data to the computer system 800. The graphics adapter 812 displays images and other information on the display device 818. In one embodiment, the display device 818 includes touch screen functionality for receiving user inputs and selections. The network adapter 816 couples the computer system 800 to a network. An embodiment of the computer 800 has different and / or other components than those shown in FIG. 8. For example, the computer system 800 can be a server that lacks a display device, a keyboard, and other elements.

[0120] The computer 800 is adapted to execute computer program modules for providing the functions described herein. As used herein, the term "module" refers to computer program instructions and / or other logic used to provide a specified function. Thus, a module may be implemented in hardware, firmware, and / or software. In one embodiment, a program module formed of executable computer program instructions is stored in the storage device 808, loaded into the memory 806, and executed by the processor 802.

[0121] (Other considerations) The disclosed configurations may include several benefits and / or advantages. For example, a multi-channel input signal can be output to stereo speakers while preserving or enhancing the spatial sense of the sound field. A high-quality listening experience can be achieved without the need for expensive multi-speaker sound systems such as mobile devices, sound bars, smart speakers, etc.

[0122] Upon reading this disclosure, those skilled in the art will recognize even further alternative embodiments of the principles disclosed herein. Thus, while specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the precise structures and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, can be made in the arrangement, operation and details of the methods and apparatus disclosed herein without departing from the scope of the present disclosure.

[0123] Any of the steps, operations, or processes described herein may be performed or implemented in one or more hardware or software modules, alone or in combination with other devices. In one embodiment, the software modules are implemented in a computer program product that includes a computer-readable medium (e.g., a non-transitory computer-readable medium) containing computer program code and can be executed by a computer processor that performs any or all of the steps, operations, or process descriptions. [Explanation of symbols]

[0124] 200 Audio System 210A Left Input Channel 210B Right Input Channel 210C Center Input Channel 210D Low Frequency Input Channel 210E Left Surround Input Channel 210F Right Surround Input Channel 210G Left Surround Rear Input Channel 210H Right surround rear input channel 215A gain 215B Gain 215C gain 215D gain 215E Gain 215F gain 215G gain 215H gain 220 High Shelf Filter 230A Subband Spatial Processor 230B Subband Spatial Processor 230C Subband Spatial Processor 240 Distributor 250A Binaural Filter 250B Binaural Filter 250C Binaural Filter 250D Binaural Filter 260A Left Channel Combiner 260B Right Channel Combiner 260C Left Channel Combiner 260D Right Channel Combiner 270 Crosstalk Cancellation Processor 280 Output Gain 290L Left Output Channel 290R Right Output Channel

Claims

1. 1. A system for processing a multi-channel input audio signal, comprising: receiving said multi-channel input audio signal including a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel; generating a left combined channel and a right combined channel from the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel; performing subband spatial processing on the left combined channel and the right combined channel to generate a spatially enhanced channel; performing crosstalk cancellation on the spatial enhancement channel to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel; generating a left output channel from the left crosstalk cancellation channel and a right output channel from the right crosstalk cancellation channel; A system comprising a circuit configured to:

2. The circuitry configured to perform the subband spatial processing comprises: Gain-adjusting the mid subband components and the side subband components of the left combined channel and the right combined channel 2. The system of claim 1, further comprising a circuit configured to:

3. 2. The system of claim 1, wherein the circuitry is further configured to provide gain to the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel.

4. The circuitry for generating the left and right combined channels comprises: applying a binaural filter to each of the left peripheral input channel and the right peripheral input channel; combining a left output channel of the binaural filter with the left input channel to generate the left combined channel; Combining the right output channel of the binaural filter with the right input channel to generate the right combined channel.

2. The system according to claim 1, configured to:

5. 5. The system of claim 4, wherein the binaural filters include a first binaural filter and a second binaural filter, the first binaural filter adjusting an angular position associated with the left peripheral input channel and the second binaural filter adjusting an angular position associated with the right peripheral input channel.

6. the spatial enhancement channels include a left spatial enhancement channel and a right spatial enhancement channel, and the circuitry configured to perform the crosstalk cancellation comprises: splitting the left spatial enhancement channel into a left in-band signal and a left out-of-band signal; splitting the right spatial enhancement channel into a right in-band signal and a right out-of-band signal; generating a left crosstalk cancellation component by filtering and time delaying the left in-band signal; generating a right crosstalk cancellation component by filtering and time delaying the right in-band signal; generating the left crosstalk cancellation channel by combining the left in-band signal and the left out-of-band signal with the right crosstalk cancellation component; generating the right crosstalk cancellation channel by combining the right in-band signal and the right out-of-band signal with the left crosstalk cancellation component; 2. The system of claim 1, further comprising a circuit configured to:

7. 2. The system of claim 1, wherein the left peripheral input channel is a left surround input channel of the multi-channel input audio signal, and the right peripheral input channel is a right surround input channel of the multi-channel input audio signal.

8. 2. The system of claim 1, wherein the left peripheral input channel is a left surround rear input channel of the multi-channel input audio signal, and the right peripheral input channel is a right surround rear input channel of the multi-channel input audio signal.

9. 2. The system of claim 1, wherein the circuitry is further configured to combine a center channel and a low frequency channel of the multi-channel input audio signal with the left crosstalk cancellation channel and the right crosstalk cancellation channel.

10. 10. The system of claim 9, wherein the circuitry is further configured to apply a binaural filter to each of the left input channel, the right input channel, the left peripheral input channel, the right peripheral input channel, and the center channel.

11. 10. The system of claim 9, wherein the circuitry is further configured to apply a high shelf filter to the center input channel prior to combining the center input channel with the left crosstalk cancellation channel and the right crosstalk cancellation channel.

12. The circuit comprises: combining at least one of a center channel and a low frequency channel with the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel to generate a combined channel; performing the subband spatial processing and the crosstalk cancellation on the combined channel; 10. The system of claim 1 further configured to:

13. When executed by the processor, receiving a multi-channel input audio signal including a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel; generating a left combined channel and a right combined channel from the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel; performing subband spatial processing on the left combined channel and the right combined channel to generate a spatially enhanced channel; performing crosstalk cancellation on the spatial enhancement channel to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel; generating a left output channel from the left crosstalk cancellation channel and a right output channel from the right crosstalk cancellation channel; A non-transitory computer readable medium storing program code that causes the processor to:

14. The program code for causing the processor to perform the subband spatial processing comprises: Gain-adjusting the mid subband components and the side subband components of the left combined channel and the right combined channel 14. The computer readable medium of claim 13, comprising program code that causes the processor to:

15. 14. The computer readable medium of claim 13, wherein the program code further causes the processor to apply gain to the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel.

16. The program code further causing the processor to generate the left combined channel and the right combined channel, applying a binaural filter to each of the left peripheral input channel and the right peripheral input channel; combining a left output channel of the binaural filter with the left input channel to generate the left combined channel; Combining the right output channel of the binaural filter with the right input channel to generate the right combined channel.

14. The computer-readable medium of claim 13, configured to cause the processor to:

17. The binaural filter includes a first binaural filter and a second binaural filter, and the program code includes: adjusting an angular position associated with the left peripheral input channel by the first binaural filter; Adjusting the angular position associated with the right peripheral input channel by the second binaural filter.

20. The computer-readable medium of claim 16, further comprising:

18. The spatial enhancement channels include a left spatial enhancement channel and a right spatial enhancement channel, and the program code for causing the processor to perform the crosstalk cancellation further comprises: splitting the left spatial enhancement channel into a left in-band signal and a left out-of-band signal; splitting the right spatial enhancement channel into a right in-band signal and a right out-of-band signal; generating a left crosstalk cancellation component by filtering and time delaying the left in-band signal; generating a right crosstalk cancellation component by filtering and time delaying the right in-band signal; generating the left crosstalk cancellation channel by combining the left in-band signal and the left out-of-band signal with the right crosstalk cancellation component; generating the right crosstalk cancellation channel by combining the right in-band signal and the right out-of-band signal with the left crosstalk cancellation component; 14. The computer readable medium of claim 13, comprising program code that causes the processor to:

19. 14. The computer-readable medium of claim 13, wherein the left peripheral input channel is a left surround input channel of the multi-channel input audio signal, and the right peripheral input channel is a right surround input channel of the multi-channel input audio signal.

20. 14. The computer-readable medium of claim 13, wherein the left peripheral input channel is a left surround rear input channel of the multi-channel input audio signal, and the right peripheral input channel is a right surround rear input channel of the multi-channel input audio signal.

21. 14. The computer-readable medium of claim 13, wherein the program code further causes the processor to combine a center channel and a low frequency channel of the multi-channel input audio signal with the left crosstalk cancellation channel and the right crosstalk cancellation channel.

22. 22. The computer-readable medium of claim 21, wherein the program code further causes the processor to apply a binaural filter to each of the left input channel, the right input channel, the left peripheral input channel, the right peripheral input channel, and a center channel.

23. 22. The computer-readable medium of claim 21, wherein the program code further causes the processor to apply a high shelf filter to the center input channel before combining the center input channel with the left crosstalk cancellation channel and the right crosstalk cancellation channel.

24. The program code comprises: combining the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel with at least one of a center channel and a low frequency channel to generate a combined channel; 4. Perform the subband spatial processing and the crosstalk cancellation on the combined channel 14. The computer-readable medium of claim 13, further comprising the processor.

25. 1. A method for processing a multi-channel input audio signal, comprising the steps of: receiving the multi-channel input audio signal including a left input channel, a right input channel, a left peripheral input channel, and a right peripheral input channel; generating a left combined channel and a right combined channel from the left input channel, the right input channel, the left peripheral input channel, and the right peripheral input channel; performing subband spatial processing on the left combined channel and the right combined channel to generate a spatially enhanced channel; performing crosstalk cancellation on the spatial enhancement channel to generate a left crosstalk cancellation channel and a right crosstalk cancellation channel; generating a left output channel from the left crosstalk cancellation channel and a right output channel from the right crosstalk cancellation channel; 23. A method comprising:

Citation Information

Patent Citations

  • Sound field effect device

    JP1997200897A

  • Acoustic processor

    JP2003092799A

  • Sound signal processing equipment

    JP2008228225A

  • Subband spatial and crosstalk cancellation for audio reproduction

    US20170208411A1