Acoustic echo cancellation unit

Abstract

An acoustic echo cancellation unit comprising: an audio processor configured to receive a multi-channel audio signal and comprising: a first stage configured to process the multi-channel audio signal to obtain a first set of spatial audio components; and a second stage configured to process the first set of spatial audio components to obtain a second set of spatial audio components; an echo cancellation processor configured to perform echo cancellation using the first set of spatial audio components or a warped version of the first set of spatial audio components as a reference signal.

Description

TECHNICAL FIELD

[0001] Embodiments of the invention relate to an acoustic echo cancellation unit and to a corresponding method. Other embodiments relate to a computer program. Another embodiment provides a soundbar or another playback device comprising an acoustic echo cancellation unit. Preferred embodiments of the invention relate to the field of processing audio signals, more specifically to a method for acoustic echo cancellation (AEC) for multi-channel audio playback. BACKGROUND

[0002] Acoustic echo cancellation is desirable for a wide range of applications. For example, it facilitates human-machine interaction by far-field sound capture and strong insertion functionality, and enables full-duplex voice communication. For performing AEC, typically a microphone signal or a plurality of microphone signals and an AEC reference signal or an AEC reference signal comprising one, two or more channels are used. Generally, all methods in the literature use the loudspeaker drive signal as reference signal, as shown in Figure 3

[0003] Figure 3 An audio processing path 10 as well as an acoustic echo cancellation path 20 are shown.

[0004] The audio processing path 10 comprises an audio processor 12 and one or more loudspeakers 14. The loudspeakers 14 can be formed by a conventional multi-loudspeaker setup (5.1 or 7.2 or the like) or by a soundbar having at least one, preferably two or three or more transducers.

[0005] The audio processor 12 receives a multi-channel audio (e.g. a 5.2 or 7.2 signal) and processes it such that a surround sound can be reproduced by using the loudspeakers 14. For example, the audio processor 12 is configured to process a multi-channel audio signal MS to obtain a spatial component SC via which the soundbar 14 is controlled.

[0006] The echo cancellation path 20 comprises an acoustic echo cancellation unit 22 which is configured to compute an echo cancellation component based on an input signal IS and a reference signal RS. As input signal IS, a microphone signal from one or more microphones (see reference sign 24) is used. Typically, the spatial component SC is used as reference signal RS. Thus, the echo cancellation unit 22 comprises inputs for the reference signal RS and for the one or more microphones 24, the reference signal RS being connected to the audio processor 12. An echo cancellation parameter is output to a back-end 26.

[0007] ​A significant AEC performance degradation can be expected when there is a high correlation between the loudspeaker driving signals using a large number of loudspeakers 14 and / or when the loudspeaker driving signals. The correlation between the loudspeaker driving signals can be reduced by applying decorrelation methods, but this is at the expense of reproduction fidelity, and thus is not desirable for some applications. Therefore, there is a need for improved methods. SUMMARY

[0008] It is an object of the present invention to provide a concept for acoustic echo cancellation that has an improved trade-off between echo cancellation quality and application field. In particular, it is an object to provide an acoustic echo cancellation concept that is suitable for use in a soundbar.

[0009] This object is achieved by the content of the independent claims.

[0010] Embodiments of the present invention provide an acoustic echo cancellation unit comprising an audio processor configured to receive a multi-channel audio signal and comprising a first stage configured to process the multi-channel audio signal to obtain a first set of spatial audio components and a second stage configured to process the first set of spatial audio components to obtain a second set of spatial audio components, and an echo cancellation processor configured to perform an acoustic echo cancellation by using the first set of spatial audio components or a variant version of the first set of spatial audio components as a reference signal.

[0011] According to other embodiments, the echo cancellation processor comprises a spatial component combiner configured to process the first set of spatial audio components to obtain a variant version of the first set of spatial components to be used as a reference signal.

[0012] Embodiments of the present invention are based on the finding that it is beneficial to apply a two-stage processing to the sound processing, wherein the output signal / spatial component signal of the first stage is preferably used as a reference signal for the acoustic echo cancellation (compared to the second stage). Here, the first spatial component signal can be further processed to obtain the final spatial component and is more suitable for the acoustic echo cancellation than the final spatial component. For example, the transition signal can comprise component signals associated with room directions to be rendered as a 3D sound scene by the soundbar. According to a preferred embodiment, the first spatial component is further processed before being used as a reference signal. Thus, also a variant version of the first set of spatial components can be used. This variant version is obtained by using a spatial component combiner, e.g. by performing a linear processing or another processing.

[0013] In other words, this means that the first spatial components are accessed and processed using the spatial component combiner to obtain the reference signals. The resulting reference signals are used for the AEC. The use of the intermediate signals (i.e. the first spatial components) in combination with the spatial component combiner enables to obtain or make applicable in practice a proper AEC reference signal or AEC.

[0014] According to other embodiments, an acoustic echo cancellation unit is provided, wherein the spatial component combiner is configured to output at least one reference signal or a reference signal comprising one, two or more channels. According to embodiments, the spatial component combiner is configured to perform a linear combination of the first set of spatial components to obtain the reference signal and / or to apply a time-invariant downmix matrix to the first set of spatial components to obtain the reference signal. For example, the spatial component combiner can be configured to perform its processing based on the following formula:

[0015]

[0016] wherein, is the i-th reference signal, is a weight, is the k-th spatial component signal. Here, each reference signal is obtained from a subset of the first set of spatial components. In addition, each spatial component channel can be used at most for one reference signal channel.

[0017] According to embodiments, the spatial component combiner is configured to reduce the number of spatial components to obtain the at least one reference signal.

[0018] With respect to the echo cancellation processor, it should be noted that the echo cancellation is typically based on a microphone signal received via a microphone input. According to other embodiments, an acoustic echo cancellation unit is provided, wherein the echo cancellation processor is configured to perform the echo cancellation using a single adaptive filter or a multi-channel adaptive filter or a single adaptive filter or a multi-channel adaptive filter that can be configured based on a comparison between the reference signals and the microphone signals.

[0019] According to another embodiment, an acoustic echo cancellation unit is provided, wherein the first stage is configured to perform a non-linear processing or a time-varying processing or a highly time-varying processing to obtain the first set of spatial audio components. According to another embodiment, the second stage is configured to output the second set of spatial audio components to a playback device or a soundbar. It is noted that according to other embodiments, an acoustic echo cancellation is provided, wherein the set of spatial audio components enables to directly control one or more transducers of the playback device or to control one or more transducers of the playback device or soundbar by using one or more amplifiers. For example, an acoustic echo cancellation unit is provided, wherein the second stage is configured to perform a linear processing.

[0020] Another embodiment provides a playback device or soundbar comprising an acoustic echo cancellation unit.

[0021] According to another embodiment, a method for acoustic echo cancellation is provided:

[0022] - receiving a multi-channel audio signal;

[0023] - processing the multi-channel audio signal to obtain a first set of spatial audio components;

[0024] - processing the first set of spatial audio components to obtain a second set of spatial audio components;

[0025] - performing echo cancellation by using the first set of spatial audio components or a variant version of the first set of spatial audio components as a reference signal.

[0026] According to other embodiments, the method can be performed by using a computer. BRIEF DESCRIPTION OF DRAWINGS

[0027] Embodiments of the present application will be discussed later on with reference to the disclosed figures, in which:

[0028] Figure 1a a schematic block diagram is shown illustrating an echo cancellation unit according to the basic embodiment;

[0029] Figure 1b a schematic flow diagram is shown illustrating an improved echo cancellation concept according to the basic embodiment;

[0030] Figure 2 a schematic block diagram is shown illustrating a sound cancellation method according to the enhanced method; and

[0031] Figure 3 a schematic block diagram is shown illustrating a sound cancellation method according to the prior art. DETAILED DESCRIPTION

[0032] In the following, embodiments of the present application will be discussed later on with reference to the enclosed figures, in which identical reference signs are provided for objects having the same or similar function, so that their description is interchangeable and mutually applicable.

[0033] Figure 1aA sound cancellation unit 30 is shown which comprises two paths 31 and 41. Within path 31, the multi-channel audio signal MS is processed so as to enable playback of the multi-channel audio signal MS by using a sound reproduction device 14 (e.g. a soundbar). Here, the processing as described in the context of the prior art is subdivided into two stages 32 and 34. The first stage 32 processes the multi-channel audio signal MS so as to obtain a transition signal labeled with reference SC_1. This transition signal SC_1 represents a first spatial component signal set. Then, this transition signal SC_1 is further processed by using the second stage 34 so as to obtain a second spatial component signal set SC_2 which can directly drive the loudspeakers 14. Directly drive means that no further spatial processing is used. I.e. the transducers of the soundbar 14 can be driven by the second spatial component set (e.g. after amplifying the spatial components).

[0034] For example, the first stage 32 performs a non-linear signal processing and / or a (highly) time-variant processing. The second processing stage 34 can mainly perform linear time-invariant processing steps. The background for the subdivision into two stages is that the processing steps performed by the second stage 34 (e.g. linear time-invariant processing steps) can negatively influence the applicability of the echo cancellation.

[0035] The echo cancellation path 41 performs echo cancellation based on the first spatial component set SC_1. To this end, the echo cancellation unit 40 receives the signal SC_1 from the first stage of the audio processing 32. The echo cancellation unit 40 comprises at least an echo cancellation processor 42 which performs echo cancellation based on a reference signal RS. According to an embodiment, the first spatial component set SC_1 can be used as the reference signal RS. According to another (preferred) embodiment, a variant version of the first spatial component set SC_1 can be used as the reference signal RS. Thus, the echo cancellation unit 40 can optionally comprise a processor 44 (e.g. a so-called combiner). This combiner 44 performs a processing (e.g. a linear processing) based on the first spatial component set SC_1 so as to obtain the reference signal RS.

[0036] For the sake of completeness only, it should be noted that the echo cancellation typically performed by the entity 42 uses another input signal received e.g. via one or more microphones (not shown) in addition to the reference signal RS as will be discussed below.

[0037] Before discussing the enhanced embodiment, the concept will be discussed with respect to the method steps of the improved echo cancellation concept.

[0038] Figure 1b Method steps of the echo cancellation method 100 are shown. Within the basic implementation, the acoustic echo cancellation 100 comprises three basic steps 132, 142 and 134.

[0039] Steps 132 and 134 represent the audio processing performed within the first stage. Within the first stage 132, the multi-channel audio signal (see Figure 1a reference signal RS. The second stage 134 is arranged to perform a further processing of the first set of spatial components SC_1 to obtain a second set of spatial components SC_2. The echo cancellation processing is performed in parallel with the second stage processing 132 plus 134. The echo cancellation processing uses the signal SC_1 as input, such that a first step of the echo cancellation is performed after step 132. Here, the basic step of the echo cancellation is marked by reference numeral 142, which performs the echo cancellation based on the signal SC_1 used as a reference signal. As mentioned above, this processing step 142 can use additional signals (e.g. microphone signals) as input signals. Optionally, a further processing step 144 can be arranged before step 142. This step 144 enables the processing of the first set of component signals SC_1 to obtain the reference signal RS.

[0040] In the following, with respect to Figure 2 optional elements of the echo cancellation processing will be discussed.

[0041] Figure 2 The echo cancellation unit 30' is shown to comprise an audio processor 31 having two stages 32 and 34, and the echo cancellation unit 40' is shown to comprise two main stages 42 and 44 (see Figure 1a ) and one or more microphones 24 and a back-end processing 26.

[0042] With respect to the microphones 24 and the loudspeakers 14, it should be noted that they can be combined in one common housing or in multiple common housings.

[0043] With respect to the back-end processing 26, it should be noted that it can be used as a human-machine interface, e.g. by using speech recognition / far-field sound capture, or for applications such as full-duplex communication.

[0044] Previously, the functionality of the entire system 30' will be discussed.

[0045] Due to the problems encountered with the state-of-the-art approach, we suggest to use the mid signal of a soundbar processing instead of the signal driving the loudspeakers as AEC reference signal. Figure 2A general approach is illustrated in Fig. 1. The audio processor 31 receives a multi-channel audio signal 115, e.g. in 5.1 surround format or 7.1 + 4H immersive format. The audio input signal 115 is processed by a first processing block 32 to generate a first set of spatial component signals SC_1. The resulting set of spatial component signals SC_1 is not suitable for direct playback via the loudspeakers 14 of the soundbar (or any other multi-loudspeaker playback system), but serves as a basis for further processing by an audio processing stage 39. The first spatial component signals SC_1 are input into a second audio processing block 34, in which a second set of spatial component signals SC_2 is generated. Typically, these component signals SC_2 are then reproduced by the loudspeakers 14 of the soundbar, i.e. they represent the loudspeaker playback signals.

[0046] To generate suitable reference signals RS for the MC-AEC 42, the first set of spatial component signals SC_1 is further processed by a spatial component combiner 44. Typically, the spatial component combiner 44 determines the AEC reference signals RS by a linear combination of the first set of spatial component signals. In some embodiments, certain AEC reference signals RS can also correspond to one of the spatial component signals SC_1 without any further modification. In typical embodiments, the number of AEC reference signals RS is smaller than the number of spatial component signals SC_1, i.e. the spatial component combiner 44 reduces the number of signals. One advantage is that the configuration and computational complexity of the MC-AEC 42 does not directly depend on the number of loudspeakers 14. If the number of loudspeakers 14 included in the soundbar is significantly larger than the number of AEC reference signals RS, it is particularly important to reduce the computational complexity. Another advantage is that due to the high correlation between different loudspeaker channels, the statistical properties of the loudspeaker drive signals are typically not suitable for direct use as AEC reference signals RS, whereas the AEC reference signals RS derived from the first set of spatial component signals SC_1 typically have properties that are more suitable for the adaptive filtering of the MC-AEC 42.

[0047] According to embodiments, the echo paths to be modeled by the MC-AEC 42 based on the AEC reference signals RS are preferably (only) slowly time-varying and linear. It is therefore important to appropriately distribute different processing steps between the first and second processing blocks 32, 34 of the audio processor 31 / soundbar processing chain 31. For example, any non-linear or highly time-varying processing steps should be applied in the first processor 32, whereas the second processing block should mainly contain linear time-invariant processing steps 34.

[0048] In some embodiments, the first processor generates spatial component signals SC_1 associated with left, right, center, low frequency, top and back of a 3D sound scene rendered with the soundbar 14. An appropriate implementation of the spatial component combiner 44 will generate the AEC reference signals 25 as a linear combination of the first set of spatial components, e.g., by applying a time-invariant downmix matrix to the spatial components SC_1, as shown in Equation 1:

[0049]

[0050] wherein, is the i-th AEC reference signal, is a weight, is the k-th spatial component signal. For example, let left, right, center, low frequency, top and back be denoted by C1, C2, C3, C4, C5 and C6, respectively. If a 2-channel AEC reference signal is desired, the spatial component combiner can combine the components by applying the following weights:

[0051]

[0052] In this case, the 2-channel AEC reference signal can be obtained by:

[0053] R1 = C1 + C2 + C4

[0054] R2 = C3 + C s + C6

[0055] The processing performed by the echo cancellation unit 30' can be described as follows:

[0056] 1) receiving a multi-channel audio signal 115 (at least 2 channels) and at least one microphone signal.

[0057] 2) processing the received multi-channel audio signal 115 with a first audio processor 32 to obtain a first set of spatial component signals SC_1. This processing can include time-varying and / or non-linear processing steps.

[0058] 3) processing the first set of spatial component signals SC_1 with a second audio processor 34 to obtain a second set of spatial component signals SC_2.

[0059] 4) outputting the second set of spatial component signals SC_2 on a playback device 14 (e.g., a soundbar) having multiple loudspeakers.

[0060] 5) combining the first set of spatial component signals SC_1 to obtain a set of AEC reference signals RS.

[0061] This step can be performed by using the processor 44.

[0062] 6) performing echo cancellation based on the AEC reference signal RS and the received microphone signals, e.g. using a multi-channel adaptive filter.

[0063] As shown, this step is performed by entity 42.

[0064] According to another embodiment, an additional step 5a (after step 5, before step 6) can be performed: processing the first set of spatial components SC_1 with a spatial component combiner 44, resulting in a number of AEC reference signals RS smaller than the number of spatial components SC_1.

[0065] Although some aspects have been described in the context of an apparatus, it is clear that separate aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps can be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or electronic circuit. In some embodiments, some one or more of the most important method steps can be executed by such an apparatus.

[0066] The novel encoded audio signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium (e.g. the Internet).

[0067] Depending on certain implementation requirements, embodiments of the application can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium can be computer readable.

[0068] Some embodiments according to the application comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0069] Generally, embodiments of the present application can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code can for example be stored on a machine readable carrier.

[0070] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0071] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0072] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non- transitory.

[0073] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be configured to be transferred via a data communication connection, for example, via the Internet.

[0074] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.

[0075] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0076] A further embodiment according to the application comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0077] In some embodiments, a programmable logic device (for example, a field programmable gate array) can be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0078] Although in the above described embodiments the audio processor 31 has been described as having only the first stage 32 and the second stage 34, it should be noted that the audio processor 31 can have additional stages, for example, an amplification stage at the output of 34, an input stage at the input of 32, and / or a stage between 32 and 34.

[0079] The above examples are merely illustrative of the principles of the application. It will be readily apparent to those skilled in the art that modifications and variations of the arrangements and details described herein can be made without departing from the spirit and scope of the application. Accordingly, it is intended that the scope of the application be limited only by the scope of the claims attached hereto.

Claims

1. An acoustic echo cancellation unit, comprising: an audio processor (31) configured to receive a multi-channel audio signal (MS) and comprising: a first stage (32) configured to process the multi-channel audio signal (MS) to obtain a first set of spatial audio components (SC_1) representing a transition signal, wherein the first stage (32) is configured to perform a non-linear processing or a time-variant processing or a highly time-variant processing to obtain the first set of spatial audio components (SC_1); and a second stage (34) configured to process the first set of spatial audio components (SC_1) to obtain a second set of spatial audio components (SC_2) representing a loudspeaker playback signal; an echo cancellation processor (42) configured to perform an echo cancellation by using the first set of spatial audio components (SC_1) or a variant version of the first set of spatial audio components (SC_1) as a reference signal (RS) and by using at least one received microphone signal, wherein the variant version is obtained by using a spatial component combiner performing a linear processing or another processing.

2. The acoustic echo canceling unit (30, 30') according to claim 1, wherein the echo cancellation processor (42) comprises a spatial component combiner (44) configured to process the first set of spatial audio components (SC_1) to obtain the variant version of the first set of spatial components (SC_1) to be used as the reference signal (RS).

3. The acoustic echo canceling unit (30, 30') according to claim 2, wherein the spatial component combiner (44) is configured to output one reference signal or multiple reference signals (RS) or a reference signal (RS) comprising one, two or more channels.

4. The acoustic echo canceling unit (30, 30') according to claim 2, wherein the spatial component combiner (44) is configured to perform a processing to output one or more linear combinations of the first set of spatial components (SC_1) as the reference signal (RS) and / or to apply a time-invariant downmix matrix to the first set of spatial components (SC_1) to obtain the reference signal (RS).

5. The acoustic echo canceling unit (30, 30') according to claim 2, wherein, the spatial component combiner (44) is configured to perform a processing based on the following equation: wherein is the ith reference signal (RS), is a weight, is the kth spatial component signal.

6. The acoustic echo canceling unit (30, 30') according to claim 1, wherein the reference signal (RS) is a subset of the first set of spatial audio components (SC_1).

7. The acoustic echo canceling unit (30, 30') according to claim 1, wherein each spatial component of the first set of spatial audio components (SC_1) is included at most in one signal of the reference signal (RS).

8. The acoustic echo canceling unit (30, 30') according to claim 2, wherein, the spatial component combiner (44) is configured to output one or more signals of the reference signal (RS) having a reduced number compared to the number of spatial audio components in the first set of spatial audio components (SC_1).

9. The acoustic echo canceling unit (30, 30') according to claim 1, wherein, the echo cancellation processor (42) performs the echo cancellation based at least on a microphone signal received via a microphone (24) input.

10. The acoustic echo canceling unit (30, 30') according to claim 1, wherein, the echo cancellation processor (42) is configured to perform the echo cancellation using a single adaptive filter or a multi-channel adaptive filter or a single adaptive filter or a multi-channel adaptive filter configurable based on a comparison between the reference signal (RS) and a microphone signal.

11. The acoustic echo canceling unit (30, 30') according to claim 1, wherein, The second stage (34) is configured to output the second set of spatial audio components (SC_2) to a playback device (14) or a soundbar.

12. The acoustic echo canceling unit (30, 30') according to claim 11, wherein The set of spatial audio components enables direct control of one or more transducers of the playback device (14) or control of one or more transducers of the playback device (14) or soundbar by using one or more amplifiers.

13. The acoustic echo canceling unit (30, 30') according to claim 1, wherein, The second stage (34) is configured to perform linear processing.

14. A playback device (14) or soundbar comprising the acoustic echo canceling unit (30, 30') according to claim 1.

15. A method for acoustic echo canceling, the method comprising the steps of: receiving a multi-channel audio signal (MS); processing (132) the multi-channel audio signal MS to obtain a first set of spatial audio components (SC_1) representing a transition signal, the first set of spatial audio components (SC_1) being obtained by performing a non-linear processing or a time-varying processing or a highly time-varying processing; processing (134) the first set of spatial audio components (SC_1) to obtain a second set of spatial audio components (SC_2) representing a loudspeaker playback signal; performing echo canceling (142) by using the first set of spatial audio components (SC_1) or a variant version of the first set of spatial audio components (SC_1) as a reference signal (RS) and by using at least one received microphone signal, wherein the variant version is obtained by performing linear processing or another processing using a spatial component combiner.

16. A computer program product having stored thereon instructions for carrying out, when running on a computer, the method according to claim 15.

Images