Separation of dry sound and ambient sound

Abstract

To provide a method for separating a dry sound from an environmental sound.SOLUTION: The present invention relates to a method of separating a speech input signal into a dry signal component and an environmental signal component, and the method includes : generating a conversion input signal which includes converting a speech input signal into a frequency space; applying a smoothing filter to the conversion input signal to generate an estimated environmental signal component; identifying a dry signal component based on the estimated environmental signal component; and identifying the environmental signal component based on the identified dry signal component and the speech input signal.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a method for separating an audio input signal into a dry signal component and an ambient signal component, and a corresponding audio module used to perform the method.Furthermore, a computer program and a carrier containing the computer program are provided. [Background technology]

[0002] Audio signals reproduced from different sources contain a dry signal component and an ambient signal component, which includes an indirect signal component generated when the dry signal component interacts with the room in which the audio signal was recorded. In the audio industry, applications are known that require mainly the dry signal component, or only the ambient signal component. As an example, in a stereo input signal to be output in a 5.1 system, it is desirable for the dry signal component to be output substantially on the front speakers and the ambient signal component to be output on the rear speakers. Another application amplifies the ambient signal component in the audio signal.

[0003] There is therefore a need to be able to effectively identify either the dry signal component or the ambient signal component in an audio input signal that contains both dry and ambient signal components. Summary of the Invention [Means for solving the problem]

[0004] This need is met by the features of the independent claims. Further aspects are set out in the dependent claims.

[0005] According to a first aspect, there is provided a method for separating an audio input signal into a dry signal component and an ambient signal component, in which a transformed input signal is generated, comprising transforming the audio input signal into a frequency space. A smoothing filter is applied to the transformed input signal in the frequency space to generate an estimated ambient signal component. Based on the estimated ambient signal component, a dry signal component is identified, and then based on the identified dry signal component and the audio input signal, the ambient signal component is identified.

[0006] Since the rate of change in amplitude of the audio input signal due to the ambient signal component is slow, a smoothing filter can be used to extract the ambient signal component. Applying the smoothing filter can suppress the dry signal component and generate an estimated ambient signal component. The dry signal component can be identified based on the estimated ambient signal component, and then the ambient signal component can be identified using the identified dry signal component and the audio input signal, for example by subtracting the identified dry signal component from the audio input. It should be understood that although this subtraction may be performed in frequency space, this identification can be done in the time domain.

[0007] Further provided is an audio module configured to separate an audio input signal into a dry signal component and an ambient signal component, the audio module comprising a memory and at least one processing unit, the memory including instructions executable by the at least one processing unit, the audio module then being operable to perform the steps set out above or discussed in more detail below.

[0008] Further, a computer program product is provided comprising program code, which upon execution of the program code causes at least one processing device to perform the methods discussed above or in more detail below.

[0009] A carrier containing the computer program is provided, the carrier being one of an electronic signal, an optical signal, a radio signal, or a computer readable storage medium.

[0010] It is to be understood that the features mentioned above and those to be described below can be used not only in the respective combinations shown, but also in other combinations or alone, without departing from the scope of the present invention. The above and below features mentioned in the aspects and embodiments described below can be combined with each other in other embodiments, unless otherwise specified. For example, the present application provides the following: (Item 1) A method for separating an audio input signal (10) into a dry signal component (11) and an ambient signal component (12), comprising the steps of: generating a transformed input signal, the transformed input signal including transforming the audio input signal into a frequency space; applying a smoothing filter (130) to the transformed input signal to generate an estimated environmental signal component; determining the dry signal component (11) based on the estimated environmental signal component; determining the environmental signal component (12) based on the determined dry signal component (11) and the audio input signal (10); The above method. (Item 2) the audio input signal (10) includes a composite signal period (15) in which the ambient signal component (12) and the dry signal component (11) are present, and a reverb signal period (16) following the composite signal period (15) in which substantially only the ambient signal is present; The above method is detecting said reverb signal period (16) in said audio input signal; identifying an estimated reverb signal component within the estimated ambient signal component for the reverb signal period; comparing the estimated reverberation signal component with the audio input signal during the reverberation signal period to determine a smoothing coefficient to be used in the smoothing filter for generating the estimated ambient signal component; The method according to the above item, comprising: (Item 3) 13. The method of any one of the preceding claims, wherein the smoothing filter (130) comprises a first order recursive filter. (Item 4) determining an amplitude of the converted input signal; attenuating the determined amplitude of the transformed input signal before applying the smoothing filter (130); The method according to any one of the preceding claims, further comprising: (Item 5) A ratio of said environmental signal component (12) to said audio input signal (10) is determined; 13. The method of claim 1, wherein the determined amplitude is attenuated based on the determined ratio. (Item 6) identifying an onset (17) in said audio input signal (10) at which a significant amount of said dry signal component (11) begins to be present in said audio input signal (10); identifying an offset (18) in said audio input signal at which said substantial amount of said dry signal component (11) is no longer present in said audio input signal; The method according to any one of the above items, comprising: (Item 7) identifying an onset (17) in said audio input signal (10) at which a significant amount of said dry signal component (11) begins to be present in said audio input signal (10); identifying an offset (18) in said audio input signal at which said substantial amount of said dry signal component (11) is no longer present in said audio input signal; Including, A period of the reverberation signal (16) is identified, the period beginning after the offset (18) and ending when the output of the smoothing filter meets a predetermined criterion; 13. The method of claim 1, wherein comparing the estimated reverberation signal component with the audio input signal for the reverberation signal period is performed on a portion of the audio input signal after the determined offset (18). (Item 8) identifying an onset (17) in said audio input signal (10) at which a significant amount of said dry signal component (11) begins to be present in said audio input signal (10); identifying an offset (18) in said audio input signal at which said substantial amount of said dry signal component (11) is no longer present in said audio input signal; Including, Determining the ratio of the environmental signal component (12) to the audio input signal (10) comprises: determining an amplitude of the audio input signal (10) at a last sample of the audio input signal before the offset; determining an amplitude of the audio input signal (10) at a first sample of the audio input signal after the offset; determining said ratio taking into account said amplitude at said last sample and said amplitude at said first sample; The method according to any one of the above items, comprising: (Item 9) to determine an average ratio used to attenuate the amplitude of the converted input signal; said step of determining said ratio taking into account said last sample and said first sample is repeated several times for different offsets in said audio input signal; lengths of the dry signal components are determined for the different offsets; 2. The method of claim 1, wherein the longer the length of the identified dry signal component, the greater the contribution of the corresponding ratio to the average ratio. (Item 10) 13. A method according to any one of the preceding claims, wherein it is determined whether the reverberation signal period (16) is currently present in the audio input signal (10), and if so, the audio input of the reverberation signal period is used for the estimated ambient signal component, and if not, the output of the smoothing filter is used for the estimated ambient signal component. (Item 11) extracting a variation from the attenuated amplitude of the transformed input signal; adding the extracted variation to an output of the smoothing filter to identify the estimated environmental signal component; The method according to any one of the preceding claims, further comprising: (Item 12) The method further includes identifying an exponentially decreasing peak subsequent signal from the transformed input signal, the peak subsequent signal being thereafter attenuated by a factor less than one; 13. A method according to any one of the preceding claims, wherein the offset (18) is detected at the point where the audio input signal decreases when the attenuated peak-following signal crosses the input signal. (Item 13) 13. The method of claim 1, wherein the smoothing coefficients are determined such that the estimated reverberation signal components correspond to the audio input signal during the reverberation signal period. (Item 14) 2. The method of claim 1, wherein the estimated ambient signal component generated from the output of the smoothing filter is subtracted from the audio input signal to identify the dry signal component. (Item 15) 1. An audio module (100) configured to separate an audio input signal into a dry signal component and an ambient signal component, said audio module comprising a memory and at least one processor, said memory including instructions executable by said at least one processor, said audio module comprising: generating a transformed input signal, the transformed input signal including transforming the audio input signal into a frequency space; applying a smoothing filter to the transformed input signal to generate an estimated environmental signal component; determining the dry signal component based on the estimated environmental signal component; determining said ambient signal components based on said determined dry signal components and said audio input signal; The above voice module is operable. (Summary) The present invention relates to a method for separating an audio input signal into a dry signal component and an ambient signal component, the method comprising: generating a transformed input signal including transforming the audio input signal into a frequency space; applying a smoothing filter to the transformed input signal to generate an estimated ambient signal component; identifying the dry signal component based on the estimated ambient signal component; and identifying the ambient signal component based on the identified dry signal component and the audio input signal.

[0011] The foregoing and additional features and advantages of the present application will become apparent from the following detailed description, read in conjunction with the accompanying drawings, in which like reference numbers refer to like elements and in which: [Brief description of the drawings]

[0012] [Figure 1] 1 shows a schematic diagram of an audio input signal over time including environmental amplitudes at a single frequency bin; [Diagram 2] 1 shows an overview of a module having the components necessary to separate dry signal components from ambient signal components in an audio input signal. [Diagram 3] FIG. 2 shows a detailed view of the reverberation characteristic estimation performed. [Figure 4] FIG. 3 shows further details of the calculation of the environmental amplitude performed in FIG. [Diagram 5] FIG. 2 shows a schematic diagram of a smoothing recursive filter used to separate an audio input signal into a dry signal component and an ambient signal component. [Figure 6] 2 shows an exemplary flow chart of a method used to process an audio input signal to identify dry and ambient signal components. [Figure 7] 4 shows another exemplary flow chart of a method of processing steps performed on an audio signal to identify dry and ambient signal components; [Figure 8] 2 shows an exemplary flow chart of a method including processing reverb components present in an ambient sound signal component. [Figure 9]4 shows an exemplary flow chart of a method performed to separate an input signal into a dry signal component and an ambient signal component. [Figure 10] 1 illustrates an exemplary structural representation of an audio module configured to separate an audio signal into different components as discussed in connection with FIGS. 1-9. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] In the following, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the following description of the embodiments should not be interpreted in a limiting sense. The scope of the present invention is not intended to be limited to the embodiments or drawings described below, which are merely illustrative.

[0014] The drawings should be considered as schematic representations, and the elements illustrated in the drawings are not necessarily drawn to scale. Rather, the various elements are represented so that their function and general purpose are apparent to a person skilled in the art. Any connection or coupling between components of the functional blocks, devices, physical or functional units shown in the drawings and described below may also be implemented by indirect connections or couplings. Coupling between components may be established by wired or wireless connections. The functional blocks may be implemented in hardware, software, firmware, or a combination thereof.

[0015] FIG. 1 shows a schematic diagram of an audio input signal 10 transformed into frequency space and separated into different bins, where the signal of one frequency bin is shown over time, including the input amplitude 10, including the ambient amplitude 12. There is an onset 17 when the sound starts to be present in the input signal, and an offset 18 when the sound stops. As can be seen, in the period between the onset and offset, the input signal includes a dry signal component, which is not known and is not shown, and an ambient signal component 12. This composite signal period 15 is followed by a reverb signal period 16, where mainly the ambient signal is present. A portion of the reverb signal period is also called the reverb tail, and in most rooms this reverb tail has an exponentially decreasing characteristic. Similarly, when acoustic energy is continuously fed into the room, the reverb amplitude increases exponentially, as shown in FIG. 1. When a sound is played by an instrument or through a speaker, the direct sound and the ambient sound overlap until the sound stops at the offset 18, and the ambient signal component becomes evident in the reverb portion. In a recording that contains both dry and ambient signal components, the ambient component can be extracted by limiting the rate of amplitude change for all frequency bins, for example using a first order recursive filter, to obtain the exponential rise and fall due to the acoustics in a real room.

[0016] FIG. 2 shows a schematic diagram of the main steps performed during signal processing of an audio input signal 10 in order to separate the ambient signal components from the dry signal components.

[0017] As shown in Fig. 2, the audio input signal 10 is processed in several processing modules shown in Fig. 2, the processing being carried out separately for each channel of the input signal. Thus, the proposed method accommodates any number of audio channels, including mono. After Fourier transforming the audio input signal into the frequency domain using a short-time Fourier transform (STFT) in module 50, the amplitude is calculated for each time frame in module 60 and smoothed in module 100 based on parameters obtained from the analysis of the input signal estimated in estimation module 200. For this purpose, overlapping windows of the time signal may be used. In module 100, as will be explained in more detail below, a smoothing filter is applied to the signal, which suppresses the dry signal components, so that the output represents the ambience part of the signal. By subtracting the ambience part from the input signal in adder 75, the estimated dry amplitude is present at the output of adder 75. In module 70, an amplitude adjustment is applied to the signal such that the amplitude of the dry signal content is extracted and applied to the input signal in the frequency domain, which is then transformed back to the time domain in module 80. Thus, at the output of module 80, there are N dry output signal components. Once the dry component is known, the ambience component can be generated by subtracting the dry signal component from the input signal to obtain N ambience output channels 12 as shown in FIG.

[0018] In the following, the signal flow of the different modules is explained in more detail. In module 50, a short-time Fourier transform (STFT) is performed, converting the input signal from the time domain to the frequency domain. For computational efficiency, a fast Fourier transform can be used, but other types of transforms can also be applied, such as other implementations of the digital Fourier transform, the cosine transform of the wavelet transform, etc. The time-domain input can be divided into blocks of samples of length N / 2. Windowing, optional zero padding, and the actual fast Fourier transform operation are performed. The output of the transform of one of the N input samples is a block of N complex spectral bins in real / imaginary representation, if no zero padding is applied.

[0019] In module 60, the amplitudes are calculated for each time-frequency bin from the complex input spectrum in real and imaginary representation.

[0020] In module 200, some parameters used in module 100 to estimate the environmental signal components are determined. As shown in Fig. 3, the input amplitudes output from module 60 are used for onset detection module 210 and offset detection module 220. In offset detection module 220, a peak trailing curve with an exponential decrease is calculated for each frequency based on the input amplitudes. The peak trailing curve is adjusted by an adjustable factor a, such as 0.4. pf The decay time of the peak trailing curve corresponds to the current estimated reverb time or an adjustable coefficient, or a combination of both, as shown in Figure 1. An offset is detected when the input amplitude crosses the decayed peak trailing curve downwards, as shown in Figure 1.

[0021] In the onset detection module 210, an onset is observed and detected when the smoothed actual input amplitude increases steeply above an adjustable threshold, e.g., when the ratio between two consecutive samples exceeds a certain limit. The threshold may be automatically adjusted based on the current estimated rate of decline. The threshold may also be frequency dependent. The threshold must be greater than 1.

[0022] In module 230, sound and reverb tail detection is performed. The reverb tail, i.e. the reverb signal period 16, starts after the offset and continues until the smoothed amplitude starts to increase again. This is the case when the new value of the smoothed amplitude is greater than the previous value. The input amplitude can also be used to identify the end of the reverb signal period 16, but this is more difficult, since the amplitude of the reverb signal period 16 does not have a clear exponential decrease but has some fluctuations, which make it more difficult to detect the end of the reverb signal period. Between the onset 17 and the offset 18, the sound is detected. Based on the nature of the observation method, several onsets can be detected without an offset in between. In this case, neither a sound period nor a reverb signal period is detected. In the following, the sound period is also referred to as the composite signal period in which the ambient signal component and the dry signal component are present. Using several STFT blocks between the latest onset and the following onset, the duration of the sound is measured.

[0023] In module 240, the smoothing coefficients that are subsequently used in the smoothing filter included in module 100 are updated. For this purpose, the deviation between the estimated amplitude output from the smoothing filter in module 100 and the audio input amplitude is evaluated separately for each frequency bin during the reverberation signal period 16. Once the offset detection 220 indicates the beginning of a reverberation signal period, i.e. the reverb tail, the progression of both the estimated reverb tail and the smoothed actual reverb tail are recorded in a buffer. This is done for all transformed signal blocks until one of the following criteria is met: - The reverb tail, i.e. the end of the reverb signal period, is detected. - Both the estimated reverb tail and the smoothed actual reverb tail are reduced by more than a defined threshold, for example 10 dB, compared to the first bin after offset detection. A 10 dB reduction may be used to identify the initial decay time of the room's reverb, and although 10 dB may be a preferred value, other values ​​may be used. - The maximum number of samples is reached, which means the buffer is full.

[0024] If at least one of the above criteria is met, a correction factor is applied to the estimated ambient signal component so that the first sample of the reverb signal period has the same amplitude as the smoothed real reverb tail. The two reverb tails are then compared, i.e., the estimated reverb signal component calculated by the smoothing filter in module 100 and the audio input signal are compared during the reverb signal period by calculating the difference for each time frame in the current frequency bin. The average of the difference can be multiplied by an adjustable scaling factor and smoothed over time to avoid sudden changes. The current recursion coefficients can then be updated by adding the resulting value as a correction value to the current recursion coefficients.

[0025] Thus, the output of module 240 is an amplitude smoothing factor α, which is discussed in more detail in connection with FIG.

[0026] In module 250, the environment-to-input ratio is determined and updated. After a particularly long sound, the environment-to-input ratio can be estimated by looking at the amplitude decrease at offset 18 of the sound. After the dry signal stops, only the environment signal component is present. The difference between the last sample before offset 18 and the sample when the offset is detected can be used to estimate the environment-to-input ratio. The longer the sound is played, the more accurate this estimate is, since the environmental energy in the room grows exponentially and approaches saturation. This is taken into account by recording the length of the last sound and weighting the contribution of the latest measurement to the determination of the average environment-to-input ratio according to the sound length. The weighting of the contribution can be done by applying a recursive filter to the environment-to-input ratio estimation, with recursion coefficients that depend on the sound length. The longer the sound, the more influence the latest measurement has, and vice versa. Furthermore, to avoid updating the estimate based on very short sounds, a minimum required sound length can be used to update the environment-to-input ratio estimate.

[0027] Thus, the output of module 250 is the average environment-to-input ratio and the output of module 230 is information about when the reverb signal period 16 begins.

[0028] As will be explained in more detail in relation to FIG. 4, these three pieces of information are used in the calculation 100 of the environmental amplitude, which will be discussed in more detail in relation to FIG.

[0029] As shown in Fig. 4, a recursive filter 130 is used for smoothing. However, the original input amplitude is attenuated by the environment-to-input ratio estimated as described above in connection with Fig. 3 before being fed into the smoothing filter 130. Alternatively, for long sounds, the output of the smoothing filter converges to the original input amplitude. However, in a real room, the environment energy is usually much lower than the input energy, since it is absorbed by the walls.

[0030] However, the environmental content in a real room does not show a clear exponential rise or fall, but has small fluctuations such as noise due to random phase superposition. Therefore, in module 120, the small fluctuations are extracted from the damped amplitude and added after filter smoothing 130 to create a more natural sound. The fluctuations can be extracted, for example, by smoothing the amplitude with a moving average filter or another recursive filter with a short time constant and subtracting the filtered signal from the input signal. During the detected reverberation signal period 16, the original amplitude can be used to estimate the environmental amplitude, or the smoothed damped amplitude plus the fluctuations, which is particularly relevant for periods where sound is present.

[0031] An important aspect of the separation is the smoothing by the recursive filter 130. This smoothing is applied to all amplitudes separately. This limits the rise and fall times according to the smoothed latest estimate of the reverb characteristic estimation stage, producing amplitudes that rise and fall with an exponential shape and individual time constants according to the actual reverb in the room. The output of the recursive filter 130 is therefore a signal similar to signal 12 shown in FIG. 1.

[0032] A first order recursive low pass filter with a feedback or smoothing factor α between 0 and 1 can be used as shown in Figure 5, and a forward factor 1-α can be used to provide exponential rise and fall and can be implemented with low processing cost. A diagram of a recursive filter, also called a leaky integrator, is shown in Figure 5.

[0033] The corresponding formula is: y(n)=(1-α)·x(n)+α·y(n-1)

[0034] Although a leaky integrator is a preferred and efficient way to perform the amplitude smoothing operation, other types of filters capable of smoothing the amplitude in a controlled manner can also be used.

[0035] Finally, the estimated environment amplitude is constrained in module 150 so that it can never be larger than the original input amplitude. This is a safety measure since the environment content in the signal cannot be larger than the input.

[0036] Returning again to Figure 2, the output of module 100 is the estimated environment amplitude, as described above in relation to Figure 4. Once the environment amplitude has been subtracted from the input amplitude in summer 75, there is a dry amplitude after summer 75.

[0037] The phase value of the input signal in the frequency domain can be combined with the output of the amplitude shaping block. However, it is computationally more efficient to multiply the real and imaginary parts of the complex input spectrum in real and imaginary representation by the ratio of the dry amplitude to the original input amplitude in module 70. Finally, in module 180, an inverse short-time Fourier transform is performed to convert the signal back to the time domain. An inverse Fourier transform IFFT may also be used, but other transformation implementations can be used. The N complex frequency spectrum bins in real and imaginary representation are converted to the time domain. This module 80 can also include the overlap-add functionality necessary to correctly reconstruct the time domain signal. As disclosed in more detail in US 2013 / 0208895 A1, the overlap-add method of digital filtering can include using a series of overlapping Hanning windowed segments of the input waveform and filtering each segment separately in the frequency domain. After filtering, the segments can be recombined by combining the overlapping portions together. The overlap-add method can make it possible to perform frequency domain filtering in real time on continuous signals without excessive memory requirements.

[0038] Figure 6 shows an overview and more detailed signal flow diagram from the input channel 10 on the left side of Figure 2 to only the dry output channel 11 and N ambient output channels 12 on the right side of Figure 2. As described above in relation to Figure 2, the signal is transformed into frequency space in step S61 and the amplitude is calculated in step S62. The calculated amplitude is then used for offset detection in step S63 and onset detection in step S64. Once the offset and onset are known, it is possible to separate the audio input signal into composite signal periods 15 where the sound is present and reverberation signal periods 16 where the ambient signal is mainly present. In the case of the composite signal periods 15, the duration of the sound is determined in step S66 and the ambient to input ratio estimate is updated in step S67 as described above in relation to module 250. In step S68, a smoothing of the ambient to input ratio is performed, so that the output after step S68 is the ambient to input ratio used to attenuate the input amplitude in step S70. In the case of a reverb signal period, a decay rate estimation is performed in step S71, whereby a decay rate is obtained after this step. In step S72, a smoothing of the decay rate is performed, which provides a recursion coefficient α used in the smoothing filter. In response, a first-order recursive filtering is performed in step S73, using the recursion or smoothing coefficient α. Furthermore, as previously described in relation to FIG. 4, the fluctuations are extracted and added to the smoothed signal. In step S75, the original input amplitude or the smoothed amplitude is selected from the input signal, depending on the fact that the audio signal is in a composite signal period or in a reverb signal period 16. As explained in relation to FIG. 7, in step S75, if the signal is in a reverb signal period, the original input amplitude is used, whereas if the signal is in a composite signal period, the smoothed amplitude of the signal is used. Based on this information, it is possible to calculate the dry amplitude by subtracting the estimated ambient signal component output from the module 100 from the input signal (S76). As described above in relation to FIG. 2, after amplitude adjustment has been performed in step S77 and transformation back to time space, a dry signal component is present in the signal and by subtracting the dry signal component from the input signal, the ambient signal component can be identified.

[0039] Fig. 7 shows a flow chart including only the actual audio processing steps. After the start of the method, the signal is transformed to the frequency domain in step S81 and the input amplitude is calculated in step S82, for example in module 60 discussed in relation to Fig. 2. As discussed in relation to Fig. 4, the signal is attenuated by the ambience-to-input ratio in step S83 and small fluctuations are identified in step S84. As discussed in relation to Fig. 4, smoothing with a recursive filter is performed in step S85 and small fluctuations are added in step S86. Then, in step S87, it is checked whether the current signal is in the reverb signal period 16 or in the composite signal period 15. If it is in the composite signal period 15 where the dry signal and the ambience signal are present, the smoothed amplitude is used in step S88. However, if the signal is in the reverb tail, the original input signal is used in step S89. Thus, after steps S88 and S89, an estimated ambience sound signal exists and it is possible to update the estimates of the reverb parameters such as the ambience-to-input ratio and the amplitude smoothing factor in step S90. In step S91 the dry amplitude is calculated and in step S92 the amplitude change is applied, corresponding to the adjustments performed in module 70 discussed in relation to FIG.

[0040] After the inverse Fourier transformation in step S93, in addition to the dry output signal, an ambient signal can be calculated in adder 85, as discussed in relation to FIG.

[0041] FIG. 8 shows a more detailed flow chart of the processing of reverb parameters. In step S101, offset detection is performed in module 220 as described above in relation to FIG. 3. Furthermore, in step S102, onset detection is performed. In step S103, it is checked whether an offset can be detected. If an offset is detected, it can be determined in step S104 that the actual signal is within the reverb signal period, and therefore, in step S105, signal buffering should be performed. In order to store the estimated reverb tail and the actual reverb tail, the buffer is reset in step S106, and the amplitude is saved in the buffer. In step S108, the environment-to-input ratio is updated.

[0042] Returning to the determination in step S103, if no offset is detected, it is checked in step S109 whether buffering has been performed. If buffering has been performed, it is checked in step S110 whether the smoothed amplitude has increased. If the smoothed amplitude has increased, it may be determined in step S111 that the signal is not in the reverb tail, and therefore it may be determined in step S112 that there is no need to perform buffering. In step S113, the amplitude is updated accordingly. Returning again to step S110, if it is determined that the amplitude has not increased, it is determined in step S114 whether the buffer is full or the amplitude has decreased by more than 10 dB. If the buffer is not full and the amplitude has not decreased by more than 10 dB, it may be inferred that the signal is still in the reverb tail, so the current original amplitude and the smoothed amplitude may be saved in the buffer in step S115. As discussed in connection with FIG. 2 and FIG. 3, the two reverb tails are then compared by calculating the difference between the actual reverb tail and the estimated reverb tail for each time frame.

[0043] Returning to step S109, if buffering is not occurring, it is determined whether an onset has been detected in the audio input signal in step S116. If an onset has been detected, the note length is reset in step S117, and if an onset has not been detected, the note length is incremented in step S118.

[0044] Figure 9 summarises some of the main steps performed on the audio input signal. In step S121 a transformed input signal is generated from the audio input signal. In step S122 a smoothing filter is applied to generate an estimated ambient signal component, which is the component after module 100 shown in Figure 2. Then in step S123 it is possible to determine the dry signal component, for example by subtracting the estimated ambient component from the input signal. Once the dry signal component is known, in step S124 the actual ambient signal component can be determined, for example by subtracting the dry signal component from the input signal with adder 85 shown in Figure 2.

[0045] Figure 10 shows a schematic block diagram of an audio module capable of carrying out the processing steps described above in relation to Figures 2 to 8. The audio module is capable of carrying out the aforementioned separation and comprises an input / output module 110 receiving N input channels and outputting N dry output channels 11 and N ambient output channels 12. The module 100 comprises a processing unit 120 comprising one or more processors and capable of executing instructions stored in a memory 130, which may comprise a read only memory, a random access memory, a mass storage device, a hard disk or the like. The memory may further comprise suitable program code executed by the processing unit 120 to carry out the aforementioned functions of separating an audio input signal into a dry signal component and an ambient signal component.

[0046] From the above, some general conclusions can be drawn: The audio input signal may include a composite signal period 15 including both components as shown in Figure 2, and a reverb signal period 16 following the composite signal period 15. It is possible to detect the reverb signal period 16 in the audio input signal. An estimated reverb signal component is then identified within the estimated ambient signal component of the reverb signal period output from the filter, and the estimated reverb signal component is compared with the audio input signal in the reverb signal period to determine a smoothing coefficient to be used in the smoothing filter for generating the estimated ambient signal component.

[0047] As mentioned above, the filter output corresponding to the estimated environmental signal component is compared to the input signal of period 16 and based on this comparison, a smoothing factor can be determined. This smoothing factor can be between 0 and 1 and as can be inferred from the above equation for the smoothing filter, a high smoothing factor α means that more smoothing is applied. As shown in Figure 5, in a recursive filter a high α means that the high part of the old signal is used and only the low part of the new signal is used.

[0048] The smoothing filter may be a first order recursive filter, although other filters may be used.

[0049] Furthermore, the amplitude of the transformed input signal is determined, and the determined amplitude is attenuated before applying the smoothing filter. The attenuation is performed because the estimated environment signal component determined using the smoothing filter can never be larger than the input signal. The attenuation can be performed using an estimate of the environment-to-input ratio.

[0050] Furthermore, it is possible to identify an onset 17 in the audio signal where a significant amount of dry signal content begins to be present in the audio input signal, in other words where there is sound in the audio input signal. Furthermore, it is possible to identify an offset 18 where a significant amount of dry signal content no longer exists in the audio input signal.

[0051] A reverberation signal period may then be identified that begins after the offset 18 and ends when the output of the smoothing filter meets a predetermined criterion, such as when the output of the smoothing filter begins to increase again. A comparison of the estimated reverberation signal components to the audio input signal for the reverberation signal period is then performed in the portion of the audio signal after the identified offset, i.e., the reverberation signal period.

[0052] Further, an amplitude of the audio input signal at the last sample of the audio input before the offset and an amplitude of the audio input signal at the first sample after the offset can be determined. A ratio of the ambient signal component to the audio input signal is then determined based on the amplitude of the first sample and the amplitude of the last sample.

[0053] To determine the average ratio used to attenuate the amplitude of the transformed input signal, this ratio determination based on the last and first samples can be repeated several times for different offsets in the input signal. Furthermore, the length of the dry signal component is determined from different offsets, and the longer the determined dry signal component, the more the corresponding ratio contributes to the average ratio.

[0054] As mentioned above, the longer the note is played, the more accurate the determination of the environment-to-input ratio will be.

[0055] Further, it may be determined whether a reverberation signal period is currently present in the audio input signal, and if so, the audio input of the reverberation signal period is used for the estimated ambient signal component, and if not, the output of a smoothing filter is used for the estimated ambient signal component.

[0056] Furthermore, to identify the estimated environmental signal component, it is possible to extract the variance from the attenuated amplitude of the transformed input signal and add the extracted variance to the output of the smoothing filter.

[0057] Furthermore, from the input signal, an exponentially decreasing peak-following signal can be identified, which is then attenuated by a factor between 0 and 1, and an offset is detected at the point where the input signal decreases when the attenuated peak-following signal intersects with the audio input signal.

[0058] A smoothing factor, detected as discussed in connection with FIG. 3, may be determined such that the estimated reverb signal component corresponds to the audio input signal in the reverb tail.

[0059] The estimated environmental signal components may be identified based on the output after the smoothing filter, i.e. the dry output channel is provided after subtraction performed in summer 75, which is then subtracted from the input signal to identify the environmental signal components present in the audio input signal.

[0060] In summary, using the above method, effective separation of dry and ambient signal components in an audio input signal is possible.

Claims

1. A method for separating an audio input signal (10) into a dry signal component (11) and an ambient signal component (12), the method comprising the steps of: generating a transformed input signal, the transformed input signal including transforming the audio input signal into a frequency space; applying a smoothing filter (130) to the transformed input signal to generate an estimated ambient signal component; determining the dry signal component (11) based on the estimated ambient signal component, wherein determining the dry signal component (11) comprises subtracting the estimated ambient signal component from the transformed input signal in frequency space and then transforming a result of the subtraction into time space to obtain the determined dry signal component (11); determining said ambient signal components (12) in time space by subtracting said determined dry signal components (11) from said audio input signal (10); Including, 2. The method of claim 1, wherein applying the smoothing filter (130) to the transformed input signal to generate the estimated environmental signal component comprises suppressing the dry signal component (11) of the audio input signal (10) using the smoothing filter (130) to generate the estimated environmental signal component.

2. the audio input signal (10) comprises a composite signal period (15) in which the ambient signal component (12) and the dry signal component (11) are present, and a reverberation signal period (16) following the composite signal period (15) in which substantially only the ambient signal is present; The method comprises: detecting said reverb signal periods (16) in said audio input signal; identifying an estimated reverb signal component within the estimated ambient signal component for the reverb signal period; comparing the estimated reverberation signal component with the audio input signal during the reverberation signal period to determine smoothing coefficients to be used in the smoothing filter for generating the estimated ambient signal component; The method of claim 1 , comprising:

3. The method of claim 1 , wherein the smoothing filter (130) comprises a first order recursive filter.

4. determining an amplitude of the converted input signal; attenuating the determined amplitude of the transformed input signal before applying the smoothing filter (130); The method of claim 1 further comprising:

5. A ratio of the environmental signal component (12) to the audio input signal (10) is determined; The method of claim 4 , wherein the determined amplitude is attenuated based on the determined ratio.

6. identifying an onset (17) in the audio input signal (10) at which a significant amount of the dry signal component (11) begins to be present in the audio input signal (10); identifying an offset (18) in the audio input signal at which said substantial amount of said dry signal component (11) is no longer present in the audio input signal; The method of claim 1 , comprising:

7. identifying an onset (17) in the audio input signal (10) at which a significant amount of the dry signal component (11) begins to be present in the audio input signal (10); identifying an offset (18) in the audio input signal at which said substantial amount of said dry signal component (11) is no longer present in the audio input signal; Including, A reverberation signal period (16) is identified, the period beginning after the offset (18) and ending when the output of the smoothing filter meets a predetermined criterion; The method of claim 2 , wherein comparing the estimated reverberation signal component to the audio input signal for the reverberation signal period is performed on a portion of the audio input signal after the identified offset (18).

8. identifying an onset (17) in the audio input signal (10) at which a significant amount of the dry signal component (11) begins to be present in the audio input signal (10); identifying an offset (18) in the audio input signal at which said substantial amount of said dry signal component (11) is no longer present in the audio input signal; Including, Determining the ratio of the environmental signal component (12) to the audio input signal (10) comprises: determining an amplitude of the audio input signal (10) at a last sample of the audio input signal before the offset; determining an amplitude of the audio input signal (10) at a first sample of the audio input signal after the offset; determining said ratio taking into account said amplitude at said last sample and said amplitude at said first sample; The method of claim 5 , comprising:

9. to determine an average ratio used to attenuate the amplitude of the converted input signal; said step of determining said ratio taking into account said last sample and said first sample is repeated several times for different offsets in said audio input signal; lengths of the dry signal components are determined for the different offsets; The method of claim 8 , wherein the longer the length of the identified dry signal component, the greater the contribution of the corresponding ratio to the average ratio.

10. 8. The method of claim 7, further comprising determining whether the reverberation signal period (16) is currently present in the audio input signal (10), and if so, using the audio input signal for the reverberation signal period for the estimated ambient signal component, and if not, using the output of the smoothing filter for the estimated ambient signal component.

11. extracting a variation from the attenuated amplitude of the transformed input signal; adding the extracted variation to an output of the smoothing filter to identify the estimated environmental signal component; The method of claim 4 further comprising:

12. The method further comprises identifying an exponentially decreasing peak subsequent signal from the transformed input signal, the peak subsequent signal being thereafter attenuated by a factor less than one; 7. The method of claim 6, wherein the offset (18) is detected at a point where the input signal decreases when the attenuated peak-following signal and the audio input signal intersect.

13. The method of claim 2 , wherein the smoothing coefficients are determined such that the estimated reverberation signal components correspond to the audio input signal during the reverberation signal.

14. 1. An audio module (100) configured to separate an audio input signal into a dry signal component and an ambient signal component, the audio module comprising a memory and at least one processing unit, the memory including instructions executable by the at least one processing unit, the audio module comprising: generating a transformed input signal, the transformed input signal including transforming the audio input signal into a frequency space; applying a smoothing filter to the transformed input signal to generate an estimated environmental signal component; determining the dry signal component based on the estimated ambient signal component, wherein the estimated ambient signal component is subtracted from the transformed input signal in frequency space to determine the dry signal component, and then a result of the subtraction is transformed into time space to obtain the determined dry signal component; identifying the ambient signal components in time-space by subtracting the identified dry signal components from the audio input signal; is operable, an audio module, wherein applying the smoothing filter to the transformed input signal to generate the estimated environmental signal component comprises suppressing the dry signal component of the audio input signal with the smoothing filter to generate the estimated environmental signal component.

Images