Time-reversed audio subframe error concealment

Abstract

A method and decoder device for generating hidden audio subframes of an audio signal are provided. The method includes generating a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following characteristics: an applied window shape of a first subframe in the consecutive subframes is a mirrored version or a time-reversed version of a second subframe in the consecutive subframes. For the hidden subframes, peaks of a signal spectrum of a previously received audio signal are detected, and a phase of each peak is estimated. A time-reversed phase adjustment is derived based on the estimated phase, and the time-reversed phase adjustment is applied to the peaks of the signal spectrum to form time-reversed phase-adjusted peaks.

Description

Technical Field

[0001] The present disclosure relates generally to communications, and more particularly to methods and apparatus for controlling packet loss concealment for mono, stereo, or multi-channel audio encoding and decoding. Background Art

[0002] Modern telecommunication services usually provide reliable connections between end users. However, such services still need to handle changing channel conditions, under which occasional data packets may be lost due to, for example, network congestion or poor cell coverage. In order to overcome the problem of transmission errors and lost packets, telecommunication services can use packet loss concealment techniques (PLC). In the case where data packets are lost due to poor connections, network congestion, etc., the missing information of the lost packets on the receiver side can be replaced by a synthetic signal in the decoder. PLC technology can usually be closely associated with the decoder, where internal states can be used to generate signal continuation or extrapolation to compensate for packet loss. For multi-mode codecs with multiple operating modes for different signal types, there are usually multiple PLC technologies to handle concealment. Many different terms are used for packet loss concealment techniques, including frame error concealment (FEC), frame loss concealment (FLC), and error concealment unit (ECU).

[0003] For speech coding modes based on linear prediction (LP), PLC can adjust the glottal pulse position based on the use of estimated frame-end pitch information and a replica of the pitch period of the previous frame [1]. The gain of the long-term predictor (LTP) converges to zero, and the convergence speed depends on the number of consecutive lost frames and the stability of the last good (i.e., error-free) frame [2]. Frequency domain (FD) based coding modes are designed to handle general or complex signals, such as music. Different techniques can be used depending on the characteristics of the last received frame. Such analysis can include the number of detected tonal components and the periodicity of the signal. If frame loss occurs during a highly periodic signal (such as active speech or a single instrumental), time domain PLC (similar to LP-based PLC) may be appropriate. In this case, FD PLC can emulate an LP decoder by estimating the LP parameters and the excitation signal based on the last received frame [2]. If frame loss occurs during a non-periodic or noise-like signal, the last received frame can be repeated in the spectral domain, where the coefficients are multiplied with a random sign signal to reduce the metallic sound of the repeated signal. For stationary tonal signals, it has been found to be advantageous to use a method based on prediction and extrapolation of detected tonal components. More details on the above techniques can be found in [1][2][3].

[0004] A common error concealment method operating in the frequency domain is the Phase ECU (Error Concealment Unit) [4]. The Phase ECU is a standalone tool that operates on a buffer of previously decoded and reconstructed time domain signals. The framework of the Phase ECU is based on the sinusoidal analysis and synthesis paradigm. In this approach, the sinusoidal components of the last good frame can be extracted and phase shifted. When a frame is lost, the sinusoidal frequencies are obtained from the past decoded synthesis in the DFT (Discrete Fourier Transform) domain. First, the corresponding frequency bins are identified by finding the peaks in the amplitude spectrum plane. Then, the fractional frequency of the peak is estimated using the peak frequency bins. The frequency bins corresponding to the peak and the adjacent peaks are phase shifted using the fractional frequency. For the rest of the frame, the amplitude of the past synthesis is retained, while the phase is randomized. Burst errors are also handled so that the estimated signal is smoothly muted by converging the estimated signal to zero. More details about the Phase ECU can be found in [4].

[0005] The concept of phase ECU can be used for decoders operating in the frequency domain. This concept includes encoding and decoding systems that perform decoding in the frequency domain, such as Figure 1 As shown, it also includes a decoder that performs time domain decoding and additional frequency domain processing, such as Figure 2 As shown. Figure 1 In the present invention, a time domain input audio signal (sub)frame is windowed 100 and transformed to the frequency domain by means of a DFT 101. An encoder 102 performs the coding in the frequency domain and provides the coding parameters for transmission 103. A decoder 104 decodes the received frame or applies a PLC 109 in case of frame loss. In the construction of concealed frames, the PLC can use a memory 108 of previously decoded frames. The decoded or concealed frame is transformed to the time domain by means of an inverse DFT 110 and then the output audio signal is reconstructed by means of an overlap-add operation 111. Figure 2 An encoder and decoder pair is shown, where the decoder applies a DFT transform to facilitate frequency domain processing. The received and decoded time domain signal is first windowed 105 per (sub)frame and then transformed to the frequency domain by DFT 106 for frequency domain processing 107, which can be done before PLC 109 or after PLC 109 (in the case of frame loss).

[0006] Since the frequency domain spectrum has been generated for each frame, the raw material for the phase ECU can be easily obtained by simply storing the last decoded spectrum in the memory. However, if the decoded spectrum corresponds to a frame of the time domain signal with a different windowing function (see Figure 1), the efficiency of the algorithm may be reduced. This may happen when the decoder divides the synthesized frame into shorter subframes, for example to process transient sounds that require higher temporal resolution. In order to obtain good results, the ECU should generate the desired window shape for each frame, otherwise there may be transition artifacts at each frame boundary. One solution is to store the spectrum of each frame corresponding to a specific window and apply the ECU on them individually. Another solution may be to store a single spectrum for the ECU and correct the windowing in the time domain. This can be achieved by applying an inversion window and then reapplying a window with the desired shape. These solutions have some disadvantages discussed below.

[0007] One disadvantage of applying frequency domain ECU for individual subframes is that there may be differences between subframes that will be replicated for each subframe during a lost frame. For consecutive frame losses, this may lead to repetition artifacts, as each subframe may have slightly different spectral characteristics. Another issue is the increased memory requirements, as the spectrum of each subframe needs to be stored.

[0008] The window rescaling solution, where the windowing is inverted and reapplied, overcomes the problem of different spectral characteristics, since the ECU can be based on a single subframe. However, applying the inverted window and applying the new window involves division and multiplication for each sample, where division is a computationally complex operation and is computationally expensive. This solution can be improved by storing the pre-computed rescaling windows in memory, but this will increase the required table memory. If the ECU is applied for a sub-portion of the spectrum, the full spectrum may also need to be rescaled, since the full spectrum needs to have the same window shape. Summary of the invention

[0009] According to a first aspect, a method for generating hidden audio subframes of an audio signal in a decoding device is provided. The method comprises: generating a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following characteristics: the applied window shape of a first subframe in the consecutive subframes is a mirrored version or a time-reversed version of a second subframe in the consecutive subframes. The method also comprises: detecting peaks of a signal spectrum of a previously received audio signal on a fractional frequency scale; estimating the phase of each of the peaks; and deriving a time-reversed phase adjustment to be applied to the peaks of the signal spectrum to form time-reversed phase-adjusted peaks based on the estimated phases. The method also comprises: applying time reversal to the hidden audio subframes.

[0010] A potential advantage provided is the generation of multi-subframe ECUs from a single subframe spectrum by applying inverted time synthesis. This generation can be suitable for cases where the subframe windows are time-reversed versions of each other. Generating all ECU frames from a single stored decoded frame ensures that the subframes have similar spectral characteristics while keeping memory usage and computational complexity to a minimum.

[0011] According to a second aspect, there is provided a decoder device configured to generate hidden audio subframes of an audio signal. The decoder device is configured to generate a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following characteristics: the applied window shape of a first subframe in the consecutive subframes is a mirrored version or a time-reversed version of a second subframe in the consecutive subframes. The decoder device is also configured to detect peaks of a signal spectrum of a previously received audio signal on a fractional frequency scale; and estimate the phase of each of the peaks. The decoder device is also configured to derive a time-reversed phase adjustment to be applied to the peaks of the signal spectrum based on the estimated phase; and form a time-reversed phase-adjusted peak by applying the time-reversed phase adjustment to the peaks of the signal spectrum. The decoder device is also configured to apply time reversal to the hidden audio subframe.

[0012] According to a third aspect, there is provided a computer program. The computer program comprises program code to be executed by processing circuitry of a decoder device configured to operate in a communications network, whereby execution of the program code causes the decoder device to perform operations according to the first aspect.

[0013] According to a fourth aspect, there is provided a computer program product. The computer program product comprises a non-transitory storage medium storing program code to be executed by processing circuitry of a decoder device configured to operate in a communication network, whereby execution of the program code causes the decoder device to perform operations according to the first aspect.

[0014] According to a fifth aspect, a method for generating hidden audio subframes of an audio signal in a decoding device is provided. The method comprises: generating a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following characteristics: the applied window shape of the first subframe in the consecutive subframes is a mirrored version or a time-reversed version of the second subframe in the consecutive subframes. A signal spectrum corresponding to the second subframe in the first two consecutive subframes is stored. The method further comprises: receiving a bad frame indicator for the second two consecutive subframes. The method further comprises: obtaining the signal spectrum; detecting peaks of the signal spectrum on a fractional frequency scale; estimating the phase of each of the peaks; and deriving a time-reversed phase adjustment to be applied to the peaks of the stored spectrum for the first subframe in the second two consecutive subframes based on the estimated phase. The method further comprises: applying the time-reversed phase adjustment to the peaks of the signal spectrum to form time-reversed phase-adjusted peaks. The method also includes: applying time reversal to the hidden audio subframe; combining the time-reversed phase-adjusted peak with a noise spectrum of the signal spectrum to form a combined spectrum for the first subframe in the second two consecutive subframes; and generating a synthetic hidden audio subframe based on the combined spectrum.

[0015] According to a sixth aspect, there is provided a decoder device configured to generate a concealed audio subframe of an audio signal. The decoder device comprises a processing circuit and a memory operatively coupled to the processing circuit, wherein the memory comprises instructions which, when executed by the processing circuit, cause the decoder device to perform operations according to the first or fifth aspect.

[0016] According to a seventh aspect, a decoder device is provided. The decoder device is configured to generate a concealed audio subframe of an audio signal, wherein the decoder device is adapted to perform the method according to the fifth aspect.

[0017] According to an eighth aspect, there is provided a computer program. The computer program comprises program code to be executed by processing circuitry of a decoder device configured to operate in a communications network, whereby execution of the program code causes the decoder device to perform operations according to the fifth aspect.

[0018] According to a ninth aspect, there is provided a computer program product comprising a non-transitory storage medium storing program code to be executed by a processing circuit of a decoder device configured to operate in a communication network, whereby execution of the program code causes the decoder device to perform operations according to the fifth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments. In the drawings:

[0020] Figure 1 is a block diagram showing an encoder and decoder pair, wherein encoding is performed in the DFT domain;

[0021] Figure 2 is a block diagram illustrating an encoder and decoder pair, wherein the decoder applies a DFT transform to facilitate frequency domain processing;

[0022] Figure 3 is an illustration of two subframe windows of a decoder, wherein the window applied on the second subframe is a time-reversed or mirrored version of the window applied on the first subframe;

[0023] Figure 4 is a block diagram illustrating an encoder and decoder system including a PLC method that performs phase estimation and applies ECU synthesis in reversed time using a time-reversed phase calculator according to some embodiments;

[0024] Figure 5 is a flow chart illustrating the operation of a decoder device performing time-reversed ECU synthesis according to some embodiments;

[0025] Figure 6 is an illustration of a time-reversed window on a sine wave according to some embodiments;

[0026] Figure 7 is a diagram of how a window of inversion time affects DFT coefficients in the complex plane according to some embodiments;

[0027] Figure 8 According to some embodiments, ε - Graphical representation of frequency f;

[0028] Fig. 9 is a block diagram illustrating a decoder device according to some embodiments;

[0029] Fig.10 is a flow chart illustrating the operation of a decoder device according to some embodiments;

[0030] Fig.11 is a flow chart illustrating the operation of a decoder device according to some embodiments. DETAILED DESCRIPTION

[0031] Various aspects of the present disclosure will now be described more fully below with reference to the accompanying drawings, in which examples of embodiments are shown. However, embodiments may be embodied in a variety of different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided to make the present disclosure detailed and complete, and to fully convey the scope of the present embodiments to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be defaulted to being present / used in another embodiment.

[0032] The following description provides various embodiments of the disclosed subject matter. These embodiments are provided as teaching examples and are not to be construed as limiting the scope of the disclosed subject matter. For example, specific details of the described embodiments may be modified, omitted, or expanded without departing from the scope of the described subject matter.

[0033] Fig. 9 900 is a block diagram showing units of a decoder device 900 configured to provide wireless communication according to an embodiment, which may be part of a mobile terminal, a mobile communication terminal, a wireless communication device, a wireless terminal, a wireless communication terminal, a user equipment UE, a user equipment node / terminal / device, etc. As shown, the decoder 900 may include a network interface circuit 906 (also referred to as a network interface) configured to provide communication with other devices / entities / functions, etc. The decoder 900 may also include a processor circuit 902 (also referred to as a processor) operatively coupled to the network interface circuit 906 and a memory circuit 904 (also referred to as a memory) operatively coupled to the processor circuit. The memory circuit 904 may include a computer-readable program code that, when executed by the processor circuit 902, causes the processor circuit to perform operations according to the embodiments disclosed herein.

[0034] According to other embodiments, the processor circuit 902 can be defined as including a memory so that a separate memory circuit is not required. As discussed herein, the operation of the decoder 900 can be performed by the processor 902 and / or the network interface 906. For example, the processor 902 can control the network interface 906 to send a communication to a multi-channel audio player and / or receive a communication from one or more other network nodes / entities / servers (e.g., encoder nodes, repository servers, etc.) through the network interface 906. In addition, modules can be stored in the memory 904, and these modules can provide instructions so that when the instructions of the modules are executed by the processor 902, the processor 902 performs the corresponding operation.

[0035] In the following description, embodiments will be described using subframe notation. Here, a subframe represents a portion of a larger frame, where the larger frame includes a group of subframes. The described embodiments may also be used with frame notation. In other words, subframes may form a frame group having the same window shape as described herein, and subframes need not be part of a larger frame.

[0036] Consider a decoder in an encoder and decoder pair, where the decoding method generates a spectrum on a subframe basis. Consecutive subframes may have the following properties: the applied window shapes are mirrored or time-reversed versions of each other, such as Figure 3 As shown, subframe 2 is a mirrored or time-reversed version of subframe 1. The decoder obtains the reconstructed subframe of each frame m. In one embodiment, the spectrum of The subframe spectrum is obtained in , where n is the sample index. Figure 2 The dashed boxes in the figure indicate that frequency domain processing can be performed before or after the memory and PLC modules. The spectrum is obtained by multiplying with the subframe windowing functions w1(n) and w2(n) and applying a DFT transform according to the following formula:

[0037]

[0038]

[0039] Where N represents the length of the subframe window, N step12 is the sample distance between the starting points of the first subframe and the second subframe. The subframe windowing functions w1(n) and w2(n) are mirrored or time-reversed versions of each other. Here, the subframe spectrum is obtained from the decoder time domain synthesis, similar to Figure 2 It should be noted that the embodiments are also applicable to the system as outlined in Figure 1 The system outlined in , where the decoder directly reconstructs the subframe spectrum. For each correctly received and decoded audio frame m, the spectrum corresponding to the second subframe is stored in memory.

[0040]

[0041] For correctly received frames, the decoder device 900 may proceed with the frequency domain processing steps, perform an inverse DFT transform, and reconstruct the output audio using an overlap-add strategy. Missing or damaged frames may be identified by the transport layer processing the connection and signaled to the decoder as "bad frames" via a bad frame indicator (BFI), which may take the form of a flag. When the decoder device 900 detects a bad frame via the bad frame indicator (BFI), the PLC algorithm is activated. PLC follows the principle of phase ECU [4]. The stored spectrum is input to a peak detector algorithm that detects peaks on a fractional frequency scale. A set of peaks can be detected

[0042] F={f i},i=1,2,…N peaks

[0043] These peaks are given by their estimated fractional frequencies f i Indicates that, and where N peaks is the number of detected peaks. Similar to the sine encoding paradigm, the peaks of the spectrum are modeled using a sine wave with a specific amplitude, frequency, and phase. Fractional frequencies can be represented as fractions of DFT bins so that, for example, the Nyquist frequency is found at f=N / 2+1. Each peak can be associated with multiple frequency bins representing the peak. This is done by rounding the fractional frequencies to the nearest integer and including adjacent bins (e.g., N on each side). near peaks) to find these intervals:

[0044]

[0045] Where [·] represents rounding operation, G i It means that at frequency f i The number of intervals N is the peak value at near is a tuning constant that can be determined when designing the system. near Provides greater accuracy in each peak representation, but also introduces greater distances between peaks that can be modeled. near Suitable values ​​of can be 1 or 2. The hidden spectrum can be formed by using these interval groups The peak values ​​of , where a phase adjustment has been applied to each group. The phase adjustment takes into account the phase change in the underlying sine wave, assuming that the frequency remains the same between the last correctly received and decoded frame and the hidden frame. The phase adjustment is based on the fractional frequency and the number of samples between the analysis frame of the previous frame and the start of the current frame. Figure 3 As shown, between the start of the second subframe of the last received frame and the start of the first subframe of the first ECU frame, the number of samples is N step21, and between the first subframe of the last received frame and the first subframe of the first ECU frame, the number of samples is N full . Note that N full The distance between the second subframe of the last received frame and the second subframe of the first ECU frame is also given.

[0046] Figure 4 An encoder and decoder system according to the embodiment described below is shown where the PLC block 109 uses a phase estimator 112 to perform phase estimation, and uses a time-reversed phase calculator to apply ECU synthesis in reversed time.

[0047] Figure 5 is a flow chart showing the steps of the time-reversed ECU synthesis described below. For the concealment of the first subframe, the ECU synthesis can be performed in reversed time to obtain the desired window shape. For the first subframe, the phase adjustment or phase correction or phase progression (these terms can be used interchangeably throughout the specification) of the peak i can be expressed as

[0048] Δφ i =-2φ i -2πf i (N+N step21 +(N lost -1)N full ) / N,

[0049] Where N lost represents the number of consecutive lost frames, and φ i Indicates that at frequency f i The phase of the sine wave at . lost -1)N full The phase progression for handling burst errors is as large as the frame length N of the full frame. full For the first lost frame, N lost =1. For the spectrum The frequency interval centered on the frequency can easily obtain the phase φ by simply extracting the angle i :

[0050]

[0051] where k i =[f i ].

[0052] Generally speaking, the frequency f i is a fraction, and the phase needs to be estimated in operation 501. One estimation method is to use linear interpolation of the phase spectrum.

[0053]

[0054] in and denote the operators for rounding down and rounding up, respectively. However, this estimation method was found to be unstable. This estimation method also requires two phase extractions, which requires a computationally complex arctan function if the spectrum is represented using complex numbers in the standard form a+bi. Another phase estimate that was found to be reliable at relatively low computational complexity is:

[0055]

[0056] f frac =f i -k i

[0057] Among them, f frac is the rounding error, and φ C is the tuning constant, which depends on the window shape applied. For the window shape of this embodiment, a suitable value was found to be φ C = 0.33. For another window shape, a suitable value is found to be φ C = 0.48. Generally speaking, it is expected that a suitable value can be found in the range [0.1, 0.7].

[0058] In operation 502, the time-reversed phase adjustment Δφ is derived as described above. i .

[0059] The peak of the hidden spectrum may be formed by applying a phase adjustment to the stored spectrum in operation 503 .

[0060]

[0061] The asterisk "*" indicates complex conjugation, which gives a time reversal of the signal in operation 504. This results in a time reversal of the first ECU subframe. It should be noted that the reversal can also be performed in the time domain after the inverse DFT. However, if If only a part of the complete spectrum is represented, this requires pre-processing of the remaining spectrum, for example by time reversal before DFT analysis.

[0062] The unpeaked interval G i The remaining intervals occupied may be referred to as the noise spectrum or noise component of the spectrum. These remaining intervals may be filled using the coefficients of the stored spectrum with a random phase applied:

[0063]

[0064] where φ randrepresents a random phase value. The remaining intervals may also be filled with spectral coefficients that preserve the desired characteristics of the signal (e.g., correlation with a second channel in a multi-channel decoder system). In operation 505, the peak spectrum (where k∈G i ) and the noise spectrum (in ) are combined to form a combined spectrum.

[0065] In embodiments where the noise is generated in the time domain and is windowed and transformed, time reversal of the noise (to match the windowing of the peak component) and combination with the peak spectrum should be performed before applying the time reversal described above.

[0066] For the generation of the second subframe synthesized in normal (non-inverted) time, conventional phase adjustment may be used.

[0067] Δφ i =2πf i N full N lost / N

[0068] The ECU synthesis for the second subframe may be formed similarly to the first subframe, but omitting the complex conjugation on the peak coefficients.

[0069]

[0070]

[0071] Once the combined hidden spectrum is generated in operation 505 , it may be fed to subsequent processing steps in operation 506 , including inverse DFT and overlap-add operations resulting in an output audio signal.

[0072] The output audio signal may be sent to one or more speakers, such as speakers for playback. The speakers may be part of the decoding device, a separate device, or part of another device.

[0073] Derivation of Phase Correction Formula for Time-Reversed ECU Synthesis

[0074] Assume that the initial phase of the sinusoidal component is φ0 and the frequency of the sinusoidal wave is f. step The expected phase φ1 of the sine wave after samples is then:

[0075] φ1=φ0+2πfN step / N

[0076] For the time-reversed continuation of the sine wave, the phase needs to be mirrored in the real axis by applying a complex conjugate or by simply taking the negative phase -φ1. Since this phase angle now represents the end point of the ECU synthesis frame, the phase needs to wrap around the length of the analysis frame to reach the desired starting phase φ2.

[0077] φ2=-φ1-2πf(N-1) / N

[0078] In order to obtain the phase correction Δφ, the starting phase needs to be subtracted, that is,

[0079]

[0080] Substituting into φ2 gives

[0081] Δφ=-2φ0-2πf(N step +N-1) / N

[0082] To add the progression of consecutive frame losses (burst losses), a factor N corresponding to the number of samples between the start points of a full frame can be added offset =(N lost -1)N full . This provides the final phase correction:

[0083] Δφ=-2φ0-2πf(N+N step -1+(N lost -1)N full ) / N,

[0084] By using complex conjugation and a single sample cyclic shift, the desired time reversal can be achieved in the DFT domain. This cyclic shift can be achieved by a phase correction of 2πk / N, which can be included in the final phase correction.

[0085] Δφ=-2φ0-2πf(N+N step -1+(N lost -1)N full ) / N+2πk / N

[0086] For coefficients representing a single peak, the frequency interval k of the cyclic shift can be approximated as a fractional frequency k≈f, and the phase correction can be simplified to:

[0087] Δφ=-2φ0-2πf(N+N step -1+(N lost -1)N full ) / N+2πf / N=-2φ0-2πf(N+N step +(N lost -1)N full ) / N

[0088] The window can be designed so that N = N full , in this case, the expression can be further simplified to:

[0089] Δφ=-2φ0-2πf(N step +N lost ·N) / N

[0090] Alternative embodiment of reversed time ECU synthesis

[0091] In another embodiment, the phase correction is performed in two steps. In the first step the phase is advanced, ignoring the window mismatch.

[0092] Δφ=2πf(N step +(N lost -1)N full )

[0093]

[0094] In the second step, the phase can be adjusted by -φ m Return, apply complex conjugation, and use φ m Restore the phase to achieve windowed time reversal:

[0095]

[0096] Through research such as Figure 6 The motivation for this operation can be seen in the effect of a time-reversed window on a sine wave. Figure 6 In , the upper figure shows the window applied in a first direction, and the lower figure shows the window applied in the reverse direction. Figure 7 The three coefficients representing a sine wave are shown in Figure 2, which shows how reversing the window in time affects the DFT coefficients in the complex plane. Figure 6 The three DFT coefficients of the sine wave in the figure above are marked with circles, and Figure 6 The corresponding coefficients of the figure below are marked with asterisks. The diamonds represent the positions of the original phases of the sine waves, and the dashed lines show the observed mirror plane through which the coefficients of the time-reversed window are projected. The time-reversed window gives the coefficients mirrored in the mirror plane with an angle of φ m .

[0097] φ m =φ0+φ frac

[0098] Through experiments, it was found that frac can be expressed as:

[0099] φ frac =πf frac

[0100] f frac =f i -k i

[0101] k i =[f i ]

[0102] where [·] indicates rounding operation. It is also found that φ ε (expressed as a positive angle) can be approximated by f frac The linear relationship between Figure 8 In the middle, the angle φ ε is expressed as a function of frequency f. Figure 8 The sawtooth shape of ε A good approximation of is:

[0103] φ ε =-f fac φ C

[0104] where φ C is a constant. In one embodiment, φ C Can be set to φ C = 0.33, which yields a close approximation. Since φ0 is not explicitly known, φ m An alternative approximation of can be written as:

[0105]

[0106] Among them, φ ki is the rounded frequency interval k after the first phase adjustment step i The phase of the maximum peak factor found at

[0107]

[0108] The operations of aligning the mirror plane with the real axis, applying the complex conjugation, and inverting the phase again can be understood as adjusting the phase of the shaped sinusoid to a phase position (0 or π) that is neutral for the complex conjugation, thereby only inverting the temporal shape of the signal. The two-step approach is computationally more complex than the previously described embodiments. However, observation can also lead to an approximation of φ0. From Figure 7 It can be seen that φ0 can be expressed as:

[0109]

[0110] This is the phase approximation used above.

[0111] Reference will now be made to some embodiments of the Fig.10 The decoder device 900 (using Fig. 9 For example, the module can be stored in Fig. 9 The modules may be stored in the memory 904, and these modules may provide instructions so that when the instructions of the modules are executed by the corresponding decoder device processing circuit 902, the processing circuit 902 performs the corresponding operations of the flowchart.

[0112] In operation 1000, the processing circuit 902 generates a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following characteristics: the applied window shape of a first subframe in the consecutive subframes is a mirrored version or a time-reversed version of a second subframe in the consecutive subframes. For example, generating a spectrum for each of the first two consecutive subframes includes determining:

[0113]

[0114]

[0115] Where N represents the length of the subframe window, and the subframe windowing function w1(n) is the first subframe in the continuous subframes. The subframe windowing function of w2(n) is the second subframe in the continuous subframes. The subframe windowing function, and N step12 is the number of samples between a first subframe in the first two consecutive subframes and a second subframe in the first two consecutive subframes.

[0116] In operation 1002, the processing circuit 902 determines whether a bad frame indicator (BFI) has been received. The bad frame indicator provides an indication that an audio frame has been lost or corrupted.

[0117] In operation 1004, for each correctly decoded audio frame, the processing circuit 902 stores the spectrum corresponding to the second subframe in the memory. For example, for a correctly decoded frame m, the spectrum corresponding to the second subframe is stored in memory, e.g. For correctly received frames, the decoder device 900 may continue to perform the frequency domain processing steps, perform an inverse DFT transform, and reconstruct the output audio using an overlap-add strategy, as described above and Figure 4 As shown. Note that the principle of overlap-add is the same for both subframes and frames. The creation of a frame requires applying overlap-add to the subframes, while the final output frame is the result of the overlap-add operation between frames.

[0118] When the processing circuit 902 detects a bad frame through a bad frame indicator (BFI) in operation 1002 , PLC operations 1006 to 1030 are performed.

[0119] In operation 1006, the processing circuit 902 obtains a signal spectrum corresponding to the second subframe of the first two consecutive subframes that were previously correctly decoded and processed. For example, the processing circuit 902 may obtain the signal spectrum from the memory 904 of the decoding device.

[0120] In operation 1008 , the processing circuit 902 detects a peak of a signal spectrum of a previously received audio frame of the audio signal on a fractional frequency scale, the previously received audio frame being received before the bad frame indicator is received.

[0121] In operation 1010 , the processing circuit 902 determines whether a concealed frame is used for a first subframe of two consecutive subframes.

[0122] If a concealed frame is used for the first subframe, then in operation 1012, the processing circuit 902 estimates the phase of each peak. In one embodiment, the phase estimate is calculated for the peak of the time-reversed phase-corrected peaks according to the following equation:

[0123]

[0124] f frac =f i -k i

[0125] Among them, φ i is at frequency f i The estimated phase at is in the frequency interval k i The spectrum at The angle, f frac is the rounding error, φ C is the tuning constant, and k i Yes i ]. Tuning constant φ C It can be a value in the range between 0.1 and 0.7.

[0126] In operation 1014 , the processing circuit 902 derives a time-reversed phase correction to be applied to the peak of the signal spectrum based on the estimated phase.

[0127] In operation 1016 , the processing circuit 902 applies a time-reversed phase correction to the peaks of the signal spectrum to form time-reversed phase-corrected peaks.

[0128] In operation 1018, the processing circuit 902 applies time reversal to the concealed audio sub-frame. In one embodiment, the time reversal may be applied by applying a complex conjugate to the concealed audio sub-frame.

[0129] In operation 1020 , the processing circuit 902 combines the time-reversed phase-corrected peak with the noise spectrum of the signal spectrum to form a combined spectrum of the concealed audio sub-frame.

[0130] Go to Fig.11 In one embodiment, the processing circuit 902 may perform 1016 and 1018 by associating each peak with a plurality of peak frequency bins in operation 1100. The processing circuit 902 may apply the time-reversed phase correction by applying the time-reversed phase correction to each of the plurality of frequency bins in operation 1102. In operation 1104, the remaining bins are filled with coefficients of the signal spectrum to which the random phase is applied.

[0131] Back to Fig.10 In operation 1022, the processing circuit 902 generates a synthetic concealed audio subframe based on the combined spectrum.

[0132] If it is determined in operation 1010 that a concealment frame is not used for the first subframe, the processing circuit 902 derives a non-time-reversed phase correction to be applied to the peak of the signal spectrum for a second concealment subframe of at least two consecutive concealment subframes in operation 1024 .

[0133] In operation 1026 , the processing circuit 902 applies a non-time-reversed phase correction to the peak of the signal spectrum for the second subframe to form a non-time-reversed phase-corrected peak.

[0134] In operation 1028, the processing circuit 902 combines the non-time-reversed phase-corrected peak with the noise spectrum of the signal spectrum to form a combined spectrum for the second concealed subframe.

[0135] In operation 1030 , the processing circuit 902 generates a second synthesized concealed audio sub-frame based on the combined spectrum.

[0136] Go to Fig.11 In one embodiment, the processing circuit 902 may perform 1026 and 1028 by associating each peak with a plurality of peak frequency bins in operation 1100. The processing circuit 902 may apply the non-time-reversed phase correction by applying the non-time-reversed phase correction to each of the plurality of frequency bins in operation 1102. In operation 1104, the remaining bins are filled with coefficients of the signal spectrum to which the random phase is applied.

[0137] For some embodiments of decoder devices and related methods, from Fig.10 Various operations of the flowcharts may be optional. For example, with respect to the method of Example Embodiment 1 (described below), Fig.10The operations of blocks 1004 and 1022-1030 may be optional. For example, with respect to the method of example embodiment 19 (described below), Fig.10 The operations of blocks 1010 and 1022 - 1030 may be optional.

[0138] Example embodiments are discussed below.

[0139] 1. A method for generating a concealed audio subframe of an audio signal in a decoding device, the method comprising:

[0140] generating (1000) a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following property: an applied window shape of a first subframe in the consecutive subframes is a mirrored version or a time-reversed version of a second subframe in the consecutive subframes;

[0141] receiving (1002) a bad frame indicator;

[0142] detecting (1008) a peak of a signal spectrum of a previously received audio frame of the audio signal on a fractional frequency scale, the previously received audio frame being received before receiving the bad frame indicator;

[0143] Estimate (1012) the phase of each peak;

[0144] Based on the estimated phase, deriving (1014) a time-reversed phase correction to be applied to a peak of the signal spectrum;

[0145] applying (1016) a time-reversed phase correction to a peak of the signal spectrum to form a time-reversed phase-corrected peak;

[0146] applying (1018) time reversal to the hidden audio subframe;

[0147] combining (1020) the time-reversed phase-corrected peak with the noise spectrum of the signal spectrum to form a combined spectrum for concealing the audio subframe; and

[0148] Based on the combined spectrum, a synthetic concealed audio subframe is generated (1022).

[0149] 2. The method of embodiment 1, wherein the synthesized concealed audio frame comprises at least two consecutive concealed subframes, and wherein deriving the time-reversed phase correction, applying the time-reversed phase correction, applying the time reversal, and combining the time-reversed phase corrected peaks are performed for a first concealed subframe of the at least two consecutive concealed subframes, the method further comprising:

[0150] deriving (1024) a non-time-reversed phase correction to be applied to a peak of a signal spectrum for a second concealed subframe of at least two consecutive concealed subframes;

[0151] For a second subframe, applying (1026) a non-time-reversed phase correction to a peak of the signal spectrum to form a non-time-reversed phase-corrected peak;

[0152] combining (1028) the non-time-reversed phase-corrected peak with the noise spectrum of the signal spectrum to form a combined spectrum for a second concealed subframe; and

[0153] Based on the combined spectrum, a second synthesized concealed audio subframe is generated (1030).

[0154] 3. The method of any one of embodiments 1-2, wherein the concealed audio subframe comprises a concealed audio subframe for one of a lost audio frame and a damaged audio frame.

[0155] 4. The method of any one of embodiments 1-3, wherein the bad frame indicator provides an indication that the audio frame is lost or damaged.

[0156] 5. The method according to any one of embodiments 1-4 further includes: obtaining a signal spectrum of a previously received audio signal frame from a memory of a decoder.

[0157] 6. The method of any one of embodiments 1-5, wherein applying time reversal comprises applying complex conjugation to the hidden audio subframe.

[0158] 7. The method according to any one of embodiments 1-6, further comprising:

[0159] Each peak of the plurality of peaks is associated with a plurality of peak frequency bins representing the peak (1100).

[0160] 8. The method of embodiment 7, wherein for each peak of the plurality of peaks, one of a time-reversed phase correction and a non-time-reversed phase correction is applied (1102) to the peak.

[0161] 9. The method of any one of embodiments 8, further comprising:

[0162] The remaining intervals of the signal spectrum are filled (1104) using coefficients of the stored signal spectrum to which the random phase is applied.

[0163] 10. The method of any one of embodiments 1-9, wherein estimating the phase of each peak comprises:

[0164] The phase estimate is calculated for the peak of the time-reversed phase-corrected peaks according to:

[0165]

[0166] f frac =f i -k i

[0167] Among them, φ i is at frequency f i The estimated phase at is in the frequency interval k i The spectrum at The angle, f frac is the rounding error, φ C is the tuning constant, and k i Yes i ].

[0168] 11. The method of embodiment 10, wherein φ C has a value in the range between 0.1 and 0.7.

[0169] 12. The method of embodiment 10, wherein the peak-calculated phase estimate for the non-time-reversed phase correction is calculated according to the following formula:

[0170] Δφ i =2πf i N full N lost / N

[0171] Among them, Δφ i Indicates that at frequency f i Phase correction of the sine wave at full represents the number of samples between two frames, N lost represents the number of consecutive lost frames, and N represents the length of the subframe window.

[0172] 13. The method according to any one of embodiments 1-12 further includes: applying a random phase to the noise spectrum of the signal spectrum.

[0173] 14. The method of embodiment 13, wherein applying the random phase to the noise spectrum comprises applying the random phase to the noise spectrum before combining the non-time-reversed phase-corrected peaks with the noise spectrum.

[0174] 15. A decoder device (900) configured to generate concealed audio subframes of a received audio signal, wherein a decoding method of the decoding device generates a spectrum on a subframe basis, wherein consecutive subframes have the following property: the applied window shapes are mirrored versions or time-reversed versions of each other, the decoder device comprising:

[0175] processing circuit (902); and

[0176] A memory (904) coupled to the processing circuit, wherein the memory includes instructions that, when executed by the processing circuit, cause the decoder device to perform operations according to any one of embodiments 1-14.

[0177] 16. A decoder device (900) configured to generate hidden audio subframes of a received audio signal, wherein a decoding method of the decoding device generates a spectrum on a subframe basis, wherein consecutive subframes have the following characteristics: the applied window shapes are mirror versions or time-reversed versions of each other, wherein the decoder device is suitable for performing according to any one of embodiments 1-14.

[0178] 17. A computer program comprising program codes to be executed by processing circuitry (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program codes causes the decoder device (900) to perform operations according to any one of embodiments 1-14.

[0179] 18. A computer program product comprising a non-transitory storage medium storing program codes to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program codes causes the decoder device (900) to perform operations according to any one of embodiments 1-14.

[0180] 19. A method for generating a concealed audio subframe of an audio signal in a decoding device, the method comprising:

[0181] generating (1000) a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following property: an applied window shape of a first subframe in the consecutive subframes is a mirrored version or a time-reversed version of a second subframe in the consecutive subframes;

[0182] storing (1004) a signal spectrum corresponding to a second subframe of the first two consecutive subframes;

[0183] Receiving a bad frame indicator for a second two consecutive subframes (1002);

[0184] Obtaining (1006) a signal spectrum;

[0185] detecting (1008) peaks of the signal spectrum on a fractional frequency scale;

[0186] Estimate (1012) the phase of each peak;

[0187] deriving (1014) a time-reversed phase correction to be applied to a peak of the stored spectrum for a first subframe of a second two consecutive subframes based on the estimated phase;

[0188] applying (1016) a time-reversed phase correction to a peak of the signal spectrum to form a time-reversed phase-corrected peak;

[0189] applying (1018) time reversal to the hidden audio subframe;

[0190] combining (1020) the time-reversed phase-corrected peak with the noise spectrum of the signal spectrum to form a combined spectrum for a first subframe of a second two consecutive subframes; and

[0191] Based on the combined spectrum, a synthetic concealed audio subframe is generated (1022).

[0192] 20. The method of embodiment 19, wherein the synthesized concealed audio frame comprises at least two consecutive concealed subframes, and wherein deriving the time-reversed phase correction, applying the time-reversed phase correction, and combining the time-reversed phase corrected peaks are performed for a first concealed subframe of the at least two consecutive concealed subframes, the method further comprising:

[0193] deriving (1024) a non-time-reversed phase correction to be applied to a peak of a spectrum of the signal for a second subframe of a second two consecutive subframes;

[0194] For a second subframe of the second two consecutive subframes, applying (1026) a non-time-reversed phase correction to a peak of the signal spectrum to form a non-time-reversed phase-corrected peak;

[0195] combining (1028) the non-time-reversed audio subframe with the noise spectrum of the signal spectrum to form a second combined spectrum for a second subframe of a second two consecutive subframes; and

[0196] Based on the second combined spectrum, a second synthesized audio subframe is generated (1030).

[0197] 21. The method of any one of embodiments 19-20, wherein the concealed audio subframe comprises a concealed audio subframe of one of a lost audio frame and a damaged audio frame.

[0198] 22. The method of any one of embodiments 19-21, wherein the bad frame indicator provides an indication that the audio frame is lost or corrupted.

[0199] 23. The method according to any one of embodiments 19-22 also includes: obtaining a signal spectrum from a memory of a decoder.

[0200] 24. The method of any one of embodiments 19-23, wherein applying time reversal comprises applying complex conjugation to the hidden audio subframe.

[0201] 25. The method of any one of embodiments 18-24, further comprising:

[0202] Each peak is associated with a plurality of peak frequency bins representing the peak.

[0203] 26. The method according to embodiment 25 also includes: for each peak among the multiple peaks, applying one of a time-reversed phase correction and a non-time-reversed phase correction to the peak.

[0204] 27. The method of embodiment 26, further comprising:

[0205] The remaining intervals of the signal spectrum are filled using the coefficients of the stored spectrum to which a random phase is applied.

[0206] 28. The method of any one of embodiments 19-27, wherein estimating the phase comprises:

[0207] The phase estimate is calculated for the time-reversed phase-corrected peak according to:

[0208]

[0209] f frac =f i -k i

[0210] Among them, φ i is at frequency f i The estimated phase at is in the frequency interval k i The spectrum at The angle, f frac is the rounding error, φ C is the tuning constant, and k i Yes i ].

[0211] 29. The method of embodiment 28, wherein φ C has a value in the range between 0.1 and 0.7.

[0212] 30. The method of embodiment 28, further comprising: calculating a phase estimate for the non-time-reversed phase-corrected peak according to the following equation:

[0213] Δφ i =2πf i N full N lost / N

[0214] Among them, Δφ i Indicates that at frequency f iPhase correction of the sine wave at full represents the number of frame samples between two frames, N lost represents the number of consecutive lost frames, and N represents the length of the subframe window.

[0215] 31. The method of any one of embodiments 19-30, wherein generating a spectrum for each of the first two consecutive subframes includes determining:

[0216]

[0217]

[0218] Where N represents the length of the subframe window, and the subframe windowing function w1(n) is used for the first subframe in consecutive subframes. The subframe windowing function is w2(m), and w2(m) is used for the second subframe in the continuous subframes. The subframe windowing function, and M step12 is the number of samples between a first subframe in the first two consecutive subframes and a second subframe in the first two consecutive subframes.

[0219] 32. The method according to any one of embodiments 19-31 also includes: applying a random phase to the noise spectrum of the signal spectrum.

[0220] 33. The method of embodiment 32, wherein applying a random phase to the noise spectrum comprises applying a random phase to the noise spectrum before combining the non-time-reversed phase-corrected peaks with the noise spectrum.

[0221] 34. A decoder device (900) configured to generate concealed audio subframes of a received audio signal, wherein a decoding method of the decoding device generates a spectrum on a subframe basis, wherein consecutive subframes have the following property: the applied window shapes are mirrored versions or time-reversed versions of each other, the decoder device comprising:

[0222] processing circuit (902); and

[0223] A memory (904) coupled to the processing circuit, wherein the memory includes instructions that, when executed by the processing circuit, cause the decoder device to perform operations according to any of embodiments 19-33.

[0224] 35. A decoder device (900) configured to generate hidden audio subframes of a received audio signal, wherein a decoding method of the decoding device (900) generates a spectrum on a subframe basis, wherein consecutive subframes have the following characteristics: the applied window shapes are mirror versions or time-reversed versions of each other, wherein the decoder device is suitable for performing according to any one of embodiments 19-33.

[0225] 36. A computer program comprising program codes to be executed by processing circuitry (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program codes causes the decoder device (900) to perform operations according to any one of embodiments 19-33.

[0226] 37. A computer program product comprising a non-transitory storage medium storing program codes to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program codes causes the decoder device (900) to perform operations according to any one of embodiments 19-33.

[0227] A description of the various abbreviations / acronyms used in this disclosure is provided below.

[0228] Abbreviation

[0229] DFT Discrete Fourier Transform

[0230] IDFT Inverse Discrete Fourier Transform

[0231] LP Linear Prediction

[0232] PLC Packet Loss Concealment

[0233] ECU Error Hiding Unit

[0234] FEC Frame Error Correction / Concealment

[0235] References are provided below.

[0236] [1] T. Vaillancourt, M. Jelinek, R. Salami, and R. Lefebvre, “Efficient FrameErasure Concealment in Predictive Speech Codecs using Glottal Pulse Resynchronisation,” in 2007 IEEE International Conference on Acoustics, Speech, and Signal Processing—ICASSP’07, Honolulu, Hawaii, 2007, pp. IV-1113–IV-1116.

[0237] [2] J. Lecomte et al., “Packet-loss concealment technology advances in EVS,” in 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Brisbane, Queensland, 2015, pp. 5708–5712.

[0238] [3] 3GPP TS 26.447, Codec for Enhanced Voice Services (EVS); Error Concealment of Lost Packets (Release 12)

[0239] [4] S. Bruhn, E. Norvell, J. Svedberg, and S. Sverrisson, “A novel sinusoidal approach to audio signal frame loss concealment and its application in the new evs codec standard,” in 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Brisbane, Queensland, 2015, pp. 5142–5146.

[0240] Generally, unless different meanings are clearly given and / or different meanings are implied in the context of using the term, all terms used in this article will be interpreted according to their common meanings in the relevant technical field. Unless clearly stated, all references to one / an / the element, device, assembly, part, step, etc. should be publicly interpreted as referring to at least one instance of the element, device, assembly, part, step, etc. Unless a step is explicitly described as being after or before another step and / or a step must be after or before another step implicitly, the steps of any method disclosed herein do not have to be performed in the exact order disclosed. Where appropriate, any feature of any embodiment disclosed herein may be applied to any other embodiment. Similarly, any advantage of any embodiment may be applicable to any other embodiment, and vice versa. By the following description, other purposes, features and advantages of the attached embodiments will be apparent.

[0241] In the above description of various embodiments, it will be understood that the terms used herein are only for describing specific embodiments and are not intended to be limiting. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of this specification and related art, and will not be interpreted in an idealized or overly formal sense, unless explicitly defined herein.

[0242] When a unit is referred to as being "connected to," "coupled to," "responsive to" (or variations thereof) another unit, it may be directly connected to, coupled to, or responsive to the other unit, or there may be intermediate units. In contrast, when a unit is referred to as being "directly connected to," "directly coupled to," "directly responsive to" (or variations thereof) another unit, there are no intermediate units. The same numbers herein refer to the same units. In addition, as used herein, "coupling," "connecting," "responsiveness" or variations thereof may include wireless coupling, connection, or response. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. For the sake of brevity and / or clarity, well-known functions or structures may not be described in detail. The term "and / or" includes any and all combinations of one or more listed associated items.

[0243] It will be understood that although the terms first, second, third, etc. may be used herein to describe various units / operations, these units / operations should not be limited by these terms. These terms are only used to distinguish one unit / operation from another unit / operation. Therefore, the first unit / operation in some embodiments may be referred to as the second unit / operation in other embodiments without departing from the teachings of the present disclosure. The same reference numerals or the same reference designators in this specification represent the same or similar units.

[0244] As used herein, the terms "comprises," "including," "having," or variations thereof are open ended and include one or more stated features, integers, units, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integers, units, steps, components, functions, or combinations thereof. In addition, as used herein, the general abbreviation "such as," derived from the Latin phrase "exempli gratia," may be used to introduce or specify one or more general examples of previously mentioned items, and is not intended as a limitation of such items. The general abbreviation "that is," derived from the Latin phrase "id est," may be used to specify a specific item from a more general description.

[0245] Example embodiments are described herein with reference to block diagrams and / or flow charts of computer-implemented methods, apparatus (systems and / or devices), and / or computer program products. It will be understood that the blocks of the block diagrams and / or flow charts, and the combination of the blocks in the block diagrams and / or flow charts, can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to a processor circuit of a general-purpose computer circuit, a special-purpose computer circuit, and / or other programmable data processing circuit to produce a machine so that these instructions, when executed via a processor of a computer and / or other programmable data processing device, transform and control transistors, values ​​stored in storage cells, and other hardware components within such circuits to implement the functions / operations specified in one or more blocks in the block diagrams and / or flow charts, thereby generating a device (function) and / or structure that implements the functions / operations specified in the blocks in the block diagrams and / or flow charts.

[0246] These computer program instructions may also be stored in a tangible computer-readable medium, which may cause a computer or other programmable data processing device to operate in a specific manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / operations specified in one or more blocks in the block diagram and / or flow chart. Therefore, the embodiments of the present disclosure may be embodied in hardware and / or software (including firmware, resident software, microcode, etc.), which runs on a processor such as a digital signal processor, and the hardware and / or software may be collectively referred to as "circuits", "modules" or variations thereof.

[0247] It should also be noted that in some alternative implementations, the function / operation marked in the square frame can occur in a sequence different from that marked in the flow chart. For example, two continuous square frames can actually be performed substantially in parallel, and they can also be performed in reverse order sometimes, depending on the function / operation involved. In addition, the function of the given square frame of flow chart and / or block diagram can be divided into a plurality of square frames, and / or the function of two or more square frames of flow chart and / or block diagram can be integrated at least in part. Finally, other square frames can be added / inserted between the square frames shown, and / or square frame / operation can be omitted without departing from the scope of the embodiment. In addition, although some figures include arrows on the communication path to illustrate the main direction of communication, it will be understood that communication can occur in the direction opposite to the arrows shown.

[0248] Many changes and modifications can be made to the embodiments without substantially departing from the principles of the present disclosure. In this article, all these changes and modifications are intended to be included in the scope of the present disclosure. Therefore, the subject matter disclosed above is considered to be illustrative rather than restrictive, and the examples of the embodiments are intended to cover all these modifications, enhancements and other embodiments that fall within the spirit and scope of the present disclosure. Therefore, to the maximum extent permitted by law, the scope of the present disclosure will be determined by the most widely permitted interpretation of the present disclosure (including examples of the embodiments and their equivalents), and should not be limited or restricted by the above detailed description.

Claims

1. A method for generating a hidden audio subframe of an audio signal in a decoding device, characterized in that: The method comprises: generating (1000) a frequency spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following property: an applied window shape of a first subframe of the consecutive subframes is a mirrored version or a time-reversed version of a window applied on a second subframe of the consecutive subframes; detecting (1008) peaks of a signal spectrum of a previously received audio signal on a fractional frequency scale; estimating (1012) a phase of each of said peaks; deriving (1014) a time-reversed phase adjustment to be applied to the peak of the signal spectrum based on the estimated phase; applying (1016) the time-reversed phase adjustment to the peak of the signal spectrum to form a time-reversed phase adjusted peak; applying (1018) time reversal to the concealed audio subframe; combining (1020) the time-reversed, phase-adjusted peak with a noise spectrum of the signal spectrum to form a combined spectrum for the concealed audio subframe; and Based on the combined spectrum, a synthetic concealed audio sub-frame is generated (1022).

2. The method according to claim 1, wherein: The synthesized concealed audio frame comprises at least two consecutive concealed subframes, and wherein deriving the time-reversed phase adjustment, applying the time-reversed phase adjustment, applying the time reversal, and combining the time-reversed phase adjusted peaks are performed for a first concealed subframe of the at least two consecutive concealed subframes, the method further comprising: deriving (1024) a non-time-reversed phase adjustment to be applied to the peak of the signal spectrum for a second concealed subframe of the at least two consecutive concealed subframes; for the second subframe, applying (1026) the non-time-reversed phase adjustment to the peak of the signal spectrum to form a non-time-reversed phase adjusted peak; combining (1028) the non-time-reversed phase adjusted peak with a noise spectrum of the signal spectrum to form a combined spectrum for the second concealed subframe; and Based on the combined spectrum, a second synthesized concealed audio subframe is generated (1030).

3. The method according to claim 1, further comprising: The signal spectrum of the previously received audio signal is obtained (1006) from a memory of the decoding device.

4. The method according to claim 1, wherein: Applying the time reversal includes applying a complex conjugate to the time-reversed phase adjusted peak.

5. The method according to claim 2, further comprising: Each of the detected peaks is associated with a plurality of peak frequency bins representing the peak (1100).

6. The method according to claim 5, wherein: For each peak frequency bin of the plurality of peak frequency bins, one of the time-reversed phase adjustment and the non-time-reversed phase adjustment is applied (1102) to the peak frequency bin.

7. The method according to claim 6, further comprising: The remaining intervals of the signal spectrum are filled (1104) using coefficients of the stored signal spectrum that retain desired characteristics of the signal.

8. The method according to claim 7, wherein: The desired characteristic includes a correlation with a second channel in a multi-channel decoder system.

9. The method according to claim 1, wherein: Estimating the phase of each of the peaks comprises: A phase estimate is calculated for the peak of the time-reversed, phase-adjusted peaks according to: f frac =f i -k i Among them, φ i is at frequency f i The estimated phase at is in the frequency interval k i The spectrum at The angle, f frac is the rounding error, φ C is the tuning constant, and k i Yes i ].

10. The method according to claim 9, wherein: The phase adjustment for the peak of the time-reversed concealed audio subframe is calculated according to: Df i =-2φ i -2πf i (N+N step +(N lost -1)N full ) / N Among them, N represents the length of the full frame, N step is the number of samples between the start of the second subframe of the last received frame and the start of the first subframe of the concealed audio frame, N lost Indicates the number of consecutive lost frames, N fill Indicates the number of samples between two frames.

11. The method according to claim 9, wherein: The phase adjustment for the peak of the time-reversed concealed audio subframe is calculated according to: Δφ=-2φ0-2πf(N step +N lost ·N) / N Where f is the frequency of the sine wave, φ0 is the starting phase of the sine component, and N step is the number of samples between the start of the second subframe of the last received frame and the start of the first subframe of the concealed audio frame, N lost It indicates the number of consecutive lost frames, and N indicates the length of a full frame.

12. The method according to claim 2, further comprising: A random phase is applied to the noise spectrum of the signal spectrum.

13. The method according to claim 12, wherein: Applying the random phase to the noise spectrum includes applying the random phase to the noise spectrum before combining the non-time-reversed phase adjusted peaks with the noise spectrum.

14. A decoder device (900) configured to generate a concealed audio subframe of an audio signal, the decoder device comprising: Processing circuit (902); as well as A memory (904) operatively coupled to the processing circuit, characterized in that the memory stores instructions which, when executed by the processing circuit, cause the decoder device to perform a method according to any one of claims 1-13.

15. A decoder device (900) configured to generate a concealed audio subframe of an audio signal, characterized in that The decoder device is suitable for: generating a spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following property: an applied window shape of a first subframe in the consecutive subframes is a mirrored version or a time-reversed version of a window applied on a second subframe in the consecutive subframes; detecting peaks of a signal spectrum of a previously received audio signal on a fractional frequency scale; estimating a phase of each of said peaks; deriving a time-reversed phase adjustment to be applied to the peak of the signal spectrum based on the estimated phase; applying the time-reversed phase adjustment to the peak of the signal spectrum to form a time-reversed phase adjusted peak; applying time reversal to the hidden audio subframe; combining the time-reversed, phase-adjusted peak with a noise spectrum of the signal spectrum to form a combined spectrum for the concealed audio subframe; and Based on the combined spectrum, a synthetic concealed audio sub-frame is generated.

16. The decoder device according to claim 15, wherein: The synthesized concealed audio frame comprises at least two consecutive concealed sub-frames, and wherein deriving the time-reversed phase adjustment, applying the time-reversed phase adjustment, applying the time reversal and combining the time-reversed phase adjusted peaks are performed for a first concealed sub-frame of the at least two consecutive concealed sub-frames, the decoder device being further adapted to: deriving, for a second concealed subframe of the at least two consecutive concealed subframes, a non-time-reversed phase adjustment to be applied to the peak of the signal spectrum; for the second subframe, applying the non-time-reversed phase adjustment to the peak of the signal spectrum to form a non-time-reversed phase-adjusted peak; combining the non-time-reversed phase adjusted peak with a noise spectrum of the signal spectrum to form a combined spectrum for the second concealed subframe; and Based on the combined spectrum, a second synthesized concealed audio sub-frame is generated.

17. Decoder device according to claim 15, further adapted to obtain the signal spectrum of the previously received audio signal from a memory of the decoder device.

18. A decoder device according to claim 15, adapted to apply the time reversal by applying a complex conjugate to the time-reversed phase adjusted peak.

19. The decoder device of claim 16, further adapted to associate each of the detected peaks with a plurality of peak frequency bins representing the peak.

20. The decoder device of claim 19, further adapted to apply one of the time-reversed phase adjustment and the non-time-reversed phase adjustment to each peak frequency bin of the plurality of peak frequency bins.

21. The decoder device according to claim 20, further adapted to: The remaining intervals of the signal spectrum are filled using coefficients of the stored signal spectrum that retain desired characteristics of the signal.

22. The decoder device according to claim 21, wherein The desired characteristic includes a correlation with a second channel in a multi-channel decoder system.

23. A decoder device according to claim 15, adapted to estimate the phase of each of the time-reversed phase adjusted peaks by calculating a phase estimate for the peak in accordance with: f frac =f i -k i in, φ i is at frequency f i The estimated phase at is in the frequency interval k i The spectrum at The angle, f frac is the rounding error, φ C is the tuning constant, and k i Yes i ].

24. A decoder device according to claim 23, adapted to calculate the phase adjustment for the peak of the time-reversed concealed audio subframe according to: Df i =-2φ i -2πf i (N+N step +(N lost -1)N full ) / N in, N represents the length of the full frame, N step is the number of samples between the start of the second subframe of the last received frame and the start of the first subframe of the concealed audio frame, N lost Indicates the number of consecutive lost frames, N full Indicates the number of samples between two frames.

25. A decoder device according to claim 23, adapted to calculate the phase adjustment for the peak of the time-reversed concealed audio subframe according to: Δφ=-2φ0-2πf(N step +N lost ·N) / N in, f is the frequency of the sine wave, φ0 is the starting phase of the sine component, N step is the number of samples between the start of the second subframe of the last received frame and the start of the first subframe of the concealed audio frame, N lost It indicates the number of consecutive lost frames, and N indicates the length of a full frame.

26. The decoder device of claim 16, further adapted to apply a random phase to the noise spectrum of the signal spectrum.

27. The decoder device of claim 26, further adapted to apply the random phase to the noise spectrum before combining the non-time-reversed phase adjusted peaks with the noise spectrum.

28. A computer program product, characterized in that The computer program product comprises program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program code causes the decoder device (900) to perform a method according to any one of claims 1-13.

29. A non-transitory storage medium, characterized in that: The non-transitory storage medium stores program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program code causes the decoder device (900) to perform a method according to any one of claims 1-13.

30. A method for generating a hidden audio subframe of an audio signal in a decoding device, characterized in that: The method comprises: generating (1000) a frequency spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the following property: an applied window shape of a first subframe of the consecutive subframes is a mirrored version or a time-reversed version of a window applied on a second subframe of the consecutive subframes; storing (1004) a signal spectrum corresponding to a second subframe of the first two consecutive subframes; Receiving a bad frame indicator for a second two consecutive subframes (1002); Obtaining (1006) the signal spectrum; detecting (1008) peaks of the signal spectrum on a fractional frequency scale; estimating (1012) a phase of each of said peaks; deriving (1014) a time-reversed phase adjustment to be applied to the peak of the stored spectrum for a first of the second two consecutive subframes based on the estimated phase; applying (1016) the time-reversed phase adjustment to the peak of the signal spectrum to form a time-reversed phase adjusted peak; applying (1018) time reversal to the concealed audio subframe; combining (1020) the time-reversed, phase-adjusted peak value with a noise spectrum of the signal spectrum to form a combined spectrum for the first subframe of the second two consecutive subframes; and Based on the combined spectrum, a synthetic concealed audio sub-frame is generated (1022).

31. The method according to claim 30, wherein: The synthesized concealed audio frame comprises at least two consecutive concealed subframes, and wherein deriving the time-reversed phase adjustment, applying the time-reversed phase adjustment, and combining the time-reversed phase adjusted peaks are performed for a first concealed subframe of the at least two consecutive concealed subframes, the method further comprising: deriving (1024) a non-time-reversed phase adjustment to be applied to a peak of the signal spectrum for a second subframe of the second two consecutive subframes; for the second subframe of the second two consecutive subframes, applying (1026) the non-time-reversed phase adjustment to the peak of the signal spectrum to form a non-time-reversed phase adjusted peak; combining (1028) the non-time-reversed audio subframe with a noise spectrum of the signal spectrum to form a second combined spectrum for the second subframe of the second two consecutive subframes; and Based on the second combined spectrum, a second synthesized audio subframe is generated (1030).

32. The method of claim 30, further comprising: The signal spectrum is obtained from a memory of a decoding device.

33. The method of claim 30, wherein: Applying the time reversal includes applying a complex conjugate to the time-reversed phase adjusted peak.

34. The method of claim 31 , further comprising: Each peak is associated with a plurality of peak frequency bins representing the peak.

35. The method of claim 34, further comprising: For each peak frequency bin of the plurality of peak frequency bins, one of the time-reversed phase adjustment and the non-time-reversed phase adjustment is applied to the peak frequency bin.

36. The method of claim 35, further comprising: The remaining intervals of the signal spectrum are filled using coefficients of the stored spectrum, the spectral coefficients preserving desired characteristics of the signal.

37. The method of claim 36, wherein: The desired characteristic includes a correlation with a second channel in a multi-channel decoder system.

38. The method of claim 30, wherein: Estimating the phase comprises: A phase estimate is calculated for the time-reversed, phase-adjusted peak according to: f frac =f i -k i Among them, φ i is at frequency f i The estimated phase at is in the frequency interval k i The spectrum at The angle, f frac is the rounding error, φ C is the tuning constant, and k i Yes i ].

39. The method of claim 38, wherein: φ C has a value in the range between 0.1 and 0.

7.

40. The method of claim 38, wherein: The phase adjustment for the peak of the time-reversed concealed audio subframe is calculated according to: Df i =-2φ i -2πf i (N+N step +(N lost -1)N full ) / N Among them, N represents the length of the full frame, N step is the number of samples between the start of the second subframe of the last received frame and the start of the first subframe of the concealed audio frame, N lost Indicates the number of consecutive lost frames, N full Indicates the number of samples between two frames.

41. The method of claim 38, wherein: The phase adjustment for the peak of the time-reversed concealed audio subframe is calculated according to: Δφ=-2φ0-2πf(N step +N lost ·N) / N Where f is the frequency of the sine wave, φ0 is the starting phase of the sine component, and N step is the number of samples between the start of the second subframe of the last received frame and the start of the first subframe of the concealed audio frame, N lost It indicates the number of consecutive lost frames, and N indicates the length of a full frame.

42. The method of claim 30, wherein: Generating a spectrum for each of the first two consecutive subframes includes determining: Wherein N represents the length of the subframe window, and the subframe windowing function w1(n) is used for the first subframe in the continuous subframes. The subframe windowing function is as follows: w2(n) is the second subframe in the continuous subframes. The subframe windowing function, k is the frequency interval, N step12 is the sample distance between the starting points of the first subframe in the first two consecutive subframes and the second subframe in the first two consecutive subframes, is the reconstructed time domain synthesis of the first subframe, m is the frame index, n is the sample index, is a reconstructed time domain synthesis of the second subframe.

43. The method of claim 31 , further comprising: A random phase is applied to the noise spectrum in the signal spectrum.

44. The method of claim 43, wherein: Applying the random phase to the noise spectrum includes applying the random phase to the noise spectrum before combining the non-time-reversed phase adjusted peaks with the noise spectrum.

45. A decoder device (900) configured to generate a concealed audio subframe of an audio signal, the decoder device comprising: Processing circuit (902); as well as A memory (904) operatively coupled to the processing circuit, wherein the memory stores instructions which, when executed by the processing circuit, cause the decoder device to perform a method according to any one of claims 30-44.

46. ​​A decoder device (900) configured to generate a concealed audio subframe of an audio signal, characterized in that The decoder device is adapted to perform a method according to any of claims 30-44.

47. A computer program product, characterized in that The computer program product comprises program code to be executed by processing circuitry (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program code causes the decoder device (900) to perform a method according to any one of claims 30-44.

48. A non-transitory storage medium, characterized in that The non-transitory storage medium stores program code to be executed by processing circuitry (902) of a decoder device (900) configured to operate in a communication network, whereby execution of the program code causes the decoder device (900) to perform a method according to any one of claims 30-44.

Images