Time-reversed audio subframe error concealment

Abstract

This invention provides a method for generating hidden audio subframes of an audio signal and a decoder device for doing so. [Solution] A method for generating a frequency spectrum on a subframe basis, wherein a sequence of subframes of an audio signal generates a frequency spectrum having the characteristic that the applied window shape of a first subframe among those sequence of subframes is a mirrored or time-reversed version of a second subframe among those sequence of subframes; the process of detecting the peaks of the signal spectrum of a previously received audio signal with respect to the occluding subframes, estimating the phase of each of the peaks, deriving a time-reversed phase adjustment based on the estimated phase, and applying it to the peaks of the signal spectrum to form a time-reversed phase-adjusted peak.

Description

[Technical Field]

[0001] This disclosure relates more broadly to communications, and more particularly to methods and apparatus for controlling packet loss concealment in relation to mono, stereo, or multi-channel audio encoding and decoding. [Background technology]

[0002] Modern telecommunications services generally provide reliable connectivity between end users. However, such services still need to deal with fluctuating channel conditions, in which data packets may occasionally be lost, for example, due to network congestion or poor cell coverage. To overcome the problems of transmission errors and lost packages, telecommunications services can utilize packet loss concealment (PLC) techniques. In cases where data packets are lost due to poor connectivity, network congestion, etc., the missing information of the lost packets at the receiver can be replaced by a synthesized signal in the decoder. PLC technology is often closely tied to the decoder, in which case it is possible to use internal states to bring about signal continuation or extrapolation to cover packet loss. For multimode codecs with several operating modes for various signal types, there are often several PLC technologies to handle concealment. There are many different terms used for packet loss concealment techniques, including frame error concealment (FEC), frame loss concealment (FLC), and error concealment unit (ECU).

[0003] For linear prediction (LP) based speech coding modes, the PLC can be based on adjusting the glottal pulse position using estimated frame-end pitch information and a copy of the pitch cycle from the previous frame [1]. The gain of the long-term predictor (LTP) converges to zero at a speed corresponding to the number of consecutive lost frames and the stability of the last good, i.e., error-free frame [2]. Frequency-domain (FD) based coding modes are designed to handle common or complex signals such as music. A variety of techniques can be used depending on the characteristics of the last received frame. Such analysis can include the number of timbre components detected and the periodicity of the signal. When frame loss occurs in highly periodic signals such as lively speech or music of a single instrument, a time-domain PLC similar to an LP-based PLC may be suitable. In this case, the FD PLC can mimic an LP decoder by estimating LP parameters and excitation signals based on the last received frame [2]. In cases where lost frames occur in aperiodic or noise-like signals, the last received frame can be repeated in the spectral domain, in which case the coefficients are multiplied together to reduce the metallic tone of the repeated signal, resulting in a random sine wave signal. For stationary timbral signals, it has been found to be advantageous to use an approach based on the prediction and extrapolation of detected timbral components. Further details on the techniques described above can be found in [1][2][3].

[0004] A common error masking method that works in the frequency domain is the phase ECU (error masking unit) [4]. The phase ECU is a standalone tool that operates on a buffer of previously decoded and reconstructed time-domain signals. The framework of the phase ECU is based on the sinusoidal analysis and synthesis paradigm, in which the sinusoidal component of the last good frame can be extracted and phase-shifted. If a frame is lost, the sinusoidal frequencies in the DFT (Discrete Fourier Transform) domain are obtained from a previously decoded synthesis. First, the corresponding frequency bins are identified by finding the peaks in the magnitude spectral plane. Then, the fractional frequencies of those peaks are estimated using the peak frequency bins. The frequency bins corresponding to those peaks, along with adjacent peaks, are phase-shifted using the fractional frequencies. For the remainder of the frame, the magnitude of the previous synthesis is preserved while the phase is randomized. Burst errors are also handled so that the estimated signal is smoothly muted by converging it to zero. Further details about the phase ECU can be found in [4].

[0005] The concept of a phase ECU can be used in decoders operating in the frequency domain. This concept includes an encoding / decoding system that performs decoding in the frequency domain, as shown in Figure 1, and a decoder that performs time-domain decoding with further frequency-domain processing, as shown in Figure 2. In Figure 1, a time-domain input audio signal (sub)frame is windowed at 100 and converted to the frequency domain by DFT 101. An encoder 102 performs encoding in the frequency domain and provides encoded parameters for transmission 103. A decoder 104 decodes the received frame or applies PLC 109 in the case of frame loss. In constructing a hidden frame, the PLC can use memory 108 of previously decoded frames. The decoded or hidden frame is converted to the time domain by inverse DFT 110, and the output audio signal is then reconstructed by overlap addition 111. Figure 2 shows an encoder-decoder pair, in which case the decoder applies DFT transformation to facilitate frequency-domain processing. The received and decoded time-domain signal is first windowed frame by frame by frame 105, then converted to the frequency domain by DFT 106 for frequency domain processing 107, which can be performed either before or after PLC 109 (in the case of frame loss).

[0006] Since frequency-domain spectra have already been generated for each frame, the raw materials for the phase ECU can be easily obtained by simply storing the last decoded spectrum in memory. However, the efficiency of the algorithm may be reduced if the decoded spectra correspond to frames of time-domain signals with separate windowing functions (see Figure 1). This can occur if the decoder divides the composite frame into shorter subframes to handle transients that require higher time resolution, for example. To achieve good results, the ECU should generate the desired window shape for each frame; otherwise, transition artifacts may be present at each frame boundary. One solution is to store the spectra of each frame corresponding to a specific window and apply the ECU to them individually. Another solution may be to store a single spectrum for the ECU and correct the windowing in the time domain. This can be done by applying the inverse window and then reapplying the window with the desired shape. These solutions have several drawbacks, which are discussed below.

[0007] One drawback of applying frequency domain ECUs to individual subframes is that there may be differences between subframes that would be replicated for each subframe during a lost frame. For consecutive frame losses, this can lead to recurring artifacts because each subframe may have slightly different spectral signatures. Another issue is increased memory requirements because the spectrum of each subframe needs to be stored.

[0008] A window correction solution that reverses and reapplies the windowing process overcomes the problem of separate spectral signatures because the ECU can be based on a single subframe. However, applying the reversed window and then the new window involves division and multiplication for each sample, and in this case, the division is a computationally complex and costly operation. This solution can be improved by storing a pre-calculated correction window in memory, but this would increase the required table memory. In cases where the ECU is applied to the lower part of the spectrum, it may be necessary to correct the full spectrum as well, because the full spectrum needs to have the same window shape. [Overview of the project]

[0009] According to a first aspect, a method for generating concealed audio subframes of an audio signal in a decoding device is provided. This method includes generating a frequency spectrum on a subframe basis, wherein a sequence of subframes of an audio signal has the property that the applied window shape of a first subframe among those sequence of subframes is a mirrored or time-inverted version of a second subframe among those sequence of subframes. This method further includes detecting peaks in the signal spectrum of a previously received audio signal on a fractional frequency scale, estimating the phase of each of the peaks, and deriving a time-inverted phase adjustment to be applied to the peaks of the signal spectrum based on the estimated phases to form a time-inverted phase-adjusted peak. This method further includes applying the time inversion to the concealed audio subframes.

[0010] The potential benefit offered is that multiple subframe ECUs can be generated from a single subframe spectrum by applying inverted time synthesis. This generation may be suitable for cases where subframe windows are time-inverted versions of each other. Generating all ECU frames from a single stored decoded frame ensures that the subframes have similar spectral signatures while minimizing memory footprint and computational complexity.

[0011] According to a second embodiment, a decoder device configured to generate hidden audio subframes of an audio signal is provided. This decoder device is configured to generate a frequency spectrum on a subframe basis, wherein a sequence of subframes of an audio signal generates a frequency spectrum having the property that the applied window shape of a first subframe among those sequence of subframes is a mirrored or time-inverted version of a second subframe among those sequence of subframes. This decoder device is further configured to detect peaks in the signal spectrum of a previously received audio signal on a fractional frequency scale and to estimate the phase of each of the peaks. This decoder device is further configured to derive a time-inverted phase adjustment to apply to the peaks of the signal spectrum based on the estimated phase and to form time-inverted phase-adjusted peaks by applying the time-inverted phase adjustment to the peaks of the signal spectrum. This decoder device is further configured to apply the time inversion to the hidden audio subframes.

[0012] According to a third aspect, a computer program is provided. This computer program includes program code which will be executed by a processing circuit of a decoder device configured to operate in a communication network, thereby causing the decoder device to perform the operation according to the first aspect.

[0013] According to a fourth aspect, a computer program product is provided. This computer program product includes a non-temporary storage medium containing program code which will be executed by a processing circuit of a decoder device configured to operate in a communication network, thereby causing the decoder device to perform the operation according to the first aspect.

[0014] A fifth aspect provides a method for generating hidden audio subframes relating to an audio signal in a decoding device. This method includes generating a frequency spectrum on a subframe basis, wherein a sequence of subframes of an audio signal has the property that the applied window shape of a first subframe among those sequence of subframes is a mirrored or time-reversed version of a second subframe among those sequence of subframes. The signal spectrum corresponding to the second subframe of the first two sequence of subframes is stored. This method further includes receiving a bad frame indicator relating to the second two sequence of subframes. This method further includes acquiring the signal spectrum, detecting the peaks of the signal spectrum on a fractional frequency scale, estimating the phase of each of the peaks, and deriving a time-reversed phase adjustment to apply to the stored spectrum peak with respect to the first subframe of the second two sequence of subframes, based on the estimated phases. This method further includes applying the time-reversed phase adjustment to the peak of the signal spectrum to form a time-reversed phase-adjusted peak. This method further includes applying time inversion to a hidden audio subframe, combining the time-inverted, phase-adjusted peak with the noise spectrum of the signal spectrum to form a combined spectrum relating to the first subframe of a second set of two consecutive subframes, and generating a synthesized hidden audio subframe based on the combined spectrum.

[0015] According to a sixth aspect, a decoder device configured to generate hidden audio subframes of an audio signal is shown. This decoder device includes a processing circuit and a memory operably coupled to the processing circuit, which includes instructions, which, when executed by the processing circuit, cause the decoder device to perform an operation according to the first or fifth aspect.

[0016] According to a seventh aspect, a decoder device is provided. This decoder device is configured to generate hidden audio subframes of an audio signal, and is adapted to perform the method according to a fifth aspect.

[0017] According to the eighth aspect, a computer program is provided. This computer program includes program code which will be executed by a processing circuit of a decoder device configured to operate in a communication network, thereby causing the decoder device to perform the operation according to the fifth aspect.

[0018] According to the ninth aspect, a computer program product is provided. This computer program product includes a non-temporary storage medium containing program code which will be executed by a processing circuit of a decoder device configured to operate in a communication network, thereby causing the decoder device to perform an operation according to the fifth aspect.

[0019] The accompanying drawings, included to provide a further understanding of this disclosure and incorporated into and constituting part thereof, illustrate certain non-limiting embodiments. [Brief explanation of the drawing]

[0020] [Figure 1] This is a block diagram showing an encoder-decoder pair, in which case encoding is performed in the DFT domain. [Figure 2] This is a block diagram showing an encoder-decoder pair, in which case the decoder applies a DFT transform to facilitate frequency domain processing. [Figure 3]This diagram shows two subframe windows of the decoder, where the window applied to the second subframe is a time-reversed or mirrored version of the window applied to the first subframe. [Figure 4] This block diagram shows an encoder / decoder system including a PLC method that performs phase estimation and applies ECU synthesis at inversion time using a time-reversal phase computer, according to several embodiments. [Figure 5] This flowchart shows the operation of a decoder device that performs time-reverse ECU synthesis according to several embodiments. [Figure 6] This is a diagram of a time-reversal window on a sine wave according to several embodiments. [Figure 7] This figure illustrates how the inversion time window affects the DFT coefficients in the complex plane in several embodiments. [Figure 8] This is a diagram showing φε versus frequency f in several embodiments. [Figure 9] This is a block diagram showing a decoder device according to several embodiments. [Figure 10] This flowchart shows the operation of a decoder device according to several embodiments. [Figure 11] This flowchart shows the operation of a decoder device according to several embodiments. [Modes for carrying out the invention]

[0021] Aspects of the present disclosure are then described more fully with reference to the accompanying drawings, which illustrate examples of embodiments. However, embodiments can be embodied in many different forms and should not be construed as being limited to the embodiments shown herein. Rather, these embodiments are provided so that the present disclosure may be thorough and complete, and so as to fully convey the scope of these embodiments to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be implicitly assumed to exist / be used in another embodiment.

[0022] The following description presents various embodiments of the disclosed subject matter. These embodiments are presented as teaching examples and should not be construed as limiting the scope of the disclosed subject matter. For example, certain details of the embodiments described may be modified, omitted, or expanded without departing from the scope of the described subject matter.

[0023] Figure 9 is a block diagram showing elements of a decoder device 900 configured to provide wireless communication according to an embodiment (which can be part of a mobile terminal, mobile communication terminal, wireless communication device, wireless terminal, wireless communication terminal, user equipment (UE), user equipment node / terminal / device, etc.). As shown, the decoder 900 may include a network interface circuit 906 (also called a network interface) configured to provide communication with other devices / entities / functions, etc. The decoder 900 may also include a processor circuit 902 (also called a processor) operably coupled to the network interface circuit 906 and a memory circuit 904 (also called memory) operably coupled to the processor circuit. The memory circuit 904 may contain computer-readable program code which, when executed by the processor circuit 902, causes the processor circuit to perform operations according to the embodiments disclosed herein.

[0024] In other embodiments, the processor circuit 902 may be defined to include memory, thereby eliminating the need for a separate memory circuit. As discussed herein, the operation of the decoder 900 can be performed by the processor 902 and / or the network interface 906. For example, the processor 902 can control the network interface 906 to send communications to a multichannel audio player and / or to receive communications through the network interface 906 from one or more other network nodes / entities / servers such as encoder nodes, depositary servers, etc. Furthermore, modules can be stored in the memory 904, and these modules can provide instructions, thereby allowing the processor 902 to perform their respective operations when the instructions of the modules are executed by the processor 902.

[0025] In the following descriptions, subframe notation will be used to describe embodiments. Here, a subframe means a part of a larger frame, in this case, the larger frame consists of a set of subframes. The embodiments described may also be used with frame notation. In other words, subframes can form a group of frames having the same window shape as described herein, and subframes do not need to be part of a larger frame.

[0026] Consider the decoder in an encoder-decoder pair when the decoding method generates a frequency spectrum on a subframe basis. Sequential subframes may have the characteristic that the applied window shape is a mirrored or time-reversed version of each other, as shown in Figure 3, in which case subframe 2 is a mirrored or time-reversed version of subframe 1. The decoder reconstructs the subframe for each frame m. Obtain the spectrum of TIFF2026053413000002.tif8170. In one embodiment, the subframe spectrum is reconstructed using time-domain synthesis. It can be obtained from TIFF2026053413000003.tif6170, in which case n is the sample index. The dashed box in Figure 2 indicates that frequency domain processing can be performed either before or after the memory and PLC module. Using the subframe window processing functions w1(n) and w2(n) Multiply TIFF2026053413000004.tif6170, The spectrum can be obtained by applying a DFT transformation according to TIFF2026053413000005.tif34170, in which case N indicates the length of the subframe window, and N step12is the distance between samples between the starting points of the first and second subframes. The subframe windowing functions w1(n) and w2(n) are mirrored or time-reversed versions of each other. Here, the subframe spectrum is obtained from decoder time-domain synthesis, as in the system outlined in Figure 2. Note that these embodiments are equally applicable to systems in which the decoder directly reconstructs the subframe spectrum, as outlined in Figure 1. For each correctly received and decoded audio frame m, the second subframe The spectrum corresponding to TIFF2026053413000006.tif7170 is stored in memory. TIFF2026053413000007.tif7170

[0027] For frames that are received correctly, the decoder device 900 can proceed to perform frequency domain processing steps to perform an inverse DFT transform and reconstruct the output audio using an overlap summing strategy. Missing or corrupted frames can be identified by the transport layer handling the connection and signaled to the decoder as "bad frames" through a Bad Frame Indicator (BFI), which can be in the form of a flag. If the decoder device 900 detects a bad frame through the Bad Frame Indicator (BFI), the PLC algorithm is activated. The PLC follows the principle of phase ECU.[4] Stored spectrum TIFF2026053413000008.tif7170 is input to a peak detector algorithm that detects peaks on a fractional frequency scale. Peak setting F={f i},i=1,2,…N peaks It is possible to detect these, and they have their estimated fractional frequency f i It is represented by, in this case, N peaksis the number of detected peaks. Similar to the sinusoidal coding paradigm, the peaks of the spectrum are modeled using sinusoids with specific amplitudes, frequencies, and phases. Fractional frequencies can be represented as fractions of DFT bins, so that, for example, the Nyquist frequency is found at f = N / 2 + 1. Each peak can be associated with the number of frequency bins that represent that peak. These are rounded to the nearest integer for fractional frequencies, such as TIFF2026053413000009.tif13170, and found by including N near peaks on each side, where [·] represents the rounding operation and G i is the group of bins representing the peak at frequency f i . The number N near is a tuning constant that can be specified when designing the system. A larger N near provides higher accuracy in each peak representation but also results in a larger distance between the peaks that can be modeled. Appropriate values for N near may be 1 or 2. The peaks of the hidden spectrum TIFF2026053413000010.tif7170 can be formed by using these groups of bins, where a phase adjustment is applied to each group. The phase adjustment takes into account the change in phase in the underlying sinusoid, assuming that the frequency remains the same between the last correctly received and decoded frame and the hidden frame. The phase adjustment is based on the fractional frequency and the number of samples between the analysis frame of the previous frame and where the current frame will start. As shown in Figure 3, this number of samples is N step21 between the start of the second subframe of the last received frame and the start of the first subframe of the first ECU frame, and N full between the start of the first subframe of the last received frame and the start of the first subframe of the first ECU frame. Nfull It should also be noted that this gives the distance between the second subframe of the last received frame and the second subframe of the first ECU frame.

[0028] Figure 4 shows an encoder / decoder system according to an embodiment described later, in which a PLC block 109 performs phase estimation using a phase estimator 112 and applies ECU synthesis at inversion time using a time-reversal phase computer 113.

[0029] Figure 5 is a flowchart showing the steps of the time-reversed ECU synthesis described below. To conceal the first subframe, the ECU synthesis is performed in reverse time to obtain the desired window shape. Phase adjustment, or phase correction or phase advance (these terms are used interchangeably throughout this description), with respect to the first subframe with respect to peak i is performed. Δφ i = -2φ i -2πf i (N+N step21 +(N lost -1)N full ) / N It can be written as, in this case, N lost This indicates the number of consecutive lost frames, φ i is, frequency f i This shows the phase of the sine wave at (N). lost -1)N full This term deals with phase progression related to burst errors, in which case the step is the frame length N of the full frame. full It is incremented by N for the first lost frame. lost = 1. Spectrum Regarding the frequencies centered on the frequency bins of TIFF2026053413000011.tif7170, the phase φ i is simply an angle It is easily available by extracting TIFF2026053413000012.tif7170, in this case, k i =[fi ]

[0030] Generally, frequency f i This is a fraction, and the phase needs to be estimated in Operation 501. One estimation method is to use linear interpolation of the phase spectrum. TIFF2026053413000013.tif11170 In this case, TIFF2026053413000014.tif6170 represents the operators for truncation and rounding up, respectively. However, this estimation method has been found to be unstable. This estimation method further requires two-phase extraction, which requires a computationally complex arctan function in the case where the spectrum is represented using complex numbers in the standard form a+bi. Another phase estimation method that has been found to be reliable with relatively low computational complexity is: TIFF2026053413000015.tif12170f frac =f i -k i And in this case, f frac φ is a rounding error. C is an adjustment constant that depends on the window shape to be applied. For the window shape of this embodiment, the appropriate value is φ C It was found that the value is =0.33. For other window shapes, the appropriate value is φ C It was found to be = 0.48. In general, it is expected that a suitable value can be found in the range [0.1, 0.7]. In Operation 502, the time-reversed phase adjustment Δφ i However, it is derived as explained above.

[0031] The peaks in the occlusion spectrum can be formed in Operation 503 by applying phase adjustment to the stored spectrum. The asterisk "*" in TIFF2026053413000016.tif12170 indicates the complex conjugate, which gives a time reversal of the signal in operation 504. This results in a time reversal of the first ECU subframe. Note that it may also be possible to perform a reversal in the time domain after the inverse DFT. However, If TIFF2026053413000017.tif7170 represents only a portion of the complete spectrum, this requires that the remaining spectrum be preprocessed, for example, by time reversal before DFT analysis.

[0032] Peak Bin G i Not occupied by The remaining bins in TIFF2026053413000018.tif7170 may be called the noise spectrum or noise components of the spectrum. They can be populated using coefficients of the stored spectrum with random phases applied. TIFF2026053413000019.tif12170 In this case, φ rand This represents a random phase value. The remaining bins can also be populated with spectral coefficients that preserve the desired characteristics of the signal, such as the correlation with the second channel in a multi-channel decoder system. In operation 505, the peak spectrum TIFF2026053413000020.tif7170(In this case, k∈G i ) but the noise spectrum TIFF2026053413000021.tif7170(In this case, It is combined with TIFF2026053413000022.tif6170 to form a combined spectrum.

[0033] In embodiments where noise is generated in the time domain, windowed, and transformed, time reversal of the noise to match the windowing of peak components and the combination with the peak spectrum should be performed before applying the time reversal described above.

[0034] For the generation of the second subframe, which is synthesized at normal (uninverted) time, ordinary phase adjustment can be used. Δφ i =2πf i N full N lost / N

[0035] The ECU synthesis for the second subframe can be formed in the same way as for the first subframe, but the complex conjugate of the peak coefficient is omitted. TIFF2026053413000023.tif13170

[0036] Once the combined opacity spectrum is generated in operation 505, it can be fed into subsequent processing steps in operation 506, which include an inverse DFT and an overlap sum operation that yields the output audio signal.

[0037] The output audio signal can be sent to one or more speakers, such as loudspeakers, for playback. These speakers can be part of the decoding device, a separate device, or part of another device.

[0038] Derivation of the phase correction formula for time-reverse ECU synthesis Assume that the starting phase of the sinusoidal component is φ0 and the frequency of the sinusoidal wave is f. step The desired phase φ1 of the sine wave after advancing by this many samples is as follows: φ1 = φ0 + 2πfN step / N

[0039] For the time-reversed continuation of a sine wave, the phase needs to be mirrored on the real axis by applying the complex conjugate or simply by taking the negative phase -φ1. Since this phase angle now represents the endpoint of the ECU composite frame, the phase needs to be rewound by the length of the analysis frame in order to reach the desired starting phase φ2. φ2 = -φ1 - 2πf(N-1) / N

[0040] In order to obtain the phase correction Δφ, the starting phase must be subtracted, as follows: φ0 + Δφ = φ2 ⇒ Δφ = φ2 - φ0

[0041] Replacing φ2 gives the following: Δφ = -2φ0 - 2πf(N step (+N-1) / N

[0042] To add a sequence of consecutive frame losses (burst losses), it is possible to add a coefficient corresponding to the number of samples between the starting points of full frames, N offset =(N lost -1)N full This provides the final phase correction. Δφ = -2φ0 - 2πf(N + N) step -1+(N lost -1)N full ) / N,

[0043] By using complex conjugate along with a one-sample cyclic shift, the desired time reversal can be achieved in the DFT domain. This cyclic shift can be performed with a 2πk / N phase correction, which can be included in the final phase correction. Δφ = -2φ0 - 2πf(N + N) step -1+(N lost -1)N full ) / N+2πk / N

[0044] Regarding the coefficient representing a single peak, the frequency bin k of the cyclic shift can be approximated by a fractional frequency k ≈ f, and the phase correction can be simplified as follows. Δφ = -2φ0 - 2πf(N + N) step -1+(N lost -1)N full ) / N+2πf / N= -2φ0-2πf(N+N step +(N lost -1)N full ) / N

[0045] The window is N=N full It is possible to design it in such a way that, in which case the formula can be further simplified as follows. Δφ = -2φ0 - 2πf(N step +N lost ·N) / N

[0046] Alternative Embodiment of Reversal Time ECU Synthesis In another embodiment, phase correction is performed in two steps. In the first step, the phase correction proceeds while ignoring window mismatches. TIFF2026053413000024.tif26170

[0047] In the second step, the phase is -φ m We return only that, apply the complex conjugate, and φ m By restoring the phase, it is possible to achieve time reversal in window processing. TIFF2026053413000025.tif8170

[0048] The motivation for this operation can be found by examining the effect of a time-reversal window on a sine wave, as shown in Figure 6. In Figure 6, the upper plot shows the window applied in the first direction, and the lower plot shows the window applied in the opposite direction. Three coefficients representing the sine wave are shown in Figure 7, illustrating how the reversal time window affects the DFT coefficients in the complex plane. The three DFT coefficients approximating the sine wave in the upper plot of Figure 6 are marked with circles, while the corresponding coefficients in the lower plot of Figure 6 are marked with stars. The diamonds indicate the original phase positions of the sine wave, and the dashed lines indicate the observed mirroring plane through which the coefficients of the time-reversal window are projected. The time-reversal window is angle φ m This provides mirroring of coefficients in the mirroring plane. φ m =φ0+φ frac

[0049] Through the experiment, φ frac It was found that it can be expressed as follows: φ frac =πf frac f frac =f i -k i k i =[f i ] In this case, [·] indicates rounding. φ is expressed as a positive angle. ε is, f frac It was also found that it is possible to approximate this by a linear relationship with φ. In Figure 8, the angle φ ε This is expressed as a function of frequency f. Observing the sawtooth shape in Figure 8, φ ε The following were found to be good approximations. φ ε =-f frac φ C In this case, Cis a constant. In one embodiment, φ C is φ C = 0.33 can be set, which produces a very close approximation. Since φ0 is not explicitly known, the alternating approximation of φ m can be written as follows. TIFF202605​​​​​​​​​​​​​​​​​​​In operation 1000, the processing circuit 902 generates a frequency spectrum on a subframe basis, wherein consecutive subframes of the audio signal have the characteristic that the applied window shape of the first subframe among those consecutive subframes is a mirrored or time-reversed version of the second subframe among those consecutive subframes. For example, generating a frequency spectrum for each of the first two consecutive subframes includes specifying the following: In this case, N represents the length of the subframe window, and the subframe window processing function w1(n) is the first subframe among the consecutive subframes. This is a subframe windowing function for TIFF2026053413000031.tif7170, where w2(n) is the second subframe of a sequence of subframes. This is a subframe windowing function for TIFF2026053413000032.tif7170, and N step12 This is the number of samples between the first of the first two consecutive subframes and the second of the first two consecutive subframes.

[0053] In operation 1002, processing circuit 902 determines whether a bad frame indicator (BFI) has been received. The bad frame indicator indicates that an audio frame is missing or corrupted.

[0054] In operation 1004, the processing circuit 902 stores in memory the spectrum corresponding to the second subframe for each correctly decoded audio frame. For example, for a correctly decoded frame m, the second subframe The spectrum corresponding to TIFF2026053413000033.tif7170 is: The file, such as TIFF2026053413000034.tif7170, is stored in memory. For correctly received frames, the decoder device 900 can proceed to perform frequency domain processing steps, as described above and shown in Figure 4, to perform an inverse DFT transform and reconstruct the output audio using an overlap summing strategy. Note that the principle of overlap summing is the same for both subframes and frames. Frame creation requires applying overlap summing on subframes, while the final output frame is the result of overlap summing operations between frames.

[0055] If the processing circuit 902 detects a faulty frame through the faulty frame indicator (BFI) in operation 1002, PLC operations 1006 to 1030 are executed.

[0056] In operation 1006, the processing circuit 902 obtains the signal spectrum corresponding to the second subframe of the first two consecutive subframes that were previously correctly decoded and processed. For example, the processing circuit 902 can obtain the signal spectrum from the memory 904 of the decoding device.

[0057] In operation 1008, the processing circuit 902 detects the peak of the signal spectrum of a previously received audio frame of the audio signal on a fractional frequency scale, and that previously received audio frame was received before the bad frame indicator was received.

[0058] In operation 1010, the processing circuit 902 determines whether the concealed frame is for the first of two consecutive subframes.

[0059] When the concealment frame is for the first sub-frame, in operation 1012, the processing circuit 902 estimates each phase of the peaks. In one embodiment, according to the following, the phase estimation regarding the peaks of the time-reversed and phase-corrected peaks is calculated. TIFF2026053413000035.tif21170 In this case, φ i is the estimated phase at frequency f i and, TIFF2026053413000036.tif7170 is the angle of the spectrum at frequency bin k i and, TIFF'2026053413000037.tif7170 is the angle of the spectrum at frequency bin k frac is the rounding error, φ C is the tuning constant, k i is [f i . The tuning constant φ C can be a value in the range between 0.1 and 0.7.

[0060] In operation 1014, the processing circuit 902 derives a time-reversed phase correction to apply to the peaks of the signal spectrum based on the estimated phase.

[0061] In operation 1016, the processing circuit 902 applies the time-reversed phase correction to the peaks of the signal spectrum to form time-reversed and phase-corrected peaks.

[0062] In operation 1018, the processing circuit 902 applies time reversal to the concealment audio sub-frame. In one embodiment, time reversal can be applied by applying the complex conjugate to the concealment audio sub-frame.

[0063] In operation 1020, the processing circuit 902 combines the time-reversed and phase-corrected peaks with the noise spectrum of the signal spectrum to form the combined spectrum of the concealment audio sub-frame.

[0064] Looking at Figure 11, in one embodiment, 1016 and 1018 can be performed by the processing circuit 902 associating each peak with a plurality of peak frequency bins in operation 1100. The processing circuit 902 that performs the association can apply time-inverted phase correction in operation 1102 by applying time-inverted phase correction to each of those plurality of frequency bins. In operation 1104, the remaining bins are populated using coefficients of the signal spectrum with random phase applied.

[0065] Returning to Figure 10, in operation 1022, the processing circuit 902 generates a synthesized hidden audio subframe based on the combined spectrum.

[0066] If the concealed frame is not for the first subframe when identified in operation 1010, the processing circuit 902 derives in operation 1024 an uninverted phase correction to be applied to the peak of the signal spectrum relating to the second concealed subframe of at least two consecutive concealed subframes.

[0067] In operation 1026, the processing circuit 902 applies a non-time-reversed phase correction to the peak of the signal spectrum relating to the second subframe to form a non-time-reversed phase-corrected peak.

[0068] In operation 1028, the processing circuit 902 combines the time-inverted, phase-corrected peak with the noise spectrum of the signal spectrum to form a combined spectrum relating to the second concealed subframe.

[0069] In operation 1030, the processing circuit 902 generates a second synthesized hidden audio subframe based on the combined spectrum.

[0070] Looking at Figure 11, in one embodiment, 1026 and 1028 can be performed by the processing circuit 902 associating each peak with a plurality of peak frequency bins in operation 1100. The processing circuit 902 performing the association can apply non-reversed phase correction in operation 1102 by applying non-reversed phase correction to each of those frequency bins. In operation 1104, the remaining bins are filled using coefficients of the signal spectrum with random phase applied.

[0071] The various operations from the flowchart in Figure 10 can be optional with respect to some embodiments of the decoder device and related methods. With respect to the method of exemplary embodiment 1 (shown hereafter), for example, the operations in blocks 1004 and 1022-1030 in Figure 10 can be optional. With respect to the method of exemplary embodiment 19 (shown hereafter), for example, the operations in blocks 1010 and 1022-1030 in Figure 10 can be optional.

[0072] Exemplary embodiments are discussed below.

[0073] 1. A method for generating hidden audio subframes of an audio signal in a decoding device, Generating a frequency spectrum on a subframe basis (1000), wherein a sequence of subframes of an audio signal has the property that the applied window shape of the first subframe among those sequence of subframes is a mirrored or time-reversed version of the second subframe among those sequence of subframes (1000), Receiving a faulty frame indicator (1002), Detecting the peak of the signal spectrum of a previously received audio frame of an audio signal on a fractional frequency scale (1008), wherein the previously received audio frame was received before the bad frame indicator was received, and detecting on a fractional frequency scale (1008), Estimating the phase of each of the peaks (1012), Deriving a time-reversed phase correction to apply to the peak of the signal spectrum based on the estimated phase (1014), Applying time-reversed phase correction to the peaks of the signal spectrum to form time-reversed phase-corrected peaks (1016), Applying time reversal to the hidden audio subframe (1018), The time-reversed, phase-corrected peak is combined with the noise spectrum of the signal spectrum to form a combined spectrum relating to the hidden audio subframe (1020), A method comprising generating a synthesized hidden audio subframe based on a combined spectrum (1022).

[0074] 2. The synthesized hidden audio frame includes at least two consecutive hidden subframes, and the following is performed with respect to the first hidden subframe of the at least two consecutive hidden subframes: deriving time-inverted phase correction, applying time-inverted phase correction, applying time inversion, and combining time-inverted phase-corrected peaks, and this method is further performed. To derive a time-inverted phase correction to apply to the peak of the signal spectrum relating to the second of at least two consecutive occlusion subframes (1024), Applying a non-time-reversed phase correction to the peak of the signal spectrum relating to the second subframe to form a non-time-reversed phase-corrected peak (1026), Combining the phase-corrected peaks that have not been time-reversed with the noise spectrum of the signal spectrum to form a combined spectrum relating to the second hidden subframe (1028), A method of Embodiment 1, comprising generating a second synthesized hidden audio subframe based on a combined spectrum (1030).

[0075] 3. The method of Embodiment 1 or 2, wherein the concealed audio subframe includes a concealed audio subframe relating to one of the lost audio frames and the corrupted audio frames.

[0076] 4. Any one of embodiments 1 to 3, wherein a faulty frame indicator provides an indication that an audio frame is missing or corrupted.

[0077] 5. Any one of embodiments 1 to 4, further comprising obtaining the signal spectrum of a previously received audio signal frame from the decoder's memory.

[0078] 6. Any one of Embodiments 1 to 5, wherein applying time inversion includes applying complex conjugate to the hidden audio subframe.

[0079] 7. Associating each of the multiple peaks with a peak frequency bin representing the peak (1100) A method which further includes any one of embodiments 1 to 6.

[0080] 8. The method of Embodiment 7, wherein for each of the multiple peaks, either time-reversed phase correction or time-unreversed phase correction is applied to the peak (1102).

[0081] 9. Fill the remaining bins of the signal spectrum using the coefficients of the stored signal spectrum with a random phase applied (1104). Any one of the methods of Embodiment 8, further including the following.

[0082] 10. Estimating the phase of each of the peaks is This includes calculating a phase estimate for the peak of a time-reversed, phase-corrected peak, as described below: TIFF2026053413000038.tif20170 In this case, φ i However, frequency f i This is the estimated phase at TIFF2026053413000039.tif7170 is a frequency bin k i Spectrum This is the angle of TIFF2026053413000040.tif7170, and f frac This is the rounding error, φ C The adjustment constant is k i ga [f i One of the methods of Embodiments 1 to 9, wherein the method is as follows:

[0083] 11. φ C The method of Embodiment 10, wherein the value is in the range between 0.1 and 0.7.

[0084] 12. The phase estimate for the phase-corrected peak that has not been time-reversed is calculated according to the following: Δφ i =2πf i N full N lost / N In this case, Δ φ i However, frequency f i This shows the phase correction of a sine wave at N full However, this indicates the number of samples between two frames, N lost The method of Embodiment 10, wherein n indicates the number of consecutive lost frames, and N indicates the length of the subframe window.

[0085] 13. Any one of embodiments 1 to 12, further comprising applying a random phase to the noise spectrum of the signal spectrum.

[0086] 14. The method of Embodiment 13, wherein applying a random phase to the noise spectrum includes applying a random phase to the noise spectrum before combining the noise spectrum with a time-reversed, phase-adjusted peak.

[0087] 15. A decoder device (900) configured to generate hidden audio subframes of a received audio signal, wherein the decoding method of the decoding device generates a frequency spectrum on a subframe basis, in which consecutive subframes have the characteristic that the applied window shape is a mirrored or time-reversed version of each other, and this decoder device Processing circuit (902), A decoder device (900) including a memory (904) coupled to the processing circuit, which includes instructions, and when those instructions are executed by the processing circuit, causes the decoder device to perform an operation according to any one of embodiments 1 to 14.

[0088] 16. A decoder device (900) configured to generate concealed audio subframes of a received audio signal, wherein the decoding method of the decoding device generates a frequency spectrum on a subframe basis, wherein consecutive subframes are characterized in that the applied window shape is a mirrored or time-reversed version of the other, and the decoder device (900) is adapted to perform according to any one of embodiments 1 to 14.

[0089] 17. A computer program including program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform an operation according to any one of embodiments 1 to 14.

[0090] 18. A computer program product including a non-temporary storage medium containing program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform an operation according to any one of embodiments 1 to 14.

[0091] 19. A method for generating hidden audio subframes relating to an audio signal in a decoding device, Generating a frequency spectrum on a subframe basis (1000), wherein a sequence of subframes of an audio signal has the property that the applied window shape of the first subframe among those sequence of subframes is a mirrored or time-reversed version of the second subframe among those sequence of subframes (1000), To store the signal spectrum corresponding to the second subframe of the first two consecutive subframes (1004), Receiving a bad frame indicator for two consecutive subframes (1002), Obtaining the signal spectrum (1006), Detecting the peaks in the signal spectrum on a fractional frequency scale (1008), Estimating the phase of each of the peaks (1012), Based on the estimated phase, derive a time-reversed phase correction to apply to the peak of the spectrum stored with respect to the first of two consecutive subframes (1014), Applying time-reversed phase correction to the peaks of the signal spectrum to form time-reversed phase-corrected peaks (1016), Applying time reversal to the hidden audio subframe (1018), The time-reversed, phase-corrected peak is combined with the noise spectrum of the signal spectrum to form a combined spectrum relating to the first subframe of the second set of two consecutive subframes (1020), A method comprising generating a synthesized hidden audio subframe based on a combined spectrum (1022).

[0092] 20. The synthesized hidden audio frame includes at least two consecutive hidden subframes, and the process of deriving time-inverted phase correction, applying time-inverted phase correction, and combining time-inverted phase-corrected peaks is performed with respect to the first hidden subframe of at least two consecutive hidden subframes, and this method is further performed To derive a time-unreversed phase correction to apply to the peak of the signal spectrum relating to the second subframe of the second set of two consecutive subframes (1024), Applying a non-time-reversed phase correction to the peak of the signal spectrum relating to the second subframe of the second set of two consecutive subframes to form a non-time-reversed phase-corrected peak (1026), Combining the audio subframes that have not been time-reversed with the noise spectrum of the signal spectrum to form a second combined spectrum relating to the second subframe of the second set of two consecutive subframes (1028), The method of Embodiment 19, comprising generating a second synthesized audio subframe based on a second combined spectrum (1030).

[0093] 21. The method of Embodiment 19 or 20, wherein the concealed audio subframe includes a concealed audio subframe relating to one of the lost audio frames and the corrupted audio frames.

[0094] 22. Any one of embodiments 19 to 21, wherein a faulty frame indicator provides an indication that an audio frame is missing or corrupted.

[0095] 23. Any one of embodiments 19 to 22, further comprising obtaining the signal spectrum from the decoder's memory.

[0096] 24. Any one of embodiments 19 to 23, wherein applying time inversion includes applying complex conjugate to the hidden audio subframe.

[0097] 25. Associate each peak with multiple peak frequency bins that represent the peak. Any one of embodiments 18 to 24 further includes the following.

[0098] 26. The method of Embodiment 25, further comprising applying either time-reversed phase correction or time-unreversed phase correction to each of the multiple peaks.

[0099] 27. Using the coefficients of the stored spectrum, which have a random phase applied, to fill in the remaining bins of the signal spectrum. Any one of the methods of Embodiment 26, further including the following.

[0100] 28. Estimating the phase is This includes calculating a phase estimate for a time-reversed, phase-corrected peak, as described below: TIFF2026053413000041.tif9170f frac =f i -k i In this case, i However, frequency f i This is the estimated phase at TIFF2026053413000042.tif7170 has a frequency f i Spectrum This is the angle of TIFF2026053413000043.tif7170, and f frac This is the rounding error, φ C The adjustment constant is k i ga [f i One of the embodiments 19 to 27, wherein the method is as follows:

[0101] 29. φ C However, the method of Embodiment 28 has a value in the range between 0.1 and 0.7.

[0102] 30. Further includes calculating the phase estimate for a phase-corrected peak that has not been time-reversed, according to the following: Δφ i =2πf i N full N lost / N In this case, Δ φ i However, frequency f i This shows the phase correction of a sine wave at N full However, this indicates the number of frame samples between two frames, N lost The method of Embodiment 28, where n indicates the number of consecutive lost frames, and N indicates the length of the subframe window.

[0103] 31. Generating frequency spectra for each of the first two consecutive subframes includes identifying the following: TIFF2026053413000044.tif35170 In this case, N indicates the length of the subframe window, and the subframe window processing function w1(n) is the first subframe among the consecutive subframes. This is a subframe windowing function for TIFF2026053413000045.tif7170, where w2(n) is the second subframe of a series of subframes. This is a subframe windowing function for TIFF2026053413000046.tif7170, and N step12 The method according to any one of embodiments 19 to 30, wherein the number of samples is between the first subframe of the first two consecutive subframes and the second subframe of the first two consecutive subframes.

[0104] 32. Any one of the embodiments 19 to 31, further comprising applying a random phase to the noise spectrum of the signal spectrum.

[0105] 33. The method of Embodiment 32, wherein applying a random phase to the noise spectrum includes applying a random phase to the noise spectrum before combining it with a time-reversed, phase-adjusted peak.

[0106] 34. A decoder device (900) configured to generate hidden audio subframes of a received audio signal, wherein the decoding method of the decoding device generates a frequency spectrum on a subframe basis, in which consecutive subframes have the characteristic that the applied window shape is a mirrored or time-reversed version of each other, and this decoder device Processing circuit (902), A decoder device (900) including a memory (904) coupled to the processing circuit, the memory (904) containing instructions, which, when executed by the processing circuit, cause the decoder device to perform an operation according to any one of embodiments 19 to 33.

[0107] 35. A decoder device (900) configured to generate concealed audio subframes of a received audio signal, wherein the decoding method of the decoding device (900) generates a frequency spectrum on a subframe basis, wherein consecutive subframes are characterized in that the applied window shape is a mirrored or time-reversed version of the applied subframe, and the decoder device (900) is adapted to perform according to any one of embodiments 19 to 33.

[0108] 36. A computer program including program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform an operation according to any one of embodiments 19 to 33.

[0109] 37. A computer program product including a non-temporary storage medium containing program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform an operation according to any one of embodiments 19 to 33.

[0110] Explanations of the various abbreviations and acronyms used in this disclosure are provided below. Abbreviations and Explanations DFT (Discrete Fourier Transform) IDFT (Inverse Discrete Fourier Transform) LP Linear Prediction PLC packet loss concealment ECU Error Concealment Unit FEC Frame Error Correction / Concealment

[0111] The references are listed below. [1] T. Vaillancourt, M. Jelinek, R. Salami and R. Lefebvre, "Efficient Frame Erasure Concealment in Predictive Speech Codecs using Glottal Pulse Resynchronisation," 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07, Honolulu, HI, 2007, pp. IV-1113-IV-1116. [2] J. Lecomte et al., "Packet-loss concealment technology advances in EVS," 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QLD, 2015, pp. 5708-5712. [3] 3GPP TS 26.447, Codec for Enhanced Voice Services (EVS); Error Concealment of Lost Packets (Release 12) [4] S. Bruhn, E. Norvell, J. Svedberg and S. Sverrisson, "A novel sinusoidal approach to audio signal frame loss concealment and its application in the new evs codec standard," 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QLD, 2015, pp. 5142-5146.

[0112] In general, all terms used herein should be interpreted according to their common meanings in the relevant art (unless otherwise explicitly stated, and / or implied by the context in which the term is used). All references to elements, apparatus, components, means, steps, etc., should be openly interpreted as referring to at least one example of an element, apparatus, component, means, step, etc., unless otherwise explicitly stated. The steps of any method disclosed herein do not need to be performed in the strict order disclosed (unless otherwise explicitly stated, and / or implied, that a step must follow or precede another). Any feature of any embodiment disclosed herein can, where appropriate, be applied to any other embodiment. Similarly, any advantage of any embodiment can apply to any other embodiment, and vice versa. Other purposes, features, and advantages of the embodiments included will be apparent from the following description.

[0113] In the above-mentioned descriptions of various embodiments, it should be understood that the terms used herein are for the purpose of describing specific embodiments only and are not intended to be limiting. Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as they would be generally understood by a standard technician in the art to which this disclosure belongs. It should be further understood that terms such as those defined in commonly used dictionaries should be interpreted in a way that is consistent with their meaning in the context of this specification and the relevant art, and not to be interpreted in an idealized or overly formal sense (unless such a provision is explicitly made herein).

[0114] Where an element is referred to as “connected,” “combined,” “responding to,” or a variation thereof, to another element, that element may be directly connected, combined, or responding to any other element or intervening element that may be present. In contrast, where an element is referred to as “directly connected,” “directly combined,” “directly responding to,” or a variation thereof, to another element, there are no intervening elements present. Similar numbers refer to similar elements throughout. Furthermore, as used herein, “combined,” “connected,” “responding,” or their variations may include being wirelessly combined, connected, or responding. Where used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (unless the context clearly indicates otherwise). Well-known features or structures may not be described in detail for the sake of brevity and / or clarity. The term “and / or” includes any combination of one or more of the enumerated items being associated.

[0115] While terms such as "first," "second," and "third" may be used herein to describe various elements / operations, it will be understood that these elements / operations should not be limited by these terms. These terms are used merely to distinguish one element / operation from another. Thus, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments without departing from the teachings of this disclosure. The same reference number or the same reference designator refers to the same or similar elements throughout this specification.

[0116] As used herein, the terms “comprise,” “comprising,” “comprises,” “include,” “including,” “includes,” “have,” “has,” and “having,” or their variations thereof, are open-ended and include one or more described features, completes, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, completes, elements, steps, components, functions, or groups thereof. Furthermore, as used herein, the general abbreviation “eg,” derived from the Latin phrase “exempli gratia,” may be used to introduce or specify one or more general examples of the aforementioned items and is not intended to be an limitation on such items. The general abbreviation “ie,” derived from the Latin phrase “id est,” may be used to specify a particular item from a more general enumeration.

[0117] In this specification, exemplary embodiments are described with reference to block diagrams and / or flowcharts of computer implementations, apparatus (systems and / or devices), and / or computer program products. It is understood that blocks in block diagrams and / or flowcharts, and combinations of blocks in block diagrams and / or flowcharts, can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to processor circuits of general-purpose computer circuits, dedicated computer circuits, and / or other programmable data processing circuits to create machines, thereby enabling instructions executed via the processor of a computer and / or other programmable data processing device to translate and control transistors, values ​​stored in memory locations, and other hardware components in such circuits to perform functions / actions specified in block diagrams and / or flowchart blocks, and thereby create means (functionality) and / or structures for performing functions / actions specified in block diagrams and / or flowchart blocks.

[0118] These computer program instructions may also be stored in tangible computer-readable media that can instruct a computer or other programmable data processing device to function in a particular manner, thereby producing a product containing instructions that perform functions / actions specified in block diagrams and / or one or more flowchart blocks. Embodiments of the present disclosure can therefore be embodied in hardware and / or in software (including firmware, resident software, microcode, etc.) running on a processor such as a digital signal processor, which may collectively be referred to as “circuits,” “modules,” or variations thereof.

[0119] It should also be noted that in some alternative embodiments, the functions / actions described in a block may occur in a different order than those described in the flowchart. For example, two consecutively shown blocks may actually be executed substantially simultaneously, or they may be executed in the opposite order depending on the functions / actions they contain. Furthermore, the functionality of a given block in a flowchart and / or block diagram may be separated into multiple blocks, and / or the functionality of two or more blocks in a flowchart and / or block diagram may be integrated at least partially. Finally, without departing from the scope of the embodiment, other blocks may be added / inserted between shown blocks, and / or blocks / operations may be omitted. Moreover, while some of the diagrams include arrows on the communication path to indicate the primary direction of communication, it should be understood that communication may occur in the opposite direction to the indicated arrows.

[0120] Many variations and modifications can be made to the embodiments without substantially departing from the principles of this disclosure. All such variations and modifications are intended to be included in the scope of this disclosure as herein. Accordingly, the subject matter disclosed above should be considered illustrative rather than restrictive, and the examples of embodiments are intended to cover all such modifications, enhancements, and other embodiments that fall within the spirit and scope of this disclosure. Therefore, to the maximum extent permitted by law, the scope of this disclosure should be specified by the broadest acceptable interpretation of this disclosure, including examples of embodiments and their equivalents, and not limited or restricted by the detailed description above.

Claims

1. A method for generating hidden audio subframes of an audio signal in a decoding device, When a series of subframes of the audio signal has the characteristic that the applied window shape of the first subframe among the series of subframes is a mirrored or time-reversed version of the second subframe among the series of subframes, generating a frequency spectrum on a subframe basis (1000), Detecting the peaks in the signal spectrum of previously received audio signals on a fractional frequency scale (1008), Estimating the phase of each of the aforementioned peaks (1012), (1014) Deriving a time-reversed phase adjustment to apply to the peak of the signal spectrum based on the estimated phase, Applying the time-reversed phase adjustment to the peak of the signal spectrum to form a time-reversed phase-adjusted peak (1016), A method comprising applying time reversal to the concealed audio subframe (1018).

2. (1020) The time-reversed and phase-adjusted peak is combined with the noise spectrum of the signal spectrum to form a combined spectrum relating to the hidden audio subframe. Based on the aforementioned combined spectrum, a synthesized hidden audio subframe is generated (1022) The method according to claim 1, further comprising:

3. The synthesized hidden audio frame includes at least two consecutive hidden subframes, and the process of deriving the time-inverted phase adjustment, applying the time-inverted phase adjustment, applying the time inversion, and combining the time-inverted phase-adjusted peaks is performed with respect to the first hidden subframe of the at least two consecutive hidden subframes, and the method further, To derive a time-inverted phase adjustment to apply to the peak of the signal spectrum relating to the second concealed subframe of the at least two consecutive concealed subframes (1024), Applying the non-time-reversed phase adjustment to the peak of the signal spectrum relating to the second subframe to form a non-time-reversed phase-adjusted peak (1026), The phase-adjusted peak that has not been time-reversed is combined with the noise spectrum of the signal spectrum to form a combined spectrum relating to the second concealed subframe (1028), The method according to claim 1 or 2, further comprising generating a second synthesized hidden audio subframe based on the combined spectrum (1030).

4. The method according to any one of claims 1 to 3, further comprising obtaining the signal spectrum of the previously received audio signal from the memory of the decoding device (1006).

5. The method according to any one of claims 1 to 4, wherein applying the time reversal includes applying a complex conjugate to the time-reversed phase-adjusted peak.

6. The method according to any one of claims 1 to 5, further comprising associating each of the detected peaks with a plurality of peak frequency bins representing the peak (1100).

7. The method according to claim 6, wherein with respect to each of the plurality of peak frequency bins, one of the time-inverted phase adjustment and the time-uninverted phase adjustment is applied to the peak frequency bin (1102).

8. Filling the remaining bins of the signal spectrum using the coefficients of the stored signal spectrum (1104), wherein the spectral coefficients maintain the desired characteristics of the signal (1104). The method according to claim 7, further comprising:

9. The method according to claim 8, wherein the desirable characteristics include a correlation with a second channel in a multi-channel decoder system.

10. Estimating the phase of each of the aforementioned peaks is This includes calculating a phase estimate of the time-reversed and phase-adjusted peak according to the following: f frac =f i -k i In this case, ｉ However, frequency f ｉ This is the estimated phase at However, frequency bin k ｉ Spectrum is the angle, and f ｆｒａｃ is the rounding error, and φ Ｃ is the tuning constant, and k ｉ is [f ｉ , the method according to any one of claims 1 to 9.

11. The phase adjustment with respect to the peak of the time-reversed hidden audio subframe is performed as follows: Df i =-2φ i -2πf i (N+N step21 +(N lost -1)N full ) / N The method according to claim 10, calculated according to the method described in claim 10.

12. The phase adjustment of the peak of the time-reversed hidden audio subframe is below Δφ=-2φ 0 -2πf(N step21 +N lost ・N) / N The method according to claim 10, calculated according to the method described in claim 10.

13. The method according to any one of claims 2 to 12, further comprising applying a random phase to the noise spectrum of the signal spectrum.

14. The method according to claim 13, wherein applying the random phase to the noise spectrum includes applying the random phase to the noise spectrum before combining the time-inverted, phase-adjusted peak with the noise spectrum.

15. A decoder device (900) configured to generate hidden audio subframes of an audio signal, Processing circuit (902), A decoder device (900) comprising a memory (904) operably coupled to the processing circuit, the memory (904) including an instruction, which, when the instruction is executed by the processing circuit, causes the decoder device to perform the operation described in any one of claims 1 to 14.

16. A decoder device (900) configured to generate hidden audio subframes of an audio signal, When a series of subframes of the audio signal have the characteristic that the window shape to which the first subframe of the series of subframes is applied is a mirrored or time-reversed version of the second subframe of the series of subframes, generating a frequency spectrum on a subframe basis, Detecting the peaks in the signal spectrum of previously received audio signals on a fractional frequency scale, To estimate the phase of each of the aforementioned peaks, Deriving a time-reversed phase adjustment to apply to the peak of the signal spectrum based on the estimated phase, Applying the time-reversed phase adjustment to the peak of the signal spectrum to form a time-reversed phase-adjusted peak, A decoder device (900) adapted to apply time reversal to the concealed audio subframe.

17. The time-reversed, phase-adjusted peak is combined with the noise spectrum of the signal spectrum to form a combined spectrum relating to the hidden audio subframe. Based on the aforementioned combined spectrum, a synthesized hidden audio subframe is generated. The decoder device according to claim 16, further adapted to perform the following:

18. The synthesized hidden audio frame includes at least two consecutive hidden subframes, and the deriving of the time-inverted phase adjustment, the application of the time-inverted phase adjustment, the application of the time inversion, and the combination of the time-inverted phase adjustment peaks are performed with respect to the first hidden subframe of the at least two consecutive hidden subframes, and the decoder device further, To derive a time-inverted phase adjustment to apply to the peak of the signal spectrum relating to the second concealed subframe of the at least two consecutive concealed subframes, Applying the non-time-reversed phase adjustment to the peak of the signal spectrum relating to the second subframe to form a non-time-reversed phase-adjusted peak, The phase-adjusted peak that has not been time-reversed is combined with the noise spectrum of the signal spectrum to form a combined spectrum relating to the second concealed subframe, The decoder device according to claim 16 or 17, which is adapted to generate a second synthesized hidden audio subframe based on the combined spectrum.

19. The decoder device according to any one of claims 16 to 18, further adapted to acquire the signal spectrum of the previously received audio signal from the memory of the decoder device.

20. A decoder device according to any one of claims 16 to 19, which is adapted to apply the time reversal by applying a complex conjugate to the time-reversed, phase-adjusted peak.

21. The decoder device according to any one of claims 16 to 20, further adapted to associate each of the detected peaks with a plurality of peak frequency bins representing the peak.

22. The decoder device according to claim 21, further adapted to apply either the time-reversed phase adjustment or the non-time-reversed phase adjustment to each of the multiple peak frequency bins.

23. Filling the remaining bins of the signal spectrum using the coefficients of the stored signal spectrum, wherein the spectral coefficients maintain the desired characteristics of the signal. The decoder device according to claim 22, further adapted to perform the following:

24. The decoder device according to claim 23, wherein the desirable characteristics include a correlation with a second channel in a multi-channel decoder system.

25. The system is adapted to estimate the phase of each of the peaks by calculating the phase estimate of the time-reversed and phase-adjusted peaks according to the following: f frac =f i -k i In this case, ｉ However, frequency f ｉ This is the estimated phase at However, frequency bin k ｉ Spectrum The angle is f ｆｒａｃ This is the rounding error, φ Ｃ The adjustment constant is k ｉ ga [f ｉ A decoder device according to any one of claims 16 to 24.

26. Phase adjustment with respect to the peak of the time-inverted hidden audio subframe Df i =-2φ i -2πf i (N+N step21 +(N lost -1)N full ) / N A decoder device according to claim 25, which is adapted to perform calculations according to the following:

27. Phase adjustment with respect to the peak of the time-inverted hidden audio subframe Δφ=-2φ 0 -2πf(N step21 +N lost ・N) / N A decoder device according to claim 25, which is adapted to perform calculations according to the following:

28. A decoder device according to any one of claims 16 to 27, further adapted to apply random phases to the noise spectrum of the signal spectrum.

29. The decoder device according to claim 28, further adapted to apply the random phase to the noise spectrum before combining the time-reversed, phase-adjusted peak with the noise spectrum.

30. A computer program comprising program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform the operation described in any one of claims 1 to 14.

31. A computer program product including a non-temporary storage medium containing program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform the operation described in any one of claims 1 to 14.

32. A method for generating hidden audio subframes for an audio signal in a decoding device, When a series of subframes of the audio signal has the characteristic that the applied window shape of the first subframe among the series of subframes is a mirrored or time-reversed version of the second subframe among the series of subframes, generating a frequency spectrum on a subframe basis (1000), (1004) storing the signal spectrum corresponding to the second subframe of the first two consecutive subframes, Receiving a bad frame indicator for two consecutive subframes (1002), The acquisition of the signal spectrum (1006), The peaks of the signal spectrum are detected on a fractional frequency scale (1008), Estimating the phase of each of the aforementioned peaks (1012), Based on the estimated phase, derive a time-reversed phase adjustment to apply to the peak of the spectrum stored with respect to the first subframe of the two second consecutive subframes (1014), Applying the time-reversed phase adjustment to the peak of the signal spectrum to form a time-reversed phase-adjusted peak (1016), Applying time reversal to the concealed audio subframe (1018), The time-reversed and phase-adjusted peak is combined with the noise spectrum of the signal spectrum to form a combined spectrum relating to the first subframe of the two second consecutive subframes (1020), A method comprising generating a synthesized hidden audio subframe based on the combined spectrum (1022).

33. The synthesized hidden audio frame includes at least two consecutive hidden subframes, and the process of deriving the time-inverted phase adjustment, applying the time-inverted phase adjustment, and combining the time-inverted phase adjustment peaks is performed with respect to the first hidden subframe of the at least two consecutive hidden subframes, and the method further, To derive a time-inverted phase adjustment to apply to the peak of the signal spectrum relating to the second subframe of the two consecutive subframes (1024), Applying the non-time-reversed phase adjustment to the peak of the signal spectrum relating to the second subframe of the two consecutive second subframes to form a non-time-reversed phase-adjusted peak (1026), The non-time-reversed audio subframe is combined with the noise spectrum of the signal spectrum to form a second combined spectrum relating to the second subframe of the two consecutive second subframes (1028), The method according to claim 32, further comprising generating a second synthesized audio subframe based on the second combined spectrum (1030).

34. The method according to claim 32 or 33, further comprising obtaining the signal spectrum from the memory of the decoding device.

35. The method according to any one of claims 32 to 34, wherein applying the time reversal includes applying a complex conjugate to the time-reversed phase-adjusted peak.

36. Each peak is associated with a plurality of peak frequency bins that represent the aforementioned peak. The method according to any one of claims 32 to 35, further comprising:

37. The method according to claim 36, further comprising applying either the time-inverted phase adjustment or the time-uninverted phase adjustment to each of the multiple peak frequency bins.

38. Filling the remaining bins of the signal spectrum using the stored spectral coefficients, wherein the spectral coefficients retain the desired characteristics of the signal. The method according to claim 37, further comprising:

39. The method according to claim 38, wherein the desirable characteristics include a correlation with a second channel in a multi-channel decoder system.

40. Estimating the aforementioned phase This includes calculating a phase estimate for the time-reversed and phase-adjusted peak, according to the following: f frac =f i -k i In this case, ｉ However, frequency f ｉ This is the estimated phase at However, frequency f ｉ Spectrum The angle is f ｆｒａｃ This is the rounding error, φ Ｃ The adjustment constant is k ｉ ga [f ｉ The method according to any one of claims 32 to 39.

41. φ Ｃ The method according to claim 40, wherein the value is in the range between 0.1 and 0.

7.

42. The phase adjustment with respect to the peak of the time-reversed hidden audio subframe is performed as follows: Df i =-2φ i -2πf i (N+N step21 +(N lost -1)N full ) / N The method according to claim 40, calculated according to the method described in claim 40.

43. The phase adjustment with respect to the peak of the time-reversed hidden audio subframe is performed as follows: Δφ=-2φ 0 -2πf(N step21 +N lost ・N) / N The method according to claim 40, calculated according to the method described in claim 40.

44. Generating the frequency spectrum for each of the two consecutive subframes includes specifying the following: In this case, N represents the length of the subframe window, and the subframe window processing function w １ (n) is the first subframe among the continuous subframes. This is a subframe window processing function related to w ２ (n) is the second subframe of the continuous subframes. This is a subframe window processing function related to N ｓｔｅｐ１２ The method according to any one of claims 32 to 43, wherein the number of samples is between the first subframe of the first two consecutive subframes and the second subframe of the first two consecutive subframes.

45. The method according to any one of claims 32 to 44, further comprising applying a random phase to the noise spectrum of the signal spectrum.

46. The method according to claim 45, wherein applying the random phase to the noise spectrum includes applying the random phase to the noise spectrum before combining the time-inverted, phase-adjusted peak with the noise spectrum.

47. A decoder device (900) configured to generate hidden audio subframes of an audio signal, Processing circuit (902), A decoder device (900) comprising a memory (904) operably coupled to the processing circuit, the memory (904) including an instruction, which, when the instruction is executed by the processing circuit, causes the decoder device to perform an operation according to at least one of claims 1 to 14 or 32 to 46.

48. A decoder device (900) configured to generate hidden audio subframes of an audio signal, and adapted to perform at least one of the methods of claims 32 to 46.

49. A computer program comprising program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform an operation described in any one of claims 32 to 46.

50. A computer program product including a non-temporary storage medium containing program code to be executed by a processing circuit (902) of a decoder device (900) configured to operate in a communication network, wherein the execution of the program code causes the decoder device (900) to perform an operation according to any one of claims 32 to 46.

Images