Adaptive coding of transient audio signals.

Abstract

A method and encoder for adjusting coding scheme selection upon detecting a transient in an input sound signal. The encoder encodes the input signal in frames using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme. The method includes detecting one or more transient attacks and transient releases in the input signal and the locations of the one or more transient attacks and releases in at least one of a current frame and a previous frame. Whether to force a TD coding scheme is determined based on a plurality of conditions related to the one or more transient attacks and releases, and the method further includes selecting a TD coding scheme in response to determining that a TD coding scheme is forced to be used.

Description

[Technical Field]

[0001] The present disclosure relates generally to communications, and more particularly to encoding and decoding of transient audio signals and related devices and nodes that support the encoding and decoding. [Background technology]

[0002] Modern audio codecs, such as 3GPP-EVS (Third Generation Partnership Project - Enhanced Voice Services) and MPEG-USAC (Moving Picture Experts Group - Integrated Speech and Audio Coding), consist of multiple compression schemes optimized for signals with different characteristics. Generally, speech-like signals are processed using time-domain (TD) coding schemes, e.g., using ACELP (Algebraic Code Excited Linear Prediction), while music signals are processed using frequency-domain (FD) coding schemes, e.g., based on the modified discrete cosine transform (MDCT). In the following description, the terms compression scheme, coding scheme, and encoding scheme are used interchangeably.

[0003] For FD coding schemes, transform windows with different lengths and tapering at the beginning and end of the window may be used, as shown in FIG. 1, for example. The windows may also be zero-padded before transformation. Operating at low bit rates, FD coding schemes are generally limited to using wide (or long) transform blocks to save bits. In transition coding schemes that switch from TD coding to FD coding, the MDCT transform length may be temporarily increased to keep up with the normal MDCT framing, thus reducing the bit rate / sample. For the EVS codec, the FD transition coding mode synthesizes 25 ms instead of the 20 ms synthesis in the normal TCX20 frame, thereby providing a 25% reduction in bit rate / sample.

[0004] To select the optimal compression scheme, an audio codec performs an analysis on the input signal. This analysis typically includes a transient detector and a speech / music classifier. The input signal is divided into segments called frames, and each frame is processed consecutively by the codec and placed into a bitstream. A transient detector, such as that used by the EVS codec (3GPP TS26.445 V16.1.1 (December 2020), "Codec for Enhanced Voice Services (EVS); Detailed Algorithmic Description", Section 5.1.8), typically operates at the subframe level; that is, the transient detector divides a 20 ms frame into eight non-overlapping subblocks of 2.5 ms. If there is a significant increase in energy in one of the subframes, an attack flag is set. This transient detector works as follows: 1. Frame F N Given a frame, we define it as S i Divide into 8 subframes denoted as [j], where j is the jth sample in the ith subframe. A negative index for i indicates the preceding subframe belonging to the previous frame. 2. Energy of each subframe TIFF2025540695000002.tif7170, where k is the number of samples in each subframe. 3. The low-pass filtered maximum energy envelope accE for each subframe i Calculate. If i==0, initialize tmpE with the last subframe of the previous frame, i.e., accE -1 TIFF2025540695000003.tif39170Here, α is less than 1, for example 0.8125. 4. Detect whether there is an attack in the main part of the windowed signal (i∈{-2,-1,0,1,2,3,4,5}) as shown in Figure 1 by checking whether the subframe energy substantially exceeds accE by a threshold θ, where θ can be 8.5. TIFF2025540695000004.tif54170

[0005] For speech / music classification, several features are used. For example, the classifier used by EVS (GPP TS26.445 V16.1.1 (December 2020), "Codec for Enhanced Voice Services (EVS); Detailed Algorithmic Description," section: 5.1.13.6 Speech / Music Classification" (also known as the SMC speech / music classifier)) employs a two-stage speech / music classifier. The first stage uses features such as line spectral frequencies (LSFs), mel-frequency cepstral coefficients (MFCCs), spectral stationarity, and correlation map sums to build a Gaussian Mixture Model (GMM) that models the speech, music, and noise probabilities of a frame. The first-stage decision is based on the voice activity detection flag and the smoothed GMM scores. The second stage speech / music classifier refines its decision by analyzing the signal for stability, calculating correlation variance, analyzing attacks on a high resolution of 32 subframes, detecting tonal signals, and calculating the spectral peak-to-mean ratio.

[0006] Some adaptations can be made to the FD coding scheme to handle the coding of transient signals, for example, the time resolution of the transform block can be increased based on a transient detector, as done in the ITU-T G.719 codec (ITU-T, "G.719: Low-complexity, full-band audio coding for high-quality, conversational applications", June 13, 2008), but there are other methods as described herein.

[0007] Currently, there is one or more issues.

[0008] For signals with strong transients, when the existing classification scheme selects a coding scheme, the codec may switch from TD (ACELP) coding to FD (TCX) coding, or may stay in FD coding mode if already operating in FD coding mode. This has been found to result in audible and annoying time-domain (TD) smearing in the decoded signal of the FD compression scheme, especially when there are strong transients at some time positions of the frequency-domain analysis frame.

[0009] TD smearing results in an increase in noise level temporally before a transient or after that transient. The human ear is significantly more sensitive to smearing before the transient because it is often perceived as a bothersome pre-echo artifact. When smearing occurs after the transient (also known as a post-echo artifact), the smearing is better perceptually masked by the encoded transient signal, but may still be perceived as bothersome, depending, for example, on the amount of smearing.

[0010] If the transform window covers samples in both the current frame and parts of the previous frame (see Figure 1), there can be troublesome pre-echo and post-echo artifacts in the decoded signal if FD coding is used during strong transients in the current frame or parts of the previous frame. In particular, when a period of low-energy dynamics is followed by a transient or vice versa in the same coding block, a wider transform block will increase the amount of audible quantization noise and cause time-domain smearing effects.

[0011] Time-domain smearing in FD transform coding schemes is generally handled by four FD methods. See references, 3GPP TS26.445 V16.1.1 (December 2020), "Codec for Enhanced Voice Services (EVS); Detailed Algorithmic Description," Section: "5.3.2.3 Transient location dependent overlap and transform length," and Fuchs et al., "LOW DELAY LPC AND MDCT-BASED AUDIO CODING IN THE EVS CODEC," 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015.

[0012] The four FD methods are: a. Using techniques that switch to shorter FD transform blocks (at the expense of reduced frequency resolution). b. Applying temporal noise shaping (TNS) to reduce smearing in the time domain, for example by linear prediction in the FD domain. c. Use a technique to adjust the window shape while maintaining the transform block length and frequency resolution. The front end of the MDCT analysis window can be adapted on the fly without incurring additional delay. However, a sharper front-end analysis MDCT window implies a reduction in the energy separation capability of the transform, so the default is generally to use a smooth window with a longer overlap to obtain better energy separation. For further details, see Section 5.3.2.3 of 3GPP TS26.445 V16.1.1 (December 2020). d. Applying a decoder-side post-filter that attenuates areas before and / or after the temporal transient.

[0013] Methods a) and b) increase the bit rate, and therefore alternative methods are desirable for low-bit-rate coding. However, method d) only serves as a stopgap measure, generally does not provide very high fidelity for smeared sections, and can introduce distortion even at high bit rates. Finally, method c) can only handle a few possible transient locations, i.e., when the transient is located within a certain portion of the look-ahead section of the MDCT analysis window. Therefore, method c) generally must be combined with one of {a), b), or d)} to better handle all locations of strong transients. In addition to only handling front-end transients, there is a bit rate cost for method c) due to the required signaling of the front-end transform window shape(s). Summary of the Invention

[0014] Instead of mitigating the effects of strong transients in FD, TD coding techniques can be used to obtain better control of the time shape of the coded transient signal. Typically, a multimode codec utilizing both TD and FD coding techniques will select TD coding when speech is detected and switch to FD coding when a music or non-speech signal is detected. However, because both speech and music signals can contain transients (and attacks), the speech / non-speech or speech / music distinction does not always result in the subjectively best quality.

[0015] Some aspects of the present disclosure and their embodiments may provide solutions to these or other problems. According to some embodiments, a multi-mode codec adaptively forces the selection of a TD coding scheme (e.g., ACELP) for transient encoding, even though the signal may have been initially classified by a speech / music classification stage to be encoded using an FD coding scheme (e.g., TCX MDCT mode in an EVS codec). Moreover, the solution is not closed-loop or emulates a closed-loop solution, where the decision regarding the coding scheme is based on selecting the best-performing coding mode, e.g., by calculating an SNR (signal-to-noise ratio) value, based on combining (or approximated) the encoding and decoding outputs of both the FD and TD schemes.

[0016] According to a first aspect, a method in an encoder is presented for adjusting coding scheme selection upon detecting a transient in an input sound signal. The encoder encodes the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme. The method includes detecting one or more transient attacks and transient releases in the input signal and locations of the one or more transient attacks and transient releases in at least one of a current frame and a previous frame. Whether to force a TD coding scheme is determined based on a plurality of conditions related to the one or more transient attacks and transient releases, and selecting a TD coding scheme in response to determining that the TD coding scheme is forced to be used.

[0017] According to a second aspect, there is provided an apparatus comprising means for carrying out the method according to the first aspect.

[0018] According to a third aspect, an encoder is presented, the encoder comprising: a processing circuit; and a memory coupled to the processing circuit, the memory including instructions that, when executed by the processing circuit, cause the encoder to perform operations, the operations including detecting one or more of transient attacks and transient releases in an input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in the frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme. The operations include determining whether to force the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attacks and transient releases, and selecting a TD coding scheme in response to determining that the use of a TD coding scheme is forced.

[0019] According to a fourth aspect, an encoder is presented that is adapted to perform operations including detecting one or more of transient attacks and transient releases in an input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in the frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme. The encoder is adapted to determine whether to force the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attacks and transient releases, and to select a TD coding scheme in response to determining that the use of a TD coding scheme is forced.

[0020] According to a fifth aspect, there is presented a computer program comprising program code to be executed by a processing circuit of an encoder, whereby execution of the program code causes the encoder to perform operations including: detecting, in at least one of a current frame and a previous frame, one or more transient attacks and transient releases and locations of the one or more of the transient attacks and transient releases in an input signal while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to force the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attacks and transient releases; and selecting a TD coding scheme in response to determining that the use of a TD coding scheme is forced.

[0021] According to a sixth aspect, a computer program product is presented, comprising: a non-transitory storage medium comprising program code to be executed by a processing circuit of an encoder, whereby execution of the program code causes the encoder to perform operations, the operations including detecting one or more of transient attacks and transient releases in an input signal and locations of the one or more of the transient attacks and transient releases, in at least one of a current frame and a previous frame while encoding the input signal in the frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme. The operations include determining whether to force the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attacks and transient releases, and selecting a TD coding scheme in response to determining that the use of a TD coding scheme is forced.

[0022] Some embodiments may provide one or more of the following technical advantage(s): An advantage that may be achieved is improved coding and synthesis quality for signals with strong transients, such as percussive single instruments (e.g., castanets), compared to compression schemes that do not use the described embodiments. Another advantage that may be achieved is that embodiments may be adapted when the input signal contains a harmonic background, such that the resulting quality is not harmed.

[0023] The accompanying drawings, which are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application, illustrate several non-limiting embodiments of the inventive concepts. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 10 illustrates how transient analysis subframe slots align with various TCX transform windows and past and present ACELP frames, according to some embodiments. [Figure 2] FIG. 1 is a block diagram of an example operating environment in an encoder and decoder in which adaptive mode selection may be implemented, according to some embodiments. [Figure 3A-3C] 1 is a graph illustrating how an input signal exhibiting pre-echo is decoded using a conventional solution and according to some embodiments of the present disclosure. [Figures 4A-4C] 1 is a graph illustrating how an input signal showing post-echoes is decoded using a conventional solution and how it is decoded according to some embodiments of the present disclosure. [Figure 5] FIG. 10 illustrates how transient analysis subframe slots align with various TCX transform window(s), the ACELP end of the past synthesis line, and the subframe to be analyzed, according to some embodiments. [Figure 6] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 7] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 8] FIG. 10 is a graph showing a strong transient that falls between transient analysis subframe-5 and transient analysis subframe-4, where a normal transient detector (not operating in the reversed time direction) would detect the transient in only subframe-5, even though some of the energy from that transient also falls in subframe-4. [Figure 9] 10 is a flowchart illustrating the operation of an encoder according to some embodiments. [Figure 10] 10 is a flowchart illustrating the operation of an encoder according to some embodiments. [Figure 11] 10 is a flowchart illustrating the operation of an encoder according to some embodiments. [Figure 12] 10 is a flowchart illustrating the operation of an encoder according to some embodiments. [Figure 13]1 is a block diagram of an encoder and decoder showing where adaptive mode selection may be implemented in a stereo codec according to some embodiments. [Figure 14] 1A and 1B are block diagrams of an encoder and a decoder illustrating where adaptive mode selection may be implemented in an audio codec, such as a multi-channel or mono codec, according to some embodiments. [Figure 15] FIG. 2 is a block diagram of an encoder according to some embodiments. [Figure 16] FIG. 2 is a block diagram of a decoder according to some embodiments. [Figure 17] FIG. 2 is a block diagram of a host computer, according to some embodiments. [Figure 18] FIG. 1 is a block diagram of a virtualized environment, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0025] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Examples of embodiments of the inventive concepts are shown, and the embodiments are provided as examples to convey the scope of the subject matter to those skilled in the art. However, the inventive concepts may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concepts to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. It may be implicitly assumed that a component from one embodiment is present / used in another embodiment.

[0026] In this disclosure, The term "attack" generally refers to a low-to-high energy change in an audio signal characterized by a sudden increase in energy within a speech signal segment, such as a voiced onset (including a transition from an unvoiced speech segment to a voiced speech segment) and other speech onsets, transitions, plosives, etc. - The term "release" refers to a decline in energy towards lower energy after being preceded by a change from low to high energy. - The term "transient" refers to a change in energy from low to high in an audio signal that is followed by a relatively fast decay back to low energy, i.e., an attack can be transient if it is followed by a release (a drop in energy).

[0027] Before describing various embodiments for adjusting compression scheme selection, an example operating environment will be described. FIG. 2 illustrates an example operating environment in which various embodiments of the present disclosure may be implemented. Referring to FIG. 2, in the exemplary operating environment 200, an encoder 202 having an audio mode selector 2041 described herein receives data, such as an audio file, to be encoded from an entity such as a host 208 over a network 206 and / or from storage 210. In some embodiments, the host 208 may communicate directly with the encoder 202. The encoder 202 either encodes the audio file as described herein and stores the encoded audio file in storage 210, or transmits the encoded audio file over a network 212 to a decoder 214 having an audio mode selector 2042. The decoder 214 decodes the audio file using the audio mode selector 2042 therein and transmits the decoded audio file to an audio player 216 for playback. For example, the audio player 216 may play the decoded audio file for spatial audio presentations, such as virtual reality conferences or computer games. The audio player 216 may be or be included in a user device, terminal, mobile phone, etc. In other embodiments, the host 208 may transmit the encoded audio file to the decoder 214 over the network 212.

[0028] As previously indicated, the present disclosure enables adaptively forcing the selection of a TD coding scheme (e.g., ACELP) for transient encoding, even though the signal may have been initially classified by a speech / music classification stage to be coded using an FD coding scheme (e.g., TCX MDCT mode in an EVS codec). Moreover, the solution is not closed-loop or emulates a closed-loop solution, where the decision regarding the coding scheme is based on selecting the best-performing coding mode, e.g., by calculating an SNR value, based on combining (or approximated) the encoding and decoding outputs of both the FD and TD schemes.

[0029] This disclosure describes adjusting compression scheme selection upon detecting a transient or attack in a sound signal to be coded, such as music or speech, or in an audio signal.

[0030] In one embodiment, various embodiments operate on a stereo encoder and decoder. The stereo encoder processes input signals for the left and right channels in 20 ms frames. A transient detector is run on each of the signals in those channels to capture the location of transients in each channel. In the stereo encoder, the left and right channels may be downmixed to a mid channel accompanied by side information containing additional side signals and / or parameters describing the stereo image. The mid channel, referred to as the downmix channel, generally has more energy than the side channels and generally consumes more bits for encoding than those spent on the side information.

[0031] In some embodiments, a decision is made to identify whether the input signal contains a problematic transient (attack or release) based on forward and reverse time direction signal analysis. The adaptive selection of the TD coding scheme avoids the smearing artifacts normally caused by the FD block transform, as seen in FIGS. 3C and 4C, while maintaining the quality benefits of FD coding. In FIG. 3C, the signal energy before the transient attack is significantly lower than the reference solution in FIG. 3B, which better matches the input signal in FIG. 3A. In FIG. 4C, the signal energy following the transient release is significantly lower than the reference solution in FIG. 4B, which better matches the input signal in FIG. 4A. While TD coding schemes handle transients better, it may come at the expense of somewhat poorer compression performance, especially in higher frequency regions. This is because most TD compression schemes focus their error minimization on the low-frequency region and cannot efficiently compress all types of signals. Therefore, it is not desirable to always utilize the TD coding scheme, and adaptive selection of the coding mode is desirable for some signals containing strong transients.

[0032] The adaptive selection of the coding mode is based on detecting transients and their locations in the current and past frames and on analyzing the harmonicity of the input signal. A transient detection threshold is based on the harmonicity of the signal. Two main (i.e., high-level) conditions are required to force the selection of the TD coding scheme. These two main conditions are: 1) the transform block of the FD coding scheme contains transients (transient attacks and / or transient releases), and 2) the signal is not considered to be harmonic.

[0033] In one embodiment, the two main conditions are evaluated using three conditions: c1, c2, and c3, which are evaluated to arrive at a decision on whether to force the selection of a TD coding scheme. The first condition c1 determines whether the transient is present in the current frame F except for the last subframe, as shown in FIG. 5. N The second condition c2, which should be checked especially when the previous frame is a TD frame, is whether the transient is detected in the previous frame F N-1 The third condition c3 is whether the signal is harmonic. The decision to force the TD coding scheme is given by forceTD = (c1|c2) & !c3. The third condition c3 being satisfied (true) indicates that the signal is harmonic, while !c3, i.e., c3 not being satisfied (false), indicates that the signal is not harmonic. In other words, the TD coding scheme is forced when conditions c1 or c2 are satisfied and condition c3 is not satisfied (i.e., c3 indicates that the signal is not harmonic). The decision regarding the coding scheme is set to TD coding if forceTD is set; otherwise, the coding scheme is determined by the speech / music classifier.

[0034] 6, in which, in block 601, the encoder 202 detects one or more of transient attacks and transient releases in the input signal and one or more locations of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme. In other words, the encoder 202 detects a transient attack or a transient release, or a transient attack and a transient release.

[0035] In block 603, the encoder 202 determines whether to force the TD coding scheme to be used based on multiple conditions related to transients or attacks. In block 605, in response to determining to force the TD coding scheme, the encoder 202 switches to the TD coding scheme. If the current coding scheme is the TD coding scheme, the switch to the TD coding scheme is to continue using the TD coding scheme. Additional conditions may be used.

[0036] The first condition c1 addresses both forward and backward spreading (and smearing) caused by bit shortage in the low-rate FD TCX20 compression scheme, where TCX20 is a conventional MDCT frame type that produces a 20 ms synthesized output signal. The reason for not including the last subframe, index 7 in Figure 5, is that if there is to be a transient in this subframe, it is assumed that the front end of the window function is shortened. If not, this subframe should also be part of the analysis for the first condition c1.

[0037] The second condition c2 addresses smearing caused by the suboptimal transition window (TCX25) used when switching from TD (ACELP) coding to FD (TCX) coding. TCX25 is an MDCT frame type that can produce a 25 ms synthesized output signal. Compared to TCX20, an additional 5 ms of synthesis is required to fully fill the MDCT overlap-add (OLA) buffer used by the TCX operation in the transition from ACELP coding to TCX coding. If there is a strong transient at the end of a TD-coded frame, some of its energy may be contained at the beginning of the FD-coded frame, which then causes smearing.

[0038] The third condition c3 restricts switching to TD for signals with harmonic content when low-rate TD coding modes may not perform as well as FD coding, e.g., due to limited high-frequency encoding quality and / or because switching to TD coding mode causes perceptually harmful switching artifacts.

[0039] The speech / music classifier is used to make an initial decision on the coding scheme to be used for the channel, either TD or FD. A harmonic flag is calculated to indicate whether the signal is harmonic. Long-term harmonicity can be indicated by analyzing the spectral peak-to-average (P2A) or spectral peak-to-noise (P2N) long-term correlation between frames. Short-term harmonicity can be indicated by analyzing P2A or P2N for a set of spectral peaks within the current frame to determine whether they are harmonically related.

[0040] A preferred variant of the peak-to-noise analysis, similar in scope to the harmonic detection method used in the EVS codec and shown in the flowchart of FIG. 7, is a method for determining harmonic flags by analyzing the long-term evolution of energy spectral peaks over a frame, as follows: 1. Calculate the log bin energy spectrum for both the channel (e.g., downmix channel) of the current frame and the channel (e.g., downmix channel) of the previous frame. This is shown in block 701 of Figure 7, where the encoder 202 calculates the log bin energy spectrum for the signal (e.g., downmix channel) of the current frame and the signal (e.g., downmix channel) of the previous frame. 2. Subtract the estimated noise floor from the bin energy and calculate the correlation between the current frame and the previous frame in the band centered on each peak to obtain a correlation map. This is shown in block 703 of Figure 7, where the encoder 202 subtracts the estimated noise floor from the log bin energy spectrum and calculates the correlation between the current frame and the previous frame in the band centered on each peak to obtain a correlation map. 3. Summing the correlation map values ​​and low-pass filtering the correlation map sum over the frame: This is shown in block 705 of Figure 7, where the encoder 202 sums the correlation map values ​​and low-pass filters the correlation map sum over the frame. 4. Long-term correlation map total CMS LT There is a threshold θ harm If the long-term correlation map sum CMS LT is a predetermined threshold θ harm If the predetermined threshold θ is greater than θ , the signal is classified as harmonic and a harmonic flag is set to indicate that the signal is harmonic. harm is another predetermined threshold Θ harm Based on θ harm =βΘ harm where β may be set to 1 or 0.9<β<1.4. 5. θ harm This is shown in block 709 of FIG. 7, where the encoder 202 updates θ harm For example, update θ harm =βΘ harm When , it is Θ harm and then updating θ harm =βΘ harm Using θ harm and determining Θ. harm can be updated by: Θ harm is a strict threshold θ hard If it is less than Θ harmIncrement by step δ, otherwise, Θ harm Decrement by a step δ and Θ harm The updated value of high and harm low θ hard ,δ,harm high , harm low may be set to, for example, 56, 0.2, 60, and 49, respectively. harm The initial value of can be set to 56.

[0041] Figure 5 shows the alignment of the FD analysis window to subframes in the current and previous frames. Transient locations in the previous frame, such as subframe {-3}, can lead to troublesome post-echo artifacts, and it is generally preferable to select the TD coding mode instead. The main reason for the post-echo smearing-like artifacts in the ACELP-TCX frame (TCX25) with high energy at positions -3 (and -4) is due to the abrupt transition from the last 2-3 ms synthesis of the previous rectangular ACELP to the initial few (4-5) ms synthesis of the TCX25 FD-domain frame, and the transition near the TCX25 MDCT rear folding line.

[0042] However, for signals with a high degree of harmonic content, switching to another coding mode can cause undesirable distortion, making it better not to always switch coding modes. Therefore, various embodiments take the harmonic content of a signal into account when selecting a coding mode for a transient signal.

[0043] FIG. 9 illustrates operations that the encoder 202 performs based on the harmonic flag in some embodiments. If the harmonic flag is set, meaning that there is a high degree of harmonicity in the signal, the coding mode is not changed with respect to potential transients. Thus, as shown in block 901 of FIG. 9 , the encoder 202 does not change the coding scheme in response to the harmonic flag being set. However, if the harmonic flag is not set, a transient analysis is performed to determine whether to force the selection of a TD coding scheme, taking into account the location and strength of one or more of transient attacks and transient releases. This is shown in block 903 of FIG. 9 , where the encoder 202 performs a transient analysis in response to the harmonic flag not being set, taking into account the location and strength of one or more of transient attacks and transient releases, to determine whether to force the selection of a TD coding scheme. The subframes analyzed for each audio channel for transients are those that fall within the transform window to be coded if an FD coding scheme is to be used.

[0044] In the example of Figure 1, this corresponds to subframe {-4,6}, as shown in Figure 8. In the case of a transient in subframe 7, the handling of the front-end transient energy is postponed to the next frame by using the existing minimum, half and full frame window adaptation, as in the EVS case. For subframe {-2,6}, the threshold θ, which depends on the harmonic nature of the signal, fwd A transient detector similar to that described above is used, with θ fwd is the long-term correlation map total CMS LT When θ is close to the harmonic threshold, fwd1 otherwise, θ fwd is θ fwd2 is set to CMS LT For example, θ harm If it exceeds 80% of θ fwd θ fwd1 otherwise, θ fwd θ fwd2A preliminary flag, forceTD, to force the selection of TD coding if a transient is detected. prel is set.

[0045] For subframes {-3} and {-4}, additional analysis is performed to detect potentially harmful transient releases, where a transient (attack) was actually detected in the previous frame, but its energy may spread too much into the current frame to be coded. Figure 8 shows an example where a transient is detected in subframe {-5}, which belongs to the previous frame, but some of the energy spreads into subframe {-4}. Note that even a strong transient that starts as early as in subframe {-6} and is detected by the forward transient detector to be located in subframe {-6} may cause a significant amount of energy to fall into the subsequent subframes {-4} and {-3}.

[0046] For this purpose, for example, a transient detector as described above, but operated in the reverse time direction, and preferably a threshold θ 1 used for transient attack detection may be used. fwd1 , θ fwd2 lower than the threshold θ rev_high , θ rev_low An improved transient detector scheme is used to detect transient releases using θ rev_high and θ rev_low is determined based on the harmonic nature of the signal, i.e., the long-term correlation sum CMS LT is the harmonic threshold θ harm If it exceeds 60% of θ rev_high and θ rev_low θ rev1_high and θ rev1_low otherwise, θ rev_high and θ rev_low θ rev2_high and θ rev2_low Set to.

[0047] 10 illustrates operations that the encoder 202 performs in some embodiments to detect one or more of a transient attack and a transient release in an input signal in at least one of a current frame and a previous frame using an improved transient detector. Referring to FIG. 10, in block 1001, the encoder 202 converts at least one of a current frame and a previous frame into a signal having a transient attack and a transient release. i [j], where j is the jth sample in the ith subframe. In block 1003, the encoder 202 calculates the energy TIFF2025540695000005.tif7170, where k is the number of samples in the subframe.

[0048] In block 1005, the encoder 202 calculates the low-pass filtered maximum energy envelope accE for each subframe. i In block 1007, the encoder 202 calculates whether the subframe energy is substantially equal to the threshold θ fwd Detect whether there is one or more of a transient attack and a transient release in the main part of the windowed signal by checking whether accE is exceeded by a threshold θ fwd depends on the harmonic content of the signal, and θ fwd is the long-term correlation map total CMS LT is the harmonic threshold θ harm If the setpoint is greater than or equal to the setpoint, θ fwd1 otherwise, θ fwd is θ fwd2 In some embodiments, the predetermined set point is 80%.

[0049] 11 illustrates detecting a transient release according to some embodiments. Referring to FIG. 11, in block 1101, the encoder 202 detects a threshold θ rev_high , θ rev_lowDetect the transient release by using a transient detector in the reverse time direction using θ rev_high and θ rev_low is determined based on the harmonic nature of the signal.

[0050] Figure 12 shows the rev_high and θ rev_low 12, in block 1201, the encoder 202 determines the long-term correlation sum CMS LT is the harmonic threshold θ harm and responsive to the second predetermined threshold being exceeded or equal to θ rev_high θ rev1_high and set θ rev_low θ rev1_low In block 1202, the encoder 202 sets the long-term correlation sum CMS LT responsive to θ falling below a second predetermined threshold. rev_high θ rev2_high and set θ rev_low θ rev2_low Set to.

[0051] For the reverse analysis of subframe {-3}, the transient release energy is θ rev_low It is checked whether it exceeds θ rev_low If the FD coding scheme is used, it may lead to smearing. Therefore, we force the coding scheme to be TD(ACELP). prel is set.

[0052] For the reverse analysis of subframe {-4}, the transient release is at threshold θ rev_low and threshold θ rev_high It is checked whether it is detected using both θ rev_high is preferably θ rev_low The transient release is higher than the threshold θ rev_high If detected using forceTD prel The flag is set. The transient release exceeds the threshold θ rev_lowIf only ∇ ... prel The reason for the additional high-resolution time-domain analysis in subframe {-4} is that only a portion of the first half of that subframe actually falls within the TCX25 transform window, and the signal is weighted by the TCX25 window, so if most of the energy is located in the second part of subframe {-4}, the energy will fall within the limit θ rev_high Although the value is lower than the reference value, smearing may occur.

[0053] In all other cases, forceTD prel The auxiliary flag forceTD from each channel is not set. prel If any of the above is set, then a final decision can be made as to whether to force the TD encoding: forceTD = (forceTD L |forceTD R ) where forceTD L and forceTD R are auxiliary flags forceTD for the left and right channels, respectively. prel The calculation of forceTD is summarized in the following pseudocode: forceTD=f(ch left )|f(ch right ) where | corresponds to the logical OR operation, and f(ch n ) is the transient analysis block for the nth channel, where n is either left or right. Alternatively, the final decision forceTD may be based on a logical combination between the preliminary flags determined for each audio channel. For example, in a multi-channel scenario, the logical combination may be a weighted sum based on the energy of each channel. In another embodiment, the analysis may be performed on the downmix channel, and the final decision is based on this analysis. nis the subframe vector for each channel {S i [j],CMS LT ,θ harm}. The function f is given by: TIFF2025540695000006.tif104170TIFF2025540695000007.tif197170TIFF2025540695000008.tif18170TIFF2025540695000009.tif18170TIFF2025540695000010.tif56170Where, ch n is the subframe vector {S i [j],CMS LT ,θ harm}. E i is the energy of subframe i. accE i is the low-pass filtered maximum energy envelope up to subframe i, as explained in paragraph

[0004] . accRevE i is the buffer subframe energy E up to subframe i i Using accE i is the calculated low-pass filtered maximum energy envelope, similar to, but in the opposite time direction, i.e., i==m start If Initialize tmpE with TIFF2025540695000011.tif5170, where m start is the subframe index, and the energy envelope filter state is initialized according to: TIFF2025540695000012.tif41170, Here, α is smaller than 1, for example, 0.8125. Here, m stop is the index to stop just before the last subframe of interest, e.g., {-4}. start and mstop are 3 and -5, respectively. start can be varied, but when fewer subframes are used, m start It should not be too close to the large region of interest (which is {-4,-3}) as this will result in inaccurate energy envelope estimates around m start is the envelope energy estimate m stop It cannot be chosen too far away as it would not be accurate for the area around it. k is the number of samples in each subframe. S i [j] is the jth sample in the ith subframe. θ fwd1 , θ fwd2 , θ rev1_high , θ rev1_low , θ rev2_high , θ rev2_low are thresholds in the range between 4 and 9, which may be set to, for example, 8.5, 8.0, 5.5, 4.5, 5.25, and 4.25, respectively.

[0054] If the forceTD flag is set (true), the TD encoding scheme is selected regardless of the preceding FD / TD (ACELP / TCX) classifier decision.

[0055] The coded downmix channels and the coded side information are put together into a bitstream and sent to a decoder. The decoder decodes the bitstream to extract the side information and the downmix signal. Stereo upmixing is performed to obtain left and right channel audio signals.

[0056] The proposed method can be implemented in a stereo codec, as shown in Figure 13, or in either a multi-channel codec or a mono codec, as shown in Figure 14, where the adaptive mode selector blocks in Figures 13 and 14 refer to the above-described embodiments of the present disclosure.

[0057] In one embodiment, the harmonic flag is calculated with respect to the downmixed channels. Similarly, the calculation of the forceTD flag may be based directly on the downmix channels rather than the left and right channels.

[0058] In another embodiment where the codec operates on a multi-channel signal, the speech / music classifier, harmonic analysis and calculation forceTD _prel is performed for each channel. Then the final forceTD is calculated by subtracting the preliminary flag forceTD from any of the channels. _prel is set, or alternatively based on another combination of the channel's spare flags.

[0059] 15 illustrates an encoder 202, according to some embodiments. As used herein, an encoder refers to a device capable of, set up, configured, and / or operable to encode files and communicate wirelessly with network nodes, decoders, and / or other encoders. Examples of encoders include, but are not limited to, smartphones, mobile phones, cell phones, voice-over-IP (VoIP) phones, wireless local loop phones, desktop computers, personal digital assistants (PDAs), wireless cameras, gaming consoles or devices, music storage devices, playback appliances, wearable terminal devices, wireless endpoints, mobile stations, tablets, laptop computers, laptop embedded appliances (LEEs), laptop mounted appliances (LMEs), smart devices, wireless customer premises equipment (CPEs), vehicle-mounted or vehicle-embedded / integrated wireless devices, etc.

[0060] An encoder may support device-to-device (D2D) communications, for example, by implementing 3GPP standards for sidelink communications, dedicated short-range communications (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, an encoder does not necessarily have a user in the sense of a human user who owns and / or operates an associated device. Instead, an encoder may represent a device that is intended for sale to or operation by a human user, but that may not be associated with or initially associated with a particular human user. Alternatively, an encoder may represent a device that is not intended for sale to or operation by an end user, but that may be associated with or operated for the benefit of a user.

[0061] The encoder 202 includes a processing circuit 1502 operably coupled to an input / output interface 1506, a power supply 1508, a memory 1510, a communication interface 1512, and / or any other components, or any combination thereof, via a bus 1504. Some encoders may utilize all or a subset of the components shown in FIG. 15. The level of integration between components may vary from encoder to encoder. Furthermore, some encoders may include multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc. In its simplest form, the encoder 202 may have a processing circuit 1502, a memory 1510, and a communication interface 1512.

[0062] The processing circuit 1502 is configured to process instructions and data and may be configured to implement any sequential state machine operable to execute instructions stored in memory 1510 as a machine-readable computer program. The processing circuit 1502 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.), programmable logic together with appropriate firmware, one or more stored computer programs such as a microprocessor or digital signal processor (DSP) together with appropriate software, a general-purpose processor, or any combination of the above. For example, the processing circuit 1502 may include multiple central processing units (CPUs).

[0063] In this example, the input / output interface 1506 may be configured to provide one or more interfaces to an input device, an output device, or one or more input and / or output devices. Examples of output devices include a speaker, a sound card, a video card, a display, a monitor, an actuator, an emitter, a smart card, another output device, or any combination thereof. An input device may allow a user to capture information into the encoder 202. Examples of input devices include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a webcam, etc.), a microphone, a sensor, a directional pad, a trackpad, a scroll wheel, a smart card, etc. A presence-sensitive display may include a capacitive or resistive touch sensor for detecting input from a user. The sensor may be, for example, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, a light sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as the input device. For example, a universal serial bus (USB) port may be used to accommodate input and output devices.

[0064] In some embodiments, the power supply 1508 is structured as a battery or battery pack. Other types of power sources may be used, such as an external power source (e.g., an electrical outlet), a photovoltaic device, or a battery. The power supply 1508 may further include power circuitry for delivering power to various portions of the encoder 202 from the power supply 1508 itself and / or from an external power source via an interface such as an input circuit or a power cable. Delivering power may be for charging the power supply 1508, for example. The power circuitry may perform any formatting, converting, or other modification on the power from the power supply 1508 to make it suitable for the respective components of the encoder 202 being powered.

[0065] The memory 1510 may be or be configured to include memory, such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disk, optical disk, hard disk, removable cartridge, flash drive, etc. In one example, the memory 1510 includes one or more application programs 1514, such as an operating system, a web browser application, a widget, a gadget engine, or other applications, and corresponding data 1516. The memory 1510 may store any of a variety of different operating systems or combinations of operating systems for use by the encoder 202.

[0066] The memory 1510 may be configured to include several physical drive units, such as a redundant array of independent disks (RAID), flash memory, a USB flash drive, an external hard disk drive, a thumb drive, a pen drive, a key drive, a high-density digital versatile disc (HD-DVD) optical disc drive, an internal hard disk drive, a Blu-ray optical disc drive, a holographic digital data storage (HDDS) optical disc drive, an external mini dual in-line memory module (DIMM), a synchronous dynamic random access memory (SDRAM), an external micro-DIMM SDRAM, a smart card memory such as a tamper-resistant module in the form of a universal integrated circuit card (UICC) containing one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may be, for example, an embedded UICC (eUICC), an integrated UICC (iUICC), or a removable UICC, commonly known as a "SIM card." The memory 1510 may enable the encoder 202 to access, offload, or upload data, instructions, application programs, and the like stored on a temporary or non-transitory memory medium. An article of manufacture, such as an article of manufacture utilizing a communication system, may be tangibly embodied as or in the memory 1510, which may be or comprise a device-readable storage medium.

[0067] The processing circuit 1502 may be configured to communicate with an access network or other networks using a communication interface 1512. The communication interface 1512 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1522. The communication interface 1512 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another encoder or decoder or network node in the access network). Each transceiver may include a transmitter 1518 and / or a receiver 1520 suitable for providing network communication (e.g., optical, electrical, frequency allocation, etc.). Moreover, the transmitter 1518 and receiver 1520 may be coupled to one or more antennas (e.g., antenna 1522) and may share circuit components, software, or firmware, or may alternatively be implemented separately.

[0068] In the illustrated embodiment, the communication capabilities of communication interface 1512 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communication such as Bluetooth, near-field communication, location-based communication such as using a Global Positioning System (GPS) to determine location, another similar communication capability, or any combination thereof. Communication may be implemented in accordance with one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Synchronous Optical Networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), etc.

[0069] 16 illustrates a decoder 214, according to some embodiments. As used herein, a network node refers to a device capable of, set up, configured, and / or operable to communicate, directly or indirectly, with UEs and / or other network nodes or devices in a communication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., wireless access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs), and NR Node Bs (gNBs)).

[0070] Base stations may be categorized based on the amount of coverage they provide (or, stated another way, their transmit power level) and may therefore be referred to as femto, pico, micro, or macro base stations depending on the amount of coverage provided. A base station may be a relay node or a relay donor node that controls a relay. A network node may also include one or more (or all) parts of a distributed radio base station, such as a centralized digital unit and / or a remote radio unit (RRU), sometimes referred to as a remote radio head (RRH). Such remote radio units may or may not be integrated with an antenna, as in an antenna-integrated radio. Portions of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0071] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, MSR equipment such as a multi-standard radio (MSR) BS, a network controller such as a radio network controller (RNC) or base station controller (BSC), a base transceiver station (BTS), a transmission point, a transmitting node, a multi-cell / multicast coordination entity (MCE), an operation and maintenance (O&M) node, an operation support system (OSS) node, a self-organizing network (SON) node, a positioning node (e.g., an evolved serving mobile location center (E-SMLC)), and / or a minimization drive test (MDT).

[0072] The decoder 214 includes a processing circuit 1602, a memory 1604, a communication interface 1606, and a power supply 1608. The decoder 214 may be assembled from multiple physically separate components (e.g., a Node B component and an RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In some scenarios in which the decoder 214 comprises multiple separate components (e.g., a BTS component and a BSC component), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple Node Bs. In such scenarios, each unique Node B and RNC pair may, in some cases, be considered a single separate network node. In some embodiments, the decoder 214 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1604 for different RATs) and some components may be reused (e.g., the same antenna 1610 may be shared by different RATs). The decoder 214 may also include multiple sets of the various shown components for different wireless technologies, such as GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, radio frequency identification (RFID), or Bluetooth wireless technologies, integrated into the decoder 214. These wireless technologies may be integrated into the same or different chips or sets of chips and other components within the decoder 214.

[0073] The processing circuit 1602 may comprise one or more combinations of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software, and / or coded logic operable to provide the decoder 214 functionality, either alone or in conjunction with other decoder 214 components such as memory 1604.

[0074] In some embodiments, the processing circuit 1602 comprises a system on a chip (SOC). In some embodiments, the processing circuit 1602 includes one or more of a radio frequency (RF) transceiver circuit 1612 and a baseband processing circuit 1614. In some embodiments, the radio frequency (RF) transceiver circuit 1612 and the baseband processing circuit 1614 may be on separate chips (or sets of chips), boards, or units, such as a radio unit and a digital unit. In alternative embodiments, some or all of the RF transceiver circuit 1612 and the baseband processing circuit 1614 may be on the same chip or set of chips, board, or unit.

[0075] Memory 1604 may comprise any form of volatile or non-volatile computer-readable memory, including, but not limited to, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (e.g., hard disk), removable storage media (e.g., flash drive, compact disc (CD) or digital video disc (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory device that stores information, data, and / or instructions that may be used by processing circuit 1602. Memory 1604 may store any suitable instructions, data, or information, including applications including one or more of computer programs, software, logic, rules, code, tables, and / or other instructions that can be executed by processing circuit 1602 and utilized by decoder 214. Memory 1604 may be used to store computations performed by processing circuit 1602 and / or data received via communications interface 1606. In some embodiments, the processing circuit 1602 and the memory 1604 are integrated.

[0076] The communication interface 1606 is used for wired or wireless communication of signaling and / or data between the encoder, network node, access network, and / or decoder. As shown, the communication interface 1606 comprises port(s) / terminal(s) 1616 for sending and receiving data to and from a network, e.g., over a wired connection. The communication interface 1606 also includes radio front-end circuitry 1618, which is coupled to an antenna 1610 or, in some embodiments, may be part of the antenna 1610. The radio front-end circuitry 1618 comprises a filter 1620 and an amplifier 1622. The radio front-end circuitry 1618 may be connected to the antenna 1610 and the processing circuit 1602. The radio front-end circuitry may be configured to condition signals communicated between the antenna 1610 and the processing circuit 1602. The radio front-end circuitry 1618 may receive digital data to be sent to another network node or UE via a wireless connection. The radio front-end circuitry 1618 may convert the digital data into radio signals having appropriate channel and bandwidth parameters using a combination of filters 1620 and / or amplifiers 1622. The radio signals may then be transmitted via the antenna 1610. Similarly, when receiving data, the antenna 1610 may collect the radio signals, which are then converted to digital data by the radio front-end circuitry 1618. The digital data may be passed to the processing circuitry 1602. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0077] In some alternative embodiments, the decoder 214 does not include a separate radio front-end circuit 1618; instead, the processing circuit 1602 includes the radio front-end circuitry and is connected to the antenna 1610. Similarly, in some embodiments, all or a portion of the RF transceiver circuitry 1612 is part of the communications interface 1606. In still other embodiments, the communications interface 1606 includes one or more ports or terminals 1616, the radio front-end circuitry 1618, and the RF transceiver circuitry 1612 as part of a radio unit (not shown), and the communications interface 1606 communicates with baseband processing circuitry 1614 that is part of a digital unit (not shown).

[0078] The antenna 1610 may include one or more antennas or an antenna array configured to send and / or receive wireless signals. The antenna 1610 may be coupled to the wireless front-end circuitry 1618 and may be any type of antenna capable of wirelessly transmitting and receiving data and / or signals. In some embodiments, the antenna 1610 is separate from the decoder 214 and may be connectable to the decoder 214 through an interface or port.

[0079] The antenna 1610, the communication interface 1606, and / or the processing circuit 1602 may be configured to perform any receiving operation and / or some obtaining operation described herein as being performed by a network node. Any information, data, and / or signals may be received from a UE, another network node, and / or any other network equipment. Similarly, the antenna 1610, the communication interface 1606, and / or the processing circuit 1602 may be configured to perform any transmitting operation described herein as being performed by a network node. Any information, data, and / or signals may be transmitted to a UE, another network node, and / or any other network equipment.

[0080] The power supply 1608 provides power to the various components of the decoder 214 in a form suitable for each component (e.g., at the voltage and current levels required for each respective component). The power supply 1608 may further include, or be coupled to, a power management circuit for supplying power to the components of the decoder 214 to perform the functions described herein. For example, the decoder 214 may be connectable to an external power source (e.g., a power grid, an electrical outlet) via an input circuit or interface, such as an electrical cable, whereby the external power source provides power to the power circuit of the power supply 1608. As a further example, the power supply 1608 may include a power source in the form of a battery or battery pack connected to or integrated in the power circuit. The battery may provide backup power if the external power source fails.

[0081] 16 to provide certain aspects of the functionality of a network node, including any of the functionality described herein and / or functionality necessary to support the subject matter described herein. For example, the decoder 214 may include user interface devices to enable input of information into the decoder 214 and output of information from the decoder 214. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the decoder 214.

[0082] 17 is a block diagram of a host 208. As used herein, a host 208 may be or comprise various combinations of hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, a container, or processing resources in a server farm. A host 208 may provide one or more services to one or more encoders and decoders.

[0083] The host 208 includes a processing circuit 1702 operably coupled to an input / output interface 1706, a network interface 1708, a power supply 1710, and a memory 1712 via a bus 1704. In other embodiments, other components may be included. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as FIGS. 15 and 16, and therefore, those descriptions are generally applicable to the corresponding components of the host 208.

[0084] The memory 1712 may include one or more computer programs, including one or more host application programs 1714 and data 1716, which may include user data, e.g., data generated by the UE for the host 208 or data generated by the host 208 for the UE. Embodiments of the host 208 may utilize only a subset or all of the shown components. The host application programs 1714 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711, EVS), including transcoding for multiple different classes, types, or implementations of UE (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application program 1714 may also provide user authentication and license checks, and may periodically report health, route, and content availability to a central node, such as a device in the core network or a device on the edge of the core network. Thus, the host 208 may select and / or indicate a different host for over-the-top services for the UE. The host application program 1714 may support various protocols, such as HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0085] FIG. 18 is a block diagram illustrating a virtualization environment 1800 in which functionality implemented by some embodiments may be virtualized. In this context, virtualizing means creating a virtual version of an apparatus or device, which may include virtualizing a hardware platform, storage devices, and networking resources. Virtualization, as used herein, may apply to any device described herein, or components thereof, and relates to implementations in which at least a portion of functionality is implemented as one or more virtual components. Some or all of the functionality described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1800 hosted by one or more of the hardware nodes, such as a network node, a UE, a core network node, or a hardware computing device acting as a host. Furthermore, in embodiments in which the virtual node does not require wireless connectivity (e.g., to a core network node or host), the node may be fully virtualized.

[0086] An application 1802 (which may alternatively be referred to as a software instance, a virtual appliance, a network function, a virtual node, a virtual network function, etc.) is run in the virtualized environment 1800 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0087] Hardware 1804 includes processing circuitry, memory that stores software and / or instructions executable by the hardware processing circuitry, and / or other hardware devices described herein, such as network interfaces, input / output interfaces, etc. Software is executed by the processing circuitry to instantiate one or more virtualization layers 1806 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1808A and 1808B (one or more of which may be referred to generically as VMs 1808), and / or implement any of the functions, features, and / or benefits described with respect to some embodiments described herein. Virtualization layer 1806 may present to VMs 1808 a virtual operating platform that appears to be networking hardware.

[0088] The VMs 1808 may comprise virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and may be run by a corresponding virtualization layer 1806. Different embodiments of the virtual appliance 1802 instances may be implemented on one or more of the VMs 1808, and the implementations may be done in different ways. Hardware virtualization is referred to in some contexts as network functions virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry-standard high-volume server hardware, physical switches, and physical storage that may be located in data centers and customer premises equipment.

[0089] In the context of NFV, a VM 1808 may be a software implementation of a physical machine that runs programs as if those programs were running on a physical, non-virtualized machine. Each VM 1808 and the portion of the hardware 1804 on which it runs, whether hardware dedicated to that VM and / or hardware shared by that VM with other VMs, form a separate virtual network element. Further, in the context of NFV, a virtual network function is responsible for handling a particular network function running in one or more VMs 1808 on the hardware 1804 and corresponds to the application 1802.

[0090] The hardware 1804 may be implemented in a standalone network node with general or specific components. The hardware 1804 may implement some functions via virtualization. Alternatively, the hardware 1804 may be part of a larger cluster of hardware (e.g., as in a data center or CPE) where many hardware nodes cooperate and are managed via a management and orchestration 1810 that, among other things, oversees the lifecycle management of the application 1802. In some embodiments, the hardware 1804 is coupled to one or more radio units, each including one or more transmitters and one or more receivers, which may be coupled to one or more antennas. The radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with virtual components to provide a virtual node with wireless capabilities, such as a wireless access node or base station. In some embodiments, some signaling may be provided using a control system 1812, which may alternatively be used for communication between the hardware nodes and the radio units.

[0091] While the computing devices described herein (e.g., encoders, decoders, UEs, network nodes, hosts) may include the depicted combinations of hardware components, other embodiments may comprise computing devices with different combinations of components. It should be understood that these computing devices may comprise any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determining, calculating, obtaining, or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting obtained information to other information, comparing the obtained or converted information with information stored in a network node, and / or performing one or more operations based on the obtained or converted information and as a result of the processing making a decision. Moreover, while a component is depicted as a single box located within a larger box or nested within multiple boxes, in reality the computing device may comprise multiple different physical components that make up the single depicted component, and functionality may be partitioned among the separate components. For example, a communications interface may be configured to include any of the components described herein, and / or the functionality of those components may be partitioned between the processing circuitry and the communications interface. In another example, non-computationally intensive functionality of any of such components may be implemented in software or firmware, and computationally intensive functionality may be implemented in hardware.

[0092] In some embodiments, some or all of the functionality described herein may be provided by a processing circuit executing instructions stored in a memory, which in some embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuit without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hardwired manner. In any of these particular embodiments, the processing circuit may be configured to perform the described functionality, regardless of whether or not it executes instructions stored on a non-transitory computer-readable storage medium. Benefits provided by such functionality are not limited to the processing circuit alone or to other components of the computing device, but are enjoyed by the computing device as a whole and / or by end users and wireless networks generally.

[0093] Exemplary embodiments: 1. A method in an encoder (202, 1802) for adjusting compression scheme selection upon detecting a transient or attack in an audio signal, the encoder encoding an input signal in frames, the method comprising: Detecting (601) one or more of transient attacks and transient releases in the input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to one or more of a transient attack and a transient release (603); In response to determining to force the TD coding scheme, switching to the TD coding scheme for encoding one or more of the transient attack and the transient release (605); A method comprising: 2. determining a coding scheme by the speech / music classifier in response to determining not to enforce the TD coding scheme (607); 2. The method of embodiment 1, further comprising: 3. The method of embodiment 1 or 2, wherein the plurality of conditions includes at least two main conditions. 4. The method of embodiment 3, wherein a first main condition of the at least two main conditions includes determining whether a transform block of the FD coding scheme includes one or more of a transient attack and a transient release. 5. Determining whether a transform block of the FD coding scheme includes one or more of a transient attack and a transient release includes: determining a first condition c1, which includes determining whether a transient or an attack is detected in the current frame; determining a second condition c2, which includes determining whether a transient or attack was detected in the last half of the previous frame; 5. The method of embodiment 4, comprising: 6. The method of embodiment 5, wherein determining whether a transient or attack is detected in the current frame includes determining whether a transient or attack is detected in the current frame excluding the last subframe. 7. The method of any one of embodiments 3 to 6, wherein the second condition of the at least two conditions includes determining whether the signal is harmonic. 8. The method of embodiment 7, wherein determining whether the signal is harmonic includes determining whether the third condition c3 includes determining whether the signal is harmonic. 9. The method of embodiment 8, wherein determining whether to force the TD coding scheme to be used includes determining to force the TD coding scheme in response to c1 or c2 being satisfied and c3 indicating that the signal is not harmonic. 10. Determining whether a signal is harmonic can be done by examining the long-term evolution of the energy spectral peaks over the frames. Calculating (701) a logarithmic bin energy spectrum of a signal of a current frame and a signal of a previous frame; subtracting an estimated noise floor from the logarithmic bin energy spectrum and calculating the correlation between the current frame and the previous frame in a band centered on each peak to obtain a correlation map (703); summing the correlation map values ​​and low-pass filtering the sum of the correlation map values ​​over the frame (705); Long-term correlation map total CMS LT is a predetermined threshold θ harm classifying the signal as harmonic and setting a harmonic flag indicating that the signal is harmonic if θ harm Update (709) 10. The method of any one of embodiments 7 to 9, comprising analyzing by: 11. In response to the harmonic flag being set, not changing the coding mode (901); In response to the harmonic flag not being set, performing a transient analysis (903) to determine whether to force selection of a TD coding scheme, taking into account the location and strength of one or more of a transient attack and a transient release; 11. The method of embodiment 10, further comprising: 12. Detecting one or more of a transient attack and a transient release in the input signal in at least one of a current frame and a previous frame using a transient detector that performs an operation, the operation comprising: At least one of the current frame and the previous frame is i Dividing (1001) at least one of a current frame and a previous frame into a plurality of subframes denoted as [j], where j is the jth sample in the ith subframe; For each subframe, the energy of the subframe 1003, calculating the energy of the subframe, k, where k is the number of samples in the subframe. Calculating TIFF2025540695000014.tif7170 (1003) The low-pass filtered maximum energy envelope accE for each subframe i Calculating (1005) If the subframe energy is substantially below the threshold θ fwd Detecting whether there is one or more of a transient attack and a transient release in the main part of the windowed signal by checking whether accE is exceeded by a threshold θ fwd depends on the harmonic content of the signal, and θ fwd is the long-term correlation map total CMS LT is the harmonic threshold θ harm If the setpoint is greater than or equal to the setpoint, θ fwd1 otherwise, θ fwd is θ fwd2 Detecting whether there is one or more of a transient attack and a transient release in the main portion of the windowed signal (1007), 12. The method of any one of embodiments 1 to 11, comprising: 13. The method of embodiment 12, wherein the predetermined set point comprises 80%. 14. Detecting one or more of a transient attack and a transient release in an input signal is performed by a threshold θ rev_high , θ rev_low Detecting a transient release by using a transient detector in a reverse time direction using θ rev_high and θ rev_low 14. The method of embodiment 12 or 13, further comprising detecting (1101) a transient release, wherein is determined based on the harmonic nature of the signal. 15. θ rev_high and θ rev_low teeth, Long-term correlation total CMSLT is the harmonic threshold θ harm and responsive to the second predetermined threshold being exceeded or equal to θ rev_high θ rev1_high and set θ rev_low θ rev1_low (1201) and Long-term correlation total CMS LT responsive to θ falling below a second predetermined threshold. rev_high θ rev2_high and set θ rev_low θ rev2_low Set it to (1203) 15. The method of embodiment 14, wherein the method is determined by 16. The method of embodiment 15, wherein the second predetermined threshold comprises 60%. 17. An encoder (202, 1802), A processing circuit (1502); a memory (1510) coupled to the processing circuit; wherein the memory contains instructions that, when executed by the processing circuit, cause the encoder (202, Q1802) to perform operations, the operations including: Detecting (601) one or more of transient attacks and transient releases in the input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to one or more of a transient attack and a transient release (603); In response to determining to force the TD coding scheme, switching to the TD coding scheme for encoding one or more of the transient attack and the transient release (605); Including,Encoder(202, 1802). 18. An encoder (202, 1802) as described in embodiment 17, wherein the memory includes further instructions that, when executed by the processing circuit, cause the encoder (202, 1802) to perform the operations described in any one of embodiments 2 to 16. 19. An encoder (202, 1802) adapted to perform an operation, said operation comprising: Detecting (601) one or more of transient attacks and transient releases in the input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to one or more of a transient attack and a transient release (603); In response to determining to force the TD coding scheme, switching to the TD coding scheme for encoding one or more of the transient attack and the transient release (605); Including,Encoder(202, 1802). 20. An encoder (202, 1802) according to embodiment 17, further adapted to perform according to any one of embodiments 2 to 16. 21. A computer program comprising program code to be executed by a processing circuit (1502) of an encoder (202, 1802), whereby execution of the program code causes the encoder (202, 1802) to perform an operation, the operation being: Detecting (601) one or more of transient attacks and transient releases in the input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to one or more of a transient attack and a transient release (603); In response to determining to force the TD coding scheme, switching to the TD coding scheme for encoding one or more of the transient attack and the transient release (605); a computer program comprising: 22. The computer program of embodiment 21, comprising further program code, whereby execution of the further program code causes the encoder (202, 1802) to perform the operations of any one of embodiments 2 to 16. 23. A computer program product comprising a non-transitory storage medium comprising program code to be executed by a processing circuit (1502) of an encoder (202, 1802), whereby execution of the program code causes the encoder (202, 1802) to perform operations, the operations being: Detecting (601) one or more of transient attacks and transient releases in the input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to one or more of a transient attack and a transient release (603); In response to determining to force the TD coding scheme, switching to the TD coding scheme for encoding one or more of the transient attack and the transient release (605); a computer program product, 24. The computer program of embodiment 23, wherein a non-transitory storage medium comprises further program code, whereby execution of the further program code causes the encoder (202, 1802) to perform the operations of any one of embodiments 2 to 16.

Claims

1. 1. A method in an encoder (202, 1802) for adjusting coding scheme selection upon detecting a transient in an input sound signal, the encoder encoding the input signal in frames using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme, the method comprising: Detecting (601) one or more of a transient attack and a transient release in the input signal and a location of the one or more of the transient attack and the transient release in at least one of a current frame and a previous frame; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attack and the transient release (603); selecting (605) the TD coding scheme in response to determining that the TD coding scheme is to be forced to be used; A method comprising:

2. determining (607) the coding scheme by a speech / music classifier in response to determining that the TD coding scheme is not mandated to be used. The method of claim 1 further comprising:

3. The method of claim 1 or 2, wherein the plurality of conditions includes at least two main conditions.

4. 4. The method of claim 3, wherein a first main condition of the at least two main conditions includes determining whether a transform block of an FD coding scheme includes one or more of the transient attack and the transient release.

5. Determining whether the transform block of an FD coding scheme includes one or more of the transient attack and the transient release includes: a first condition c including determining whether the transient or attack is detected in the current frame; １ and determining a second condition c including determining whether the transient or the attack was detected in the last half of the previous frame; ２ and determining The method of claim 4, comprising:

6. 6. The method of claim 5, wherein determining whether the transient or the attack is detected in the current frame comprises determining whether the transient or the attack is detected in the current frame excluding a last subframe.

7. 7. The method of claim 3, wherein a second of the at least two main conditions comprises determining whether the signal is harmonic.

8. A third condition c, wherein determining whether the signal is harmonic includes indicating whether the signal is harmonic. ３ The method of claim 7, comprising determining:

9. Determining whether the TD coding scheme should be forced to be used includes: １ or c ２ is satisfied, and c ３ and determining to enforce the TD coding scheme in response to the signal being non-harmonic.

10. Determining whether the signal is harmonic may be performed by examining the long-term evolution of the energy spectral peaks over a frame, Calculating (701) the logarithmic bin energy spectrum of the current frame signal and the previous frame signal; subtracting an estimated noise floor from the logarithmic bin energy spectrum and calculating the correlation between the current frame and the previous frame in bands centered around each peak to obtain a correlation map (703); summing correlation map values ​​and low-pass filtering (705) the sum of correlation map values ​​over a frame; Long-term correlation map total CMS ＬＴ is a predetermined threshold θ ｈａｒｍ if so, classifying the signal as harmonic and setting a harmonic flag indicating that the signal is harmonic (707); θ ｈａｒｍ (709) and 10. The method of claim 7, comprising analyzing by:

11. determining (901) that the TD coding scheme is not mandated to be used in response to the harmonic flag being set; In response to the harmonic flag not being set, performing a transient analysis (903) to determine whether the selection of a TD coding scheme should be forced, taking into account the location and strength of the one or more of the transient attack and the transient release; The method of claim 10 further comprising:

12. Detecting the one or more of the transient attack and the transient release in the input signal in at least one of the current frame and the previous frame includes: The at least one of the current frame and the previous frame is ｉ Dividing (1001) the at least one of the current frame and the previous frame into a plurality of subframes denoted as [j], where j is the jth sample in the ith subframe; For each subframe, the energy of said subframe Calculating (1003) the energy of the subframe, Calculating (1003) The filtered maximum energy envelope accE for each subframe ｉ Calculating (1005) The energy of the subframe is substantially below a threshold θ ｆｗｄ Detecting whether there is one or more of the transient attacks and the transient releases in the main part of the windowed signal by checking whether accE is exceeded by accE (1007); 12. The method of claim 1, further comprising:

13. The threshold value θ ｆｗｄ depends on the harmonic content of the signal, and θ ｆｗｄ is the long-term correlation map sum CMS ＬＴ is the harmonic threshold θ ｈａｒｍ If the setpoint is greater than or equal to the setpoint, θ ｆｗｄ１ otherwise, θ ｆｗｄ is θ ｆｗｄ２ The method of claim 12, wherein the value is set to

14. 14. The method of claim 12 or 13, wherein the predetermined set point comprises 80%.

15. Detecting the one or more of the transient attack and the transient release in the input signal is performed by adjusting a threshold θ ｒｅｖ＿ｈｉｇｈ , θ ｒｅｖ＿ｌｏｗ Detecting a transient release by using a transient detector in a reverse time direction using θ ｒｅｖ＿ｈｉｇｈ and θ ｒｅｖ＿ｌｏｗ 15. The method of claim 12, further comprising detecting (1101) a transient release, wherein is determined based on the harmonic nature of the signal.

16. θ ｒｅｖ＿ｈｉｇｈ and θ ｒｅｖ＿ｌｏｗ teeth, Long-term correlation total CMS ＬＴ is the harmonic threshold θ ｈａｒｍ in response to the second predetermined set point being exceeded or equal to θ ｒｅｖ＿ｈｉｇｈ θ ｒｅｖ１＿ｈｉｇｈ and set θ ｒｅｖ＿ｌｏｗ θ ｒｅｖ１＿ｌｏｗ (1201) and The long-term correlation sum CMS ＬＴ in response to θ falling below the second predetermined set point. ｒｅｖ＿ｈｉｇｈ θ ｒｅｖ２＿ｈｉｇｈ and set θ ｒｅｖ＿ｌｏｗ θ ｒｅｖ２＿ｌｏｗ (1203) and The method of claim 15, wherein the value is determined by

17. 17. The method of claim 16, wherein the second predetermined set point comprises 60%.

18. Apparatus comprising means for carrying out the method according to any one of claims 1 to 17.

19. An encoder (202, 1802), A processing circuit (1502); a memory (1510) coupled to said processing circuit; wherein the memory contains instructions that, when executed by the processing circuitry, cause the encoder (202, 1802) to perform operations, the operations including: Detecting (601) one or more of transient attacks and transient releases in an input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attack and the transient release (603); selecting (605) the TD coding scheme in response to determining that the TD coding scheme is to be forced to be used; an encoder (202, 1802) including:

20. 20. The encoder (202, 1802) of claim 19, wherein the memory includes further instructions that, when executed by the processing circuitry, cause the encoder (202, 1802) to perform an operation according to any one of claims 2 to 17.

21. An encoder (202, 1802) adapted to perform operations, said operations comprising: Detecting (601) one or more of transient attacks and transient releases in an input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attack and the transient release (603); selecting (605) the TD coding scheme in response to determining that the TD coding scheme is to be forced to be used; an encoder (202, 1802) including:

22. An encoder (202, 1802) according to claim 21, further adapted to perform a method according to any one of claims 2 to 17.

23. A computer program comprising program code to be executed by a processing circuit (1502) of an encoder (202, 1802), whereby execution of said program code causes said encoder (202, 1802) to perform operations, said operations comprising: Detecting (601) one or more of transient attacks and transient releases in an input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attack and the transient release (603); selecting (605) the TD coding scheme in response to determining that the TD coding scheme is to be forced to be used; a computer program comprising:

24. 24. The computer program of claim 23, comprising further program code, whereby execution of said further program code causes said encoder (202, 1802) to perform the operations of any one of claims 2 to 17.

25. A computer program product comprising: a non-transitory storage medium comprising program code to be executed by a processing circuit (1502) of an encoder (202, 1802), whereby execution of the program code causes the encoder (202, 1802) to perform operations, the operations including: Detecting (601) one or more of transient attacks and transient releases in an input signal and locations of the one or more of the transient attacks and transient releases in at least one of a current frame and a previous frame while encoding the input signal in a frame using a frequency domain (FD) coding scheme or a time domain (TD) coding scheme; determining whether to enforce the use of a TD coding scheme based on a plurality of conditions related to the one or more of the transient attack and the transient release (603); selecting (605) the TD coding scheme in response to determining that the TD coding scheme is to be forced to be used; a computer program product,

26. 26. The computer program product of claim 25, wherein the non-transitory storage medium comprises further program code, whereby execution of the further program code causes the encoder (202, 1802) to perform the operations of any one of claims 2 to 17.

Images