METHOD AND DEVICE FOR UNIFIED TIME DOMAIN / FREQUENCY DOMAIN CODING OF A SOUND SIGNAL

Abstract

A unified time-domain / frequency-domain coding method and a device for encoding an input sound signal comprise a classifier of the input sound signal into one of a plurality of sound signal categories, including an unclear signal type category indicating that the nature of the input sound signal is unclear. One of a plurality of coding sub-modes is selected to encode the input sound signal if the input sound signal is classified into the unclear signal type category. A mixed time-domain / frequency-domain encoder encodes the input sound signal using the selected coding sub-mode.The mixed time-domain / frequency-domain encoder comprises a frequency band selector and a bit allocator for selecting frequency bands for quantization and for distributing the available quantization bit budget among the selected frequency bands. The corresponding audio signal decoder and decoding method are also provided.

Description

METHOD AND DEVICE FOR UNIFIED TIME DOMAIN / FREQUENCY DOMAIN CODING OF A SOUND SIGNAL Field of Invention

[0001] The present description relates to a unified time-domain / frequency-domain encoding device and method that uses a mixed time-domain and frequency-domain encoding mode to encode an input sound signal, and the corresponding decoding device and decoding method.

[0002] In the present description and the accompanying claims: The term sound can be related to speech, generic audio signals such as music and reverberating speech, and any other sound. Background of the Invention

[0003] A state-of-the-art conversational codec can represent a clean speech signal with very good quality at a bit rate of approximately 8 kbps and approach transparency at a bit rate of 16 kbps. However, at bit rates below 16 kbps, low-processing-latency conversational codecs, which typically encode an input speech signal in the time domain, are not suitable for generic audio signals such as music and reverberating voice. To overcome this drawback, switched codecs have been introduced, essentially using a time-domain approach to encode speech-dominated input sound signals and a frequency-domain approach to encode generic audio signals.However, such switched solutions generally require a longer processing delay, necessary both for classifying the music from speech and for calculating a frequency domain transformation.

[0004] To overcome the above drawback related to a longer processing delay, a more unified time-domain and frequency-domain coding model has been proposed in U.S. Patent No. 9,015,038 (see Reference [1], the full content of which is incorporated herein by reference). This unified time-domain and frequency-domain coding model is part of the EVS (Enhanced Voice Services) audio codec standardized by 3GPP (Third Generation Partnership Project) as described in Reference [2], the full content of which is incorporated herein by reference.In recent years, 3GPP began working on the development of a 3D (three-dimensional) sound codec for immersive services called IVAS (Immersive Voice and Audio Services), based on the EVS codec (see reference [3] whose full content is incorporated herein by means of this reference).

[0005] To make the encoding model even more efficient for a specific type of signal, an additional encoding mode has been added to efficiently allocate the available bits between the time domain and the frequency domain, and between low and high frequencies. This additional encoding mode is activated by a new voice / music classifier whose output allows for an unclear category for signals that cannot be clearly classified as either music or voice (see reference [4], the full contents of which are incorporated herein by reference). Brief Description of the Invention

[0006] The present description relates to a unified time-domain / frequency-domain coding method for encoding an input sound signal. The method comprises: classifying the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal type category indicating that the nature of the input sound signal is unclear; selecting one of a plurality of coding sub-modes to encode the input sound signal if the input sound signal is classified into the unclear signal type category; and encoding the input sound signal in the mixed time-domain / frequency-domain using the selected coding sub-mode.

[0007] The present description also relates to a unified time-domain / frequency-domain coding method for encoding an input sound signal, comprising: classifying the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal type category indicating that the nature of the input sound signal is unclear; and encoding the input sound signal in the time-domain / frequency-domain in response to the classification of the input sound signal into the unclear signal type category.Mixed time-domain / frequency-domain encoding of the input sound signal comprises frequency band selection and bit allocation to select frequency bands for quantization and to distribute an available quantization bit budget among the selected frequency bands.

[0008] According to the present description, a unified time-domain / frequency-domain encoding device is further provided for encoding an input sound signal, comprising: a classifier of the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal type category indicating that the nature of the input sound signal is unclear; a selector of one of a plurality of encoding sub-modes for encoding the input sound signal if the input sound signal is classified into the unclear signal type category; and a mixed time-domain / frequency-domain encoder for encoding the input sound signal using the selected encoding sub-mode.

[0009] The present description further relates to a unified time-domain / frequency-domain encoding device for encoding an input sound signal, comprising: a classifier of the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal type category indicating that the nature of the input sound signal is unclear; and a mixed time-domain / frequency-domain encoder for encoding the input sound signal in response to the classification of the input sound signal into the unclear signal type category; and a mixed time-domain / frequency-domain encoder for encoding the input sound signal in response to the classification of the input sound signal into the unclear signal type category.The mixed time-domain / frequency-domain encoder comprises a frequency band selector and a bit allocator for selecting frequency bands for quantization and for distributing a budget of bits available for quantization among the selected frequency bands.

[0010] The present description provides a method for decoding sound signals comprising: receiving a bitstream carrying information usable for reconstructing a mixed time-domain / frequency-domain excitation representative of a sound signal classified in an unclear signal-type category showing that the nature of the sound signal is unclear, wherein the information includes one of a plurality of encoding submodes used to encode the sound signal classified in the unclear signal-type category; reconstructing the mixed time-domain / frequency-domain excitation in response to the information carried in the bitstream, including the encoding submode used to encode the input sound signal; converting the mixed time-domain / frequency-domain excitation into a time-domain excitation;and filter the time domain / mixed frequency domain excitation converted to time domain through a synthesis filter to produce a synthesized version of the sound signal.

[0011] The present description proposes a method for decoding sound signals comprising: receiving a bitstream carrying information usable to reconstruct a mixed time-domain / frequency-domain excitation representative of a sound signal (a) classified into an unclear signal-type category showing that the nature of the sound signal is unclear and (b) encoded using (i) frequency bands selected for quantization and (ii) a budget of bits available for quantization, distributed among the frequency bands;Reconstructing the mixed-frequency / time-domain excitation in response to the information carried in the bitstream, wherein reconstructing the mixed-frequency / time-domain excitation comprises selecting the frequency bands used for quantization and distributing the available bit budget for quantization among the frequency bands; converting the mixed-frequency / time-domain excitation into a time-domain signal; and filtering the time-domain / mixed-frequency-domain excitation converted into a time-domain signal through a synthesis filter to produce the synthesized version of the sound signal. bynonn / cznz / B / viAi

[0012] According to the present description, a sound signal decoder is provided comprising: a receiver of a bitstream carrying information usable for reconstructing a time-domain / mixed-frequency-domain excitation representative of a sound signal classified in an unclear signal-type category showing that the nature of the sound signal is unclear, wherein the information includes one of a plurality of encoding submodes used to encode the sound signal classified in the unclear signal-type category; a reconstructor of the time-domain / mixed-frequency-domain excitation in response to the information carried in the bitstream, including the encoding submode used to encode the input sound signal; and a converter of the time-domain / mixed-frequency-domain excitation into the time domain.and a synthesis filter to filter the time-domain / mixed frequency-domain excitation converted to time-domain to produce a synthesized version of the sound signal.

[0013] The present description further relates to a sound signal decoder comprising: a receiver of a bitstream carrying information usable for reconstructing a mixed time-domain / frequency-domain excitation representative of a sound signal (a) classified in an unclear signal-type category showing that the nature of the sound signal is unclear and (b) encoded using (i) frequency bands selected for quantization and (ii) a bit budget available for quantization distributed among the frequency bands; a reconstructor of the mixed time-domain / frequency-domain excitation in response to the information carried in the bitstream, wherein the reconstructor selects the frequency bands used for quantization and distributes the bit budget available for quantization among the frequency bands;a converter of the time domain / mixed frequency domain excitation into the time domain; and a filter to filter the converted mixed frequency bands of the time domain to produce a synthesized version of the sound signal.

[0014] The above and other features will become more evident after reading the following non-restrictive description of illustrative modalities of the unified time-domain / frequency-domain coding method, the unified time-domain / frequency-domain coding device, the decoding method, and the decoding device, provided by way of example only with reference to the accompanying drawings. Brief Description of the Figures

[0015] In the attached drawings:

[0016] Figure 1 is a schematic block diagram that simultaneously illustrates an overview of a unified time-domain / frequency-domain CELP (code-excited linear prediction) coding method and a corresponding unified time-domain / frequency-domain CELP coding device, e.g., coding method and device bynonn / cznz / B / viAi ACELP (linear prediction excited by algebraic code);

[0017] Figure 2 is a schematic block diagram of a more detailed structure of the unified time domain / frequency domain coding method and device of Figure 1, wherein a preprocessor performs a first level of analysis to classify the input sound signal;

[0018] Figure 3 is a schematic block diagram that simultaneously illustrates an overview of a time-domain cutoff frequency calculation device for an excitation contribution and a corresponding cutoff frequency estimation operation;

[0019] Figure 4 is a schematic block diagram illustrating a more detailed structure of the cutoff frequency calculation device of Figure 3, and of the corresponding cutoff frequency estimation operation;

[0020] Figure 5 is a schematic block diagram that simultaneously illustrates an overview of a frequency quantizer and a corresponding frequency quantization operation;

[0021] Figure 6 is a schematic block diagram of a more detailed structure of the frequency quantizer of Figure 5 and the frequency quantization operation;

[0022] Figure 7 is a schematic block diagram that simultaneously illustrates an alternative implementation of the unified time-domain / frequency-domain CELP coding method and the corresponding unified time-domain / frequency-domain CELP coding device;

[0023] Figure 8 is a schematic block diagram that simultaneously illustrates an encoding submode selection operation and a corresponding submode selector;

[0024] Figure 9 is a schematic block diagram that simultaneously illustrates a band selector and a bit allocator and a corresponding band selection and bit allocation operation for allocating the available bit budget to a frequency domain coding mode when the input sound signal is not categorized as either voice or music in the alternative implementation of Figures 7 and 8;

[0025] Figure 10 is a simplified block diagram of an example configuration of hardware components that make up the unified encoding device in the time domain / frequency domain and the method for encoding an input sound signal;

[0026] Figure 11 is a schematic block diagram simultaneously illustrating a decoder device 1100 and the corresponding decoding method 1150 for decoding a bit stream from the unified time-domain / frequency-domain encoding device and the corresponding unified time-domain / frequency-domain encoding method of Figure 7; and bjnonn / cznz / B / viAi

[0027] Figure 12 is a block schematic diagram that simultaneously illustrates a sound signal decoder and the corresponding sound signal decoding method for decoding a bit stream from the unified time domain / frequency domain encoding device and the corresponding unified time domain / frequency domain encoding method in the case of a sound signal classified into an unclear signal type category. Detailed Description of the Invention

[0028] This disclosure proposes a unified time-domain and frequency-domain coding model that improves the synthesis quality for generic audio signals such as, for example, music and / or reverberating speech, without increasing processing delay and bit rate. This unified time-domain and frequency-domain coding model comprises: - A time-domain coding mode that operates in the linear prediction (LP) residual domain where the available bits are dynamically allocated between an adaptive codebook, one or more fixed codebooks (e.g., an algebraic codebook, a Gaussian codebook, etc.), a variable-length fixed codebook; and - a frequency domain encoding mode, depending on the characteristics of the input sound signal.

[0029] To achieve a low-bitrate, low-processing-latency conversational sound codec that improves the synthesis quality of generic audio signals such as, for example, music and / or reverberating speech, the frequency-domain encoding mode is integrated as closely as possible to a CELP (code-excited linear prediction) time-domain encoding mode. For this purpose, the frequency-domain encoding mode uses a frequency transform performed in the LP (linear prediction) residual domain. This allows for almost artifact-free switching from one frame, for example, a 20 ms frame, to another. As is well known in the field of sound codecs, the input sound signal is sampled at a given sampling rate and processed by groups of these samples called structures, typically divided into several substructures.In this case, the integration of the two (2) coding modes in the time domain and in the frequency domain is close enough to allow dynamic reallocation of the bit budget to another coding mode if it is determined that the current coding mode is not efficient enough.

[0030] A feature of the proposed unified time-domain and frequency-domain coding model is a variable time support for the time-domain component, ranging from a quarter of a structure (substructure) to a full structure on a structure-by-structure basis. As a non-limiting illustrative example, one structure might represent 20 ms of input audio signal. Such a structure corresponds to 320 samples of the input audio signal if the internal sampling rate of the audio codec is 16 kHz, or 256 samples per structure if the internal sampling rate of the codec is 12.8 kHz. A substructure (quarter of a structure in this example) then represents 80 or 64 samples, depending on the internal sampling rate of the audio codec. In this non-limiting illustrative case, the internal sampling rate of the audio codec is 12.8 kHz.8 kHz giving a structure length of 256 samples and a substructure length of 64 samples of the input sound signal. Variable time support allows capturing important time events at a minimal bit rate to create a basic time-domain excitation contribution. At very low bit rates, the time support is typically the entire frame. In that case, the time-domain excitation contribution consists only of the adaptive codebook; the corresponding adaptive codebook information and gain (tone) are then transmitted once per frame. When higher bit rates are available, it is possible to capture more time events by shortening the time support and increasing the bit rate allocated to the time-domain encoding mode.Finally, when the time support is sufficiently short (shorter than a quarter of a structure (substructure)), and the available bit rate is sufficiently high, the time domain contribution of the excitation may include, for each substructure, the adaptive codebook contribution with the corresponding adaptive codebook gain, a fixed codebook contribution with a corresponding fixed codebook gain, or both the adaptive and fixed codebook contributions with the corresponding gains.Alternatively, it is also possible to transport, for each half of a structure (substructure), an adaptive codebook contribution with the corresponding adaptive codebook gain and a fixed codebook contribution with the corresponding fixed codebook gain; this has the advantage of not consuming too much bit rate while still being able to encode temporal events. The parameters describing the codebook indices and gains are then transmitted for each substructure.

[0032] At low bit rates, conversational sound codecs are unable to adequately encode higher frequencies. This results in a significant degradation of synthesis quality when the input sound signal includes music and / or reverberating speech. To address this problem, a feature is added to calculate the efficiency of the excitation contribution in the time domain. In some cases, regardless of the input bit rate and timeframe support, the excitation contribution in the time domain is not valuable. In such cases, all bits are reallocated to the next encoding stage in the frequency domain. However, most of the time, the excitation contribution in the time domain is valuable only up to a certain frequency (in this document, after the cutoff frequency).In these cases, the time-domain excitation contribution above the cutoff frequency is filtered out. This filtering operation allows valuable information encoded with the time-domain excitation contribution to be retained while eliminating non-valuable information above the cutoff frequency. In a non-restrictive, illustrative example, the filtering is performed in the frequency domain by setting the frequency intervals above a certain frequency (the cutoff frequency) to zero.

[0033] Variable time support combined with variable cutoff frequency makes the bit allocation within the unified time-domain and frequency-domain coding model highly dynamic. The bit rate after LP filter quantization can be allocated entirely to the time domain, entirely to the frequency domain, or somewhere in between. The bit rate allocation between the time and frequency domains depends on the number of substructures used for time-domain excitation contribution, the available bit budget, and the calculated cutoff frequency.To make the unified coding model in the time and frequency domains even more efficient for a specific type of input sound signal, specific coding sub-modes are added to efficiently allocate the available bits between the time domain, the frequency domain, and between low and high frequencies. These added specific coding sub-modes are determined using a new voice / music audio classifier that produces an output that allows for an unclear signal category (signals that cannot be clearly classified as either music or voice).

[0034] To create a total excitation that more efficiently matches the input LP residue, frequency-domain encoding is applied. One feature is that the frequency-domain encoding is performed on a vector containing the difference between a frequency representation (frequency transform) of the input LP residue and a frequency representation (frequency transform) of the time-domain excitation contribution filtered to the cutoff frequency, and containing a frequency representation (frequency transform) of the input LP residue itself above that cutoff frequency. A smooth spectrum transition is inserted between the two segments just above the cutoff frequency. In other words, the high-frequency portion of the time-domain frequency representation of the excitation contribution is first zeroed out above the cutoff frequency.A transition region between the unchanged portion of the spectrum and the zeroed portion of the excitation contribution spectrum in the time domain is inserted just above the cutoff frequency to ensure a smooth transition between the two parts of the spectrum. This modified spectrum of the excitation contribution in the time domain is then subtracted from the frequency representation of the input LP residue. Therefore, the resulting spectrum corresponds to the difference between the two spectra below the cutoff frequency and the frequency representation of the LP residue above it, with some transition region. The cutoff frequency, as mentioned earlier, can vary from one structure to another.

[0035] Whatever frequency quantization method (frequency-domain coding mode) is chosen, there is always the possibility of pre-echo, especially with long windows. In the technique disclosed herein, the windows used are square windows, so the additional window length compared to the encoded input sound signal is zero (0), i.e., no overlap-addition is used. While this corresponds to the best window for reducing any potential pre-echo, some pre-echoes may still be audible in time attacks. Many techniques exist for resolving such a pre-echo problem, but the present description proposes a simple feature to cancel it. This feature is based on a memoryless time-domain coding mode derived from the Transition Mode of ITU-T Recommendation G.718; Reference [5J, sections 6.8.1.4 and 6.8.4].2, the full content of which is incorporated herein by reference. The idea behind this feature is to take advantage of the fact that the proposed unified time-domain and frequency-domain coding model is integrated into LP's residual domain, allowing for artifact-free switching at almost any time. When an input sound signal is considered generic audio (music and / or reverberating voice) and a time attack is detected in a structure, then that structure is encoded using only the memoryless time-domain coding mode. This memoryless time-domain coding mode will handle the time attack, thus avoiding the pre-echo that could be introduced when using frequency-domain coding for that structure. NON-RESTRICTIVE ILLUSTRATIVE REALIZATION

[0036] In the proposed unified time-domain and frequency-domain coding model, the aforementioned adaptive codebook, one or more fixed codebooks (e.g., an algebraic codebook, a Gaussian codebook, etc.), i.e., the so-called time-domain codebooks, and frequency-domain quantization (frequency-domain coding mode) can be viewed as a codebook library, and the bits can be distributed among all available codebooks, or a subset thereof. This means, for example, that if the input sound signal is a clean voice, all bits will be allocated to the time-domain coding mode, essentially reducing the coding to the legacy CELP scheme.On the other hand, for some music segments, all the bits allocated to encoding the input LP residue are sometimes better spent in the frequency domain, for example, in the transform domain. Furthermore, specific cases can be added where (a) the time domain uses a larger portion of the total available bit rate to encode more time-domain events while still retaining bits to encode some frequency information, or (b) low-frequency content is prioritized over high-frequency content, and vice versa.

[0037] As indicated in the description above, the time support for the time-domain and frequency-domain coding modes need not be the same. While the bits spent on the different time-domain coding operations (adaptive and algebraic codebook lookups) are normally distributed on a substructure basis (usually a quarter of a structure, or 5 ms of time support), the bits allocated to the frequency-domain coding mode are distributed on a structure basis (usually 20 ms of time support) to improve frequency resolution.

[0038] The bit budget allocated to the time-domain CELP encoding mode can also be dynamically controlled depending on the input audio signal. In some cases, the bit budget allocated to the time-domain CELP encoding mode can be zero, effectively meaning that the entire bit budget is allocated to the frequency-domain encoding mode. Choosing to work in the residual LP domain for both the time-domain and frequency-domain encoding modes has two main benefits. First, it is compatible with the time-domain CELP encoding mode, which has proven efficient in encoding speech signals. Consequently, no artifacts are introduced due to switching between the two types of encoding modes (time-domain and frequency-domain).Secondly, the lower dynamic range of the LP residue with respect to the original input sound signal, and its relative flatness, facilitate the use of a square window for frequency transforms, thus allowing the use of a non-overlapping window.

[0039] In a non-limiting example where the codec's internal sampling rate is 12.8 kHz (meaning 256 samples per structure), similar to U1TT Recommendation G.718 (Reference [5]), the length of the substructures used in CELP encoding mode in the time domain can vary from a typical structure length (5 ms) to half a structure length (10 ms) or a full structure length (20 ms). The substructure length decision is based on the available bit rate and an analysis of the input audio signal, particularly its spectral dynamics. The substructure length decision can be made in a closed-loop manner. To save complexity, it is also possible to base the substructure length decision on an open-loop basis.The substructure length decision can also be controlled by the nature of the input sound signal detected by a signal classifier, for example, a speech / music classifier. The substructure length can be changed from structure to structure.

[0040] Once the length of the substructures in a current structure is chosen, a standard closed-loop pitch analysis is performed, and the first contribution to the excitation signal is selected from the adaptive codebook. Then, depending on the available bit budget and the characteristics of the input sound signal (e.g., in the case of an input speech signal), a second contribution from one or more fixed codebooks can be added before the transformation-domain conversion. The resulting excitation contribution is the time-domain excitation contribution. On the other hand, at very low bit rates and in the case of a generic audio signal, it is often better to omit the fixed codebook stage and use all the remaining bits for the transformation-domain coding.Encoding in the transformation domain can be, for example, a frequency-domain encoding mode. As described earlier, the substructure length can be one-quarter of the structure, one-half of the structure, or a full-length structure. The fixed codebook contribution is used only if the substructure length is equal to one-quarter of the structure length. If the substructure length is determined to be half of a structure or the entire structure, then only the adaptive codebook contribution is used to represent the excitation contribution in the time domain, and all remaining bits are allocated to the frequency-domain encoding mode. Alternatively, an additional encoding mode will be described where the fixed codebook can be used when the substructure length is equal to half the structure length.This addition has been made to improve the quality of particular types of input sound signals that contain a temporal event while maintaining an acceptable bit budget for encoding the excitation contribution in the frequency domain.

[0041] Once the calculation of the time-domain excitation contribution is complete, its efficiency needs to be evaluated and quantified. If the time-domain coding gain is very low, it is more efficient to eliminate the time-domain excitation contribution entirely and use all bits for the frequency-domain coding mode. On the other hand, for example, in the case of a clean input voice signal, the frequency-domain coding mode is not needed, and all bits are allocated to the time-domain coding mode. However, time-domain coding is often efficient only up to a certain frequency. This frequency corresponds to the previously mentioned cutoff frequency of the time-domain excitation contribution.Determining this cutoff frequency ensures that all time-domain coding is helping to achieve a better final synthesis rather than working against the frequency-domain coding.

[0042] The cutoff frequency can be estimated in the frequency domain. To calculate the cutoff frequency, the time-domain spectra of both the LP residual and the excitation contribution are first divided into a predefined number of frequency bands, each of which is further divided into a number of frequency bins. The number of frequency bands and the number of frequency bins covered by each frequency band can vary from one implementation to another. For each frequency band, a normalized correlation is calculated between the time-domain frequency representation of the excitation contribution and the frequency representation of the LP residual, and the correlation is smoothed between adjacent frequency bands. As a non-limiting example, the correlations per band are limited to 0.The correlations for the frequency bands are normalized between 0 and 1, and then an average correlation is calculated as the average of the correlations for all frequency bands. For a first estimate of the cutoff frequency, the average correlation is then scaled between 0 and half the internal sampling rate (half the internal sampling rate corresponding to the normalized correlation value of 1). At very low bit rates or for additional coding submodes as described below, the average correlation is doubled before finding the cutoff frequency. This is done for cases where the time-domain excitation contribution is known to be necessary even if the correlation is not very high due to the low bit rate being used, or because the type of input sound signal would not allow for a high correlation.The first estimate of the cutoff frequency is then found as the upper limit of the frequency band that is closest to the scaled mean correlation value. In one implementation example, sixteen (16) frequency bands are defined at an internal sampling rate of 12.8 kHz for the correlation calculation.

[0043] By taking advantage of the psychoacoustic properties of the human ear, the reliability of the cutoff frequency estimate can be improved by comparing the estimated position of the octave harmonic frequency of the tone with the cutoff frequency estimated by the correlation calculation. If this position is higher than the cutoff frequency estimated by the correlation calculation, the cutoff frequency is modified to correspond to the position of the eighth harmonic frequency of the tone. If one of the additional encoding sub-modes is used, the cutoff frequency has a minimum value greater than or equal to, for example, 2775 Hz (september band). The final cutoff frequency value is then quantized and transmitted to a distant decoder. In one implementation example, 3 or 4 bits are used for such quantization, giving 8 or 16 possible cutoff frequencies depending on the bit rate.

[0044] Once the cutoff frequency is known, the frequency quantization of the excitation contribution is performed in the frequency domain. First, the difference between the frequency representation (frequency transform) of the input LP residue and the frequency representation (frequency transform) of the excitation contribution in the time domain is determined. A new vector is then created, consisting of this difference up to the cutoff frequency, and a smooth transition to the frequency representation of the input LP residue for the remaining spectrum. Frequency quantization is then applied to the entire new vector. In one implementation example, the quantization consists of encoding the sign and position of the dominant (most energetic) spectral pulses.The number of pulses to be quantized per frequency band is related to the bit rate available for the encoding mode in the frequency domain. If the available bits are insufficient to cover all frequency bands, the remaining bands are filled with noise.

[0045] Frequency quantization of a frequency band using the quantization method described in the preceding paragraph does not guarantee that all frequency containers within this band are quantized. This is especially true at low bit rates where the number of quantized spectral pulses per frequency band is relatively low. To avoid audible artifacts due to these unquantized containers, some noise is added to fill these gaps. Since at low bit rates the quantized spectral pulses must dominate the spectrum rather than the inserted noise, the amplitude of the noise spectrum corresponds to only a fraction of the pulse amplitude. The amplitude of the noise added to the spectrum is greater when the available bit budget is low (allowing for more noise) and smaller when the available bit budget is high.

[0046] In frequency-domain coding mode, gains are calculated for each frequency band to match the energy of the unquantized signal with the quantized signal. The gains are vectorially quantized and applied per band to the quantized signal. When, for example, the unified time-domain and frequency-domain coding model changes the bit allocation from a time-domain-only coding mode to a mixed time-domain / frequency-domain coding mode, the per-band excitation spectrum energy of the time-domain-only coding mode does not match the per-band excitation spectrum energy of the mixed time-domain / frequency-domain coding mode. This energy mismatch can create some switching artifacts, especially at low bit rates.To reduce any audible degradation created by this bit reallocation, a long-term gain can be calculated for each band and applied to correct the power of each frequency band for a few structures after switching from time-domain-only coding mode to mixed time-domain / frequency-domain coding mode.

[0047] After the frequency-domain encoding mode is completed, the total excitation is found by adding the frequency-domain excitation contribution to the frequency-transform representation of the time-domain excitation contribution. The sum of these two excitation contributions is then transformed back into the time domain to form the total excitation. Finally, the synthesized signal is calculated by filtering the total excitation through an LP synthesis filter.

[0048] In one mode, while CELP coding memories are updated on a substructure basis using only the time-domain excitation contribution, the total excitation is used to update those memories at structure boundaries. bynonn / cznz / B / viAi

[0049] In another possible implementation, the CELP coding memories are updated on a substructure basis and also at structure boundaries using only the time-domain excitation contribution. This results in an integrated structure where the frequency-domain encoded signal constitutes an upper quantization layer independent of the core CELP layer. In this particular case, the fixed codebook is always used to update the contents of the adaptive codebook. However, the frequency-domain coding mode can be applied to the entire structure. This integrated approach works for bit rates of approximately 12 kbps and above. 1) Classification of the type of sound signal

[0050] Figure 1 is a schematic block diagram that simultaneously illustrates an overview of a unified time-domain / frequency-domain CELP coding method 150 and a corresponding unified time-domain / frequency-domain CELP coding device 100, e.g., the ACELP method and device. Of course, other types of CELP coding methods and devices can be implemented using the same concept.

[0051] Figure 2 is a schematic block diagram of a more detailed structure of the unified time-domain / frequency-domain CELP coding method 150 and device 100 of Figure 1.

[0052] The unified time-domain / frequency-domain CELP encoding device 100 comprises a preprocessor 102 (Figure 1) for performing an operation 152 of parameter analysis of the input sound signal 101 (Figures 1 and 2). With reference to Figure 2, the preprocessor 102 comprises an LP analyzer 201 for performing an operation 251 of LP analysis of the input sound signal 101, a spectral analyzer 202 for performing an operation 252 of spectral analysis, an open-loop pass analyzer 203 for performing an operation 253 of open-loop pass analysis, and a signal classifier 204 for performing an operation 254 of classification of the input sound signal. Analyzers 201 and 202 and associated operations 251 and 252 perform the LP and spectral analyses generally carried out in CELP coding, as described, for example, in ITU-T Recommendation G.718, Reference [5], sections 6.4 and 6.1.4, and therefore will not be described further in this description.

[0053] The preprocessor 102 performs a first level of analysis to classify the input sound signal 101 as either voice or non-voice (generic audio (music or reverberating voice)), for example, in a manner similar to that described in Reference [6], the full content of which is incorporated herein by reference, or with any other reliable method of voice / non-voice discrimination.

[0054] After this first level of analysis, preprocessor 102 performs a second level of analysis of the input signal parameters to allow the use of CELP encoding in the time domain (without encoding in the frequency domain) on some sound signals with strong ς Or non-voice features, but which are still best encoded with a time-domain focus. When a significant energy variation ζ occurs, this second level of analysis allows the unified time-domain / frequency-domain CELP coding device to switch to a memoryless time-domain coding mode, generally called Transition Mode. Reference [7 J, the full content of which is incorporated herein by means of this reference.

[0055] During this second level of analysis, the signal classifier 204 calculates and uses a variation of a smoothed version C„ of an open-loop pitch correlation from the open-loop pitch analyzer 203, a current total structure energy (total energy of the input sound signal in the current structure), and a difference between the current total structure energy and the previous total structure energy. First, the signal classifier 204 calculates the variation of the smoothed open-loop pitch correlation using, for example, the following relationship: Gc~ l ¿ k Γο J \;=0where: - C;: is the smoothed open-loop step correlation defined as: = 0.9 ·6\ - 0.1 -G- - is the open-loop pitch correlation calculated by the analyzer 203 using a method known to experts in the CELP coding technique, for example, as described in ITU-T Recommendation G.718, Reference [5], Section 6.6; - C!Zes is an average over the last 10 i structures of the smoothed open-loop pitch correlation G:; - <?C es la variación de la correlación de paso de bucle abierto suavizada.

[0056] When, during the first level of analysis, signal classifier 204 classifies a structure as voiceless, signal classifier 204 performs the following checks to determine, at the second level of analysis, whether it is truly safe to use a mixed time-domain / frequency-domain coding mode. Sometimes, however, it is better to encode the actual structure using only the time-domain coding mode, employing one of the time-domain approaches estimated by the time-domain coding mode preprocessing function. In particular, it might be better to use the memoryless time-domain coding mode to minimize any potential pre-echo that could be introduced with a mixed time-domain / frequency-domain coding mode.

[0057] As a non-limiting implementation of a first check of whether the mixed time-domain / frequency-domain coding mode should be used, the signal classifier 204 calculates a difference between the current total structure energy and the total energy of the previous structure. When the difference £< between the current total structure energy E,,,, and the total energy of the previous structure is greater than, for example, 6 dB, this corresponds to a so-called time attack on the input sound signal 101. In such a situation, the voice / non-voice decision and the selected coding mode are overwritten, and a memoryless time-domain coding mode is forced.More specifically, the unified time-domain / frequency-domain CELP coding device 100 comprises a time-domain / time-frequency coding selector 103 (Figure 1) for performing a selection operation 153 between time-domain-only coding and mixed time-domain / frequency-domain coding. To that end, the time-domain / time-frequency coding selector 103 comprises a generic voice / audio selector 205 (Figure 2) for performing a selection operation 255 between voice and generic audio for classifying the input sound signal 101, a time-attack detector 208 (Figure 2) for performing a time-attack detection operation 258 on the input sound signal 101, and a selector 206 (Figure 2) for performing a selection operation 256 of the memoryless time-domain coding mode. In other words: - In response to a determination of the voice signal by selector 205, a closed-loop CELP encoder 207 (Figure 2) is used to perform a CELP encoding operation 257 of the voice signal. In response to both a determination of a non-voice (generic audio) signal by selector 205 and a detection of a time attack on the input sound signal 101 by detector 208, selector 206 forces the closed-loop CELP encoder 207 (Figure 2) to use the memoryless time-domain encoding mode to encode the input sound signal. The closed-loop CELP encoder 207 is part of the time-domain-only encoder 104 in Figure 1. A closed-loop CELP encoder is well known to those skilled in the art and will not be described further herein.

[0058] As a non-limiting implementation of the second check of whether mixed-mode time-domain / frequency-domain coding should be used, when the difference £< / , / entre la energía de estructura total actual E„> and the total energy of the previous structure is less than or equal to 6 dB, but: - the smoothed open-loop pass correlation C„ is greater than 0.96; or - the smoothed open-loop pass correlation Cv is greater than 0.85 and the difference between the current total structure energy £,„, and the previous total structure energy is less than 0.3 dB; or - the smoothed open-loop step correlation variation is below 0.1 and the difference between the current total structure energy and the previous last total structure energy is below 0.6 dB; or - the current total energy of the panel E„,zes is less than 20 dB; and this is at least the second consecutive structure (cnt > 2) where the first level analysis decision is changed, then the generic voice / audio selector 205 determines that the current structure will be encoded using a time-domain only encoding mode using the closed-loop CELP encoder 207 (Figure 2).

[0059] Otherwise, the time / time-frequency encoding selector 103 selects the mixed time domain / frequency domain encoding mode as described in the following description.

[0060] The second verification can be summarized, for example, when the input sound signal without voice is music, using the following pseudocode: if (generic audio) if(Ed¡ff>6dB) codingmode = Time domain memory less cnt = l elseif^Cf, >0.96 | (Cv, >0.85 &E(<< 0.3ί / Β)|(σΓ< 0.1 &Ed¡(f <Q.6dB)\Eltil<20dB) cnt++ if (cnt > =2) coding mode = Time domain else coding mode = mix time / frequency domain cnt = 0 bynonn / cznz / B / vi where Ει,,ι is the current total structure energy expressed as: E,„, =101og where x(i) represents the samples of the input sound signal in the current structure, / Ves the The number of samples of the input sound signal per structure, and J'J] is the difference between the current total structure energy EM and the previous total structure energy.

[0061] Figure 7 is a schematic block diagram that simultaneously illustrates an alternative implementation of the unified time-domain / frequency-domain CELP coding method 750 and the corresponding unified time-domain / frequency-domain CELP coding device 700, wherein the preprocessor 702 also performs a first level of analysis to classify the input sound signal 101.

[0062] Specifically, the unified time-domain / frequency-domain CELP coding method 750 comprises an input sound signal preprocessing operation 752, as described in Reference [4], to obtain the parameters required to classify this input sound signal. To perform operation 752, the mixed time-domain / frequency-domain / frequency-domain CELP coding device 700 comprises the preprocessor 702.

[0063] The unified time-domain / frequency-domain CELP coding method 750 comprises an operation 751 of classifying the input sound signal 101 into voice, music, and unclear signal type categories using the preprocessor parameters 702 in a manner similar to that described in Reference [4], or using any other method of discriminating between voice / music and unclear signal types. The unclear signal type category indicates that the nature of the input sound signal 101 is unclear and, in particular, that the input sound signal 101 is not classified as either voice or music. To perform operation 751, the unified time-domain / frequency-domain CELP coding device 700 comprises a sound signal classifier 701.

[0064] If the sound signal classifier 701 classifies the input sound signal 101 into the music category, a frequency domain encoder 703 performs an encoding operation 753 on the input sound signal 101 using frequency domain encoding as described, for example, in Reference [2]. The frequency domain encoded music signal can then be synthesized in a music synthesis operation 754 performed by a synthesizer 704 to recover the music signal.

[0065] Similarly, if the sound signal classifier 701 classifies the input sound signal 101 into the speech category, a time-domain encoder 705 performs an encoding operation 755 on the input sound signal 101 using time-domain encoding as described, for example, in Reference [2]. The time-domain encoded speech signal can then be synthesized in a synthesis filtering operation 756 performed by a synthesizer 706 that includes a synthesis filter to recover the speech signal.

[0066] Accordingly, the 700 unified time-domain / frequency-domain encoding device and the 750 method maximize the performance of time-domain-only and frequency-domain-only encoding by respectively limiting their use to input sound signals that have clear speech characteristics and input sound signals that have clear music characteristics. This increases the overall quality of all types of input sound signals at low to medium bit rates.

[0067] The encoding submodes have been designed as part of the unified time-domain and frequency-domain encoding model to efficiently encode input sound signals that are not classified as voice or music (unclear signal type category). Two (2) bits are used to signal three (3) encoding submodes identified by corresponding submode flags. A fourth submode allows for backward interoperability with the legacy unified time-domain and frequency-domain (EVS) encoding model.

[0068] As illustrated in Figure 8, operation 751 for classifying input sound signal 101 comprises operation 850 for selecting one of the encoding sub-modes in response to the bit rate available for encoding input sound signal 101 and the characteristics of this input sound signal classified as unclear. To perform operation 850, the sound signal classifier 701 incorporates a sub-mode selector 800.

[0069] Encoding submodes are identified by a submode indicator FtfSm. In the non-limiting implementation of Figure 8, submode selector 800 selects the encoding submodes as follows: - Submode selector 800 selects the back-encoding submode mentioned above if (a) the bit rate available for encoding the input sound signal 101 is no greater than 9.2 kbps and (b) the input sound signal 101 is not classified as voice or music (see 803). The FtfSmse submode indicator is then set to 0 (see 802). Selecting the back-encoding mode causes the use of the unified time-domain and frequency-domain encoding model inherited from Figures 1 and 2 (EVS). - Submode selector 800 selects a first encoding submode if (a) the input sound signal 101 is not classified as voice or music by classifier 701 and the available bit rate is high enough to allow encoding of adaptive and fixed codebooks and gains, which generally means a bit rate greater than 9.2 kbps (see 803), (b) a probability that the input sound signal 101 is music (compensated voice / music decision tending towards music(n)) is not greater than 0 (see 804), and (c) no probability of time attack is detected in the current structure of the input sound signal (the transition counter is not greater than 0 as described in U1T-T Recommendation G.718, Reference [5], section 6.8.1.4 and section 6.8.4.2) (see 806). The FtfSmse submode indicator is then set to 1 (see 801).Although input sound signal 101 is not classified as voice or music by classifier 701, selector 800 detects voice-like characteristics in input sound signal 101 and selects the first encoding submode (submode indicator Ftf..m=\) since CELP is not optimal for encoding such a sound signal. - Submode selector 800 selects a second encoding submode if (a) the input sound signal 101 is not classified as voice or music by classifier 701 and the available bit rate is high enough to allow encoding of adaptive and fixed codebooks and gains, which generally means a bit rate greater than 9.2 kbps (see 803), (b) a probability that the input sound signal 101 is music (compensated voice / music decision tending towards music, wdZp(n)) is not greater than 0 (see 804), and (c) a probability of temporary attack is detected in the current structure of the input sound signal (the transition counter is not greater than 0 as described in Recommendation U1T-T G.718, Reference [5], section 6.8.1.4 and section 6.8.4.2) (see 806). The Ftfsmse submode indicator is then set to 2 (see 807).As will be explained in the following description, the second encoding submode (submode indicator FtfSm=2) assigns more hits to the lower part of the spectrum. - Submode selector 800 selects a third encoding submode if (a) the input sound signal 101 is not classified as either voice or music by classifier 701 and the available bit rate is high enough to allow encoding of at least the adaptive codebook and gains and still has a significant number of bits for frequency encoding, which generally means a bit rate greater than 9.2 kbps (see 803), and (b) the probability that the input sound signal 101 is music (compensated voice / music decision tending toward music, wcZZp(n)) is greater than 0) (see 804). The submode indicator FtfSmse is then set to 3 (see 808).Although input sound signal 101 is not classified as either voice or music by classifier 701, selector 800 detects music-like characteristics in input sound signal 101 and selects the third encoding submode (Submode indicator Ft^s;„ = 3). Such a sound signal segment is still considered non-musical, but the submode indicator FtfSmse is set to 3 (selection of the third encoding submode), indicating that the samples include high-frequency or tonal content. The probability that the input sound signal 101 is voice, music, or intermediate is described in Reference [4]. When the classification of voice or music is unclear, if the probability wdlp(n) is greater than 0, the signal is considered to have some musical characteristic. The following table shows the threshold where the probability would be high enough to be considered music or voice. Table 1: Probability thresholds for the unclear category bynonn / cznz / B / viAi UNTIL IMPRECISE VOICE MUSIC FROM IMPRECISE <-2.5 <2.5

[0070] The selected encoding submode, for example, the submode indicator FtfSm, is transmitted to the bitstream of a remote decoder. The path chosen within the decoder depends on the signaling bits included in the bitstream. Once the decoder detects the presence of a structure encoded using mixed time-domain / frequency-domain / frequency-domain encoding, the submode indicator Ftfsms is decoded from the bitstream. If the detected submode indicator Ftfsmes is 0, then the EVS legacy interoperable unified time-domain and frequency-domain encoding model will be used to decode the remaining portion of the bitstream. On the other hand, if the submode indicator Ftfsmes is not 0, submode decoding follows.The decoder will replicate the procedure followed by the encoder, in particular the distribution of bits between the time domain and the frequency domain and the allocation of bits in the different frequency bands as described later in section 6.2. 2) Decision on the length of the substructure

[0071] In typical CELP, input sound signal samples are processed into 10–30 ms structures, and these structures are divided into substructures for adaptive and fixed codebook analysis. For example, a 20 ms structure (256 samples when the internal sampling rate is 12.8 kHz) can be used and divided into four 5 ms substructures. A variable substructure length is a feature used to integrate the time domain and frequency domain into one encoding mode. The substructure length can range from a typical 1 / 4 of the structure length to half the structure length or a full structure length. Of course, other numbers of substructures (substructure lengths) can also be implemented.

[0072] The parameter analysis operation 152 of the unified time-domain / frequency-domain CELP coding method 150 comprises, as illustrated in Figure 2, an operation 259 for determining a high spectral dynamic range of the input sound signal 101, and an operation 260 for calculating the number of substructures per structure. To perform operations 259 and 260, the preprocessor 102 of the unified time-domain / frequency-domain CELP coding device 100 comprises, respectively, a high-spectrum dynamic analyzer 209 and a device 210 for calculating the number of substructures.

[0073] The decision regarding the length of the substructures (the number of substructures), or the time support, is determined by the computing device 210 based on the available bit rate and the analysis of the input sound signal, in particular the high spectral dynamics of the input sound signal 101 from analyzer 209 and the open-loop pitch analysis, which includes the smoothed open-loop pitch correlation CsZ, from analyzer 203. The high spectral dynamics analyzer 209 is sensitive to the information from spectral analyzer 202 to determine the high spectral dynamics of the input sound signal 101. The high spectral dynamics are calculated, for example, as described in ITU-T Recommendation G.718, Reference [5], section 6.7.2.2, as an input spectrum without background noise, which gives a representation of the dynamics of the input spectrum.When the average spectral dynamics of the input sound signal 101 in the frequency band between 4.4 kHz and 6.4 kHz, as determined by analyzer 209, is below, for example, 9.6 dB, and the last structure was considered to have high spectral dynamics, the input sound signal 101 is no longer considered to have high spectral dynamics. In that case, more bits can be allocated to the frequencies below, for example, 4 kHz, by adding more substructures to the time-domain encoding mode or by forcing more pulses in the lower frequency portion of the frequency-domain encoding mode.

[0074] On the other hand, if the increase in the average spectral dynamics of input sound signal 101 compared to the average spectral dynamics of the last structure not considered to have high spectral dynamics, as determined by analyzer 209, is greater than, for example, 4.5 dB, input sound signal 101 is considered to have high spectral dynamics content above, for example, 4 kHz. In that case, depending on the available bit rate, some additional bits are used to encode the high frequencies of input sound signal 101 to allow the encoding of one or more frequency pulses.

[0075] The substructure length determined by the calculation device 210 (Figure 2) also depends on the bit budget available for encoding the input sound signal 101. At very low bit rates, for example, bit rates below 9 kbps, only one substructure is available for time-domain encoding; otherwise, the number of available bits will be insufficient for frequency-domain encoding. At medium bit rates, for example, bit rates between 9 kbps and 16 kbps, one substructure is used when the high frequencies contain high spectral dynamic content, and two substructures are used if they do not. For medium-high bit rates, for example, bit rates of approximately 16 kbps and above, the case of four (4) substructures is also available if the smoothed open-loop pitch correlation C defined above is greater than, for example, 0.8.

[0076] While the case with one or two substructures limits the time-domain coding to an adaptive codebook contribution only (with encoded pitch delay and pitch gain), i.e., a fixed codebook is not used in that case, the case with four (4) substructures allows both adaptive and fixed codebook contributions if the available bit budget is sufficient. The four (4) substructure case is permitted at bit rates starting at approximately 16 kbps. Due to bit budget limitations, the time-domain excitation contribution consists only of the adaptive codebook contribution at lower bit rates. A fixed codebook contribution can be added at higher bit rates, for example, starting at 24 kbps.In all cases, the efficiency of time-domain coding will be evaluated later to decide up to what frequency (the cutoff frequency mentioned above) such time-domain coding is valuable.

[0077] The alternative implementation of Figures 7 and 8 uses the first, second, or third encoding submodes defined above when the input sound signal 101 is classified by classifier 701 in the unclear signal type category and the FtfSmes submode indicator is greater than zero 0.

[0078] The sound signal classifier 701 determines that the number of substructures is four (4) unless the submode indicator FtfSm is set to 1 or 2 (selection of the first or second encoding submode), which means that the content of the input sound signal 101 is closer to speech (speech-like features or the probability of a temporary attack are detected in the input sound signal 101) and the available bit rate is below 15 kbps. Specifically: - In the first or second encoding submodes (submode indicator Ftfsm set to 1 or 2), the sound signal classifier 701 determines a number of four (4) substructures unless the bit rate available to encode the input sound signal 101 is below 15 kbps; then an encoding mode using two (2) substructures will be selected.In both cases, a corresponding number of fixed code books is used, i.e., a number of two (2) or four (4) fixed code books; and. - In the third encoding mode (submode indicator FtfSm set to 3, meaning that the content of input sound signal 101 is closer to music (music-like features are detected in input sound signal 101), sound signal classifier 701 determines that the number of substructures is four (4) but a fixed codebook contribution is not used to keep more bits available for the frequency domain excitation contribution, unless the bit rate available to encode input sound signal 101 is greater than or equal to 22.6 kbps. 3) Closed-loop step analysis

[0079] In the unified time-domain / frequency-domain CELP encoding device 100 and method 150 (Figure 1), a mixed time-domain / frequency-domain encoding method 170 and a corresponding mixed time-domain / frequency-domain encoder 120 are used when generic audio is selected by selector 205 as the input sound signal classification 101 and no time attack is detected in detector 208. bynonn / cznz / B / viAi Alternatively, in the unified time-domain / frequency-domain CELP encoding device 700 and method 750 (Figure 7), a mixed time-domain / frequency-domain encoding method 770 and a corresponding mixed time-domain / frequency-domain encoder 720 are used when the sound signal classifier 701 classifies the input sound signal 101 into the unclear signal type category and one of the first, second, and third encoding submodes defined above is selected (submode indicator FtfSm set to 1, 2, or 3).

[0080] When using the mixed time-domain / frequency-domain encoding mode, a closed-loop pitch analysis is performed, followed, if necessary, by a fixed algebraic codebook search. For this purpose, the 170 / 770 mixed time-domain / frequency-domain encoding method comprises a time-domain excitation contribution calculation operation 155. To perform operation 155, the 120 / 720 mixed time-domain / frequency-domain encoder comprises a time-domain excitation contribution calculation device 105.The computing device 105 itself comprises an analyzer 211 (Figure 2) that responds to the open-loop pass analysis performed in the open-loop pass analyzer 203 (or preprocessor 702) and the substructure length (or the number of substructures in a structure) determined in the computing device 210 or sound signal classifier 701 to perform a closed-loop pass analysis operation 261. Closed-loop pass analysis is well known to those skilled in the art, and an example implementation is described, for instance, in Recommendation 1TU-T G.718, Reference [5J; Section 6.8.4.1.4.1]. Closed-loop pitch analysis results in the calculation of pitch parameters, also known as adaptive codebook parameters, which consist primarily of a pitch delay (adaptive codebook index T) and pitch gain (adaptive codebook gain b).The adaptive codebook contribution is typically the excitation passed at delay T or an interpolated version thereof. The adaptive codebook index T is encoded and transmitted to a remote decoder. The pitch gain b is also quantized and transmitted to the remote decoder.

[0081] When the closed-loop tone analysis in operation 261 is completed and a fixed codebook contribution is used, the time-domain excitation contribution calculation device 105 comprises a fixed algebraic codebook 212 searched during a fixed codebook search operation 262 to find the best fixed codebook parameters, which generally comprise a fixed codebook index and a fixed codebook gain. The fixed codebook index and gain form the fixed codebook contribution. The fixed codebook index is encoded and transmitted to the remote decoder. The fixed codebook gain is also quantized and transmitted to the remote decoder.The fixed algebraic codebook and bynonn / cznz / B / viAi search for it are believed to be well known to experts in the field of CELP coding and will therefore not be described further in this description.

[0082] The adaptive codebook index and gain, and, if used, the fixed codebook index and gain, form the CELP excitation contribution in the time domain. 4) Frequency Transform

[0083] During frequency-domain encoding of the mixed time-domain / frequency-domain coding mode, two signals are represented in the transform domain, e.g., the frequency domain. In one mode, the time-to-frequency transform can be achieved using a 256-point Type II (or Type IV) Discrete Cosine Transform (DCT), which provides a resolution of 25 Hz with an internal sampling rate of 12.8 kHz, but any other suitable transform could be used. In the event that another transform is used, the frequency resolution (defined above), the number of frequency bands, and the number of frequency containers per band (defined later) might need to be revised accordingly.

[0084] As indicated in the description above, in the unified time-domain / frequency-domain CELP encoding device 100 and method 150 (Figures 1 and 2), the mixed time-domain / frequency-domain encoding mode is used when generic audio is selected by selector 205 as the classification of the input sound signal 101 and no time attack is detected in detector 208. Alternatively, in the unified time-domain / frequency-domain CELP encoding device 700 and method 750 (Figure 7), the mixed time-domain / frequency-domain encoding mode is used when the sound signal classifier 701 classifies the input sound signal 101 in the unclear signal type category.The 120 / 720 mixed time-domain / frequency-domain encoder comprises a frequency-domain excitation contribution calculation device 107 (Figures 1 and 7) that performs a frequency-domain excitation contribution calculation operation 157 in response to the residual LP input r<,v(n) (Reference [5]) resulting from the LP analysis operation 251 of the input sound signal 101 performed by the analyzer 201 (and the preprocessor 702). As illustrated in Figure 2, the calculation device 107 can calculate a DCT 213, for example, a type II DCT of the residual LP input z„(n). The mixed time-domain / frequency-domain encoder 120 / 720 also comprises a calculation device 106 (Figures 1 and 7) for performing a calculation operation 156 of a frequency transform of the time-domain excitation contribution.As illustrated in Figure 2, the calculation device 106 can calculate a DCT 214, for example, a type II DCT of the excitation contribution in the time domain. The frequency transformations of the input LP residue 7 and the excitation contribution CELP 7c in the time domain can be calculated using, for example, the following expressions: and: bi nonn / eznz / b / y ιν· 2 *tdi ' í ccsv[ u -1 j ; j.á = o >>1 = 0 ' ' “ ' /

[0085] where Gj-OO is the input LP residue, etdú'D is the time-domain excitation contribution, and N is the structure length. In one possible mode, the structure length is 256 samples for a corresponding internal sampling rate of 12.8 kHz. The time-domain excitation contribution is determined by the following relationship: ekle(n) = bv(n) + gc(n)

[0086] where v(n) is the adaptive codebook contribution, b is the adaptive codebook gain, c(n) is the fixed codebook contribution, and yg is the fixed codebook gain. It should be noted that the excitation contribution in the time domain may consist only of the adaptive codebook contribution as described above. 5) Cutoff frequency of the contribution in the time domain

[0087] With sound signal samples classified as generic audio (Figure 1) or sound signal samples classified as unclear signal type (Figure 7), the time-domain excitation contribution does not always contribute much to the encoding improvement compared to the frequency-domain encoding. Often, it improves the encoding of the lower part of the spectrum, while the improvement of the encoding of the upper part of the spectrum is minimal. The 120 / 720 mixed time-domain / frequency-domain encoder comprises a cutoff frequency finder and filter (Figures 1 and 7) to perform an operation of determining a cutoff frequency higher than the point at which the encoding improvement provided by the time-domain excitation contribution becomes too low to be valuable.The cutoff frequency finder and filter 108 comprise, as illustrated in Figure 2, a cutoff frequency calculation device 215 and a filter 216.

[0088] A time-domain excitation contribution cutoff frequency estimation operation 265 is first completed by the calculation device 215 (Figure 2) using a computer 303 (Figures 3 and 4) that performs a normalized cross-correlation operation 353 for each frequency band between the frequency transform of the input LP residue 301 from the calculation device 107 and the time-domain excitation contribution frequency transform 302 from the calculation device 106, designated respectively ÍA-: andj~í.\c, as defined in Section 4 above. The last frequency included in each of, for example, the sixteen (16) frequency bands is defined in Hz as: f 175,375,775,1175,1575,1975,2375,2775, 1 Lf=ir ' I 3175,3575,3975,4375,4775,5175,5575,63751 bjnonn / cznz / B / viAi Bb

[0089] For this illustrative example, the number of frequency bins / per band, the bins of The cumulative frequency Cgb per band and the normalized cross-correlation 6\-C) per frequency band i are defined, for example, as follows, for a 20 ms structure at an internal sampling rate of 12.8 kHz: 0,8,16,32,48,64,80,96, 112,128,144,160,176,192,208,224 V and Where 4(0= Σ .M. / Γ .1=(-111,11]

[0090] where is the number of frequency bits / per bandb, ^Bhes the frequency bits s cumulative per band, Cc(') is the normalized cross-correlation per frequency band i, is the excitation energy for a band and similarly Jmes is the residual energy per band.

[0091] The cutoff frequency calculator 215 comprises a cross-correlation smoother 304 (Figures 3 and 4) across the frequency bands that performs certain operations 354 to smooth the cross-correlation vector between the different frequency bands. More specifically, the cross-correlation smoother 304 across the frequency bands calculates a new cross-correlation vector Cc2 using, for example, the following relationship: (min(0.5, cr-C (0)+JC (1))-0.5) for / = 0 C. ( / ) = ]22· (min (0.5, a · Cr(í)+ / ?C. (í + 1)+ / ?C. (z — 1)) -0.5) for ] < i < Nhbynonn / cznz / B / viAi where, in an illustrative embodiment, « = 0.95; ¿ = (the); Nh= 13; P = %

[0092] The cutoff frequency calculator 215 further comprises a calculation device 305 (Figures 3 and 4) that performs a calculation operation 355 of averaging the new cross-correlation vector over the first bands (e.g., =13 representing 5575 Hz).

[0093] The cutoff frequency calculator 215 also comprises a cutoff frequency module 306 (Figure 3) which includes, as illustrated in Figure 4, a cross-correlation limiter 406, a cross-correlation normalizer 407, and a frequency band finder 408 where the cross-correlation is lowest. More specifically, the limiter 406 performs an operation 456 limiting the average of the cross-correlation vector to a minimum value of 0.5, and the normalizer 407 performs an operation 457 normalizing the limited average of the cross-correlation vector C12 between 0 and 1.The search engine 408 performs an operation 458 to obtain a first estimate of the cutoff frequency by finding the last frequency L / of a frequency band i that minimizes the difference between said last frequency Lj of a frequency band i and the normalized average of the (j cross-correlation vector - v multiplied by half the internal sampling rate (FJ2) of the input sound signal 101:. where _ Z(CUO) F = 12800 Hz. and C = -^------Nhf

[0094] In the above relationships, ' represents the first estimate of the cutoff frequency. C

[0095] At low bit rates, where the normalized average is never really high (in the case of the unified time-domain / frequency-domain encoding device 100 and method 150 of Figure 1), or when the submode indicator FtfSmes is greater than 0, which means that the input sound signal is classified as an unclear signal type (in the case of the unified time-domain / frequency-domain encoding device 700 and method 750 of Figure 7), or to artificially increase the value of to give more weight to the excitation domain contribution For time intervals, it is possible to scale, using normalizer 407, the normalized average value < with a fixed scaling factor. As a non-limiting example, at a bit rate below 8 kbps, the first estimate of the cutoff frequency is multiplied by 2.

[0096] The accuracy of the cutoff frequency can be improved by adding the following component to the calculation. For that purpose, the cutoff frequency module 306 comprises an extrapolator 410 (Figure 4) of the calculated 8th harmonic, in a corresponding operation 460, from the minimum or lowest tone delay value of the time-domain excitation contribution of the structure's substructures, using, for example, the following relationship: / 8-F' 'xmin (7( / )) where F= 12300 H- is the internal sampling frequency or rate, / V,„ / > is the number of substructures in a structure, and T(i) is the adaptive codebook index or pitch delay for substructure i.

[0097] The cutoff frequency module 306 comprises a finder 409 (Figure 4) of the frequency band where the 8th harmonic is located. More specifically, for the z substructures<M„ / > , the searcher 409 performs a 459 search operation for the highest frequency band for which, for example, the following inequality still holds: (i-WLW) The index of that band will be called / χ' and indicates the band where the 8th harmonic is likely to be found.

[0098] The cutoff frequency module 306 finally comprises a selector 411 (Figure 4) of the final cutoff frequency fzc. More specifically, the selector 411 performs a hold operation 461 of the highest frequency between the first estimate f,i of the cutoff frequency of the finder 408 and the (l, ( / ,„)) last frequency of the frequency band where the 8th harmonic v v 7 of the finder 409 is located, using the following relationship: fzc = max ( i / -.! )

[0099] When using coding submodes, in the case of the unified time-domain / frequency-domain coding device 700 and method 750 of Figure 7, the cutoff frequency he is further thresholded using, for example, the following relationship: ftc = maxi^imax(¿f( / 8f / 1),2775),Acl)

[00100] As illustrated in Figures 3 and 4: - the frequency cutoff calculation device 215 further comprises a decision-maker 307 (Figure 3) to perform a decision operation 357 on the number of frequency bins of a frequency band to be zeroed out; - the decision-maker 307 itself includes an analyzer 415 (Figure 4) to perform a parameter analysis operation 465, and a selector 416 (Figure 4) to perform a frequency container selection operation 466 to be zeroed out; and - Filter 216 (Figure 2) operates in the frequency domain and comprises, for filtering operation 266, a zero 308 (Figure 3). The corresponding operation 358 zeros the frequency containers decided to be zeroed in the decision-maker 307. Zero 308 can zero (a) all frequency containers (zero 417 and corresponding zeroing operation 467 in Figure 4) or (b) the higher frequency containers located above the cutoff frequency supplemented with a smooth transition region (filter 418 and corresponding filtering operation 468 in Figure 4). The transition region is located above the cutoff frequency he and below the zeroed containers, and allows a smooth spectral transition between the unchanged spectrum below the cutoff frequency he and the zeroed containers at higher frequencies.

[00101] As a non-limiting illustrative example, when the cutoff frequency he of selector 411 is less than or equal to 775 Hz, analyzer 415 considers the cost of the time-domain excitation contribution to be too high. Selector 416 then selects all frequency bins from the time-domain excitation contribution frequency representation to be zeroed, and zero 417 forces all frequency bins to be zeroed and also forces the cutoff frequency he to zero. All bits allocated to the time-domain excitation contribution are then reassigned to the frequency-domain encoding mode. Otherwise, analyzer 415 forces selector 416 to select the high-frequency bins above the cutoff frequency he to be zeroed by filter (zero) 418.

[00102] Finally, the cutoff frequency calculation device 215 comprises a quantizer 309 (Figures 3 and 4) to perform a quantization operation 359 on the cutoff frequency h- into a quantized version fCQ of this cutoff frequency for transmission to a distant decoder. If, for example, three (3) bits are associated with the cutoff frequency parameter, a possible set of output values ​​(in Hz) can be defined as follows: f:cQ= {0. 1175. 1575. 1975.2375 2775.3175. 3575}

[00103] Selector 411 could use many mechanisms to stabilize the choice of the final cutoff frequency fie to prevent the quantized / C version from switching between 0 and 1175 in an inappropriate signal segment. To achieve this, as a non-restrictive example, analyzer 415 responds to the long-term average tone gain Gii 412 of closed-loop tone analyzer 21 I (Figure 2), the open-loop tone correlation Ct,¡ 413 of open-loop tone analyzer 203, and the smoothed open-loop tone correlation Cv / 414. To prevent switching to frequency-domain encoding only, analyzer 415 does not allow such frequency-domain encoding only when, for example, the following conditions are met, i.e., íxC cannot be set to 0: fic> 2 3 75Hz or Λ >1175 / 7< and C() / >0.7yGh>0.6 or / '>1175 / / : and C,>0.8vG>0.4 J teJst j It O .M'-U=0 and C„>0.5yC„>0.5 and C„>0.6

[00104] where Ce; is the open-loop pitch correlation 413 and corresponds to the smoothed version of the open-loop pitch correlation 414 defined as C::= 0.9 · C-· - 0.1 · C::. Furthermore, Gh (element 412 in Figure 4) corresponds to the long-term average of the pitch gain obtained by the closed-loop pitch analyzer 211 within the excitation contribution in the time domain. The long-term average of the pitch gain 412 is defined as C·:=θ·9 · G? - 0.1 · GiZ, where 6? is the average pitch gain over the current structure. To further reduce the switching rate between frequency-domain-only encoding and mixed time-domain / frequency-domain encoding, a pendant can be added. 6) Frequency domain coding 6.1) Creating a difference vector

[00105] Once the cutoff frequency fi of the excitation contribution is determined in the time domain, encoding is performed in the frequency domain. To perform such encoding in the frequency domain, the mixed time-domain / frequency-domain encoding method 170 / 770 comprises a subtraction operation 159, a frequency quantization operation 160, and an addition operation 161. The mixed time-domain / frequency-domain encoder 120 / 720 comprises a subtractor or calculating device 109, a frequency quantizer 110, and an adder 111 for performing operations 159, 160, and 161, respectively.

[00106] Figure 5 is a schematic block diagram that simultaneously illustrates an overview of a frequency quantizer 110 and the corresponding frequency quantization operation 160. In addition, Figure 6 is a schematic block diagram of a more detailed structure of the frequency quantizer 110 and the corresponding frequency quantization operation 160.

[00107] The subtractor or calculating device 109 (Figures 1, 2, 5 and 6) forms a first portion of a difference vector with the difference between the frequency transform fies502 (Figures 5 and 6) (or other frequency representation) of the input LP residue of DCT 213 (Figure 2) and the frequency transform fexc501 (Figures 5 and 6) (or other frequency representation) of the time-domain excitation contribution of DCT 214 (Figure 2) from zero to the cutoff threshold frequency f:c of the time-domain excitation contribution. A reduced scaling factor 603 (Figure 6) (see multiplier 604 and corresponding multiplication operation 654) can be applied to the frequency transform 501 for the following transition region of 2 kHz (80 frequency intervals in this implementation example) before subtracting from it the respective spectral portion of the frequency transform 502.The result of the subtraction constitutes a second part of the difference vector representing a frequency interval from the cutoff threshold frequency to the frequency transform of the input LP residue. The frequency transform of the input LP residue is used for the third remaining portion of the difference vector.

[00108] The reduced portion of the difference vector resulting from the application of the scaling factor 603 can be implemented with any type of fading function, can be shortened to just a few frequency bins, or can be omitted when the available bit budget is deemed sufficient to avoid power oscillation artifacts when the cutoff frequency is changing. For example, with a resolution of 25 Hz, corresponding to 1 frequency bin fbin = 25 Hz at 256 DCT points at an internal sampling rate of 12.8 kHz, the difference vector can be designed as: fAk) = frex(k)-feufk) where 0 <k<fJfhin 1—without bi nonn / cznz / b / y where f¡, / fnín O k (jΈ frf¡lly) / / / „„ f,i (A), on the other hand where fn.s, fXÍy fl, have been defined in the previous description. 6.2) Bit allocation in the frequency domain to encode sub-nides 6.2.1) Allocate a fraction of the available bits to lower frequencies

[0010] In the unified time / frequency domain CELP coding method 750 as illustrated in Figure 7, the mixed time / frequency domain encoder 720 comprises a band selector and a bit allocator 707 and the mixed time / frequency domain coding method 770 comprises a corresponding band selection and bit allocation detection operation 757.

[00110] Figure 9 is a schematic block diagram that simultaneously illustrates the band selector and bit allocator 707 and the corresponding band selection and bit allocation operation 757 of Figure 7 to allocate the available bit budget to the frequency quantization of the difference vector f¡ when the input sound signal 101 is not categorized as either voice or music in the alternative implementation of the unified time-domain / frequency-domain CELP coding method 150 / 750 of Figures 7 and 8.

[0011] Specifically, Figure 9 shows an innovative way in which the band selector and bit allocator 707 can distribute the available bits to frequency quantization when the input sound signal 101 is not categorized as either voice or music, but rather as an unclear signal type, depending on the previously chosen encoding modes. In Figure 9, frequency quantization is performed in a per-band manner. For simplicity, the frequency bands have the same number of frequency intervals, which is sixteen (16) frequency intervals, at an internal sampling frequency of 12.8 kHz in the current illustrative example. Frequency band 0 represents the lower part of the spectrum, while frequency band 15 represents the higher part of that spectrum.

[00112] To make the best possible use of the bits available for frequency quantization, the band selection and bit allocation operation 757 comprises a first operation 951 of prefixing a fraction of the available bit budget (see 900) to quantize the lower frequencies of the difference vector / / as a function of the quantized cutoff frequency / „ ρ of the cutoff frequency finder and filter 108. To perform operation 951, an estimator 901 uses, for example, the following relation: ^-0.125 +76^Pb1)~ {55 PBif = , 0.75), 0.5) where PBi¡ is the fraction of available bits allocated to frequency quantization of the lower frequencies of the difference vector / / . In this example, the lower frequencies refer to the first five (5) frequency bands, or the first two (2) kHz. The term Lf^ftcQ^se refers to the number of frequency containers up to the quantized cutoff threshold frequency / ,, q.

[00113] Next, estimator 901 adjusts the fraction of available bits allocated to frequency quantization of the lowest frequencies based on the coding submode indicator PBif. If the coding submode indicator Ftfsm is set to 2 (Figure 8), meaning that the probability of a temporary attack is detected in the current structure of the input sound signal 101, then the fraction of bits allocated to frequency quantization of the lowest frequencies PBif increases by 10% of the available bits. If "music-like features" are detected in the content of the current structure, indicated by a coding submode indicator Ftfsm set to 3, the fraction of bits allocated to frequency quantization of the lowest frequencies PBif is reduced by 10% of the available bits. 6.2.2) Estimation of the number of frequency bands to be quantified

[00114] Another parameter that affects the total number of bits per frequency band available for frequency quantization of the difference vector is an estimated maximum number NBmx of frequency bands of this difference vector fu to be quantized. In the illustrative example currently described, at an internal sampling rate of 12.8 kHz, the maximum total number Ntt of frequency bands is sixteen (16).

[00115] When using the coding submodes, the band selection and bit allocation operation 757 comprises a 952 operation estimating the maximum number NBmx of frequency bands of the difference vector f,ia to quantize. To perform operation 952, an estimator 902 sets, if the encoding submode indicator FtfSIII is set to 1 (selecting the first encoding submode), the maximum number NBmx of frequency bands to 10. If the encoding submode indicator FtfSm is set to 2 (selecting the second encoding submode), then the estimator 902 sets the maximum number NBmx of frequency bands to 9. If the encoding submode indicator FtfSm is set to 3 (selecting the third encoding submode), then the estimator 902 sets the maximum number NBmx of frequency bands to 13.The estimator 902 then readjusts the maximum number NBmx of frequency bands to quantize based on the bit budget available for frequency quantization of the difference vector L using, for example, the following relationships:. í 0.0125 · BF— 0.75 Ftfsm= 1&.BT< 15000, / VB„d7=j 0.02 · BF— 1.2 Ftfsm*2 &BT> 20000, otherwise Yes No NBmx=rnax(min(trunc (NBmxNBadj + 0.5), Ntt), 5) where Bf represents the number of bits available for frequency quantization of the difference vector / / (see 900), Bt is the total bit rate available to encode the channel being processed (see 900), FtfSmes the submode indicator (see 900) and Nttes the maximum total number of frequency bands.

[00116] Estimator 902 can further reduce the maximum number of frequency bands of the difference vector fi to be quantized relative to the number of bits allocated to quantizing the mid and high frequency bands of the difference vector. For the purpose of this limitation, it is assumed that the last lower frequency band and the first subsequent frequency band have a similar number of bits, or approximately 17% of the bits allocated to quantizing the lower frequencies. For the last frequency band to be quantized, a minimum of 4.5 bits is used to quantize at least one (1) frequency pulse. If the available bit rate Bres is greater than or equal to 15 kbps, then the minimum number of bits will be nine (9) to allow the quantization of more pulses per frequency band.However, if the total available bit rate Bt is below 15 kbps but the submode indicator FtfSmse is set to 3, meaning the content has similarities to music, then the number of bits mp of the last frequency band to be frequency quantized will be 6.75 to allow for more accurate quantization. The estimator 902 then calculates a corrected maximum number of frequency bands NBmx using, for example, the following relationship: Ν' — miníN 5 + ( ~Bmxmin / 0.5 (mp+ mb))j ΘΠ where NBmxcorresponds to the corrected maximum number of frequency bands to be quantized, NBmxes the estimated maximum number of frequency bands, the number 5 represents the minimum number of frequency bands, Bf represents the number of bits available for frequency quantization of the difference vector / / , PBlf is the fraction of bits assigned to quantization of the five (5) lower frequency bands, m,, is the minimum number of bits assigned to frequency quantization of a frequency band, and mi, the number of bits assigned to quantization of the first frequency band after bynonn / cznz / B / viAi of the five (5) lower frequency bands.

[00117] After calculating the maximum number of frequency bands, estimator 902 can perform an additional check so that mt> remains less than or equal to mi,. Although this additional check is an optional step, at low bit rates, it helps to allocate bits more efficiently among the frequency bands of the difference vector / / . 6.2.3) Review the number of bits allocated to lower frequencies

[00118] The band selection and bit allocation operation 757 comprises a low-frequency bit calculation operation 953. To perform operation 953, a calculation device 903 is provided. If calculating the maximum number of frequency bands NBmx results in a smaller number of frequency bands to be quantized, the calculation device 903 reassigns the portion of bits previously allocated to the higher frequency bands in such a way that it is no longer relevant for quantizing the lower frequency bands, using, for example, the following relationship: Blf= PBif · BF+ (θ,5 (mp+ mf) (NBmx— NBΒιηχ)^, where BLF corresponds to the bits assigned to the five (5) lower frequency bands, BF corresponds to the number of bits available to quantize the lower frequencies of the difference vector / / , PBif is the fraction of bits mentioned above of the estimator 901 assigned, for example, to frequency quantization of the five (5) lower frequency bands, mp is the minimum number of bits assigned to quantize a frequency band, and mi, the number of bits assigned to quantize the first frequency band after the five (5) lower frequency bands. 6.2.4) Dual classification of frequency bands

[00119] The band selection and bit allocation operation 757 comprises a frequency band characterization operation 954. To perform operation 954, the band selector and bit allocator 707 comprise a frequency band characterizer 904 which, once the bit rate is distributed among the lower frequency bands and the remaining frequency bands, performs a dual classification of the frequency bands to determine the importance of each band. The first classification involves determining whether one or more bands have lower energy compared to their neighboring frequency bands. When this occurs, the characterizer 904 marks these bands so that only the predetermined minimum number of bits can be allocated to frequency quantizing these low-energy frequency bands, even if the available bit budget is high.The second classification involves ranking the mid- and upper-energy frequency bands, for example, in order of decreasing energy. This first and second classification (double classification) is not performed for the lower frequency bands, but is carried out up to the maximum number of frequency bands, NBmx. The frequency band characterization operation 954 can be summarized as follows: Ppb (0 = {1 E(í - 1) > E(i) < E(¿ + 1)1 otherwise 7 <i<NBmxbynonn / cznz / B / viAi EP(i) = POS (maji'^EQY ,N' Ί |V'7 <i<NBmx E(i) = log10j~cBb Σ j=CBb G) where PpbCi) is set to 1 for frequency bands where only the minimum number of bits ΓΠΡ will be used, EP(i) contains the position of the middle and upper energy frequency bands in decreasing energy order, and £( / ) corresponds to the energy of each band. Cbi> and are defined above in Section 5. The difference vector / / has been defined in Section 6.1.

[00120] The energy E(?) of each frequency band of the difference vector / / is calculated in a calculation device 708 and the corresponding operation 758 of Figures 7 and 9. Calculator 708 and operation 758 also calculate a gain for each frequency band as described with reference to calculation device 615 and operation 665 of Figure 6. The energy E(?) of each frequency band of the difference vector ( / ) and the gain for each frequency band are quantized, for example, as described with reference to quantizer 616 and operation 666 of Figure 6, and both are transmitted to a distant decoder. In the case of the implementation of Figure 7 for the unified time-domain / frequency-domain coding device 700 and method 750, calculation device 708 and operation 758 replace calculation device 615 and operation 665, as well as the quantizer. 616 and operation 666. 6.2.5) Bit distribution to selected bands

[00121] The band selection and bit allocation operation 757 comprises a final frequency band bit distribution operation 955. To carry out operation 955, the band selector and bit allocator 707 comprises a final frequency band bit distributor 905.

[00122] Once the frequency bands have been characterized, the distributor 905 assigns the bit rate or number of bits Bf available to frequency quantize the difference vector between selected frequency bands.

[00123] In the non-limiting example, for the first five (5) lower frequency bands, the distributor 905 linearly distributes the Blf bits allocated to frequency quantizing the lower frequencies, with the first lower frequency band receiving 23% of the Blf bits and the fifth (5a) lower frequency band receiving the last 17% of the Blf bits. In this way, the lower frequencies of the difference vector spectrum fc¡ can be quantized with sufficient accuracy to recover a better quality synthesis of the input sound signal 101.

[00124] The distributor 905 distributes the remaining Bf bits allocated to frequency quantizing the difference vector / / over the other mid and high frequency bands as a linear function, but again taking into account the previous frequency band energy characterization (operation 954) so ​​that more bits can be allocated to higher-energy frequency bands and fewer bits to frequency bands that have lower energy compared to the energy of their neighboring frequency bands, thus making more relevant use of the available bits by more accurately quantizing larger portions of the difference vector / / spectrum. As a non-limiting example, the following relationship illustrates how the bit distribution (operation 955) can be performed: 7(0.23 -i· 0.015)· Blf0 <i<5 3 Bp(i) = j ητ,ρ 5 < i < Nfimx Ppb(.i) 1 r lmib(.O otherwise j mlb(Í) = -m- p («F-gz-F)-2 >Hp (N¿mx-5)\ _ _ where Bp(i) represents the number of bits assigned per frequency band / , BF represents the number of bits available to quantize the difference vector in frequency / / , Blf corresponds to the bit rate or bits assigned to the five (5) lower frequency bands, m;, is the minimum number of bits to quantize a frequency pulse in a frequency band. Pp&(i) contains the position where the minimum number of bits will be used, and NFmx is the maximum number of frequency bands to quantize.

[00125] If, after operation 955, there are any unassigned bits, the 905 distributor will assign them to the lowest frequency bands. As a non-limiting example, the 905 distributor will assign one remaining bit per frequency band, starting from the fifth (5a) band and returning to the first band, and repeating this procedure if necessary to assign all remaining bits.

[00126] Later, the 905 distributor may have to reduce, truncate, or round the number of bits per frequency band depending on the algorithm being used to perform the frequency pulse quantization and the potential fixed-point implementation. 6.3) Search for frequency pulses

[00127] The mixed time-domain / frequency-domain CELP coding method 170 / 770 comprises a frequency quantization operation 160 (Figures 1, 2, and 7) of the difference vector. To perform operation 160, the mixed time-domain / frequency-domain / frequency-domain / frequency-domain CELP encoder 120 / 720 comprises a frequency quantizer 110 (219 in Figure 2).

[00128] The difference vector / / can be quantized using various methods. In all cases. The frequency pulses must be searched for and quantized. In one possible implementation, the frequency quantizer 110 searches for the most energetic pulses of the difference vector across the spectrum. The method for searching for the pulses can be as simple as dividing the spectrum into frequency bands and allowing a certain number of pulses per frequency band. The number of pulses per frequency band depends on the available bit budget and the position of the frequency band within the spectrum. Generally, more pulses are allocated to lower frequencies. 6.4) Quantized difference vector

[00129] Depending on the available bit rate, frequency pulse quantization can be performed using the 110 frequency quantizer with different techniques. In one mode, at a bit rate below 12 kbps, a simple search and quantization scheme can be used to encode the position and sign of the pulses. This scheme is described below as a non-limiting example.

[00130] For frequencies below 3175 Hz, the simple search and quantization scheme uses a factorial impulse coding (FPC) approach described in the literature, e.g., in Reference [8J, the full content of which is incorporated herein by this reference.

[00131] More specifically, with reference to Figures 5 and 6, the frequency quantizer 110 comprises a selector 504 to perform an operation 554 to determine whether the entire spectrum is quantized using FPC. As illustrated in Figure 5, if the selector 504 determines that the entire spectrum is not quantized using FPC, an operation 556 of FPC encoding and pulse position and sign encoding is performed on an encoder 506.

[00132] As illustrated in Figure 6, the FPC encoding and pulse position and sign encoding operation 556 comprises a frequency pulse search operation 659, an FPC encoding operation 660, a most energetic pulse finder operation 661, and a frequency pulse position and sign quantizer operation 662. To perform operations 659-662, the encoder 506 respectively comprises a frequency pulse finder 609, an FPC encoder 610, a most energetic pulse finder 611, and a frequency pulse position and sign quantizer 612.

[00133] Searcher 609 searches for frequency pulses across all frequency bands for frequencies below 3175 Hz. Encoder FPC 610 then processes the frequency pulses. Searcher 611 determines the most energetic pulses for frequencies equal to and greater than 3175 Hz, and quantizer 612 encodes the position and sign of the most energetic pulses found. If more than one (1) pulse is allowed within a frequency band, then the amplitude of the previously found pulse is divided by 2, and the search is performed again across the entire frequency band. Each time a pulse is found, its position and sign are stored for the quantization and bit packing stage. The following pseudocode illustrates, by way of non-limiting example, this simple search and quantization scheme: for k = 0: NBD for i = 0: N Pmu. = θ for . / = CBh(k):CBh(k)+Bh(k) if > P„UIXpm<„ = fÁJ)2i- r fAj) .fAj) = —^~ pA') = j PÁ>) =wno(jA.A) end end end .end NNN where / ; / Jes is the number of frequency bands ( = 16 in the illustrative example),pes p the number of pulses to encode in a frequency band k, hes the number of frequency bits per frequency band, Bhes the cumulative frequency bits per band as defined above in Section 5), represents the vector containing the position of the pulse found, represents the vector containing the sign of the pulse found and P”u,x represents the energy of the pulse found.

[00134] At bit rates above 12 kbps, selector 504 determines that the entire spectrum must be quantized using FPC (Figures 5 and 6). As illustrated in Figure 5, an FPC encoding operation 555 is then performed on an FPC encoder 505. With reference to Figure 6, the encoder 505 comprises a frequency pulse finder 607, and the 555 operation comprises a corresponding frequency pulse search operation 667. The frequency pulse search is performed across all frequency bands. The 555 operation comprises a frequency pulse encoding operation 668, and the encoder 505 comprises, to perform the 668 operation, an FPC processor 608.

[00135] Next, the FPC 608 processor or the bynonn / cznz / B / viAi pulse position and sign quantizer 612 obtains the quantized difference vector by adding the number of pulses nb_pulses with the sign PP of pulse1' to each of the positions found. For each frequency band, the quantized difference vector f can be written using, for example, the following pseudocode: for / = 0,..., j < nb _ pulses (. / ))+ = A (. / ) 6.5) Noise Fill

[00136] Frequency bands are quantized with varying degrees of precision; the quantization method described in the previous section does not guarantee that all frequency intervals within the frequency bands will be quantized. This is especially true at low bit rates, where the number of quantized pulses per frequency band is relatively low. To avoid audible artifacts due to these unquantized frequency bins, the frequency quantizer 1 10 comprises a noise padding 507 (Figure 5) to perform a corresponding operation 557 of adding some noise to the unquantized frequency bins in order to fill these gaps. This noise addition can be performed across the entire spectrum at bit rates below 12 kbps, for example, but can be applied only above the cutoff frequency of the time-domain excitation contribution for higher bit rates.For simplicity, the noise intensity varies only with the available bit rate. At high bit rates, the noise level is low, but the noise level is higher at low bit rates.

[00137] Noise filler 507 comprises an adder 613 (Figure 6) that performs an operation f 663 of adding noise to the quantized difference vector after the intensity or energy level of the added noise has been determined. For this purpose, the frequency quantization operation 160 comprises an operation 664 of estimating the intensity or energy level of the added noise, and the frequency quantizer 110 comprises, for performing operation 664, a corresponding estimator 614 of the noise energy level. The operation 664 of estimating the intensity or energy level of the added noise is performed by the estimator 614 and prior to an operation 665 of determining a frequency band gain in a frequency band gain calculation device 615 of the frequency quantizer 110.

[00138] In the illustrative mode, in estimator 614, the noise level is directly related to the encoding bit rate. For example, at 6.60 kbps, estimator 614 sets the level N. The noise is injected at 0.4 times the amplitude of the frequency pulses encoded in a specific frequency band and progressively up to a value of 0.2 times the amplitude of the frequency pulses encoded in a frequency band at 24 kbps. The adder 613 injects noise only into the section(s) of the spectrum where a certain number of consecutive frequency containers have very low energy, for example, when the cumulative energy of the containers in half of a frequency band is below 0.5. For a specific frequency band i, the noise is injected, for example, as follows: pan j = C¡t(i)...j < C„, (<)+ Bk(i) ν Σ <°·5k = .i fork = j, ..., k < j + N. . / 4^=44)+404,. ( ) j+ = B. (i) Where N = — 2 where, for a band i, Cnh is the cumulative number of frequency bins per band of N frequency, B¡, is the number of frequency bins in a specific band i,Les is the level of added noise, ylll,des a random number generator that is limited between -1 and 1. 6.6) Quantification of gain per band

[00139] With reference to Figures 5 and 6, the frequency quantization operation 160 of the unified time-domain / frequency-domain encoding device 100 and method 150 comprises operation 665 of determining a gain per frequency band followed by operation 666 of quantizing the gain per band. The frequency quantizer 110 comprises, for performing operations 665 and 666, a gain-per-band calculation device 615 and a gain-per-band quantizer 616. F

[00140] Once the quantized difference vector is found, including noise padding if necessary, the 615 computing device calculates the per-band gain for each frequency band. The per-band gain for a specific band is defined as the ratio of the unquantized difference vector energy to the quantized difference vector energy. <jen el dominio logarítmico usando, por ejemplo, las siguientes relaciones: 5-· 4λ ó-c)="log:o——" 4.,0) bjnonn cznz b viai donde s, (: )="y" 5--.,0)=" / --Q')2;" = cr>(:'· ' ; = ύ*£>.ί;} where Cbi, and Bi, are defined above in Section 5).

[00141] The band-gain quantizer 616 quantizes the frequency gains per band. Before vector quantization, at low bit rates, the last gain (corresponding to the last frequency band) is quantized separately, and the remaining fifteen (15) band gains (when, for example, 16 frequency bands are used) are divided by the last quantized gain. Then, the remaining fifteen (15) normalized gains are vector-quantized by the quantizer 616. At higher bit rates, the average of the band gains is first quantized and then removed from all the band gains of, for example, sixteen (16) frequency bands before vector quantization of those band gains.The vector quantization used can be a standard minimization in the logarithmic domain of the distance between the vector containing the band gains and the entries of a specific codebook.

[00142] In frequency-domain coding mode, the gains are calculated in the 615 calculation device for each frequency band to match the energy of the non-quantized vector f with the quantized vector. The gains are vectorially quantized in the quantizer 616 and are applied by frequency band (operation 559) to the quantized vector through a multiplier 509 (Figures 5 and 6).

[00143] Alternatively, it is also possible to use the FPC coding scheme at a rate lower than 12 kbps for the entire spectrum by selecting only some of the frequency bands to be quantized. Before selecting the frequency bands, the energy of the unquantized difference vector frequency bands is quantized using the quantizer 616. The energy is calculated using, for example, the following relationship: M0= 1οMM0) .+<.„„( / )+ n„ ( / ) where Sd(i~)= £ ¿(j)' where Cbi, and Bi, are defined above in Section 5). £

[00144] To perform frequency band energy quantization <z, primero se cuantifica la energía promedio en las primeras 12 bandas de frecuencia dieciséis que utilizan y resta todas (16) energías banda. entonces, son vectores to 1 nonn cznz b cuantificados por grupo 3 o 4 bandas. la cuantificación vectorial utiliza puede ser una minimización estándar el dominio log distancia entre vector contiene ganancias banda entradas un libro códigos específico. si no hay suficientes bits disponibles, es posible cuantificar solo extrapolar últimas cuatro (4) utilizando tres (3) anteriores mediante cualquier otro método.

[00145] Once the energy of the frequency bands of the unquantized difference vector is quantized, it is possible to sort the energy in descending order in a way that is replicable on the decoder side. During sorting, all energy bands below 2 kHz are retained, and only the most energetic bands are then passed to the FPC scheme to encode frequency pulse amplitudes and signs. With this approach, the FPC scheme encodes a smaller vector that covers a wider frequency range. In other words, fewer bits are needed to cover significant energy events across the spectrum.

[00146] In the particular case of implementing the unified time domain / frequency domain coding device 700 and method 750 of Figure 7, the frequency band selection and bit allocation are instead performed as determined by the per-band power and per-band gain calculation device 708 and calculation operation 758 and the band selector and bit allocator 707 and the band selection and bit allocation operation 757 of Figures 7 and 9 as described above in this document.

[00147] After the pulse quantization process, noise filling similar to that described above is performed. Then, a frequency band gain adjustment factor £ ... ., f is calculated to match the energy dQ of the quantized difference vector 'dQ' with the quantized energy £ ' f ·' of the unquantized difference vector J d. This band gain adjustment factor is then applied to the quantized difference vector dQ. This can be expressed as follows: bynonn / cznz / B / viAi (1 (i) = \(}':' where Χ',,,,ι / ι+β,,ι / ι Σ 4( / )2y E / is the band-quantized energy of the unquantized difference vector fd as defined above After the completion of the encoding stage in the frequency domain, the total excitation is found in the time domain / frequency domain.For that purpose, the mixed time-domain / frequency-domain CELP coding method 170 / 770 comprises an addition operation 161, using an adder 111 (Figures 1, 2, 5, and 6) of the mixed time-domain / frequency-domain CELP encoder 120 / 720, of the frequency-quantized difference vector i,ϋ from the frequency quantizer 110 to the frequency-filtered, time-domain excitation contribution. When the unified time-domain / frequency-domain coding device 100 / 700 changes its bit allocation from a time-domain-only coding mode to a mixed time-domain / frequency-domain coding mode, the frequency-band excitation spectrum energy of the time-domain-only coding mode does not match the frequency-band excitation spectrum energy of the mixed time-domain / frequency-domain coding mode.This energy mismatch can create switching artifacts that are more audible at low bit rates. To reduce any audible degradation caused by this bit reallocation, a long-term gain can be calculated for each band and applied to the summed excitation to correct the energy of each frequency band for a few structures after the reallocation. The mixed time-domain / frequency-domain CELP coding method 170 / 770 then comprises an operation 162 (Figures 1, 5, and 6) to transform the sum of the frequency-quantized difference vector and the frequency-transformed and filtered time-domain excitation contribution / / «f to the time domain using, for example, an inverse DCT 220 (Figure 2).

[00149] The unified time-domain / frequency-domain 150 / 750 encoding method comprises a 163 / 756 operation to produce a synthesized signal by filtering the total time-domain / frequency-domain excitation from the IDCT 220 through an LP 113 / 706 synthesis filter (Figures 1, 2, and 7) of the 100 / 700 encoding device.

[00150] The quantized positions and signs of the frequency pulses that make up the quantized difference vector / jy are transmitted to the distant decoder (not shown).

[00151] In one non-limiting embodiment, while the CELP coding memories are updated on a substructure basis using only the time-domain excitation contribution, the total time-domain / frequency-domain excitation is used to update those memories at the structure boundaries. In another possible implementation, the CELP coding memories are updated on a substructure basis and also at the structure boundaries using only the time-domain excitation contribution. This results in an integrated structure where the frequency-domain quantized signal constitutes an upper quantization layer independent of the central CELP layer. This has advantages in certain applications. In this particular case, the fixed codebook is always used to maintain good perceptual quality, and the number of substructures is always four (4) for the same reason.However, frequency domain analysis can be applied to the entire structure. This integrated approach works for bit rates of approximately 12 kbps and above. 7) Decoding method and device

[00152] Figure 11 is a block schematic diagram that simultaneously illustrates a decoder device 1100 and the corresponding decoding method 1150 for decoding a bit stream 1101 from the unified time domain / frequency domain encoding device 700 described above and the corresponding unified time domain / frequency domain encoding method 750.

[00153] The decoder device 1100 comprises a receiver (not shown) for receiving bit stream 1101 from unified time domain / frequency domain encoding device 700.

[00154] If the sound signal encoded by the unified time-domain / frequency-domain encoding device 700 has been classified as music, this is indicated in the bitstream 1101 by the corresponding signaling bits and detected by the decoding device 1100 (see 1102). The received bitstream 1101 is then decoded by a music decoder 1103, for example, a frequency-domain decoder.

[00155] If the sound signal encoded by the unified time-domain / frequency-domain encoding device 700 has been classified as voice, this is indicated in the bitstream 1101 by the corresponding signaling bits and detected by the decoding device 1100 (see 1104). The received bitstream 1101 is then decoded by a voice decoder 1105, for example, a time-domain decoder using ACELP (algebraic code-excited linear prediction) or more generally CELP (code-excited linear prediction).

[00156] If the sound signal encoded by the unified time-domain / frequency-domain encoding device 700 has not been classified as music or speech (see 1102 and 1104) and the bit rate available for encoding the sound signal was equal to or less than 9.2 kbps (see 1106), this is indicated in the bitstream by the submode indicator FtfSm set to 0. The received bitstream 1101 is then decoded using the backward encoding mode, i.e., the unified time-domain and frequency-domain encoding model inherited from Figures 1 and 2 (EVS) as shown in 1107.

[00157] Finally, if the sound signal encoded by the unified time-domain / frequency-domain encoding device 700 has not been classified as music or speech (see 1102 and 1104) and the bit rate available for encoding the sound signal was greater than 9.2 kbps (see 1106), this is indicated in the bitstream 1101 by a submode indicator FtfS1 set to 1, 2, or 3. The received bitstream 1101 is then decoded using the sound signal decoder 1200 and the corresponding sound signal decoding method 1250 of Figure 12. 7.1) sound signal decoder and decoding method

[00158] Figure 12 is a block schematic diagram that simultaneously illustrates a sound signal decoder 1200 and the corresponding sound signal decoding method 1250 for decoding a bitstream from the unified time domain / frequency domain encoding device 700 described above and the corresponding unified time domain / frequency domain encoding method 750 in the case of a sound signal classified in the unclear signal type category.

[00159] As mentioned in the description above, the adaptive codebook index T and adaptive codebook gain b are quantized and transmitted, and are therefore received in the bitstream by the receiver (not shown). Similarly, when used, the fixed codebook index and fixed codebook gain are also quantized and transmitted to the decoder, and are therefore received in bitstream 1101 by the receiver (not shown). The sound signal decoding method 1250 comprises an operation 1256 of calculating a decoded time-domain excitation contribution using the adaptive codebook index and gain and, if used, the fixed codebook index and gain, as is commonly done in the CELP coding technique.To carry out operation 1256, the sound signal decoder 1200 comprises a calculation device 126 of the decoded time domain excitation contribution.

[00160] The sound signal decoding method 1250 also comprises an operation 1257 for calculating a frequency transform of the decoded time-domain excitation contribution using the same procedure as in operation 156 using a DCT transform. To carry out operation 1257, the sound signal decoder 1200 comprises a device 1207 for calculating the frequency transform of the decoded time-domain excitation contribution.

[00161] As mentioned in the description above, a quantitative version of the cutoff frequency is transmitted to the decoder and is therefore received in the bit stream 1101 by the receiver (not shown). The sound signal decoding method 1250 comprises a filtering operation 1258 of the time-domain frequency transform of the excitation contribution from the computational device 1207 using the decoded cutoff frequency f, <q recuperada del flujo de bits 1101 y un procedimiento que es el mismo o similar a la operación filtrado 266 descrita anteriormente. para completar 1258, decodificador señal sonido 1200 comprende filtro 1208 transformada frecuencia contribución excitación dominio tiempo usando corte f„y. el tiene misma estructura, al menos una estructura similar, 216 figura 2.b / nonn / cznz / B / viAi

[00162] The filtered frequency transform of the time-domain excitation contribution of filter 1208 is supplied to a positive input of a summing adder 1209 performing a corresponding addition operation 1259.

[00163] The sound signal decoding method 1250 comprises an operation 1260 of calculating the decoded energy and the gain per frequency band of the difference vector / / . To carry out operation 1260, the sound signal decoder 1200 comprises a calculation device 1210. Specifically, the calculation device 1210 dequantizes, using procedures inverse to those described herein for quantization, the quantized energy per frequency band and the quantized gain per frequency band received in the bit stream 1101 by the receiver (not shown) from the unified time-domain / frequency-domain encoding device 700.

[00164] The sound signal decoding method 1250 comprises a frequency-quantized difference vector recovery operation 1261. To perform operation 1261, the sound signal decoder 1200 comprises a calculation device 1211. The calculation device 1211 extracts the quantized positions and signs of the frequency pulses from the bit stream 1101 and replicates the selection of frequency bands to be used for quantization and bit allocation in the different frequency bands as determined by operation 757 and allocator 707 and employed by the unified time-domain / frequency-domain encoding device 700 to encode the input sound signal. The calculation device 1211 uses this replicated information to recover the frequency-quantized difference vector f¡Q from the extracted frequency pulse quantized positions and signs.Specifically, for that purpose, the 1200 sound signal decoder replicates the procedure used in the 700 unified time domain / frequency domain encoding device as illustrated in Figure 9 in response to the number of bits (bit rate) available in the 1200 decoder for the frequency quantized difference vector fdQ (see 1220), the total bit rate available for the channel being processed (see 1220), and the submode indicator (see 1220).

[00165] Specifically: - Estimator 1201 and operation 1251 in Figure 12 correspond to estimator 901 and operation 951 in Figure 9, to pre-fix a fraction of the available bit budget to quantize the lower frequencies of the difference vector / / as a function of the quantized cutoff frequency / <ρ.- The estimator 1202 and the operation 1252 of Figure 12 correspond to the estimator 902 and the operation 952 of Figure 9, to estimate the maximum number NBmx of frequency bands of the quantized difference vector / / ρ. - Calculation device 1203 and operation 1253 in Figure 12 correspond to calculation device 903 and operation 953 in Figure 9, for calculating lower frequency bits. - The characterization 1204 and operation 1254 of Figure 12 correspond to the characterization 904 and operation 954 of Figure 9, for the characterization of the frequency band. - The distributor 1205 and operation 1255 in Figure 12 correspond to the distributor 905 and operation 955 in Figure 9, for the final distribution of bits per frequency band.

[00166] The sound signal decoding method 1250 comprises an operation 1259 to add the frequency-quantized difference vector recovered f / Q from the computing device 1211 and the frequency-transformed and filtered time-domain excitation contribution f from the filter 1208 to form the mixed time-domain / frequency-domain excitation.

[00167] As can be seen, estimators 1201 and 1202, calculation device 1203, characterizer 1204, distributor 1205, calculators 1206 and 1207, filter 1208, calculators 1210 and 1211, and adder 1212 form a reconstructor of the mixed time-domain / frequency-domain excitation using information carried in bit stream 1101, including the submode indicator that identifies one of the selected encoding submodes used to encode the sound signal classified in the unclear signal type category.

[00168] Similarly, operations 1251-1261 form a method of reconstructing the mixed time domain / frequency domain excitation using the information carried in bit stream 1101.

[00169] The sound signal decoder 1200 comprises a converter 1212 for performing a 1262 transformation operation of the mixed time domain / frequency domain / frequency domain excitation 1200 to the time domain using, for example, the IDCT (inverse DCT) 220 (inverse DCT) 220.

[00170] Finally, the synthesized sound signal is calculated in the decoder 1200 by a filtering operation 1263 through an LP (linear prediction) synthesis filter 1213 of the total excitation of the converter 1212. Of course, the LP parameters required by the decoder 1200 to reconstruct the synthesis filter 1213 are transmitted from the unified time-domain / frequency-domain encoding device 700 and extracted from the bit stream 1101 as is well known in the CELP encoding technique. 8) Hardware Implementation

[00171] Figure 10 is a simplified block diagram of an example configuration of hardware components that make up the unified frequency domain 100 / 700 time domain / 150 / 750 encoding device, decoder device 1100, and decoding method 1150 described above.

[00172] The unified time-domain / frequency-domain encoding device 100 / 700 and decoding device 1100 can be implemented as part of a mobile terminal, as part of a portable media player, or in any similar device. The 100 / 700 device and the 1100 decoding device (identified as 1000 in Figure 10) comprise an input 1002, an output 1003, a processor 1001, and a memory 1004.

[00173] Input 1002 is configured to receive the sound input signal 101 / bit stream 1101 from Figures 1 and 7, in either digital or analog form. Output 1003 is configured to supply the output signal. Input 1002 and output 1003 can be implemented in a common module, for example, a serial input / output device.

[00174] Processor 1001 is functionally connected to input 1002, output 1003, and memory 1004. Processor 1001 is realized as one or more processors to execute code instructions in support of the functions of the various components of the unified time-domain / frequency-domain encoding device 100 / 700 to encode an input sound signal as illustrated in Figures 1-9, or of the decoder device 1100 of Figures 11-12.

[00175] Memory 1004 may comprise non-transient memory for storing code instructions executable by the processor(s) 1001, specifically, processor-readable memory comprising / storing non-transient instructions that, when executed, cause a processor(s) to implement the operations and components of the unified time-domain / frequency-domain encoding device 100 / 700 and method 150 / 750 and the decoding device 1100 and decoding method 1150 described herein. Memory 1004 may also comprise random-access memory or buffer memory(s) for storing intermediate processing data of the various functions performed by the processor(s) 1001. Those skilled in the field will realize that the descriptions of the 100 / 700 unified time-domain / frequency-domain encoding device and method 150 / 750, and the 1100 decoding device and decoding method 1150 are merely illustrative and not intended to be exhaustive. Other modalities will readily be suggested to those with ordinary expertise in the field who benefit from this description. Furthermore, the disclosed 100 / 700 unified time-domain / frequency-domain encoding device and method 150 / 750, the 1100 decoding device, and the 1150 decoding method can be customized to provide valuable solutions to existing sound encoding and decoding needs and problems.

[00177] For the sake of clarity, not all routine features of the bjnonn / cznz / B / viAi unified time-domain / frequency-domain encoding device implementations are shown and described. 100 / 700 and the 150 / 750 method and the 1100 decoder device and the 1150 decoding method. It will be appreciated, of course, that in the development of any actual modality of this type of unified time-domain / frequency-domain encoding device 100 / 700 and the 150 / 750 method and the 1100 decoder device and the 1150 decoding method, numerous specific implementation decisions may need to be made to achieve the specific goals of the developer, such as compliance with application, system, network, and business constraints, and that these specific goals will vary from implementation to implementation and from developer to developer. Furthermore, it will be appreciated that a development effort could be complex and time-consuming, but nevertheless would be a routine engineering task for experts in the field of sound processing who have the benefit of the present description.

[00178] According to this description, the components / processors / modules, processing operations, and / or data structures described herein may be implemented using various types of operating systems, computing platforms, network devices, software programs, and / or general-purpose machines. Furthermore, those skilled in the art will recognize that devices of a less general nature, such as hardwired devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (AS1Cs), or similar devices, may also be used. When a method comprising a series of operations and sub-operations is implemented by a processor, computer, or machine, and those operations and sub-operations can be stored as a series of non-transient code instructions readable by the processor, computer, or machine, they may be stored on a tangible and / or non-transient medium.

[00179] The 100 / 700 unified time domain / frequency domain encoding device and method 150 / 750 and the 1100 decoder device and decoding method 1150 as described herein may use software, firmware, hardware or any combination of software, firmware or hardware suitable for the purposes described herein.

[00180] In the unified time domain / frequency domain coding device 100 / 700 and method 150 / 750 and the decoding device 1100 and decoding method 1150 as described herein, the various operations and sub-operations can be performed in various orders and some of the operations and sub-operations can be optional.

[00181] Although the present description has been described above in this document by way of non-restrictive illustrative modalities thereof, these modalities may be modified at will within the scope of the appended claims without departing from the spirit and nature of the present description. bynonn / cznz / B / viAi 9) List of References

[00182] This description mentions the following references, the full content of which is incorporated herein as a reference: [1] US Patent No. 9,015,038, "Coding generic audio signals at low bit rate and low delay." [2] 3GPP TS 26.445, v. 12.0.0, "Codee for Enhanced Voice Services (EVS); Detailed Algorithmic Description, Sept. 2014. [3J 3GPP SA4 contribución S4-170749 "New WID on EVS Codee Extensión for Iinmersive Voice and Audio Services, reunión SA4 #94, 26-30 de junio, 2017, http: / / www.3gpp.org / ftp / tsg_sa / WG4_CODEC / TSGS4_94 / Docs / S4-170749.zip [4] Solicitud Provisional de Patente de EE.UU. 63 / 010,798, "Method and device for speecliinusic classification and core encoder selection in a sound codee. [5] 1TU-T Recommendation G.718 "Frame error robust narrow-band and wideband embedded variable bit-rate coding of speech and audio from 8-32 kbit / s, junio 2008. [6] T. Vaillancourt et al., "Inter-tone noise reduction in a low bit rate CELP decoder, IEEE Proceedings of International Conference on Acoustics, Speech and Signal Processing (ICASSP). Taipei, Taiwan, Abril de 2009, pp. 4113-16. [7] V. Eksler y M. Jelínek, , "Transition mode coding for source controlled CELP codees, IEEE Proceedings of International Conference on Acoustics, Speech and Signal Processing (ICASSP), marzo-abril de 2008, pp. 4001-4043. [8] U. Mittal, J.P. Ashley, y E.M. Cruz-Zeno, "Low Complexity Factorial Pulse Coding of MDCT Coefficients using Approximation of Combinatoria! Functions”, IEEE Proceedings of International Conference on Acoustics, Speech and Signal Processing (ICASSP), Taipei, Taiwan, abril de 2007, pp. 289-292. < / q> < / jen>

Claims

1. A unified time-domain / frequency-domain encoding device for encoding an input sound signal, comprising: a classifier of the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal type category indicating that the nature of the input sound signal is unclear; a selector of one of a plurality of encoding sub-modes for encoding the input sound signal if the input sound signal is classified into the unclear signal type category; and a mixed time-domain / frequency-domain encoder for encoding the input sound signal using the selected encoding sub-mode.

2. The unified time domain / frequency domain encoding device according to claim 1, wherein the sound signal categories comprise voice, music, and the unclear signal type indicating that the input sound signal is not classified as either voice or music.

3. The unified time domain / frequency domain encoding device according to claim 2, comprising a frequency domain encoder for encoding the input sound signal if the classifier classifies the input sound signal into the music category.

4. The unified time domain / frequency domain coding device according to claim 2 or 3, comprising a time domain encoder for encoding the input sound signal if the classifier classifies the input sound signal into the voice category.

5. The unified time domain / frequency domain encoding device according to any of claims 1 to 4, wherein the selector selects the encoding submode in response to a bit rate to encode the input sound signal and the input sound signal characteristics classified in the unclear signal type category.

6. The unified time domain / frequency domain encoding device according to any of claims 1 to 5, wherein the encoding submodes are identified by respective submode indicators.

7. The unified time-domain / frequency-domain encoding device according to claim 5 or 6, wherein the selector selects a back-encoding submode using a legacy unified time-domain and frequency-domain encoding model to encode the input sound signal if (a) a bit rate available for encoding the input sound signal is not greater than a given first value and (b) the input sound signal is not classified as voice or music.

8. The unified time domain / frequency domain encoding device according to any of claims 5 to 7, wherein the selector selects a first encoding submode if voice-like features are detected in the input sound signal.

9. The unified time-domain / frequency-domain encoding device according to claim 8, wherein the selector selects the first encoding submode if (a) the input sound signal is not classified as voice or music by the classifier and an available bit rate for encoding the input sound signal is greater than a second given value, (b) the probability that the input sound signal is music is not greater than a third given value, and (c) no time attack is detected in a current structure of the input sound signal.

10. The unified time domain / frequency domain encoding device according to any of claims 5 to 9, wherein the selector selects a second encoding submode if a time attack is detected in the input sound signal.

11. The unified time domain / frequency domain encoding device according to claim 10, wherein the selector selects the second encoding submode if (a) the input sound signal is not classified as voice or music by the classifier and an available bit rate for encoding the input sound signal is greater than a given fourth value, (b) the probability that the input sound signal is music is not greater than a given third value, and (c) a time attack is detected in a current structure of the input sound signal.

12. The unified time domain / frequency domain encoding device according to any of claims 5 to 11, wherein the selector selects a third encoding submode if music-like features are detected in the input sound signal.

13. The unified time domain / frequency domain encoding device according to claim 12, wherein the selector selects the third encoding submode if (a) the input sound signal is not classified as voice or music by the classifier and an available bit rate for encoding the input sound signal is greater than a given sixth value, and (b) the probability that the input sound signal is music is greater than a given seventh value.

14. The unified time-domain / frequency-domain encoding device according to any of claims 1 to 13, wherein: - the selector selects a first encoding submode if voice-like features are detected in the input sound signal; - the selector selects a second encoding submode if a time attack is detected in the input sound signal; - the selector selects a third encoding submode if music-like features are detected in the input sound signal.

15. The unified time domain / frequency domain encoding device according to claim 14, wherein the selector selects (a) in the third encoding submode, a predetermined number of substructures per structure to encode the input sound signal and (b) in the first and second encoding substructures, a number of substructures less than the predetermined number and depending on an available bit rate to encode the input sound signal.

16. A unified time-domain / frequency-domain coding method for encoding an input sound signal, comprising: classifying the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal type category indicating that the nature of the input sound signal is unclear; selecting one of a plurality of coding sub-modes to encode the input sound signal if the input sound signal is classified into the unclear signal type category; and mixed time-domain / frequency-domain coding of the input sound signal using the selected coding sub-mode.

17. The unified time domain / frequency domain coding method according to claim 16, wherein the sound signal categories comprise voice, music, and the unclear signal type indicating that the input sound signal is not classified as either voice or music.

18. The unified time domain / frequency domain coding method according to claim 17, comprising encoding the input sound signal in the frequency domain if the classifier classifies the input sound signal into the music category.

19. The unified time domain / frequency domain coding method according to claim 17 or 18, comprising encoding the input sound signal in the time domain if the classifier classifies the input sound signal into the voice category.

20. The unified time domain / frequency domain coding method according to any of claims 16 to 19, wherein selecting one from a plurality of coding submodes comprises selecting the coding submode in response to a bit rate to encode the input sound signal and the input sound signal characteristics classified in the unclear signal type category.

21. The unified time-domain / frequency-domain coding method according to any of claims 16 to 20, comprising identifying the coding submodes by means of respective submode indicators.

22. The unified time-domain / frequency-domain coding method according to bynonn / cznz / B / viAi with claim 20 or 21, wherein selecting one from a plurality of coding submodes comprises selecting a backward coding submode using a legacy unified time-domain and frequency-domain coding model to encode the input sound signal if (a) a bit rate available for encoding the input sound signal is not greater than a given first value and (b) the input sound signal is not classified as voice or music.

23. The unified time domain / frequency domain coding method according to any of claims 20 to 22, wherein selecting one from a plurality of coding submodes comprises selecting a first coding submode if speech-like features are detected in the input sound signal.

24. The unified time domain / frequency domain coding method according to claim 23, wherein the first coding submode is selected if (a) the input sound signal is not classified as voice or music and an available bit rate for encoding the input sound signal is greater than a given second value, (b) the probability that the input sound signal is music is not greater than a given third value, and (c) no time attack is detected in a current structure of the input sound signal.

25. The unified time domain / frequency domain coding method according to any of claims 20 to 24, wherein selecting one from a plurality of coding sub-modes comprises selecting a second coding sub-mode if a time attack is detected in the input sound signal.

26. The unified time domain / frequency domain coding method according to claim 25, wherein the second coding submode is selected if (a) the input sound signal is not classified as voice or music and an available bit rate for encoding the input sound signal is greater than a given fourth value, (b) the probability that the input sound signal is music is not greater than a given fifth value, and (c) a time attack is detected in a current structure of the input sound signal.

27. The unified time domain / frequency domain coding method according to any of claims 20 to 26, wherein selecting one from a plurality of coding sub-modes comprises selecting a third coding sub-mode if music-like features are detected in the input sound signal.

28. The unified time domain / frequency domain coding method according to claim 27, wherein the third coding submode is selected if (a) the input sound signal is not classified as voice or music and an available bit rate for encoding the input sound signal is greater than a given sixth value, and (b) the probability that the input sound signal is music is greater than a given seventh value.

29. The unified time-domain / frequency-domain coding method according to any of claims 16 to 28, wherein: - the first coding submode is selected if speech-like features are detected in the input sound signal; - the second coding submode is selected if a time attack is detected in the input sound signal; - the third coding submode is selected if music-like features are detected in the input sound signal.

30. The unified time domain / frequency domain coding method according to claim 29, wherein selecting one from a plurality of coding submodes comprises selecting (a) in the third coding submode, a predetermined number of substructures per structure to encode the input sound signal and (b) in the first and second coding submodes, a number of substructures less than the predetermined number and depending on an available bit rate to encode the input sound signal.

31. A unified time-domain / frequency-domain encoding device for encoding an input sound signal, comprising: at least one processor, and a memory coupled to the processor and storing non-transient instructions that, when executed, cause the processor to implement: a classifier of the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal-type category indicating that the nature of the input sound signal is unclear; a selector of one of a plurality of encoding sub-modes for encoding the input sound signal if the input sound signal is classified into the unclear signal-type category; and a mixed time-domain / frequency-domain encoder for encoding the input sound signal using the selected encoding sub-mode.

32. A unified time-domain / frequency-domain encoding device for encoding an input sound signal, comprising: at least one processor, and a memory coupled to the processor and storing non-transient instructions that, when executed, cause the processor to: classify the input sound signal into one of a plurality of sound signal categories, wherein the sound signal categories comprise an unclear signal type category indicating that the nature of the input sound signal is unclear; select one of a plurality of encoding sub-modes to encode the input sound signal if the input sound signal is classified into the unclear signal type category; and mix the input sound signal using the selected encoding sub-mode.

33. A sound signal decoder comprising: a receiver of a bitstream carrying information usable for reconstructing a mixed time-domain / frequency-domain excitation representative of a sound signal classified in an unclear signal-type category, wherein the information includes one of a plurality of coding submodes used to encode the sound signal classified in the unclear signal-type category; a reconstructor of the mixed time-domain / frequency-domain excitation in response to the information carried in the bitstream, including the coding submode used to encode the input sound signal; and a converter of the mixed time-domain / frequency-domain excitation to the time domain.and a synthesis filter to filter the mixed time domain / frequency domain / frequency domain excitation converted to time domain to produce a synthesized version of the sound signal.

34. The sound signal decoder according to claim 33, wherein the encoding submode is identified in the bit stream by a submode indicator.

35. The sound signal decoder according to claim 33 or 34, wherein the encoding submodes comprise (a) a first encoding submode if the sound signal contains voice-like characteristics, (b) a second encoding submode if the sound signal contains a time attack, and (c) a third encoding submode if the sound signal contains music-like characteristics.

36. The sound signal decoder according to any of claims 33 to 35, wherein the reconstructor recovers from the information carried in the bit stream a frequency representation of a time-domain excitation contribution, reconstructs a frequency-quantized difference vector between a frequency-domain excitation contribution and the frequency representation of the time-domain excitation contribution, and adds the frequency-quantized difference signal to the frequency representation of the time-domain excitation contribution to produce the mixed time-domain / frequency-domain excitation.

37. A sound signal decoding method comprising: receiving a bitstream carrying information usable for reconstructing a mixed time-domain / frequency-domain excitation representative of a sound signal classified in an unclear signal-type category, indicating that the nature of the sound signal is unclear, wherein the information includes one of a plurality of coding submodes used to encode the sound signal classified in the unclear signal-type category; reconstructing the mixed time-domain / frequency-domain excitation in response to the information carried in the bitstream, which includes the coding submode used to encode the input sound signal; and converting the mixed time-domain / frequency-domain excitation to the time domain.and filter the mixed time domain / frequency domain / frequency domain converted to time domain excitation through a synthesis filter to produce a synthesized version of the sound signal.

38. The sound signal decoding method according to claim 37, wherein the encoding submode is identified in the bit stream by a submode indicator.

39. The sound signal decoding method according to claim 37 or 38, wherein the encoding submodes comprise (a) a first encoding submode if the sound signal contains voice-like characteristics, (b) a second encoding submode if the sound signal contains a time attack, and (c) a third encoding submode if the sound signal contains music-like characteristics.

40. The sound signal decoding method according to any of claims 37 to 39, wherein reconstructing the mixed excitation in the time domain / frequency domain comprises retrieving from the information carried in the bit stream a frequency representation of a time domain excitation contribution, reconstructing from the information carried in the bit stream a frequency-quantized difference vector between a frequency domain excitation contribution and the frequency representation of the time domain excitation contribution, and adding the frequency-quantized difference signal to the frequency representation of the time domain excitation contribution to produce the mixed excitation in the time domain / frequency domain.

41. A sound signal decoder comprising: at least one processor, and a memory coupled to the processor and storing non-transient instructions that, when executed, cause the processor to implement: a receiver of a bit stream carrying information usable for reconstructing a mixed time-domain / frequency-domain excitation representative of a sound signal classified in an unclear signal-type category showing that the nature of the sound signal is unclear, wherein the information includes one of a plurality of encoding submodes used to encode the sound signal classified in the unclear signal-type category;a reconstructor of the mixed time-domain / frequency-domain excitation in response to the information carried in the bitstream, including the encoding submode used to encode the input sound signal; a converter of the mixed time-domain / frequency-domain excitation to the time domain; and a synthesis filter to filter the converted mixed time-domain / frequency-domain excitation to the time domain to produce a synthesized version of the sound signal.

42. A sound signal decoder comprising: at least one processor, and a memory coupled to the processor and storing non-transient instructions that, when executed, cause the processor to: receive a bitstream carrying information usable for reconstructing a mixed time-domain / frequency-domain excitation representative of a sound signal classified in an unclear signal-type category, wherein the information includes one of a plurality of encoding submodes used to encode the sound signal classified in the unclear signal-type category; reconstruct the mixed time-domain / frequency-domain excitation in response to the information carried in the bitstream, which includes the encoding submode used to encode the input sound signal;convert the mixed time-domain / frequency-domain excitation to the time domain; and filter the converted mixed time-domain / frequency-domain excitation to the time domain through a synthesis filter to produce a synthesized version of the sound signal.

Images