Optimised encoding and decoding of an audio signal using a neural network-based autoencoder

Abstract

The invention relates to a method for encoding an audio signal, comprising the following steps: - decomposing (102) the audio signal into at least amplitude components and sign or phase components; - analysing (104) the amplitude components, using a neural network-based autoencoder, in order to obtain a latent space representative of the amplitude components of the audio signal; - encoding (105) the latent space obtained; - encoding (106) at least some of the sign or phase components. The invention also relates to a corresponding decoding method, as well as to encoding and decoding devices implementing the respective encoding and decoding methods.

Description

Domaine technique

[0001] The present invention relates to the general field of audio signal encoding and decoding. The invention relates in particular to the optimized use of a neural network-based autoencoder for encoding and decoding an audio signal. Technique antérieure

[0002] In conventional audio signal encoding and decoding systems, the input audio signal is generally converted into a frequency domain, either using a filter bank or applying a short-time transform, to achieve a useful encoding gain and exploit the psychoacoustic properties of human auditory perception. Indeed, these psychoacoustic properties are exploited, for example, by distributing the bit budget non-uniformly and / or adaptively according to frequency bands. The time-frequency conversion can then be seen as a transformation towards a representation better suited for encoding at a given bit rate. The decoder, for its part, must reverse this transformation.

[0003] For a lossy compression system, the general objective is to find a signal representation that is best suited to encoding at the lowest possible bitrate for a given quality, or conversely, to achieving the best possible quality at a given bitrate. In the field of audio, perceptual considerations due to imperfections of the human ear (e.g., masking phenomena) can be exploited to obtain an even better bitrate / distortion (perceptual) trade-off than with conventional non-perceptual encoding.

[0004] Examples of classic audio codecs are given by the MPEG-Audio standards (e.g., MP3, AAC, etc.) or other standards (e.g., ITU-T G.722.1, G.719). Generally, these codecs have architectures comprising different signal processing or quantization / coding modules that are optimized separately.

[0005] Recently, new approaches to signal compression have emerged through the use of neural networks performing end-to-end learning. With the widespread adoption of GPU (Graphical Processing Unit) architectures and other processors specialized for neural networks, this type of neural network coding approach is promising and could eventually replace traditional audio codecs.

[0006] An example of neural network architecture applied to the field of image and video compression is described in the articles: "Johannes Ballé, Valero Laparra, Eero P. Simoncelli, End-to-end Optimized Image Compression, Int. Conf. on Learning Representations (ICLR), 2017" and "Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, Nick Johnston, Variational image compression with a scale hyperprior, Int. Conf. on Learning Representations (ICLR), 2018".

[0007] These methods are based on the principle of (classical) autoencoders and so-called variational autoencoders (VAE).

[0008] Autoencoders are machine learning algorithms based on artificial neural networks that construct a new, more compact (compressed) representation of a dataset. The architecture of an autoencoder consists of two parts or operators: the encoder (analysis part) f(x), which transforms the input data x into a representation z, and the decoder (synthesis part) g(z), which resynthesizes the signal from z. The encoder is made up of a set of neural layers that process the input data x to construct a new, low-dimensional representation called the "latent space" (or hidden variables) z. This latent space represents the important features of the input signal in a compact form. The decoder's neural layers receive the latent space as input and process it to try to reconstruct the original data.The differences between the reconstructed and original data allow us to measure the error made by the autoencoder. Training involves modifying the autoencoder's parameters to reduce the reconstruction error measured on the different samples of the dataset. Various error criteria are possible, such as the mean squared error (MSE).

[0009] Unlike a conventional autoencoder, a variational autoencoder (VAE) adds a representation of the latent space using a multivariate Gaussian model (means and variances) of that latent space. A VAE also consists of an encoder (or inference or recognition model) and a decoder (or generative model). A VAE attempts to reconstruct the input data like an autoencoder; however, the latent space of a VAE is continuous.

[0010] The methods of Ballé et al. use network architectures based on 2D convolutional networks (with, for example, in one implementation, a 5x5 filter and decimation or oversampling - respectively - by 2 in each encoding or decoding layer); an adaptive normalization called GDN (Generalized Divisive Network) is applied between each layer, which improves performance compared to "batch normalization".

[0011] It should be noted that the methods of Ballé et al. use an approximation of the space encoding by a simplified quantization model (addition of noise) with a Gaussian model for learning, however the latent space is in fact encoded directly, which amounts to classical (or deterministic) auto-encoder methods.

[0012] The direct application of the aforementioned methods by Ballé et al., derived from image and video compression, to audio signal compression is not satisfactory. Indeed, an image or video sequence is composed of pixels that can be viewed as random variables with integer values ​​over a predefined interval, for example [0, 255] for images or videos with a resolution of 8 bits per pixel. These pixels only have positive values.

[0013] Audio signals, on the other hand, are generally signed. Furthermore, after a time-to-frequency transformation, audio signals can be real or complex. In addition, the spectral dynamic range in the audio domain is greater than in the image or video domains, on the order of 16 bits per sample (or even more for so-called "high-resolution" audio). Directly transposing an autoencoder architecture similar to Ballé et al.'s to audio yields relatively poor results, particularly in terms of reconstruction quality.

[0014] Also noted in the prior art is patent application KR10-2021-0133551A disclosing audio encoding and decoding techniques using an autoencoder and providing a quantized representation of a latent vector representing a high-band frequency spectrum. During decoding, the sign of the spectrum is restored by copying the sign of the low-band. Audio encoding and decoding techniques are also known from patent application US2020 / 0111501A1, where the encoding transforms an original test signal in the spectral domain and uses a binary neural network of the autoencoder type. It is also known from the publication AANuraha, K. Sekiguchi, K.Yoshi "A DEEP GENERATIVE MODEL OF SPEECH COMPLEX SPECTROGRAMS" of techniques jointly modeling the amplitude and phase of a short-term Fourier transform, however does not address the encoding of these components.

[0015] Therefore, there is a need to optimize auto-encoder type coding techniques for the field of audio coding / decoding. Exposé de l'invention

[0016] The invention improves upon the existing state of the art.

[0017] To this end, the invention relates to a method for encoding an audio signal comprising the following steps: decomposition of the audio signal into at least amplitude components and sign or phase components; analysis of the amplitude components by a neural network-based autoencoder to obtain a latent space representative of the amplitude components of the audio signal; encoding of the latent space obtained; encoding of at least a part of the sign or phase components.

[0018] The invention enables the optimized application of autoencoders using neural networks for encoding / decoding audio signals. Differentiated encoding of the sign component or the phase component, depending on the signal decomposition method, ensures high audio quality. According to the invention, this quality can achieve performance levels approaching transparency at sufficient bitrates, meaning that the quality of the decoded signal is very close to that of the original signal.

[0019] The method used is an end-to-end method that does not require independent optimization of the different coding modules. Furthermore, the method does not need to take perceptual considerations into account, unlike traditional audio coding methods.

[0020] In a particular embodiment, the process further includes a step of compressing the amplitude components before their analysis by the autoencoder.

[0021] Thus, the input data to the autoencoder is restricted, in order to optimize the analysis and the obtaining of the resulting latent space.

[0022] In one example implementation, the compression of the amplitude components is carried out by a logarithmic function.

[0023] This type of less complex compression provides advantageous compression performance for reducing the dynamic range of data entering the autoencoder.

[0024] In one embodiment, the audio signal before the decomposition step is obtained by an MDCT-type transform applied to an input audio signal.

[0025] In one embodiment, the audio signal is a multichannel signal.

[0026] In another embodiment of the invention, the audio signal is a complex signal comprising a real and imaginary part resulting from a transformation of an input audio signal, the amplitude components resulting from the decomposition step corresponding to the amplitudes of the combined real and imaginary parts, and the sign or phase components corresponding to the signs or phases of the combined real and imaginary parts.

[0027] This type of frequency representation by MDCT or by another transform of the audio signal offers an advantage to the use of the coding method according to the invention, because it puts the signal in a time / frequency representation similar to a spectrogram, which makes it more natural to apply image or video compression methods to the amplitudes; the signs or phases are coded separately for better efficiency.

[0028] In one embodiment, all sign or phase components of the audio signal are coded.

[0029] This solution has the advantage of being simple but requires sufficient bandwidth.

[0030] In a particular embodiment, only the sign or phase components corresponding to the low frequencies of the audio signal are coded.

[0031] Thus, it is possible to optimize the coding rate by encoding only a portion of the signs or phases, which reduces the additional rate required to encode the signs or phases, without significantly impacting the quality of the reconstructed signal at decoding.

[0032] In one embodiment, the sign or phase components corresponding to the low frequencies of the audio signal are coded and selective coding is performed for the sign or phase components corresponding to the high frequencies of the audio signal.

[0033] This allows certain sign or phase components of high frequencies to be obtained in an optimized way while reducing the coding rate.

[0034] In one embodiment, the positions of the sign or phase components selected for selective coding are also coded, so as to recover the selection that was made during coding.

[0035] However, this solution requires additional coding bandwidth for encoding these positions.

[0036] In another embodiment, the positions of the selected sign or phase components and their associated values ​​are coded together, which optimizes the coding rate for encoding this information and retrieving it during decoding.

[0037] The invention also relates to a method for decoding an audio signal comprising the following steps: decoding of sign or phase components of the audio signal; decoding of a latent space representing amplitude components of the audio signal; synthesis of the amplitude components of the audio signal by a neural network-based autoencoder, from the decoded latent space; combination of the decoded amplitude components and the decoded sign or phase components to obtain a decoded audio signal.

[0038] The decoding process offers the same advantages as the encoding process described previously.

[0039] In a particular embodiment, where the decoded phase components correspond to a part of the phase components of the audio signal, the other part is reconstructed before the combination step.

[0040] Thus, it is possible to optimize the encoding rate of sign or phase information by encoding and decoding only a part of it and reconstructing the other part in order to recover all the sign or phase components during decoding.

[0041] The invention relates to a coding device comprising a processing circuit for implementing the steps of the coding process as described above.

[0042] The invention also relates to a decoding device comprising a processing circuit for implementing the steps of the decoding process as described above.

[0043] The invention relates to a computer program comprising instructions for implementing the encoding or decoding processes as described above, when executed by a processor.

[0044] Finally, the invention relates to a storage medium, readable by a processor, storing a computer program containing instructions for the execution of the encoding or decoding process described above. Brève description des dessins

[0045] Other features and advantages of the invention will become clearer upon reading the following description of particular embodiments, given by way of simple illustrative and non-limiting examples, and the accompanying drawings, among which: [ Fig.1a ] illustrates an encoder and a decoder implementing respectively an encoding and decoding process according to a first embodiment of the invention; [ Fig.1b ] illustrates an example of sign bit encoding and multiplexing according to the invention; [ Fig.1c ] illustrates an example of sign bit encoding and multiplexing with high-frequency bit selection according to the invention; [ Fig.1d ] illustrates an encoder and a decoder implementing respectively an encoding and decoding process according to a variant embodiment of the invention; [ Fig.2a ] illustrates an example of the implementation of the analysis and synthesis parts of an autoencoder used according to the invention; [ Fig.2b ] illustrates an example of an input / output format for the analysis part of an autoencoder used according to the invention; [ Fig.2c ] illustrates an example of an input / output format for the synthesis section of an autoencoder used according to the invention; [ Fig.3a ] illustrates an encoder and a decoder implementing respectively an encoding and decoding process according to a second embodiment of the invention; [ Fig.3b ] illustrates an encoder and a decoder implementing respectively an encoding and decoding process according to a third embodiment of the invention; and [ Fig.4 ] illustrates examples of structural realization of an encoding device and a decoding device according to an embodiment of the invention. Description des modes de réalisation

[0046] There [ Fig.1a ] describes a first embodiment of an encoder and a decoder according to the invention as well as the steps of an encoding process and a decoding process according to a first embodiment of the invention.

[0047] The codec represented in [ Fig.1a ] includes an encoder (100) and a decoder (110).

[0048] The encoder 100 receives as input an audio signal x, sampled at a frequency fs (for example, 48 kHz) and divided into successive time frames of index t and length L ≥ 1 sample(s), for example, L = 240 (5 ms). This signal x can be a mono (one-dimensional) signal denoted x(t,n) where n is the time index, or a multichannel signal denoted x(i,t,n) where i = 0, ..., C - 1 is the channel index and C > 1 is the number of channels. In an example implementation, C = 2 could be taken for a stereo signal or C = 4 for a first-order ambisonic signal.

[0049] The encoding is performed using a number of frames. N T ≥ 1, where t=T 0 , ... , T 0 +N T - 1 , Or T 0 is a frame index identifying the first frame analyzed in the group of frames analyzed. Typically T 0 can start at T 0 = 0 by convention, then T 0 is incremented by N T .L samples for each group of frames analyzed.

[0050] A time / frequency transformation is applied in 101 to the input signal x. In general this transformation can be carried out by a frequency transform described below (MDCT, STFT ...) or by a bank of filters (PQMF, CLDFB...) to obtain the transformed signal X.

[0051] In a given frame, and for the mono case with transform encoding, this transformed signal (real or complex) is denoted X(t,k) where k is a frequency index. Note that filter banks can operate by subframes and generate a set of samples (real or complex) per subframe; in this case, the transformed signal will be denoted X(t',k) where t' is a time index (of subframes) and k is a frequency index.

[0052] In the multichannel case, we generalize these notations to: X(i,t,k) where a transform is determined separately for each channel, or X(i,t',k) for the case of filter banks.

[0053] In a first embodiment, we consider the case of a modified discrete cosine transform (MDCT) for a mono signal. In this case, the signal x(t,n), n=0,...,L-1 in the current frame of index t< is analyzed with an additional segment of L future samples that correspond to the future frame x(t,n), n=L,...,2L-1, with the convention that x(t,n)= x(t+1,nL) for n=L,...,2L-1.

[0054] The MDCT transform is defined by: X t k = ∑ n = 0 2 L − 1 2 L sin π 2 L n + 0.5 cos π L n + L 2 + 0.5 k + 0.5 x t n , k = 0 , … , L − 1

[0055] With k being the frequency index and L the number of frequency indices.

[0056] In an example implementation where fs=48000 Hz, we could, for example, take L= 240 samples; in this case, we also obtain 240 frequency indices. Other values ​​of L are possible according to the invention.

[0057] In a preferred embodiment, the MDCT transform can be truncated when the high band is not relevant. For example, at 48 kHz, the 20-24 kHz band is not audible. For L = 240, the coefficients k = 200, ... , 239 can be ignored; in this case, only the N K = First 200 coefficients k= 0 , ... , N K - 1 .

[0058] Therefore, we will note the following: N K ≤ L the number of spectral coefficients actually used.

[0059] The MDCT transform can be decomposed into a windowing, time folding, and addition-over operation, followed by a discrete cosine transform (DCT-IV). Omitting the index t< to simplify the notation of the intermediate signal ( v ( n ) ), the windowing, folding and addition operations are given by: ν n = w L 2 − 1 − n x t , L 2 − 1 − n + w L 2 + n x t , L 2 + n pour 0 ≤ n ≤ L 2 − 1 And ν n + L 2 = w L − 1 − n x t , L − 1 + n − w n x t , 2 L − 1 − n pour 0 ≤ n ≤ L 2 − 1 for example, with a sinusoidal windowing given by: w n = sin π 2 L n + 0.5 , n = 0 , … , L − 1

[0060] The discrete cosine transformation of type DCT-IV is given by: X t k = ∑ n = 0 L − 1 2 2 L cos π 2 L n + 0.5 k + 0.5 ν n

[0061] In some variations, other windows are possible as long as they satisfy the conditions for (near-)perfect reconstruction. Similarly, other definitions of the MDCT transform can be used, such as the MLT (Modulated Lapped Transform) and the TDAC (Time-Domain Aliasing Cancellation) filter bank. Other fast implementation algorithms and intermediate transformations besides DCT-IV (for example, the FFT (Fast Fourier Transform)) can be employed. The advantage of the MDCT transform is that it is critically decimated. L transformed coefficients for each frame of L index samples t< . This transformation produces a transformed signal. X ( t , k ) , k = 0, ... , L - 1 in the case of a single-channel input signal.

[0062] Thus, when several successive frames are analyzed and the transformed signal is concatenated, the transformed signal X ( t, k ) , t = T 0 , ... , T 0 +N T - 1 , k = 0, ... , N K - 1, where T 0 is a frame index identifying the first frame analyzed in the group of analyzed frames, and can be seen as a two-dimensional matrix of size N T × N K with a temporal dimension (on the index t< ) and a frequency dimension (on the index k< ).

[0063] In variants where the input signal is multichannel, a transform is determined separately for each channel, to obtain a transformed signal denoted X ( i, t, k ) , où i = 0, ... , C - 1 is the channel index.

[0064] In some variations, switching analysis windows can be used, for example, based on transient detection. In this case, several shorter transforms are typically used, yielding the same total number of coefficients, but divided into subframes and with reduced frequency resolution. In the mono case, for a critical decimation filter bank, this results in a transformed signal of the form X ( t', k ) , k = 0, ... , N sub - 1 where t 'is the subframe length index L / N sub . We do not present here the generalization to the multichannel case in order to avoid making the notation cumbersome.

[0065] In this case, for a temporal support covering N T frames, the transformed signal is still a two-dimensional matrix, but of size ( N T ·N sub ) × ( N K / N sub ) , with a temporal dimension (on the index t') and a frequency dimension (on the index k ). For simplicity, we can note N T' = N T .N sub And N K' = N K / N T , with a matrix of size N T' × N K' .

[0066] In a second embodiment, the transformation has complex coefficients and it could be, for example, a short-term discrete Fourier transform.

[0067] In this embodiment, the use of the MDCT transform in block 101 of the [ Fig.1a ] can be replaced by a short-time Fourier transform (STFT).

[0068] The STFT is defined as follows: X t k = ∑ n = 0 2 L − 1 x t n w n e − jπnk L where w(n) is, for example, a sinusoidal windowing over 2 L Samples as defined in the MDCT case. In variants, other windowing options are possible.

[0069] Similarly, in variants, other complex transformations will be used, for example an MCLT (for "Modulated Complex Lapped Transform" in English) which combines an MDCT, for the real part and an MDST (for "Modified Discrete Sine Transform" in English), for the imaginary part.

[0070] Block 101 in this case yields complex coefficients. In this implementation variant, the complex coefficients of the transform (STFT, MCLT, etc.) are decomposed by block 102 into real and imaginary parts with: X r t k = Re X t k And X i t k = Im X t k

[0071] Where Re(.) is the real part and Im(.) represents the imaginary part.

[0072] The coding method according to the invention is applied with different possible variations: Either the real and imaginary parts are seen as 2 channels, with a signal X(i,t,k) which can be seen as a stereo signal, where: X 0 t k = X r t k et X 1 t k = X i t k

[0073] In this case, the transformed signal is a three-dimensional matrix with a channel dimension, a time dimension (on the index t< ) and a frequency dimension (on the index k ). Either the real and imaginary parts are combined in a sequence whose medium is doubled by interlacing: X 2 t , k = X r t , k et X 2 t + 1 , k = X i t , k or by concatenation Xt,k=Xrt,ketXt+NT,k=Xit,k

[0074] In this case, the transformed signal is still a two-dimensional matrix with a time dimension (on the index t< ), whose duration is doubled compared to the real case, and a frequency dimension (on the index k ).

[0075] The generalization of the complex case to multichannel is not developed here because it follows the same principles.

[0076] Block 102 decomposes the transformed signal, assumed to be mono and real (without loss of generality), X(t,k) into two parts: amplitudes |X(t,k)|, k=0,..., N K -1 and signs noted here s(t,k),k=0,..., N K -1 defined for example as follows: s t k = 1 si X t , k ≥ 0 − 1 si X t , k < 0

[0077] This operation is generalized for the case where the transformed signal is multidimensional; in this case, the extraction of amplitudes and signs is done separately for each coefficient.

[0078] In the case of complex coefficients, block 102 therefore provides amplitude components for both the real and imaginary parts of the signal X(t,k): X 2 t , k = Xr t , k ,

[0079] And X 2 t + 1 , k = Xi t , k and sign components corresponding to the signs of the real and imaginary parts of the signal X(t,k).

[0080] Thus, the amplitude at the output of block 102 corresponds to the amplitudes of the combined real and imaginary parts, the signs at the output of block 102 correspond to the signs of the combined real and imaginary parts.

[0081] Block 103 implements normalization and / or compression of the amplitudes |X(t,k)|. The goal is to reduce the spectral dynamic range and facilitate processing by an autoencoder. Several implementation examples for the mono case with an MDCT transform are described below.

[0082] In a particular embodiment, the compression performed by this block 103 can be carried out by a logarithmic function such as the law µ defined without loss of generality on an interval [0, 1] as follows: Y t , k = ln 1 + μ X t , k X norm ln 1 + μ , k = 0 , … , N K − 1 where the value of µ is, for example, fixed at µ=255 and the factor of X norm is a maximum value. The output value Y ( t, k ) is here normalized to [0, 1].

[0083] In an example implementation, we take X norm = 2 15< assuming that the input signals are in 16-bit PCM format, and that the transform preserves the maximum input level. In variants, other fixed (constant) values ​​of X norm are possible, in particular with scaling based on the transform used.

[0084] In another example of implementation X norm is given by: X norm = max t = T 0 , … , T 0 + N T − 1 max k = 0 , … , N K − 1 X t , k max t . representing here the maximum value across all frames (or subframes) in a sequence t = T 0 , ... , T 0 + N T - 1 of the signal to be encoded (which causes a coding delay if N T > 1) This embodiment has the disadvantage of an additional delay and requires the transmission of the factor X norm (or its inverse). In an example implementation, the (positive) value of X norm is coded on 7 bits according to a logarithmic scale - the encoding can be according to the ITU-T G.711 standard or simply according to a dictionary of the form 215 i / 127< i =0, ... 127.

[0085] In one variant, X norm can perhaps be calculated based on all elements of the input data as follows: X norm = max Ds max t = T 0 , … , T 0 + N T − 1 max k = 0 , … , N K − 1 X t , k where Ds represents the set of data for training network 120. In this case, this predetermined value from training does not have to be transmitted, but it depends on the training set and can cause saturation if | X ( t, k )| > X norm on a particular test signal.

[0086] In some cases, standardization involves coding the maximum level X norm (the link with the 107 multiplexer) is not shown so as not to complicate the [ Fig.1a ].

[0087] In other embodiment variants, other compression functions than the µ law can be used, for example an A law or a sigmoid function.

[0088] In yet another possible variant, no compression or normalization is used. In this case, module 103 does not exist, and it is assumed that analysis block 104 of autoencoder 120 uses integrated batch normalization or GSD-type layers according to the methods of Ballé et al.

[0089] In some variations, only amplitude compression is applied so that the maximum amplitude remains preserved, by normalizing the signal | X ( t, k )| depending on the maximum value: X norm = max t = T 0 , … , T 0 + N T − 1 max k = 0 , … , N K − 1 X t , k by applying compression, then multiplying the signal by X norm to maintain the maximum value equal to X norm in the current frame(s) of index t = T 0 , ... , T 0 +N T - 1.

[0090] This principle of normalization and / or amplitude compression is directly generalized to the multidimensional case, the maximum value being calculated on all the coefficients taking into account all dimensions, either separately (with a maximum value per channel), or simultaneously (with a global maximum value).

[0091] Block 104 represents the analysis section of an example autoencoder. An example implementation of block 104 is given in relation to the figure 2a described later. Here the network input corresponds to the amplitudes of the transformed and compressed signal | Y (t, k )| ; this signal - here in the mono case - corresponds to a spectrogram and can be seen as a two-dimensional image (of size N T × N K in the preferred embodiment) when several successive frames or subframes are grouped as described above.

[0092] In an example implementation, we consider the case of a group of N T = 200 frames of 240 samples (i.e., one second of signal at 48 kHz), which gives 200 x 200 coefficients if N K 200 MDCT coefficients are stored in the 0-20 kHz band. In variations, the signal can be analyzed over a shorter or longer duration, the extreme case being a single frame. N T = 1 of 20 ms to obtain N K = 800 MDCT coefficients on the 0-20 kHz to 48 kHz band (only the first 800 out of 960 coefficients per frame are kept).

[0093] In some variations, a filter bank will be used. For example, taking the case of a different sampling frequency, 20 subframes can be taken in a 20 ms frame at 32 kHz, which gives 20 x 40 MDCT coefficients over the 0-16 kHz to 32 kHz band for block 104.

[0094] The output of the encoder section 104 is the representation of the signal in a latent space denoted Z( m, p, q ), Or m< is an activation card index, and p, q< are the row (temporal) and column (frequency) indices in each activation map.

[0095] Block 104 is responsible for finding a representation of the signal in a latent space denoted Z( m, p, q< ) , such as : Z m , p , q = f a Y t k θ a ′ where fa is the function applied by the network analysis part and θ a corresponds to the parameters of the neural network. These parameters will be learned during the model training.

[0096] In a particular embodiment where the autoencoder follows the principle of a variational autoencoder during the learning phase, each latent map is assumed to follow a Gaussian distribution such that: P Z m , p , q ∼ ℵ 0 σ m 2 according to Ballé et al. (2017).

[0097] The distribution of values ​​is assumed to be homogeneous, the variance σ m 2 is estimated for each activation map (or "feature map"). In variants, a hyperlatent version according to Ballé et al (2018) is used, where P Z m , p , q ∼ ℵ 0 σ m , p , q 2 , which amounts to applying a Gaussian model to each "pixel" with index p, q< in each clue card m< .

[0098] The latent representation Z(m, p, q ) also called latent space corresponds to the bottleneck of the auto-encoder.

[0099] In the examples given previously, we will have, for example, a latent space of size 128 x 25 x 25 for the case of real input data of size 200 x 200 for the network example given in the [ Fig.2a ].

[0100] Assuming coding (block 105) by scalar quantization and entropy coding, during the training of the autoencoder, the parameters θ a (coding) and θ d (decoding) are optimized according to the following cost function: L λ = R Z ^ m , p , q + λD Y t , k , Y ^ t , k where D is a measure of distortion defined for example by: D Y t , k , Y ^ t , k = E Y t , k ∼ p Y t , k Y t , k − Y ^ t , k 2 Or p Y(t, k ) is the probability distribution of Y(t,k).

[0101] R is the estimated throughput required to transmit the latent space, defined as follows: R Z ^ m , p , q = E Y t , k ∼ p Y t , k − log 2 p Z m Z ^ m , p , q with p Z(m) the probability distribution of Z(m,p,q) (network learning phase). In practice, for the learning phase, the throughput R is evaluated by a summation over m , p, q of the entropy estimated according to the Gaussian probability model and the distortion is evaluated by a summation of the quadratic error on the different input / output data.

[0102] For the network usage phase, the R rate is replaced by the actual rate of an entropic coding (e.g., arithmetic coding).

[0103] The trade-off between reconstruction fidelity and bitrate is configurable by the value λ. A small λ will favor reconstruction quality at the expense of bitrate, a large λ will favor bitrate, but the quality of the output audio signal will be degraded.

[0104] In the case of vector quantization, the neural network is trained to minimize distortion at a given rate.

[0105] This latent space, which represents the amplitude components of the audio signal, is encoded in block 105, for example, by scalar quantization and entropy coding (e.g., arithmetic coding), as in the aforementioned articles by Ballé et al. It should be noted that during training, entropy coding is typically replaced by a theoretical quantization model and an estimation of Shannon entropy, as in the articles by Ballé et al.

[0106] In some variations, the latent space encoding (block 105) is performed by vector quantization at a given bit rate. One example of this implementation involves applying gain-shape vector quantization based on an overall bit budget allocated to latent space quantization, where an overall (scalar) gain and a block-coded shape of 8 coefficients are determined by algebraic vector quantization, as described in the article by S. Ragot et al., "Low-Complexity Multi-Rate Lattice Vector Quantization with Application to Wideband TCX Speech Coding at 32 kbit / s," Proc. ICASSP, Montreal, Canada, May 2004. This method is implemented, for example, in the 3GPP AMR-WB+ and EVS codecs.

[0107] According to the invention, the signs, noted s(t,k) for the case of a real transform of a mono signal, are coded separately, by block 106 according to embodiments described later.

[0108] The latent representation coded in 105 and the signs coded in 106 are multiplexed in the binary stream in block 107.

[0109] We now describe different embodiments of the sign coding (block 106) according to the invention. According to the invention, three main variants are developed for the current frame(s): A. Encoding of all signs. B. Encoding of all signs in low frequencies and selective encoding of signs in high frequencies (with random uncoded signs and / or signs estimated by phase reconstruction / prediction) - phases can be estimated by performing an MDST (Modified Discrete Sine Transform) of the reconstructed signal in previous frames; the uncoded signs are then deduced from the predicted phases as detailed later. C. Encoding of all signs in low frequencies and phase reconstruction / prediction for estimating signs in high frequencies.

[0110] For amplitudes encoded at a given bit rate, these three sign encoding variants allow for different bit rate / quality trade-offs: variant A provides the best quality but at a high bit rate, variant B provides intermediate quality at a lower bit rate, and finally, variant C provides more limited quality but at a reduced bit rate. The cutoff frequency, delimiting the low and high frequencies, is a parameter that allows for finer control of this trade-off; this frequency is denoted N bf The following may be fixed or adaptive.

[0111] In variants, it will be possible to combine variants B and C by defining several frequency sub-bands: a low band (where all bits are coded), an intermediate band (where a selection of bits are coded), a high band (where no bits are transmitted, and the sign bits are estimated by the decoder).

[0112] Note that the signs s ( t, k ) , t = T 0 , ... , T 0 + N T - 1, k = 0, ... , N K - 1 corresponds equivalently to a binary matrix b t , k = 0 si s t , k = 1 1 si s t , k = − 1 size N T × N K , This corresponds, for example, to 40,000 bits per second (i.e., a data rate of 40 kbit / s) in this example. N T = 200, N K = 200 of a signal sampled at 48 kHz and coded in blocks of 200 frames covering 1 second. In variants the complementary convention (which reverses the definition of bits 0 and 1) may be used.

[0113] There figure 1b illustrates a direct embodiment (variant A), where these sign bits b ( t, k ) are simply multiplexed according to a predetermined order in the binary stream, for example by writing the signs b ( t, k frame by frame, t ranging from T 0 to T 0 + N T - 1 and in a given frame according to a predetermined order, for example of k = 0 to k = N K - 1. In some variations, the signs can be written b ( t, k) in any given order that corresponds to a two-dimensional permutation of the size matrix N T × N K .

[0114] Note that this coding can easily be generalized to the multichannel case, since it is sufficient to define the sign bits b ( i, t, k corresponding to X ( i, t, k ) and to multiplex all the bits across all 3 dimensions ( i, t, k ) .

[0115] There figure 1c illustrates another embodiment (variant B) where not all the signs are coded s ( t,k ) in order to reduce the data rate required for sign encoding. In this example implementation, all low-frequency signs are encoded. k = 0, ... , N bf - 1 and a subset of N pk signs of high frequencies k = N bf , ... , N K - 1, where N bf can be set to a fixed value (for example N bf = 80 for N K = 200 in the previous example) or adaptive (depending on the signal), and N pk is also fixed at a predetermined value ( N pk = 2 to the figure 1c ). Depending on the embodiment, variant B can encode, in addition to low-frequency signs and selected high-frequency bits, metadata on a budget of B hf bits.

[0116] In some variations, a sub-band with more than two frequency bands (in addition to the low and high bands) can be used to more precisely allocate the number of sign bits encoded per sub-band. Preferably, the signs of the first frequency band will all be encoded because it is important to preserve sign information for low frequencies.

[0117] Note that this coding can easily be generalized to the multichannel case, since it is sufficient to define the sign bits b ( i, t, k corresponding to X ( i, t, k and repeat the encoding and multiplexing of the signs for each index channel i .

[0118] Different methods of selecting and / or encoding (indexing) the subset of signs are possible, first considering the simple case of 2 sub-bands and a single sub-band in high frequencies: Variant B1: In a variant (variant B1a), a search for N pk the most important peaks among N hf high-frequency coefficients ( N hf = N K -N bf ) is performed on the original amplitude spectrum. The search N pk The most important peaks can be identified simply in 2 steps, first by searching for the index lines k= N bf + 1, ... , N K - 2 that check peaks on the lines that check |X ( t, k )| > | X ( t, k- 1) |and | X ( t, k )| > | X ( t,k+ 1)|, then to order the indices k obtained to retain the positions l 0 (t),..., l N pk- 1 (t) corresponding to N pk values ​​| X ( t, k ) | the largest. In variations, other detection methods N pk peak amplitude methods may be used, for example the method described in clause 5.4.2.4.2 of the 3GPP TS 26.447 standard.

[0119] The positions l 0 ( t ) , ... , l N pk -1 ( t ) of the N pk signs among N hf High-frequency coefficients are encoded using combinational coding techniques. For example, when N pk = 2 and N hf = 200-80=120, we will have 7140 possible combinations, or B hf = 13 bits per frame (i.e., 2.6 kbit / s for 200 frames per second). The sign encoding rate is then: (80+2+13)x200 = 19 kbit / s. In variations, the high band can be divided into separate sub-bands and the sub-band method applied, or into series of interleaved positions ("tracks") and the "track" method applied (tracks are defined here as a decomposition of positions in polyphase form similar to pulse encoding in the ACELP method of the ITU-T G.729 standard). In another variant (variant B1b), a block error-correcting code is used to jointly encode the position and values ​​of the N pk signs. In this case, we use the spectrum X ( t, k ) signed and the binary error-correcting code [Nc, Kc, Dc], where Nc is the length (in bits), Kc the number of check bits, and Dc the Hamming distance, is converted to values ​​+1 and -1 instead of 1 and 0 (respectively). The sign positions and associated sign values ​​are jointly encoded. For a given frame of index t, the encoding in block 106 is then performed for sub-blocks of frequency lines (successive or interleaved) of length Kc, by dot product between X ( t,k ) and the different codewords (with values ​​of +1 / -1), and retaining the codeword that maximizes the dot product. The principle of this error-correcting coding is detailed, for example, in the document S. Ragot, "L'hexacode, le code de Golay code et le réseau de Leech: définition, construction, application en quantification" (The Hexacode, the Golay Code and the Leech Network: Definition, Construction, Application in Quantization), Master's Thesis, Department of Electrical and Computer Engineering, University of Sherbrooke, QC, Canada, Dec. 1999

[0120] In an example implementation, we can take an extended Hamming code of the type [2m< , 2m< - m - 1, 4] whose values ​​are + / -1 and not 0 / 1, which means that the signs (and their positions) of 2 m< lines are represented on 2 m< - m - 1 bit. For example, taking an extended Hamming code [8, 4, 4], we divide the N hf = 200-80=120 bits of high-frequency signs in 15 blocks of 8 bits, and we obtain by decoding (taking the signed spectrum as the value of "soft bit") a total of 15 blocks of 4 control bits, i.e. 60 bits per frame (or 12 kbit / s for 200 frames per second) to encode the signs (and their positions).

[0121] The encoding rate for the sign bits is therefore (80+60)x200 = 28 kbit / s. In some variations, other block error-correcting codes will be used. In other variations, error-correcting codes can be interleaved to facilitate the distribution of the encoded sign bits. In other sub-variants (variant B1c), sub-bands can also be classified as tonal or noise bands according to, for example, a known "spectral flatness" criterion, and then symbols will only be coded in the tonal bands. This "spectral flatness" criterion is estimated on the original amplitudes, and a pitch indication must be provided for each sub-band in addition to the positions. Variant B2: the search for N pk The most important peaks are identified on the coded amplitude spectrum, so the peak positions do not need to be transmitted, as the same information (coded amplitude spectrum) can be available to the decoder. However, this assumes that block 106 has access to the output (coded latent space) of block 105 and that the synthesis part of the autoencoder (block 113) is applied to perform local decoding.

[0122] In other sub-variants, sub-bands can also be classified as tonal or noise bands according to, for example, a "spectral flatness" criterion known from the state of the art, and then signs will only be encoded in the tonal bands. This "spectral flatness" criterion is estimated on the amplitudes decoded locally (i.e., not transmitted).

[0123] In variants, the selection of the position of the signs may be based on an estimation of a frequency masking curve to detect the most perceptually important peaks, for example as a function of a signal-to-mask ratio according to methods known in the state of the art.

[0124] There [ Fig.1d This illustrates another embodiment (variant C) where all the signs in the low frequencies are encoded (multiplexed) in block 120 and the high-frequency bits are not transmitted to the encoder. This missing data is estimated at the decoder by phase reconstruction / prediction for estimating the signs in the high frequencies.

[0125] Thus, in this variant, all the low-frequency symbols are coded. k = 0, ... , N bf - 1 and no sign of high frequencies k = N bf ... . N K - 1, where N bf can be set to a fixed or adaptive value.

[0126] For the previous example, where N T = 200, N K = For a signal sampled at 48 kHz and encoded in blocks of 200 frames covering 1 second, the required bit rate for the signs is, for example, 16,000 bits per second (i.e., 16 kbit / s) when N bf = 80 (i.e., a cutoff frequency of 8 kHz).

[0127] The above methods can be generalized to the case of several sub-bands, and also to the cases of a filter bank, a complex transform separated into real or imaginary parts, or the multichannel case.

[0128] In other variants, the different implementations of sign coding can be adapted in the case where the coefficients are divided into frequency sub-bands and the sign coding is carried out separately for each sub-band.

[0129] There [ Fig.1a ] also represents the 110 decoder now described.

[0130] Block 111 demultiplexes the binary stream to find on the one hand the coded representations of the latent space Z(m) and on the other hand, the signs s(k).

[0131] The latent space is decoded at 112. The synthesis section (block 113) of the autoencoder 120 reconstructs the spectrum Ŷ ( t , k )from the latent space decoded in the form: Y ^ t , k = g s Z ^ m , p , q ; θ d

[0132] Block 114 allows for amplitude decompression and amplitude denormalization (if block 103 has been implemented). In this case, an inverse logarithmic function such as the inverse µ law, defined by: X ^ t , k = 1 μ 1 + μ Y ^ t , k − 1

[0133] When a variant of block 103 is implemented, block 114 is adapted accordingly. In some cases, normalization involves decoding a maximum level (the link with demultiplexer 111) is not shown so as not to increase the complexity of the [ Fig.1a ].

[0134] The signal signs are decoded in block 115 as follows: In the case where all the sign bits b ( t, k ) , t = T 0 , ... , T 0 + N T - 1, k = 0, ... , N K - The bits that were multiplexed one by one in the binary stream are demultiplexed. b̂ ( t,k ) according to the writing order of block 106. When the binary stream has not suffered any binary errors, we will have b̂ ( t, k )= b ( t, k ).

[0135] As with coding, we distinguish 3 variants of decoding sign information (Variants A, B, C).

[0136] In variant A, sign decoding is reduced to demultiplexing the sign bits according to the order used in encoding and converting the value of the sign bit using, for example: s ^ t k = 1 si b ^ t k = 0 − 1 si b ^ t k = 1

[0137] In variant B, the decoding of signs is performed as in variant A for the sign bits of low frequencies. A portion of the signs at high frequencies is encoded; the demultiplexing N pk bits per frame and we decode the positions l 0 ( t ) , ..., l N pk- 1 ( t ) to find the corresponding positions. The decoding of the positions is carried out according to the coding method used, either by combinational decoding methods or by error-correcting codes.

[0138] In variants (B1a), the positions l 0, ..., l N pk- 1 are determined from the decoded amplitudes | X̂ ( t, k ) | , possibly with the estimation of a masking curve.

[0139] Therefore, we have: s ^ t k = 1 si b ^ t k = 0 − 1 si b ^ t k = 1 k = l 0 , … , l N pk − 1 .

[0140] For the remaining high-frequency signs, in one variant, all signs are given a random value, in another variant, all signs are given the same value, i.e. s ^ t k = − 1 random For k ∈ { N bf , ... , N K - 1} \ { l 0, ..., l Nsig -1} , where random() is a state-of-the-art binary random number generator.

[0141] In other variants (B1b), the signs and their positions are jointly decoded to directly obtain ŝ ( t, k), k = N bf , ..., N K - 1. For example, taking an extended Hamming code [8, 4, 4], we divide the N hf = 200-80=120 bits of high-frequency signs in 15 blocks of 8 bits. A 4-bit index is demultiplexed 8 times, and the codeword (from among 16 possibilities) with values ​​of + / -1 is obtained through "correction coding," directly giving the sequence ŝ ( t, k ) on 8 frequency lines (consecutive or interlaced).

[0142] In variant C, illustrated in the [ Fig.1d Sign decoding is performed as in variant A for the low-frequency sign bits (block 130). The high-frequency signs are missing information and are estimated, for example, by methods described in the 3GPP TS 26.447 standard, clause 5.4.2.4.3 (tonal prediction). It should be noted here that, unlike frame loss correction, amplitude information is available for high frequencies. Only the sign information is missing and is therefore estimated. One implementation example is to adapt the MDCT frame loss correction methods described in clause 5.4.2.4.3 of the 3GPP TS 26.447 standard. The phases can be estimated by performing an MDST (Modified Discrete Sine Transform) of the reconstructed signal in the preceding frames; the uncoded signs are then deduced from the predicted phases.In particular the signs can be determined at the decoder by retaining the sign of the result of equation 146 of the 3GPP TS 26.447 standard.

[0143] Block 116 allows the combination of decoded signs and amplitudes for the reconstruction of the initial frames according to the following formula: X ^ t k = X ^ t k . s ^ t k

[0144] Block 117 applies the inverse MDCT to obtain the decoded signal. x̂ ( n ) . When the N K of MDCT coefficients used is such that N K < L, block 117 will add L-N K coefficients set to zero at the end of the spectrum of each frame, in order to recover a spectrum of L of coefficients.

[0145] Each operation of the inverse MDCT operates on L coefficients to produce L audio samples in the time domain. The inverse MDCT can be decomposed into a DCT-IV followed by windowing, unfolding, and addition operations. The DCT-IV is given by: u n = ∑ k = 0 2 L − 1 2 L cos π L k + 0.5 n + 0.5 X ^ t k pour 0 ≤ n ≤ L − 1

[0146] The windowing, unfolding, and addition operations use half the samples from the DCT-IV output of the current frame with half those from the DCT-IV output of the previous frame, according to: x ^ t n = w n u L 2 − 1 − n + w L − 1 − n u old n pour 0 ≤ n ≤ L 2 − 1 x ^ t , n + L 2 = w L 2 + n u n − w L 2 − 1 − n u old L 2 − 1 − n pour 0 ≤ n ≤ L 2 − 1 Or w n = sin π 2 L n + 0.5 pour 0 ≤ n ≤ L − 1

[0147] The unused half of u() is stored as uold() to be used in the next frame: u old n = u n + L 2 pour 0 ≤ n ≤ L 2 − 1

[0148] There [ Fig.2a ] illustrates the elements of the auto-encoder 120, including the elements of the analysis part 104 and synthesis part 113.

[0149] The analysis portion of block 104 in this example consists of four convolutional layers (blocks 200, 202, 204, and 206). Each layer comprises a 2D convolution with K x K dimension filters (e.g., 5x5), followed by a 2x decimation of the activation map size. The activation map size becomes progressively smaller as the analysis progresses. However, the input and output dimensions of the layers are generally different.

[0150] There [ Fig.2b [ ] shows an example of the application of layers in blocks 200, 202, 204, and 206 of the analysis section. The first layer, represented by block 200, receives a mono signal (1,200,200). The output of this layer is an activation map of size (128, 200, 200), where N=128 is the number of activation maps considered in this layer. The following block, 202, receives a multichannel signal of size (128, 200, 200), and by decimating the size of the activation map by 2, the output is an activation map of size (128, 100, 100). The same process is applied to layer 204, where the output is an activation layer of size (128,50,50). Finally, the last block 206 gives a signal of size (128, 25, 25).

[0151] Following each of the first 3 2D convolution layers, a "Leaky ReLU" activation function is used in blocks 201, 203, and 205. The "Leaky ReLU" function is defined as follows: LeakyRelu x = x , si x ≥ 0 ax , sinon with a< a constant with a negative slope having, for example, the value a = 0.01.

[0152] For the last layer, there is no activation function so as not to limit the values ​​that y can take at the output of the layer.

[0153] In variants the ReLU function can be replaced by other functions known in the state of the art, for example an ELU (Exponential Linear Unit) function.

[0154] The synthesis section (block 113) has a mirrored architecture compared to the analysis section. It consists of four successive layers of 2D transposed convolution (blocks 216, 214, 212, and 210). The addition of transposed convolution allows for richer nonlinear interpolation than simple linear weighting of values. In the synthesis section, the last layer, block 216, has N inputs, corresponding to the number of signal channels. Y ( k) at the output, the other layers have N inputs and outputs. As with the analysis part, after each of the first 3 layers, a "Leaky ReLU" activation function is used in blocks 215, 213 and 211.

[0155] There [ Fig.2c [ ] shows an example of the application of the layers in blocks 210, 212, 214, and 216 of the synthesis section. Block 210 receives a multichannel signal of size (128, 25, 25) and produces an activation map of size (128, 50, 50). Similarly, blocks 212 and 214 produce layers of size (128, 100, 100) and (128, 200, 200), respectively. Finally, layer 216 receives a signal of size (128, 200, 200) and produces a signal of the same size as the original mono signal (1, 200, 200).

[0156] The number of activation maps, N, determines the model's degrees of freedom in representing the input signals. For training performed with only a distortion constraint and no rate constraint (λ = 0), the higher the value of N, the better the model's reconstruction quality. For a given N, training with only a distortion constraint (λ = 0) provides an estimate of the maximum reconstruction quality the model can achieve with N activation maps. With the introduction of the rate constraint (λ > 0), the reconstruction quality will necessarily be lower than this maximum quality.

[0157] There [ Fig.3a [This section] now illustrates a second embodiment of an encoder 300 and a decoder 310 according to the invention, as well as the steps of an encoding method and a decoding method according to a second embodiment of the invention. While the first embodiment decomposes the audio signal into amplitudes and signs, the second embodiment decomposes the audio signal into amplitudes and phases. The sign encoding principles described for the first embodiment are extended to the case of phases. The main difference lies in the fact that instead of having 1 bit to represent a sign (or a sign bit), there will generally be several bits per phase (for example, 7 bits at low frequencies and 5 bits at high frequencies). When the phase is encoded with 1 bit, we will return to a case similar to the first embodiment where the sign is encoded.

[0158] In this figure, the transform block 101 remains the same as that described with reference to the [ Fig.1a ], but with a complex transform (STFT or MCLT for example).

[0159] Block 302 differs from block 102 of the figure 1a . This block 302 decomposes the transformed signal X(t,k) into two parts: amplitudes |X(t,k)|, t = T 0 , ... , T 0 + N T - 1 , k=0,..., N K - 1 and phases noted here ϕ (( t , k ) =arg X(t,k) , t = T 0 , ... , T 0 + N T - 1, k=0,..., N K - 1 where arg(.) is the complex argument.

[0160] For amplitude coding, blocks 103 to 105 described with reference to the [ Fig.1a remain unchanged.

[0161] In this embodiment, block 306 separately encodes the phases thus obtained from the input signal. These encoded phases are then multiplexed in the binary stream at 307, with the latent representation encoded at 105. The main difference compared to the embodiment of the figure 1a This lies in the fact that the phase information is not encoded on 1 bit but on a larger budget, for example 7 bits per phase, for a uniform scalar quantization dictionary on [0, 2π], with a step of π / 64. In variants, the budget to encode a phase may depend on the frequency band, with for example 7 bits per phase at low frequencies and 5 bits per phase at high frequencies.

[0162] As in the first embodiment, we can define 3 variants: A. Coding of all phases ϕ t , k , t = T 0 , … , T 0 + N T − 1 , k = 0 , … , N K − 1 B. Encoding of all phases in the low frequencies and selective coding of phases in the high frequencies (with random uncoded phases and / or phases estimated by phase reconstruction / prediction). In this case, as with sign coding in the first embodiment, it will be possible to select positions of the N pk "Peaks" are important, and encode / multiplex the phases at these positions; N pk Peak positions are coded as in the first embodiment (variants B1a or B1c). C. Encoding of all phases in the low frequencies and phase reconstruction / prediction for phase estimation in the high frequencies. In this case, phases are only coded for the low frequencies, for k= 0 , ...,N bf - 1 .

[0163] During decoding, block 311 demultiplexes the binary stream to find, on the one hand, the coded representations of the latent space Z(m,p,q) representing the amplitude part |X(t,k)| and, on the other hand, the coded version of the phases ϕ ( t,k ) .

[0164] Blocks 112 to 114 remain unchanged from those described with reference to the [ Fig.1a ].

[0165] Block 315 decodes the phases according to the A and B variants used in the encoding, in order to combine them in 316 with the decoded amplitudes. Variant C is considered at the [ Fig.3b ].

[0166] Inverse transform block 117 remains unchanged from block 117 of the [ Fig.1a ].

[0167] There [ Fig.3b ] now illustrates another embodiment of an encoder 400 and a decoder 410 according to the invention as well as the steps of an encoding process and a decoding process according to an embodiment of the invention.

[0168] In this embodiment, block 401 uses a short-term Fourier transform (STFT). Block 402 performs a decomposition of the transformed signal X(t,k) into two parts: amplitudes |X(t,k)|, k=0,..., N K - 1 and phases denoted here Φ(t,k),k=0,..., N K -1.

[0169] In this embodiment, only a part of the phases, for example, only the part corresponding to the low frequencies of the transformed signal (Φ 1 ), is coded by block 406.

[0170] In one variant, part of the phase components of the high frequencies can also be encoded.

[0171] In an example implementation, with an STFT where L = 240 samples, low frequency refers to frequency lines with indices from 0 to N bf - 1 = 79 which corresponds to approximately a frequency band of 8 kHz.

[0172] For amplitude coding, blocks 103 to 105 described with reference to the [ Fig.1a remain unchanged.

[0173] The latent representation coded in 105 is then multiplexed in the binary stream in 407 with the part coded in 406 of the phases of the transformed signal.

[0174] During decoding, block 411 demultiplexes the binary stream to find the coded representations of the latent space Z(m,p,q) and part of the signal phases.

[0175] This phase part for low frequencies is decoded in 415 ( Φ ^ 1 ).

[0176] Blocks 112 to 114 remain unchanged from those described with reference to the [ Fig.1a ].

[0177] The other part of the phases ( Φ ^ 2 ) for high frequencies is reconstructed by block 418. For this, after the inverse compression of block 114, an algorithm for reconstructing the uncoded phases of the STFT is used in this block 418. This algorithm allows the inversion of the amplitude spectrogram using an algorithm as described in the paper DW Griffin and JS Lim, "Signal estimation from modified short-time Fourier transform," IEEE Trans. ASSP, vol. 32, no. 2, pp. 236–243, Apr. 1984. Given an amplitude matrix for a short-time Fourier transform | Ŷ ( t,k ) | , the algorithm randomly initializes the phases ( Φ ^ 2 corresponding to ϕ ( t, k ) , k = N bf , ... , N K - 1, then alternates between forward and inverse STFT operations. Preferably, this high-frequency phase estimation can be implemented by processing at a sampling frequency lower than the sampling frequency fs of the input / output signal.

[0178] Block 416 combines the decoded amplitudes and the decoded phases ( Φ ^ 1 ) and rebuilt ( Φ ^ 2 ), then an application of the inverse STFT is performed in block 417 for the reconstruction of the original signal.

[0179] We illustrated on the [ Fig.4 ], a DCOD encoding device and a DDEC decoding device, within the meaning of the invention, these devices being dual to each other (in the sense of "reversible") and connected to each other by a communication network RES.

[0180] The DCOD coding device includes a processing circuit typically comprising: a memory MEM1 for storing instruction data of a computer program as defined in the invention (these instructions being able to be distributed between the DCOD encoder and the DDEC decoder); an interface INT1 for receiving an audio signal of mono or multichannel origin x; a processor PROC1 for receiving this signal and processing it by executing the computer program instructions stored in the memory MEM1, for the purpose of encoding it; in particular, the processor being able to drive an analysis module of an auto-encoder based on a neural network; and a communication interface COM 1 for transmitting the encoded signals via the network.

[0181] The DDEC decoding device includes its own processing circuit, typically including: a MEM2 memory for storing instruction data for a computer program as defined in the invention (these instructions being distributed between the DCOD encoder and the DDEC decoder as previously described); a COM2 interface for receiving coded signals from the RES network for decoding by compression as defined in the invention; a PROC2 processor for processing these signals by executing the computer program instructions stored in the MEM2 memory, for decoding purposes; in particular, the processor being capable of driving a synthesis module for a neural network-based autoencoder; and an INT2 output interface for delivering the decoded audio signal. x̂ .

[0182] Of course, this [ Fig.4 ] illustrates an example of a structural implementation of a codec (coder or decoder) as defined in the invention. figures 1 à 3 The examples discussed above describe in detail the functional implementations of these codecs.

Claims

1. Method for coding an audio signal, comprising the following steps: - decomposing (102) the audio signal into at least amplitude components and sign or phase components; - analyzing (104) the amplitude components by way of a neural network-based autoencoder so as to obtain a latent space representative of the amplitude components of the audio signal; - coding (105) the obtained latent space; - coding (106) at least a portion of the sign or phase components.

2. Method according to Claim 1, furthermore comprising a step of compressing the amplitude components before they are analyzed by the autoencoder.

3. Method according to Claim 2, wherein the amplitude components are compressed by a logarithmic function.

4. Method according to Claim 1, wherein the audio signal before the decomposition step is obtained by an MDCT transform applied to an input audio signal.

5. Method according to one of the preceding claims, wherein the audio signal is a multichannel signal.

6. Method according to Claim 1, wherein the audio signal is a complex signal comprising a real and an imaginary part resulting from a transformation of an input audio signal, the amplitude components resulting from the decomposition step corresponding to the amplitudes of the combined real and imaginary parts and the sign or phase components corresponding to the signs or phases of the combined real and imaginary parts.

7. Method according to Claim 1, wherein all of the sign or phase components of the audio signal are coded.

8. Method according to Claim 1, wherein only the sign or phase components corresponding to the low frequencies of the audio signal are coded.

9. Method according to Claim 1, wherein the sign or phase components corresponding to the low frequencies of the audio signal are coded and selective coding is carried out for the sign or phase components corresponding to the high frequencies of the audio signal.

10. Method according to Claim 9, wherein the positions of the sign or phase components selected for the selective coding are also coded.

11. Method according to Claim 9, wherein the positions of the selected sign or phase components and the associated values are coded together.

12. Method for decoding an audio signal, comprising the following steps: - decoding (112) sign or phase components of the audio signal; - decoding (115) a latent space representative of amplitude components of the audio signal; - synthesizing (113) the amplitude components of the audio signal by way of a neural network-based autoencoder, from the decoded latent space; - combining (116) the decoded amplitude components and the decoded sign or phase components so as to obtain a decoded audio signal.

13. Decoding method according to Claim 12, wherein, if the decoded phase components correspond to one portion of the phase components of the audio signal, the other portion is reconstructed before the combining step.

14. Coding device comprising a processing circuit for implementing the steps of the coding method according to one of Claims 1 to 11.

15. Decoding device comprising a processing circuit for implementing the steps of the decoding method according to one of Claims 12 to 13.

16. Storage medium able to be read by a processor and storing a computer program comprising instructions for executing the coding method according to one of Claims 1 to 11 or the decoding method according to one of Claims 12 to 13.

Images