Spatialized audio coding with interpolation and quantification of rotations

Abstract

The invention relates to the compression coding of sound signals of N channels forming a succession in time of frames (t-1, t) of samples, the method comprising: - forming, from the channels for a current frame (t), a covariance matrix between channels and searching for eigenvectors of the covariance matrix to obtain an eigenvector matrix, - comparing the eigenvector matrix obtained for the current frame (t) to a rotation matrix obtained for a frame (t-1) preceding the current frame (t), and - permuting columns of the eigenvector matrix of the current frame (t) to ensure consistency with the rotation matrix of the previous frame (t-1).- test the eigenvector matrix to verify that it represents a rotation in an N-dimensional space and otherwise correct the eigenvector matrix until a rotation matrix is ​​obtained, for the current frame (t), and - apply said rotation matrix to the signals of the N channels before separate channel encoding of said signals, the process further comprising: - cutting the current frame into subframes, and - interpolating matrices to be applied to successive subframes from the matrix of the previous frame (t-1) to the matrix of the current frame (t).

Description

[0001] The present invention relates to the encoding / decoding of spatialized sound data, particularly in an ambisonic context (hereafter also referred to as "ambisonic").

[0002] The encoders / decoders (hereinafter referred to as "encoders") currently used in mobile telephony are mono (a single signal channel for playback on a single speaker). The 3GPP EVS (for "Enhanced Voice Services") encoder offers "Super-HD" quality (also called "High Definition +" or HD+ voice) with a super-wideband (SWB) audio bandwidth for signals sampled at 32 or 48 kHz, or fullband (FB) for signals sampled at 48 kHz; the audio bandwidth is 14.4 to 16 kHz in SWB mode (9.6 to 128 kbit / s) and 20 kHz in FB mode (16.4 to 128 kbit / s).

[0003] The next evolution in quality for conversational services offered by operators should be immersive services, using devices such as smartphones equipped with multiple microphones, spatial audio conferencing equipment, telepresence-type video conferencing, or even live content sharing tools, with 3D spatialized sound rendering, far more immersive than simple 2D stereo playback. With the increasingly widespread use of mobile phones for listening with headphones and the emergence of advanced audio equipment (accessories such as 3D microphones, voice assistants with acoustic antennas, virtual reality headsets, etc.) and specific tools (for example, 360° video content production), the capture and rendering of spatialized sound scenes are now common enough to offer an immersive communication experience.

[0004] As such, the future 3GPP standard "IVAS" (for "Immersive Voice And Audio Services") proposes the extension of EVS encoded audio to immersive audio by accepting as encoded input format at least the spatialized sound formats listed below (and their combinations): Multichannel format (channel-based in English) of stereo type, 5.1 where each channel feeds a speaker (for example L and R in stereo, or L, R, Ls, Rs and C in 5.1) Object-based format where sound objects are described as an audio signal (generally mono) associated with metadata describing the attributes of this object (position in space, spatial width of the source, etc.), and Ambisonic format (scene-based in English) which describes the sound field at a given point, generally captured by a spherical microphone or synthesized in the domain of spherical harmonics.

[0005] We are typically interested below in the coding of a sound in ambisonic format, as an example of an embodiment (at least some aspects presented in connection with the invention below can also be applied to formats other than ambisonic).

[0006] Ambisonics is a method for recording (acoustically "encoding") spatialized sound and a system for reproducing it (acoustically "decoding"). A first-order ambisonic microphone comprises at least four capsules (typically cardioid or subcardioid) arranged on a spherical grid, such as the vertices of a regular tetrahedron. The audio channels associated with these capsules are called the "A-format." This format is converted into a "B-format," in which the sound field is decomposed into four components (spherical harmonics) denoted W, X, Y, and Z, which correspond to four coincident virtual microphones. The W component corresponds to an omnidirectional capture of the sound field, while the more directional X, Y, and Z components can be likened to pressure gradients oriented along the three dimensions of space.An ambisonic system is flexible in that recording and playback are separate and decoupled. It allows for decoding (in the acoustic sense) on any loudspeaker configuration (for example, binaural, 5.1 surround sound, or 7.1.4 periphery (with elevation). Naturally, the ambisonic approach can be generalized to more than four channels in B-format, and this generalized representation is commonly called "HOA" (for "Higher-Order Ambisonics"). Decomposing the sound into more spherical harmonics improves the spatial accuracy of playback when rendered on loudspeakers.

[0007] An Nth-order ambisonic signal comprises (N+1) 2< components, and at the first order (if N=1), we find the four components of the original ambisonic, which is commonly called FOA (for First-Order Ambisonics). There is also a so-called "planar" variant of ambisonics, which decomposes the sound defined in a plane, generally the horizontal plane. In this case, the number of components is 2N+1 channels. First-order ambisonics (4 channels: W, X, Y, Z) and planar first-order ambisonics (3 channels: W, X, Y) are referred to hereafter as "ambisonics" interchangeably for ease of reading, as the processing described is applicable regardless of whether it is planar or not. However, if a distinction is necessary in certain passages, the terms "first-order ambisonics" and "planar first-order ambisonics" are used.It will be noted that one can derive from the B-format to the 1st order a stereo signal (2 channels) corresponding to coincident stereo recordings of the Blumlein Crossed Pair type (X+Y and XY) or Mid-Side (by combining W and X for Mid and taking Y as Side).

[0008] Subsequently, a B-format signal with a predetermined order is called an "ambisonic sound". In variations, ambisonic sound can be defined in another format such as A-format or pre-combined channels by fixed matrixing (maintaining the number of channels or reducing it to a case of 3 or 2 channels), as will be seen later.

[0009] The signals to be processed by the encoder / decoder are presented as successions of blocks of sound samples called "frames" or "subframes" below.

[0010] Furthermore, the mathematical notations below follow the following convention: Vector: u (minuscule, gras) Matrix: A (majuscule, gras)

[0011] The simplest approach to encoding a stereo or ambisonic signal is to use a mono encoder and apply it in parallel to all channels, possibly with a different bit allocation for each channel. This approach is called "multi-mono" here (although in practice the approach can be generalized to multi-stereo or the use of several parallel instances of the same core codec).

[0012] Such a creation is presented to the figure 1 The input signal is divided into (mono) channels by block 100. These channels are individually encoded by blocks 120 to 122 according to a predetermined allocation. Their binary stream is multiplexed (block 130) and after transmission and / or storage it is demultiplexed (block 140) to apply a decoding to each of the channels (blocks 150 to 152) which are then recombined (block 160).

[0013] The associated quality varies depending on the mono coding used, and is generally only satisfactory at very high bitrates, for example with a bitrate of at least 48 kbit / s per mono channel for EVS coding. Thus, at first order, a minimum bitrate of 4 x 48 = 192 kbit / s is obtained.

[0014] The solutions currently proposed for more sophisticated coded systems, particularly for ambisonic spatialization, are not satisfactory, especially in terms of complexity, delay and efficient use of bandwidth, to ensure effective decorrelation between ambisonic channels.

[0015] For example, the MPEG-H encoding for ambisonic sounds uses an add-over operation that adds delay and complexity, as well as linear interpolation on direction vectors, which is suboptimal and introduces flaws. A fundamental problem with this encoding is that it implements a decomposition into predominant and ambient components because the predominant components are supposed to be perceptually distinct from the ambient sound, but this decomposition is not fully specified. The MPEG-H encoder suffers from a mismatch problem between the directions of the main components from one frame to the next: the order of the components (signals) can be swapped, as can their associated directions. This is why the MPEG-H encoding uses a matching and overlap-add technique to solve this problem.

[0016] Furthermore, it would be possible to use frequency coding approaches (in the FFT or MDCT domain) rather than time coding as in MPEG-H coded, but signal processing in the frequency domain (sub-bands) requires transmitting data to a decoder by sub-band, thus increasing the data rate required for this transmission.

[0017] The present invention improves this situation.

[0018] To this end, it proposes a compression coding method for N-channel audio signals forming a temporal succession of (t-1, t) sample frames, the method comprising: Form, from the channels for a current frame (t), a covariance matrix between channels and search for eigenvectors in the covariance matrix to obtain an eigenvector matrix; compare the eigenvector matrix obtained for the current frame (t) to a rotation matrix obtained for a frame (t-1) preceding the current frame (t); and permute columns of the eigenvector matrix of the current frame (t) to ensure consistency with the rotation matrix of the preceding frame (t-1). Test the eigenvector matrix to verify that it represents a rotation in an N-dimensional space and, if not, correct the eigenvector matrix until a rotation matrix is ​​obtained for the current frame (t); and apply said rotation matrix to the signals of the N channels before separate-channel encoding of said signals. the process also including: divide the current frame into subframes, and interpolate matrices to be applied to successive subframes from the matrix of the previous frame (t-1) to the matrix of the current frame (t).

[0019] Thus, the present invention makes it possible to improve the decorrelation between the N channels to be encoded. Such an implementation makes it possible to maintain overall homogeneity and, in particular, to avoid audible clicks from one frame to another during sound playback.

[0020] Subframe interpolation also allows for smoothing out the difference between the matrix of the previous frame and the matrix of the current frame over time.

[0021] In one embodiment, the process may further include: encoding parameters derived from the rotation matrix for transmission via a network. These parameters can typically be quaternion values ​​and / or rotation angle and / or Euler angle as we will see later, or simply elements of this matrix for example.

[0022] However, some transformations used to obtain eigenvectors from the covariance matrix (such as the "PCA / KLT" described later) may reverse the direction of some eigenvectors. It is therefore necessary to verify both the axis consistency and the direction consistency along that axis for each eigenvector in the current frame matrix. To this end, in one embodiment, the aforementioned column permutation already ensures axis consistency for the vectors. The process further includes: check, for each eigenvector of the current frame, a consistency of direction with a corresponding position column vector of the rotation matrix of the previous frame, and in case of inconsistency, reverse the sign of the elements of this eigenvector in the eigenvector matrix of the current frame.

[0023] Typically, a permutation between columns of the eigenvector matrix reversing the sign of a determinant of the eigenvector matrix and the determinant of a rotation matrix being equal to 1, we can estimate the determinant of the eigenvector matrix, and if the latter is equal to -1, we can then reverse the signs of the elements of a chosen column of the eigenvector matrix, so that the determinant is equal to 1 and thus form a rotation matrix.

[0024] In a given implementation, the process may also include: an estimate of the difference between the rotation matrix obtained for the current frame and a rotation matrix obtained for a frame preceding the current frame, as a function of the estimated difference, determine if at least one interpolation is to be performed between the rotation matrix of the current frame and the rotation matrix of the previous frame.

[0025] Such interpolation then makes it possible to smooth out ("gradually average") the rotation matrices applied respectively to the previous frame and the current frame, thus mitigating an audible click effect from one frame to the other during playback.

[0026] In such a project: Based on the estimated difference, a number of interpolations to be performed between the rotation matrix of the current frame and the rotation matrix of the previous frame is determined; the current frame is divided into a number of subframes corresponding to the number of interpolations to be performed, and at least this number of interpolations can be coded for transmission via the aforementioned network.

[0027] In one embodiment, the ambisonic representation is of order 1 and the number N of channels is four, and the rotation matrix of the current frame is represented by two quaternions.

[0028] In one embodiment, the search for eigenvectors is carried out by principal component analysis (or "PCA") or by Karhunen Loeve transform (or "KLT"), in the time domain.

[0029] Of course, other implementations can be considered (singular value decomposition, or others).

[0030] In one embodiment, the process includes a preliminary step of forecasting the bit allocation budget per ambisonic channel, comprising: for each ambisonic channel, an estimate of the current acoustic energy in the channel, the selection in a memory of a predetermined score, of quality, a function of this ambisonic channel and a current flow rate in the network, the estimation of a weighting to be operated for the allocation of bits to this channel, by multiplying the selected score by the estimated energy.

[0031] This implementation allows for optimal bit allocation for each channel to be encoded. It is advantageous in itself and could potentially be protected separately.

[0032] The present invention also relates to a method for decoding sound signals forming a succession in time of frames (t-1, t) of samples, in each of N channels, the method comprising: receive, for a current frame (t), in addition to the signals from the N channels of this current frame, parameters of a rotation matrix, construct an inverse rotation matrix for the current frame, from said parameters, divide the current frame into subframes, and interpolate matrices to be applied to successive subframes from the matrix of the previous frame ( t -1) up to the matrix of the current frame ( t ) apply the inverse interpolated rotation matrices, by subframe, to signals from the N received channels.

[0033] Such a design also improves decoding by creating a decorrelation between the N channels and smoothing out over time the difference between the matrix of the previous frame and the matrix of the current frame.

[0034] The present invention also relates to a coding device comprising a processing circuit for implementing the coding process described above.

[0035] It also aims at a decoding device comprising a processing circuit for implementing the above decoding process.

[0036] It also refers to a computer program containing instructions for implementing the above process, when these instructions are executed by a processor in a processing circuit.

[0037] It also aims for a non-transient memory medium storing the instructions of such a computer program.

[0038] Other advantages and features of the invention will become apparent from the examples of embodiments presented in the detailed description below, and from the examination of the accompanying drawings in which: there figure 1 illustrates a multi-mono coding (state of the art), the figure 2 illustrates a succession of main steps in an example procedure within the meaning of the invention, the figure 3 presents the general structure of an example of an encoder according to the invention, the figure 4 This document details the PCA / KLT analysis and transformation performed by block 310 of the encoder. figure 3 , there figure 5 presents an example of a decoder according to the invention, the figure 6 presents the decoding and inverse PCA / KLT synthesis of the figure 4 , in decoding, the figure 7 illustrates examples of structural implementation of an encoder and a decoder within the meaning of the invention.

[0039] The invention aims to enable optimized coding by: an adaptive matrixing in time (in particular with an adaptive transformation obtained by PCA / KLT (“PCA” designating a principal component analysis and “KLT” designating a Karhunen Loeve transform), followed preferentially by a multi-mono coding.

[0040] Adaptive matrixing allows for more efficient channel decomposition than fixed matrixing. Matrixing according to the invention advantageously allows for the decoupling of channels before multi-mono coding, so that the coding noise introduced by the encoding of each channel distorts the spatial image as little as possible when the channels are recombined to reconstruct an ambisonic signal during decoding.

[0041] Furthermore, the invention allows for smooth adaptation of the matrixing parameters to avoid "click" artifacts at the edges of the frame, excessively rapid fluctuations in spatial image, or encoding artifacts due to excessive variations (for example, related to sudden permutations of sound sources between channels) in the different individual channels resulting from the matrixing, which are subsequently encoded by different instances of a mono codec. A multi-mono encoding with preferentially variable bit allocation between channels (after adaptive matrixing) is presented below, but in variations, several instances of a stereo or other core codec can be used.

[0042] To facilitate understanding of the invention, some explanatory concepts concerning rotations in dimension n, PCA / KLT type decompositions or SVD (“SVD” designating a singular value decomposition) are recalled below. Les rotations et les « quaternions »

[0043] The signals are represented by successive blocks of sound samples, these blocks being called "subframes" below.

[0044] The invention uses a representation of rotations in dimension n with parameters adapted for quantization per frame and, above all, efficient interpolation per subframe. The rotation representations used in dimensions 2, 3, and 4 are defined below.

[0045] A rotation (around the origin) is a transformation of space in dimension n that changes one vector into another vector, such that: The magnitude of the vector is preserved. The cross product of vectors defining an orthonormal frame before rotation is preserved after rotation (there is no reflection).

[0046] A matrix M size n x n is a rotation matrix if and only if M T< .M=I n Or I n denotes the identity matrix of size n × n (that is to say that M is a unitary matrix, M T< designating the transposed M) and its determinant is +1.

[0047] The invention uses several representations that are equivalent to the rotation matrix representation: In two dimensions (in a 2D plane) ( n =2): We use the angle of rotation as a representation as follows.

[0048] Given the angle of rotation θ From this, we deduce the rotation matrix: M 2 θ = cos θ − sin θ sin θ cos θ

[0049] Given a rotation matrix, we can calculate the angle θ observing that the trace of the matrix is ​​2cos θ Note that it is also possible to estimate θ directly from a covariance matrix before applying a principles component decomposition (PCA) and eigenvalue decomposition (EVD) presented later.

[0050] Interpolation between two rotations of respective angles θ 1 and θ 2 can be done by linear interpolation between θ 1 and θ 2, taking into account the shortest path constraint on the unit circle between these two angles.

[0051] In three-dimensional (3D) space ( n (3): Euler angles and quaternions are used as the representation. In variations, an axis-angle representation can also be used, which is not detailed here.

[0052] A 3x3 rotation matrix can be decomposed into a product of 3 elementary rotations of angle θ along the x, y, or z axes. M 3 , x θ = 1 0 0 0 cos θ − sin θ 0 sin θ cos θ M 3 , y θ = cos θ 0 sin θ 0 1 0 − sin θ 0 cos θ M 3 , z θ = cos θ − sin θ 0 sin θ cos θ 0 0 0 1

[0053] Depending on the combinations of axes, the angles are called Euler angles or Cardan angles.

[0054] Another representation of 3D rotations, however, is given by quaternions. Quaternions are a generalization of complex number representations with four components in the form of a number q = a + bi + cj + dk Or i 2<= j 2< = k 2< = ijk = -1.

[0055] The real a is called scalar and the three imaginary parts ( b, c, d ) form a 3D vector. The norm of a quaternion is q = a 2 + b 2 + c 2 + d 2 Unit quaternions (of norm 1) represent rotations - however, this representation is not unique; thus, if q represents a rotation, -q represents the same rotation.

[0056] Given a unit quaternion q = a + bi + cj + dk (with a 2< + b 2< + c 2< + d 2< = 1), the associated rotation matrix is: M 3 , quat q = a 2 − b 2 − c 2 − d 2 2 bc − ad 2 ac + bd 2 ad + bc a 2 − b 2 + c 2 − d 2 2 cd − ab 2 bd − ac 2 ab + cd a 2 − b 2 − c 2 + d 2

[0057] Euler angles are not suitable for accurately interpolating 3D rotations; for this purpose, quaternions or the axis-angle representation are used instead. The SLERP (spherical linear interpolation) method involves interpolating according to the following formula: slerp q 1 q 2 α = sin 1 − α Ω sin Ω q 1 + sin α Ω sin Ω q 2 where 0 ≤ α ≤ 1 is the interpolation factor for going from q 1 to q 2 and Ω is the angle between the two quaternions: Ω = arccos q 1 . q 2 Or q 1. q 2 denotes the dot product between two quaternions (identical to the dot product between two 4-dimensional vectors).

[0058] This is equivalent to interpolating by following a great circle on a 4D sphere with a constant angular velocity as a function of α It is important to ensure that the shortest path is used for the interpolant by changing the sign of one of the quaternions when q 1. q 2 < 0. Note that other quaternion interpolation methods can be used (normalized linear interpolation or nlerp, splines, ...).

[0059] Note that it is also possible to interpolate 3D rotations using the axis-angle representation; in this case, the angle is interpolated as in the 2D case, and the axis can be interpolated, for example, using the SLERP method (in 3D), ensuring that the shortest path is taken on a 3D unit sphere and taking into account that the representation given by the axis r and the angle θ is equivalent to that given by the axis in the opposite direction - r and the angle 2π - θ .

[0060] In 4 dimensions ( n =4), a rotation can be parameterized by 6 angles ( n ( n -1)12) and we show that the multiplication of two 4x4 matrices called quaternions ( Q 1) and antiquaternion ( Q 2 ⋆ ) associated with quaternions q 1 = a + bi + cj + dk And q 2 = w + xi + yj + zk gives a rotation matrix of size 4x4.

[0061] It is possible to find the associated double quaternion ( q 1 , q 2) and associated quaternion and antiquaternion matrices such as: Q 1 = a b c d − b a − d c − c d a − b − d − c b a And Q 2 ⋆ = w − x − y − z x w − z y y z w − x z − y x w

[0062] Their product yields a 4x4 matrix: M 4 , quat q 1 q 2 = Q 1 Q 2 ⋆ and it is possible to verify that this matrix satisfies the properties of a rotation matrix (unitary matrix and determinant equal to 1).

[0063] Conversely, given a 4x4 rotation matrix, we can factor this matrix into a matrix product of the form Q 1 Q 2 ⋆ ,for example with the so-called "Cayley factorization" method. This involves calculating an intermediate matrix called a "tetragonal transform" (or associated matrix) and deducing the quaternions up to an indeterminacy on the sign of the two quaternions (which can be lifted by an additional "shortest path" constraint mentioned later). Décomposition en valeurs singulières (ou « SVD ”)

[0064] Singular value decomposition (SVD) consists of factoring a real matrix A of size mxn in the form: A = U Σ V T Or U is a unitary matrix ( U T< U = I m ) of size m × m, Σ is a rectangular diagonal matrix of size m × n with real and positive coefficients σ i >_ 0 (i = 1 ... p where p = min (m, n)), V is a unitary matrix ( V T< V = I n ) of size n × n and V T< is the transposition of V. The coefficients σ i in the diagonal of Σ are the singular values ​​of the matrix A . By convention, they are generally listed in descending order, and in this case the diagonal matrix Σ associated with A is unique.

[0065] The rank r of A is given by the number of non-zero coefficients σi. We can therefore rewrite the singular value decomposition as: A = U r U ˜ r Σ r 0 0 0 V r T V ˜ r T Or U r = [ u 1, u 2, ..., u r ] are the left singular vectors (or output vectors) of A , Σ r = diag(σ 1 , ... , σ r ) and V r = [ v 1, v 2, ..., v r ] are the right singular vectors (or input vectors) of A. This matrix formulation can also be rewritten as: A = ∑ i = 1 r σ i u i v i T

[0066] If the sum is limited to an index i < rWe obtain a "filtered" matrix which only represents the "predominant" information.

[0067] We can also write: A v i = σ i u i

[0068] Which shows that the matrix A transforms v i in σ i u i.

[0069] The SVD of A has a relationship with the eigenvalue decomposition of A T< A And A A T< because : A T A = V Σ T Σ V T A A T = U ΣΣ T U T

[0070] The intrinsic values ​​of Σ T< Σ And Σ Σ T< are σ 1 2 , … , σ r 2 The columns of U are the eigenvectors of A A T< , while the columns of V are the eigenvectors of A T< A.

[0071] The SVD can be interpreted geometrically: the image of a sphere in dimension n by the matrix A is in dimension m a hyperellipse having principal axes along the directions u 1, u 2, ...,u m and of length σ 1 , ... , σ m . Transformée de Karhunen Loeve (ou « KLT » pour « Karhunen Loeve Transform »)

[0072] The Karhunen Loeve transformation (KLT) of a random vector x centered at 0 and covariance matrix R xx = E[ x x T< ] is defined by: y = V T x Or V is the matrix of eigenvectors (with the convention that eigenvectors are column vectors) obtained by eigenvalue decomposition of R xx R xx = V Λ V T where A = diag(λ 1 , ..., λ n ) is a diagonal matrix whose coefficients are the eigenvalues. The matrix V = [ v 1, v 2, ..., v n ] contains the eigenvectors (columns) of R xx such as R xx v i = λ n v i

[0073] KLT can be seen as a fundamental change, because the product V T< x expresses the vector x in the basis given by the eigenvectors.

[0074] The inverse transformation is given by: x = V y

[0075] KLT allows us to decorrelate the components of x; the variances of the transformed vector y are the intrinsic values ​​of R xx . Analyse en composantes principales (ou « PCA » pour « principal component analysis ”)

[0076] Principal component analysis (PCA) is a dimensionality reduction technique that produces orthogonal variables and maximizes the variance of the variables after projection (or equivalently minimizes the reconstruction error).

[0077] The PCA presented below, although also based on an eigenvalue decomposition like the KLT, is such that the estimated covariance matrix R̂ xx is calculated from N observed vectors x i , i = 1 ... N of dimension n: R ^ xx = 1 N − 1 ∑ i = 1 N x i x i T assuming these vectors are centered: m x = 1 N ∑ i = 1 N x i = 0

[0078] The eigenvalue decomposition of R̂ xx in the form R̂ xx = V L V T< allows us to calculate the principal components: y n = V T< x n .

[0079] PCA is a matrix transformation V T< which projects the data into a new basis to maximize the variance of the variables after projection.

[0080] Note that the PCA can also be obtained from an SVD of the signal x i put in the form of a matrix X of size n × N. In this case, we can write: X = U D V T

[0081] We check that XX T< = U DD T< U T< which corresponds to a diagonalization of XX T< . Thus, the projection vectors of the PCA correspond to the column vectors of U and the projection gives the following result U T< X = D V T< .

[0082] It should also be noted that PCA is generally seen as a dimensionality reduction technique, used to "compress" a high-dimensional dataset into one with few principal components. In the invention, PCA advantageously allows for the decoupling of the multidimensional input signal, but avoids removing channels (thus reducing the number of channels) to prevent the introduction of artifacts. This forces a minimum encoding rate to avoid "truncating" the spatial image, except in specific variants where eigenvalues ​​are so small that a zero rate can be permitted (for example, to better encode ambisonic sounds artificially created with a single synthetically spatialized source).

[0083] We now refer to the figure 2 to describe general principles of the steps implemented in a process within the meaning of the invention, for a common framework t .

[0084] Step S1 consists of obtaining the respective signals of the ambisonic channels (here four channels W, Y, Z, X in the example described using a channel order according to the ACN convention for Ambisonics Channel Number), for each frame t. These signals can be put into the form of a matrix. n × L (For n ambisonic channels (here 4) and L samples per frame).

[0085] In the next step S2, the signals from these channels can optionally be pre-processed, for example by a high-pass filter as described later with reference to the figure 3 .

[0086] In the next step S3, a principal component analysis (PCA) or, equivalently, a Karhunen-Loeve transform (KLT) is applied to these signals to obtain eigenvalues ​​and an eigenvector matrix from a covariance matrix of the n channels. In variations of the invention, a system vector degradation (SVD) may be used.

[0087] At step S4, this eigenvector matrix, obtained for the current frame t, undergoes signed permutations so that it is as aligned as possible with the matrix of the same nature from the previous frame t-1. In principle, we ensure that the column axis of the eigenvectors in the matrix of eigenvectors corresponds as closely as possible to the column axis of the eigenvectors in the same position in the matrix of the previous frame. If not, we swap the positions of the eigenvectors in the matrix of the current frame t that do not correspond. Next, we further ensure that the directions of the eigenvectors from one matrix to the other also coincide. In other words, we are initially only interested in the lines containing the eigenvectors (just the direction, not the sense) and we find, for each line, the closest line in the matrix of the previous frame t-1. To do this, we swap vectors in the matrix of the current frame. Then, in a second step, we try to match the orientation of the vectors (sense). To do this, we reverse the sign of the eigenvectors that do not have the correct sense.

[0088] Such a design ensures maximum consistency between the two matrices and thus avoids audible clicks between two frames during sound playback.

[0089] In step S5, we also ensure that the eigenvector matrix of the current frame t , thus corrected by signed permutations, accurately represents the application of a rotation (of an angle for n =2 channels, three Euler angles, an axis and an angle or a quaternion for n =3 corresponding to the planar first-order ambisonic representation W, Y, Z, and of two quaternions for n =4 in first-order ambisonic representation of type W,Y,Z,X).

[0090] To ensure that it is indeed a rotation, the determinant of the eigenvector matrix of the current frame t, corrected by permutations, must be positive and equal to (or, in practice, close to) +1 at step S6. If it is equal to (or close to) -1, then it is necessary to: swap two eigenvectors again (for example associated with low energy channels, therefore not very representative), or preferably reverse the sign of all the elements of a column (for example associated with a low energy channel) in step S6.

[0091] We then obtain a matrix of eigenvectors for the current frame t effectively corresponding to a rotation at step S7.

[0092] We can then encode parameters of this matrix (such as the angle value, the value of an axis and an angle, or the quaternion(s) of this matrix) using a number of bits allocated for this purpose at step S8. In another optional but advantageous implementation, if a significant discrepancy (greater than a threshold, for example) is observed at step S9 between the rotation matrix estimated for the current frame and the value of the matrix, the matrix can be encoded. t and the rotation matrix of the previous frame t- 1. We can determine a variable number of interpolation subframes; otherwise, we fix this number of subframes to a predetermined value. Step S10 consists of: divide the current frame into subframes, and interpolate matrices to be applied to successive subframes from the matrix of the previous frame. t -1 up to the current frame matrix t, in order to smooth out the difference between the two matrices over time.

[0093] In step S11, the interpolated rotation matrices are applied to a matrix n X ( L / K ) representing each of the K Subframes of the ambisonic channel signals from stage S1 (or optionally S2) are used to decouple these signals as much as possible before the multi-mono encoding in stage S14. It is important to remember that, according to a general approach, these signals should be decoupled as much as possible before this multi-mono transformation. A binary allocation to the separate channels is performed in stage S12 and encoded in stage S13.

[0094] At step S14, before performing the multiplexing of step S15 and thus completing the compression coding process, we can decide on a number of bits to allocate per channel based on the representativeness of that channel and the available bandwidth on the RES network ( figure 7 In one implementation, the energy in each channel for a current frame is estimated and this energy is multiplied by a predefined score for that channel and for a given data rate (this score being, for example, a MOS score explained later with reference to the figure 3 This weights the number of bits to be allocated to each channel. Such an implementation is advantageous in itself and can potentially be subject to separate protection in an ambisonic context.

[0095] We illustrated on the figure 7 a DCOD encoding device and a DDEC decoding device, as defined in the invention, these devices being dual to each other (in the sense of "reversible") and connected to each other by a communication network RES.

[0096] The DCOD coding device includes a processing circuit typically comprising: a memory MEM1 for storing instruction data of a computer program as defined in the invention (these instructions being able to be distributed between the DCOD encoder and the DDEC decoder); an interface INT1 for receiving ambisonic signals distributed on different channels (for example four channels W, Y, Z, X in order 1) for the purpose of encoding them in compression as defined in the invention; a processor PROC1 for receiving these signals and processing them by executing the computer program instructions stored in the memory MEM1, for the purpose of encoding them; and a communication interface COM 1 for transmitting the encoded signals via the network.

[0097] The DDEC decoding device includes its own processing circuit, typically including: a MEM2 memory for storing instruction data of a computer program as defined in the invention (these instructions being able to be distributed between the DCOD encoder and the DDEC decoder as previously indicated); a COM2 interface for receiving the coded signals from the RES network for decoding by compression as defined in the invention; a PROC2 processor for processing these signals by executing the computer program instructions stored in the MEM2 memory, for decoding purposes; and an INT2 output interface for delivering the decoded signals in the form of ambisonic channels W', Y', Z', X', for example for playback purposes.

[0098] Of course, this figure 7 illustrates an example of a structural implementation of a coded device (coder or decoder) as defined in the invention. figures 3 à 6 The sections discussed later describe in detail rather functional implementations of these codecs.

[0099] We now refer to the figure 3 to describe an encoding device as defined in the invention.

[0100] The encoder's strategy is to decorrelate the channels of the ambisonic signal as much as possible and encode them with a core code. This strategy helps limit artifacts in the decoded ambisonic signal. More specifically, the aim is to apply optimized decorrelation to the input channels before multi-mono encoding. Furthermore, an interpolation whose computational cost for both the encoder and decoder is limited because it is performed in a specific domain (angle in 2D, quaternion in 3D, double quaternion in 4D) allows the covariance matrices calculated for PCA / KLT analysis to be interpolated, rather than repeating an eigenvalue and eigenvector decomposition multiple times per frame.

[0101] However, before addressing core coding as defined in the invention, some advantageous features of the encoder are presented here, such as the optimization of the bit budget allocated to coding based on perceptual criteria, seen later.

[0102] In the encoder embodiment described here, the encoder can typically be an extension of the standardized 3GPP EVS (for "Enhanced Voiced Services") encoder. Advantageously, EVS encoding rates can be reused without modifying the structure of the EVS bitstream.

[0103] Thus, multi-mono coding (block 340 of the figure 3 described later) works here with a possible allocation to each transformed channel, restricted to the following bitrates for super-wideband audio coding: 9.6; 13.2; 16.4; 24.4; 32; 48; 64; 96 and 128 kbit / s.

[0104] Of course, it is possible to add additional bandwidth (for finer allocation granularity) by modifying the EVS code. A different code than EVS can also be used, for example, the OPUS® code.

[0105] Generally speaking, the finer the coding granularity, the more bits must be reserved to represent the possible data rate combinations. A compromise must be made between allocation precision and the additional information describing the binary allocation. This allocation is optimized here by block 320 of the figure 3 which is described further below. This is an advantageous feature in itself and independent of the eigenvector decomposition for establishing a rotation matrix as defined in the invention. As such, the bit allocation performed by block 320 can be subject to separate protection.

[0106] With reference to the figure 3 block 300 receives an input signal Y in the current index framework t. The index is not shown here to avoid cluttering the notation. This is a matrix of size n × L In a suitable implementation for a first-order ambisonic context, we have n =4 channels W, Y, Z, X (defined according to the ACN order) which can be normalized according to the SN3D convention. In a variant, the channel order can be alternatively, for example, W, X, Y, Z (following the FuMA convention), and the normalization can be different (N3D or FuMA). Thus, channels W, Y, Z, X correspond to the successive lines: y 1, l , y 2 ,l , y 3, l , y 4, l which will be recorded as one-dimensional signals y i ( l ) , l = 1, ..., L. This is therefore a succession of samples from 1 to L occupying the frame t.

[0107] It is assumed that the signal (in each channel) is sampled at 48 kHz, without loss of generality. The frame length is fixed at 20 ms, i.e. L =960 successive samples, without loss of generality. In variants, one could, for example, use a frame length of L = 640 samples for a sampling rate of 32 kHz.

[0108] The PCA / KLT analysis and PCA / KLT transformation described later are performed in the time domain. It is therefore understood that we remain within the time domain without necessarily needing to perform a sub-band or, more generally, a frequency-domain transform.

[0109] For each frame, block 300 of the encoder applies an (optional) preprocessing to obtain the preprocessed input signal denoted Y .This can involve high-pass filtering (with a cutoff frequency typically of 20 Hz) of each new 20 ms frame of the input signal channels. This operation removes the DC component that could bias the covariance matrix estimation, so that the signal at the output of block 300 can be considered to have a zero mean. The transfer function is denoted H pre ( z ) , Thus, for each channel we have: X i ( z ) = H pre ( z ) Y i ( z ) . If block 300 is not implemented, we have X = Y .We can also implement a low-pass filter from block 340 to perform multi-mono coding, but when block 300 is implemented, the high-pass filtering in preprocessing of the mono coding that can be used in block 340 is preferentially disabled to avoid repeating the same preprocessing and thus reduce the overall complexity.

[0110] The transfer function denoted H pre (z) The above could be of the type: H pre z = b 0 + b 1 z − 1 + b 2 z − 2 1 − a 1 z − 1 − a 2 z − 2 by applying this filter to each of the n input signal channels whose coefficients can be such as those shown in the table below: 8 kHz 16 kHz 32 kHz 48 kHz b 0 0.988954248067140 0.994461788958.195 0.997227049904470 0.998150511190452 b 1 -1.977908496134280 -1.988923577916390 -1.994454099808940 -1.996301022380904 b 2 0.988954248067140 0.994461788958195 0.997227049904470 0.998150511190452 a 1 1.977786483776764 1.988892905899653 1.994446410541927 1.996297601769122 a 2 -0.978030508491796 -0.988954249933127 -0.994461789075954 -0.996304442992686

[0111] Alternatively, another type of filter can be used, for example a 6th order Butterworth filter with a frequency of 50 Hz.

[0112] In some variations, the preprocessing may include a fixed matrixing step that can maintain the same number of channels or reduce the number of channels. An example of matrixing applied to the four channels of a B-format ambisonic signal is given below: M B → A = 1 / 2 1 6 0 1 12 1 / 2 − 1 6 0 1 12 1 / 2 0 1 6 − 1 12 1 / 2 0 − 1 6 − 1 12

[0113] Note that in this case, this preprocessing will need to be reversed during decoding by applying matrixing. M A → B = M B → A -1< of the decoded signal to recover the channels in their original format.

[0114] The following block 310 estimates each frame t a transformation matrix obtained by determining the eigenvectors using PCA / KLT and verifying that the transformation matrix formed by these eigenvectors indeed characterizes a rotation. Details regarding block operation 310 are given later with reference to the figure 4 This transformation matrix performs channel matrixing to de-correlate them, allowing the application of independent multi-mono coding by block 340. As detailed later, block 310 transmits to the multiplexer quantization indices representing the transformation matrix and optionally information encoding the number of interpolations of the transformation matrix, per subframe of the current frame t, as also detailed later.

[0115] Block 320 determines the optimal bandwidth allocation for each channel (after PCA / KLT transformation) based on a bit budget. B given. This block seeks a distribution of the bandwidth between channels by calculating a score for each possible combination of bandwidths; the optimal allocation is found by finding the combination that maximizes this score.

[0116] Several criteria can be used to define a score for each combination.

[0117] For example, the number of possible bitrates for mono encoding of a channel can be limited to the nine discrete bitrates of EVS encoding with a super-widened audio bandwidth: 9.6; 13.2; 16.4; 24.4; 32; 48; 64; 96 and 128 kbit / s. However, if the encoding according to the invention operates at a given bitrate associated with a budget of B bits in the current frame of index t, generally only a subset of these listed bit rates is usable. For example, if the bit rate of the encoded signal is set to 4 x 13.2 = 52.8 kbit / s to represent four channels, and if each channel receives a minimum budget of 9.6 kbit / s to guarantee super-wide bandwidth for each channel, the possible bit rate combinations for encoding separate channels must respect the constraint that the bit rate used remains lower than the available bit rate, which corresponds to: B multimono = B − B overhead , Or B overhead corresponds to the bit budget for the additional information encoded per frame (bit allocation + rotation data) as described later. For example, B overhead may be in the order of B overhead = 55 bits per 20 ms frame (i.e., 2.75 kbit / s) for the case of four-channel ambisonic encoding; this includes 51 bits to encode the rotation matrix and 4 bits (as described below) to encode the bit allocation for encoding the separate channels. For an overall bit rate of 4 x 13.2 = 52.8 kbit / s, this leaves a budget of B multimono =50.05 kbit / s.

[0118] This gives the following channel throughput permutations: Singleton (9.6, 9.6, 9.6, 9.6) - total = 38.4 Permutations of (13.2, 9.6, 9.6, 9.6) - total = 42 kbit / s Permutations of (13.2, 13.2, 9.6, 9.6) - total = 45.6 kbit / s Permutations of (13.2, 13.2, 13.2, 9.6) - total = 49.2 kbit / s Permutations of (16.4, 9.6, 9.6, 9.6) - total = 45.2 kbit / s Permutations of (16.4, 13.2, 9.6, 9.6) - total = 48.8 kbit / s

[0119] We observe that some combinations respecting the maximum budget limit have a much lower throughput than others, and ultimately only two relevant combinations can be retained: Permutations of (13.2, 13.2, 13.2, 9.6) – 4 cases and unused bandwidth of 50.5 - 49.2 = 1.3 kbit / s and Permutations of (16.4, 13.2, 9.6, 9.6) – 12 cases and unused bandwidth of 50.5 - 48.8 = 1.7 kbit / s. This illustrates that sixteen combinations are particularly interesting and can be encoded using 4 bits (16 values). Furthermore, a certain number of bits may remain unused depending on the chosen allocation.

[0120] It is observed that adaptive matrix coding based on PCA / KLT processing and allowing flexible binary allocation can result in unused bits and, for some channels, a lower bit rate (e.g. 9.6 kbit / s) than the equally distributed bit rate between each of the channels (e.g. 13.2 kbit / s per channel).

[0121] To improve this situation, block 320 can then evaluate all possible (relevant) data rate combinations for the 4 channels resulting from the PCA / KLT transformation (output of block 310) and assign them a score. This score is calculated based on: the energy of each channel, and an average score which can be pre-memorized and derived from subjective or objective tests, which score, denoted MOS (for "Mean Opinion Score", being an average score on a panel of testers), is associated with the allocated bandwidth.

[0122] This score can then be defined by the equation S b t , 1 ,.. , b t , n = ∑ i = 1 n Q b t , i . E i Or E i is the energy in the current frame (of index t) of the signal s ( l ), l = ···. L - 1 on the channel i , with : E i = ∑ l = 0 L − 1 s 2 l

[0123] The optimal allocation can be such that: b t , 1 opt ,.. , b t , 1 opt = arg max b t , 1 ,.. , b t , n ∑ i = 1 n b t , i ≤ B S b t , 1 ,.. , b t , n

[0124] Alternatively, the factor E i can be fixed to the value that the eigenvalue associated with channel i takes from the eigenvalue decomposition of the input signal of block 310 and after possible signed permutation.

[0125] The MOS rating Q(b i ) is preferentially the subjective quality rating of the code used for multi-mono coding in block 340 for a budget b i (in number of bits) per 20 ms frame corresponding to a data rate R i = 50 b i (in bits / sec). Initially, we can use the subjective (average) MOS scores of a standardized EVS encoder given by: κ i 0 1 2 3 4 5 6 7 8 b i 192 264 328 488 640 960 1280 1920 2560 R i 9600 13200 16400 24400 32000 48000 64000 96000 128000 Q ( b i ) 3.62 3.79 4.25 4.60 4.53 4.82 4.83 4.85 4.87

[0126] Alternatively, other MOS ratings for each of the listed bitrates can be derived from other tests (subjective or objective) predicting codec quality. It is also possible to adapt the MOS ratings used in the current frame based on a signal type classification (e.g., speech without background noise, speech with ambient noise, music, or mixed content) by reusing classification methods implemented by EVS encoding and applying them to the W channel of the input ambisonic signal before binary allocation. The MOS rating can also correspond to an average rating derived from different methodologies and rating scales: MOS (absolute) from 1 to 5, DMOS (from 1 to 5), MUSHRA (from 0 to 100).

[0127] In a variant where the EVS encoder is replaced by another codec, the list of bitrates b i and the notes Q ( b i ) can be replaced depending on this other codec. Additional encoding bitrates can also be added to the EVS encoder, thus expanding the list of bitrates and MOS notes, or the EVS encoder and potentially the associated MOS notes can be modified.

[0128] Alternatively, the allocation between channels is refined by weighting the energy by a power α Or α takes a value between 0 and 1. By varying the value of α, This allows us to control the influence of energy in the allocation: more α The closer the value is to 1, the more energy plays a significant role in the score, and therefore the more unequal the allocation between channels. Conversely, the closer the value is to 1, the more energy is allocated between channels. α The closer the score is to 0, the less important energy is and the more equally the allocation is distributed between the channels. The score is therefore expressed as: S b t , 1 ,.. , b t , n = ∑ i = 1 n Q b t , i . E i α

[0129] Alternatively, to make the allocation more stable, a second weighting can be added to the scoring function to penalize inter-frame bitrate changes. A penalty is added to the score if the bitrate combination is not the same within the frame. t that in the plot t - 1. The score is then expressed in the form: S b t , 1 ,.. , b t , n = ∑ i = 1 n Q b t , i . E i α . 1 + β i Or β i has a predetermined constant value (for example 0.1) when b t,i =b t -1, i And β i = 0 when b t,i ≠ b t -1, i .

[0130] This additional weighting helps limit excessively frequent fluctuations in throughput between channels. With this weighting, only significant energy changes result in a change in throughput. Furthermore, the value of the constant can be varied to adjust the stability of the allocation.

[0131] With further reference to the figure 3 Once the bitrate for each frame is calculated, this bitrate is exhaustively encoded by block 330, for example, for all bitrate combinations. In the case of 9 bitrates and 4 channels, the required bitrate is log 2 9 4 = 13 bits, where . corresponds to rounding up to the nearest whole number. The combination of the 4 flow rates can be coded as the index: ∑ i = 1 n 9 i κ i However, it may be preferable to enumerate (initially, offline) the different relevant bit rate combinations for the given bit budget and use the minimum bit rate to represent these combinations. The index can then be represented by a "permutation code" + "combination offset" type encoding; for example, in the example where we encode on a 4-bit index the 16 bit rate combinations comprising 4 permutations of (13.2, 13.2, 13.2, 9.6) and 12 permutations of (16.4, 13.2, 9.6, 9.6), we can use the indices 0-3 to encode the first 4 possible permutations (with an offset of 0 and a code ranging from 0 to 3) and the indices 4-15 to encode the other 12 possible permutations (with an offset of 4 and a code from 0 to 11).

[0132] With further reference to the figure 3 The multiplexing block 350 takes as input the n matrixed channels from block 310 and the bitrates allocated to each channel from block 320. It then encodes the different channels separately with a core code that corresponds to the EVS code, for example. If the core code used allows stereo or multichannel encoding, the multi-mono approach can be replaced by multi-stereo or multichannel encoding. Once the channels are encoded, the associated bitstream is sent to the multiplexer (block 350).

[0133] In frames where part of the overall budget is not fully used, the multiplexer (block 350) can add zero-based padding bits to reach the bit budget allocated to the current frame, i.e. B − ∑ i = 1 n b t , i opt bits. In some variations, the remaining bit budget can be redistributed to the encoding of the transformed channels to utilize the entire available budget, and if the multimono encoding is based on EVS-type technology, the specified 3GPP EVS encoding algorithm can be modified to introduce additional bit rates. In this case, it is also possible to integrate these additional bit rates into the table defining the correspondence between b i And Q ( b i ) .

[0134] In addition, a bit can be reserved to allow switching between two coding modes: Coding according to the invention with rotation matrix coding, and Coding according to the invention with a rotation matrix restricted to the identity matrix (therefore not transmitted) which amounts to a direct multi-mono coding if the rotation matrix of the previous frame was also an identity matrix (for example when the ambisonic signal includes very diffuse sound sources or multiple sources spread spatially around certain privileged directions, in which case the ambisonic channels are less correlated than for sounds mixing more point and isolated sources).

[0135] The choice between these two modes involves using a bit in the data stream to indicate whether the current frame uses a rotation matrix restricted to the identity matrix without transmitting rotation parameters (bit=0) or whether a rotation matrix is ​​encoded (bit=1). When bit=0, in some variants, a fixed allocation of bits to separate channels can be used, and no binary allocation can be transmitted.

[0136] We now refer to the figure 4 to describe in detail block 310 applying the PCA / KLT analysis and transformation. In this block, the encoder calculates the covariance matrix from the (pre-processed) ambisonic channels in block 400: C = 1 L − 1 X X T

[0137] Alternatively, this matrix can be replaced by the correlation matrix, where the channels are pre-normalized by their respective standard deviations, or more generally, weights reflecting relative importance can be applied to each of the channels; furthermore, the normalization term 1 / ( L - 1) can be omitted or replaced by another value (for example 1 / L ) . The values C ij correspond to the variance between x i And x j .

[0138] The encoder then performs an eigenvalue decomposition (EVD) in block 410, calculating the eigenvalues ​​and eigenvectors of the matrix C . The eigenvectors are denoted here V t to indicate the frame index t because the eigenvectors V t - 1 obtained in the previous index framework t- 1 are preferentially stored and used subsequently. Eigenvalues ​​are denoted λ 1, λ 2, ..., λ n .

[0139] In one variant, a singular value decomposition (SVD) of the preprocessed channels X can be used. This gives us the singular vectors (on the left U and right V) and the singular values ​​σ i . In this case, we can consider that the eigenvalues ​​λ i are λ i = σ i 2 and the eigenvectors V t are given by the n singular vectors (column) on the left U.

[0140] The encoder then applies a first signed permutation of the columns of the transformation matrix for the frame in block 420 t (whose columns are the eigenvectors) in order to avoid too much disparity with the transformation matrix of the previous frame t-1, which would cause click problems at the border with the previous frame.

[0141] Thus, once a draft of the transformation matrix is ​​obtained for the frame t, block 430 takes n estimated eigenvectors V t = v t, 0 , ... , v t,n of the current index framework t And n eigenvectors V t - 1 memorized from the previous index plot t - 1, and applies a signed permutation to the estimated vectors V t so that they are as close as possible to V t -1 . Thus the eigenvectors of the frame t are swapped so that the associated base is as close as possible to the frame base t - 1. This has the effect of improving the continuity of the transformed signal frames (once the transformation matrix has been applied to the channels).

[0142] Another constraint is that the transformation matrix must correspond to a rotation. This constraint ensures that the encoder can convert the transformation matrix into generalized Euler angles (block 430) for quantization (block 440) within a predetermined bit budget, as previously discussed. For this purpose, the determinant of this matrix must be positive (typically +1).

[0143] Preferably, the optimal signed permutation is obtained in two steps: The first step (S4 on the figure 2 (previously presented) matches the nearest vectors between two frames, considering only the axis and not its direction. This problem can be formulated as a combinatorial task assignment problem, where the goal is to find the configuration that minimizes a cost. The cost can be defined here as the trace of the absolute value of the cross-correlation between the eigenvector matrices of the frames. t And t - 1. C t = tr abs corr V t V t − 1 Or tr(.) denotes the trace of a matrix, abs (.) amounts to applying the absolute value operation to all the coefficients of a matrix and corr(V1,V2) gives the correlation matrix between the vectors V1 and V2.

[0144] In one embodiment, the "Hungarian" method (or "Hungarian algorithm") is used to determine the optimal assignment that results in a permutation of the frame's eigenvectors. t ; The second stage (S6 on the figure 2 This involves determining the direction / sense of each permuted eigenvector. Block 420 calculates the cross-correlation between the permuted eigenvectors. Ṽ t of the plot t and the eigenvector of the frame t - 1 Γ t = corr V ˜ t V t − 1

[0145] If a value on the diagonal of the cross-correlation matrix C t is negative, this denotes a change of sign between the directions of the eigenvectors. A sign inversion is then performed on the corresponding eigenvector in Ṽ t .

[0146] At the end of the two steps, the transformation matrix to the frame t is designated by V t so that in the next frame the stored matrix becomes V t -1 .

[0147] In one variant, the search for the optimal signed permutation can be done by calculating the change-of-basis matrix. V t − 1 − 1 V t Or V t V t − 1 − 1 which is converted to 3D or 4D, and this change-of-basis matrix is ​​converted respectively into a single unit quaternion or two unit quaternions. The search then becomes a nearest neighbor search using a dictionary representing the set of all possible signed permutations. For example, in the 4D case, the twelve possible even permutations (out of a total of 24 permutations) of 4 values ​​are associated with the following double unit quaternions written as 4D vectors: (1, 0, 0, 0) and (1, 0, 0, 0) (0, 0, 0, 1) and (0, 0, -1, 0) (0, 1, 0, 0) and (0, 0, 0, -1) (0, 0, 1, 0) and (0, -1, 0, 0)] (0.5, -0.5, -0.5, -0.5) and (0.5, 0.5, 0.5, 0.5) (0.5, 0.5, 0.5, 0.5) and (0.5, -0.5, -0.5, -0.5) (0.5, -0.5, 0.5, -0.5) and (0.5, -0.5, 0.5, 0.5) (0.5, -0.5, 0.5, 0.5) and (0.5, -0.5, -0.5, 0.5) (0.5, 0.5, -0.5, 0.5) and (0.5, 0.5, -0.5, -0.5) (0.5, -0.5, -0.5, 0.5) and (0.5, 0.5, -0.5, 0.5) (0.5, 0.5, -0.5, -0.5) and (0.5, 0.5, 0.5, -0.5) (0.5, 0.5, 0.5, -0.5) and (0.5, -0.5, 0.5, -0.5)

[0148] The search for the optimal (pair) permutation can be performed using the list above as a predefined double quaternion dictionary and by finding the nearest neighbor with respect to the double quaternion associated with the change-of-basis matrix. An advantage of this method is the reuse of quaternion and double quaternion rotation parameters.

[0149] The operation implemented in the following block 460 assumes that the transformation matrix after signed permutation is indeed a rotation matrix; the transformation matrix must necessarily be unitary, but its determinant must also be equal to 1. det V t = 1

[0150] However, the transformation matrix resulting from blocks 410 and 420 (after EVD and signed permutations) is an orthogonal (unitary) matrix that can have a determinant of -1 or 1, that is to say, a reflection or rotation matrix.

[0151] If the transformation matrix is ​​a reflection matrix (if its determinant is equal to -1), it can be changed into a rotation matrix by inverting an eigenvector (for example, the eigenvector associated with the lowest value) or by swapping two columns (eigenvectors).

[0152] Some methods of eigenvalue decomposition (e.g., by Givens rotation) or singular value decomposition can lead to transformation matrices that are intrinsically rotation matrices (with a determinant of +1); in this case, the step of verifying that the determinant is +1 will be optional.

[0153] Block 430 converts the rotation matrix into parameters. In the preferred embodiment, an angular representation is used for quantization (six generalized Euler angles for the 4D case, three Euler angles for the 3D case, and one angle for 2D). For the ambisonic case (four channels), six generalized Euler angles are obtained according to the method described in the article "Generalization of Euler Angles to N-Dimensional Orthogonal Matrices" by David K. Hoffman, Richard C. Raffenetti, and Klaus Ruedenberg, published in the Journal of Mathematical Physics 13, 528 (1972); for the planar ambisonic case (three channels), three Euler angles are obtained; and for the stereo case, one rotation angle is obtained using well-known state-of-the-art methods. The angle values ​​are quantized in block 440 with a predetermined bit budget.In the preferred embodiment, scalar quantization is used, and the quantization step is, for example, identical for each angle. For instance, in the case of 4 channels, 6 generalized Euler angles are encoded with 3x(8+9)=51 bits (3 angles defined on an interval of [-π / 2, π / 2] encoded on 8 bits with a step of π / 256, and the other 3 angles defined on an interval of [-π, π] encoded on 9 bits with a step of π / 256). The quantization indices of the transformation matrix are sent to the multiplexer (block 350). Furthermore, block 440 can convert the quantized parameters into a quantized rotation matrix. V̂ t , if the parameters used for quantization do not correspond to the parameters used for interpolation.

[0154] Alternatively, blocks 430 and 440 can be replaced as follows: The 430 block can perform a conversion of rotation matrices into a double unitary quaternion (4-channel case), into a unitary quaternion (3-channel case) and into an angle (2-channel case).

[0155] This double quaternion conversion for the 4D case can be performed for a rotation matrix whose coefficients are denoted R[i,j], i,j=0...3, using the following pseudocode: Calculation of the associated matrix A[i,j] with: A 0,0 = R 0,0 + R 1,1 + R 2,2 + R 3,3 A 1,0 = R 1,0 − R 0,1 + R 3,2 − R 2,3 A 2,0 = R 2,0 − R 3,1 − R 0,2 + R 1,3 A 3,0 = R 3,0 + R 2,1 − R 1,2 − R 0,3 A 0,1 = R 1,0 − R 0,1 − R 3,2 + R 2,3 A 1,1 = − R 0,0 − R 1,1 + R 2,2 + R 3,3 A 2,1 = − R 3,0 − R 2,1 − R 1,2 − R 0,3 A 3,1 = R 2,0 − R 3,1 + R 0,2 − R 1,3 A 0,2 = R 2,0 + R 3,1 − R 0,2 − R 1,3 A 1,2 = R 3,0 − R 2,1 − R 1,2 + R 0,3 A 2,2 = − R 0,0 + R 1,1 − R 2,2 + R 3,3 A 3,2 = − R 1,0 − R 0,1 − R 3,2 − R 2,3 A 0,3 = R 3,0 − R 2,1 + R 1,2 − R 0,3 A 1,3 = − R 2,0 − R 3,1 − R 0,2 − R 1,3 A 2,3 = R 1,0 + R 0,1 − R 3,2 − R 2,3 A 3,3 = − R 0,0 + R 1,1 + R 2,2 − R 3,3 A = A / 4

[0156] Calculating the two quaternions from the associated matrix A 2 = square A # carré de coefficents q1 = sqrt A2 .sum axis = 1 # somme sur le lignes q 2 = sqrt A2 .sum axis = 0 # somme sur les colonnes

[0157] Determining the signs For k=0..3: If sign(A[i,k])<0, Then q2[k] = -q2[k] For k=0..3: If sign(A[k,j)) !=sign(q1 [k]*q2[j)), Then q1[k] = -q1[k]

[0158] The quaternion conversion for the 3D case can be performed as follows for a 3x3 matrix R[i,j] i,j=0...2: Simplified associated matrix calculation: For i=0..3: q[i] = sqrt(q[i]) / 4

[0159] Calculation of the quaternion q Si R 2,1 − R 1 ,2 < 0 , q 1 = − q 1 If R 0,2 − R 2 ,0 < 0 , q 2 = − q 2 If R 1,0 − R 0 ,1 < 0 , q 3 = − q 3

[0160] The calculation of the angle for the case of a 2x2 matrix is ​​done according to the already known state-of-the-art methods.

[0161] In variants, the unit quaternions q1, q2 (4D case) and q (3D case) can be converted into known axis-angle representations from the state of the art. Block 440 can perform quantization in the specified range: * Case of 4 channels: the unit quaternion pair q 1 and q2 is quantized by a spherical quantization dictionary in dimension 4; by convention we quantize q 1 with a hemispherical dictionary (because q 1 and - q 1 correspond to the same 3D rotation) and q 2 is quantized with a spherical dictionary. Examples of dictionaries can be given by predefined points from 4-dimensional polyhedra; in variants, a double axis-angle representation can be quantized, which would be equivalent to the double quaternion; * Case of 3 channels: the unit quaternion is quantized by a 4-dimensional spherical quantization dictionary - examples of dictionaries can be given by predefined points from 4-dimensional polyhedra; * Case of 2 channels: the angle is quantized by uniform scalar quantization.

[0162] We now describe block 460, which interpolates rotation matrices between two successive frames. It smooths out channel discontinuities after these matrices have been applied. Typically, this is useful if two sets of angles or quaternions are too different from a previous frame. t -1 to the next one t, Audible clicks are to be expected if a smooth transition has not been performed between these two frames in subframes. A transition interpolation is then performed between the rotation matrix calculated for the frame t -1 and the rotation matrix calculated for the frame t.The encoder interpolates in block 460 the (quantized) representation of the rotation between the current frame and the previous frame to prevent excessively rapid fluctuations in the different channels after transformation. The number of interpolations can be fixed (equal to a predetermined value) or adaptive. Each frame is then divided into subframes according to the number of interpolations determined in block 450. Thus, if adaptive interpolation is used, block 450 can encode, using a chosen number of bits, the number of interpolations to be performed, and therefore the number of subframes to be generated, if this number is determined adaptively; in the case of fixed interpolation, no information needs to be encoded.

[0163] Next, block 460 converts the rotation matrices into a specific domain representing a rotation matrix. The frame is then subdivided into subframes, and interpolation is performed for each subframe within the chosen domain.

[0164] For a first-order ambisonic input signal (4 channels W, X, Y, Z), in block 460, the encoder reconstructs a quantized 4D rotation matrix from the six quantized Euler angles, and this matrix is ​​then converted into two unit quaternions for interpolation. In a variant where the encoder input is a planar ambisonic signal (3 channels W, X, Y), in block 460 the encoder reconstructs a quantized 3D rotation matrix from the three quantized Euler angles, and this matrix is ​​then converted into a unit quaternion for interpolation. In a variant where the encoder input is a stereo signal, the encoder uses the quantized 2D rotation representation with a rotation angle in block 460. In the 4-channel embodiment, the rotation matrix is ​​interpolated between the frames t and the plot t - 1, the rotation matrix calculated for the frame tis factored into 2 quaternions (a double quaternion) using Cayley factorization, and the stored double quaternion is used for the previous frame t-1 and noted

[0165] For each subframe, quaternions are interpolated in pairs within each subframe.

[0166] For the left quaternion the block determines the shortest possible path between the two ( Or ) . Depending on the case, the sign of the quaternion in the current frame is reversed. Then the interpolation is calculated for the left quaternion using spherical linear interpolation (SLERP): Or α corresponds to the interpolation factor ( α =1 / K, 2 / K, ... 1) and

[0167] For the right quaternion If there has been an inversion for the left quaternion, then parity must be respected and the sign of the right quaternion forced. This sign constraint is referred to below as the "joint shortest path constraint". The interpolation is then calculated similarly to that of the left quaternion: Or α corresponds to the interpolation factor ( α =1 / K, 2 / K, ... 1) and

[0168] Once the interpolation is calculated for the two quaternions, the 4x4 rotation matrix is ​​calculated (3x3 for planar ambisonics or 2x2 for the stereo case). This conversion to a rotation matrix can be performed using the following pseudocodes: 4D case: for a double quaternion As described previously, we calculate the quaternion and antiquaternion matrices and we calculate the matrix product.

[0169] 3D case: for quaternion q=(w,x,y,z) we obtain the matrix M[i,j], i,j=0...2, of size 3x3 xy = 2 *x*y xz = 2 *x*z yz = 2 * y * z wx = 2 * w * x wy = 2 * w * y wz = 2 * w * z xx = 2 * x * x yy = 2 * y * y zz = 2 * z * z M 0 0 = 1 − yy + zz M 0 1 = xy − wz M 0 2 = xz + wy M 1 0 = xy + wz M 1 1 = 1 − xx + zz M 1 2 = yz − wx M 2 0 = xz − wy M 2 1 = xz + wx M 2 2 = 1 − xx + yy ;

[0170] Finally, the matrices V t interp α (or their transposes) calculated per subframe in interpolation block 460 are then used in transformation block 470 which produces n channels transformed by applying the rotation matrices thus found, to the ambisonic channels which have been pre-treated by block 300.

[0171] We will return to the number below. KThe number of subframes to be determined in block 450 is for the case where this number is adaptive. The final difference between the current frame and the previous frame is measured, or directly from the angular difference of the parameters describing the rotation matrix. In the latter case, the aim is to ensure that the angular variation between successive subframes is imperceptible. Implementing an adaptive number of subframes is primarily advantageous for reducing the average complexity of the encoded data, but if reducing complexity is the primary objective, interpolation with a fixed number of subframes may be preferable.

[0172] The final difference between the frame-corrected rotation matrix t and the frame rotation matrix t- 1 provides a measure of the magnitude of the difference in channel matrixing between the two frames. The larger this difference, the greater the number of subframes required for the interpolation performed in block 460. This difference is measured by summing the absolute values ​​of the cross-correlation matrix between the transformation matrix of the current frame and the previous frame, as follows: Or I n is the identity matrix, V t the vectors specific to the frame with index t, and is a norm of the matrix M which here corresponds to the sum of the absolute values ​​of all the coefficients. Other matrix norms can be used (for example, the Frobenius norm).

[0173] If the two matrices are identical, then this difference is equal to 0. The more dissimilar the matrices are, the larger the value of the difference. δ t is high. Predetermined thresholds can be applied to δ t , Each threshold is associated with a predefined number of interpolations, for example according to the following decision logic: Thresholds: {4.0, 5.0, 6.0, 7.0} Number K subframes for interpolation: {10, 48, 96, 192}

[0174] Thus, only two bits can suffice to encode the four possible values ​​giving the number of subdivisions (subframes).

[0175] The number K The interpolation determined by block 450 is then sent to the interpolation module 460 and in the adaptive case the number of subframes is coded as a binary index which is sent to the multiplexer (block 350).

[0176] Performing the interpolation allows us to apply in fine an optimization of the decorrelation of the input channels before multi-mono encoding. Indeed, the rotation matrices calculated respectively for a previous frame t -1 and a current frame tcan be very different due to this search for decorrelation, but interpolation nevertheless makes it possible to smooth out this difference.

[0177] The interpolation used requires limited computational cost for the encoder and decoder since it is performed in a specific domain (angle in 2D, quaternion in 3D, double quaternion in 4D). This approach is more advantageous than interpolating covariance matrices calculated for PCA / KLT analysis and repeating an EVD (EigenValue Decomposition) type eigenvalue decomposition several times per frame.

[0178] Block 470 then performs the matrixing of the ambisonic channels by subframe using the transformation matrices calculated in block 460. This matrixing is equivalent to calculating by subframe V t interp α T X α , Or X ( α ) corresponds to subblocks of size nx ( L / K ) For α=1 / K, 2 / K, ... 1. The signal contained in these channels is then sent to block 340 for multi-mono encoding.

[0179] We now refer to the figure 5 to describe a decoder in an example of an embodiment of the invention.

[0180] After demultiplexing the binary stream for the current frame t In block 500, the allocation information is decoded (block 510), which allows the received binary stream(s) for each of the n transformed channels.

[0181] Block 520 uses several instances running separately from the core decoding. The core decoding can be of the EVS type, possibly modified to improve its performance. Following a multi-mono approach, each channel is decoded separately. If the previously used encoding was stereo or multichannel, the multi-mono approach can be replaced by multi-stereo or multichannel decoding. The decoded channels are then sent to block 530, which decodes the rotation matrix for the current frame and optionally the number K of subframes to use for interpolation (if the interpolation is adaptive). For each matrix, the 460 interpolation block divides the frame into subframes whose number K can be read in the stream encoded by block 610 ( figure 6 ) and interpolates the rotation matrices, the goal being to find - in the absence of transmission errors - the same matrices as in block 460 of the encoder in order to reverse the transformation which was previously done in block 470.

[0182] Block 530 performs the inverse matrixing of block 470 to reconstruct a decoded signal, as detailed below with reference to the figure 6 This matrixing process is equivalent to calculating by subframe. V t interp α X ^ α , Or X̂ ( α ) corresponds to successive sub-blocks of size nx ( L / K ) For α =1 / K, 2 / K, ... 1.

[0183] Block 530 performs the overall decoding and inverse PCA / KLT synthesis that was carried out by block 310 of the figure 3 The quantization indices of the rotation quantization parameters in the current frame are decoded in block 600. Scalar quantization can be used, and the quantization step is the same for each angle. In the adaptive case, the number of interpolation subframes is decoded (block 610) to retrieve the number K de subframes from the set {10, 48, 96, 192}; in variants where the frame length L is different, this set of values ​​can be adapted. The decoder interpolation is identical to that performed at the encoder (block 460).

[0184] Block 620 performs the inverse matrixing of the ambisonic channels by subframe using the inverses (the transposes in practice) of the transformation matrices calculated in block 460.

[0185] Thus, the invention uses a completely different approach than the MPEG-H coded addition / overlap by relying on a specific representation of the transformation matrices which are restricted to rotation matrices from one frame to another, in the time domain, allowing in particular an interpolation of the transformation matrices, with a matching which ensures consistency in direction (including taking into account the direction by the sign).

[0186] The general approach of the invention is a time-domain ambisonic encoding using PCA, notably with PCA transformation matrices forced to be rotation matrices and interpolated by subframes in an optimized manner (particularly in the quaternion / double quaternion domain) to improve quality. The interpolation step is either fixed or adaptive based on a difference criterion between an intercorrelation matrix and a reference matrix (identity) or between matrices to be interpolated. The quantization of the rotation matrices can be implemented in the domain of generalized Euler angles. However, it may be preferable to quantize the 3- and 4-dimensional matrices in the quaternion and double quaternion domains (respectively), thus remaining within the same domain for both quantization and interpolation.

[0187] In addition, eigenvector alignment is used to avoid click and channel inversion problems from one frame to another.

[0188] Of course, the present invention is not limited to the embodiments described above by way of example and extends to other variants.

[0189] Thus, the preceding description dealt with the cases of four channels.

[0190] However, in some variations, it is also possible to encode more than four channels. The implementation remains identical (in terms of functional blocks) to the case n=4, but the double quaternion interpolation is replaced by the general method described below.

[0191] The transformation matrices at frames t - 1 and t are noted V t - 1 And V t . Interpolation can be performed with a factor α between V t - 1 And V t such as : V t interp α = V t − 1 V t − 1 T V t α

[0192] The term V t − 1 T V t α can be calculated directly by eigenvalue decomposition of V t − 1 T V t Indeed, if V t − 1 T V t = QL Q T we have: V t − 1 T V t α = Q L α Q T .

[0193] It should also be noted that this variant could also replace interpolation by double unitary quaternion (4D case), unitary quaternion (3D case) or angle, however it would be less advantageous because it would require an additional diagonalization step and power calculations, whereas the embodiment described above is more efficient for these 2, 3 or 4 channel cases.

Claims

1. Method for compression coding of N-channel sound signals forming a succession in time of frames (t-1, t) of samples, the method comprising: - forming, from the channels for a current frame (t), a covariance matrix between channels and searching for eigenvectors of the covariance matrix to obtain a matrix of eigenvectors, - comparing the matrix of eigenvectors obtained for the current frame (t) with a rotation matrix obtained for a frame (t-1) preceding the current frame (t), and - permuting columns of the matrix of eigenvectors of the current frame (t) to ensure consistency with the rotation matrix of the previous frame (t-1).- testing the eigenvector matrix to verify that it represents a rotation in a space of dimension N and otherwise correcting the eigenvector matrix until a rotation matrix is ​​obtained, for the current frame (t), and - applying said rotation matrix to the signals of the N channels before encoding said signals, the method further comprising: - cutting the current frame into sub-frames, and - interpolating matrices to be applied to successive sub-frames from the matrix of the previous frame (t-1) to the matrix of the current frame (t).

2. Method according to claim 1, further comprising a step of quantizing the rotation matrix (440), said rotation matrix applied to the signals of the N channels being in a quantized representation.

3. Method according to one of the preceding claims, further comprising a step of quantizing the rotation matrix (440) and a step of interpolating the quantized matrix (460), said rotation matrix applied to the signals of the N channels being in a quantized and interpolated representation.

4. Method according to claim 3, in which the interpolation is carried out in the quaternion domain.

5. Method according to one of the preceding claims, in which said permutation of the columns makes it possible to ensure consistency of axes of the vectors, and the method further comprises - checking, for each eigenvector of the current frame (t), consistency of direction with a column vector of corresponding position of the rotation matrix of the previous frame (t-1), and - in the event of inconsistency, reversing the sign of the elements of this eigenvector in the eigenvector matrix of the current frame (t).

6. Method according to one of the preceding claims, further comprising: - an estimation of the difference between the rotation matrix obtained for the current frame (t) and a rotation matrix obtained for a frame (t-1) preceding the current frame, - depending on the estimated difference, determining whether at least one interpolation is to be carried out between the rotation matrix of the current frame (t) and the rotation matrix of the previous frame (t-1).

7. Method according to claim 6, in which: - depending on the estimated difference, a number of interpolations to be carried out between the rotation matrix of the current frame (t) and the rotation matrix of the previous frame (t-1) is determined, - the current frame is divided into a number of sub-frames corresponding to the number of interpolations to be carried out, and - at least this number of interpolations is coded for transmission via a network.

8. Method according to one of the preceding claims, in which a permutation between columns of the eigenvector matrix inverts the sign of a determinant of the eigenvector matrix and the determinant of a rotation matrix being equal to 1, if the determinant of the eigenvector matrix is ​​equal to -1, the signs of the elements of a chosen column of the eigenvector matrix are inverted, so that the determinant is equal to 1 and thus forms a rotation matrix.

9. Method according to one of the preceding claims, in which the ambisonic representation is of order 1 and the number N of channels is four, and in which the rotation matrix of the current frame is represented by two quaternions.

10. Method according to one of the preceding claims, in which the search for the eigenvectors is carried out by principal component analysis (PCA), or by Karhunen Loeve transform (KLT), in the time domain.

11. Method according to one of the preceding claims, in which a prior step of forecasting the budget for allocating bits per ambisonic channel is implemented and comprises: - for each ambisonic channel, an estimate of current acoustic energy in the channel, - the selection in a memory of a predetermined quality score (MOS), depending on this ambisonic channel and a current bit rate in the network, - the estimation of a weighting to be carried out for the allocation of bits to this channel, by multiplying the selected score by the estimated energy.

12. Method for decoding sound signals forming a succession in time of frames (t-1, t) of samples, in each of N channels, the method comprising: - receiving, for a current frame (t), in addition to the signals from the N channels of this current frame, parameters of a rotation matrix, - constructing an inverse rotation matrix for the current frame, from said parameters, - cutting the current frame into sub-frames, and - interpolating matrices to be applied to successive sub-frames from the matrix of the previous frame ( t -1) up to the matrix of the current frame (t). - apply the inverse interpolated rotation matrices, by subframe, to signals from the N received channels.

13. Coding device comprising a processing circuit for implementing the method according to one of claims 1 to 11.

14. Decoding device comprising a processing circuit for implementing the method according to claim 12.

15. Computer program comprising instructions for implementing the method according to one of claims 1 to 12, when said instructions are executed by a processor of a processing circuit.

Images