Title: spatialized audio encoding with configuration of a decorrelation processing operation

Abstract

The invention relates to a method for encoding audio signals forming in time a succession of frames (t-1, t) of samples, in each of n channels of an ambisonic representation of order higher than 0, the method comprising: - determining, for the current frame to be encoded, the binary value indicating an active or inactive mode of a decorrelation processing operation to be applied to the signals of the current frame and encoding this value into the bitstream; - in the case where the mode is determined to be active, encoding into the bitstream decorrelation-processing information; - generating an output signal to be encoded into the bitstream, depending on the mode determined for the current frame and the mode determined for the preceding frame. The invention also relates to a corresponding decoding method, as well as to encoding and decoding devices implementing the respective encoding and decoding methods.

Description

[0001] DESCRIPTION

[0002] Title: Spatialized audio coding with adaptation of decorrelation processing

[0003] The present invention relates to the coding / decoding of spatialized sound data, in particular in an ambiophonic context (hereinafter also referred to as “ambisonic”).

[0004] The codecs currently used in mobile telephony are mono (a single signal channel for playback on a single speaker). The 3GPP EVS codec (for "Enhanced Voice Services") provides "Super-HD" quality (also called "High Definition Plus" or HD+ voice) with a super-wideband (SWB) audio band for signals sampled at 32 or 48 kHz or fullband (FB) for signals sampled at 48 kHz; the audio bandwidth is 14.4 to 16 kHz in SWB mode (9.6 to 128 kbit / s) and 20 kHz in FB mode (16.4 to 128 kbit / s).

[0005] The next quality evolution in the conversational services offered by operators should be immersive services, using terminals such as smartphones equipped with several microphones or spatial audio conferencing equipment or videoconferencing such as telepresence or 360° video, or even equipment for sharing “live” audio content, with a 3D spatial sound rendering that is much more immersive than a simple 2D stereo reproduction. With the increasingly widespread use of listening on mobile phones with an audio headset and the appearance of advanced audio equipment (accessories such as a 3D microphone, voice assistants with acoustic antennas, virtual reality headsets, etc.) the capture and rendering of spatial sound scenes are now quite common to offer an immersive communication experience.

[0006] In this respect, the future 3GPP “IVAS” standard (for “Immersive Voice And Audio Services”) proposes the extension of the EVS codec to immersive by accepting as codec input format at least the spatialized sound formats listed below (and their combinations):

[0007] - Multichannel format (channel-based in English) of the stereo or 5.1 type, where each channel feeds a loudspeaker (for example L and R in stereo, or L, R, Ls, Rs and C in 5.1);

[0008] - Object-based format, where sound objects are described as an audio signal (usually mono) associated with metadata describing the attributes of this object (position in space, spatial width of the source, etc.),

[0009] - Ambisonic format (scene-based in English) which describes the sound field at a given point, generally captured by a spherical microphone or synthesized in the domain of spherical harmonics.

[0010] The following is typically concerned with the coding of a sound in Ambisonic format, as an example of an embodiment (at least certain aspects presented in connection with the invention below can also be applied to formats other than Ambisonic).

[0011] Ambisonics is a method of recording ("coding" in the acoustic sense) spatialized sound and reproducing ("decoding" in the acoustic sense). An ambisonic microphone (of order 1) comprises at least four capsules (typically cardioid or subcardioid) arranged on a spherical grid, for example the vertices of a regular tetrahedron. The audio channels associated with these capsules are called the "A-format". This format is converted into a "B-format", in which the sound field is decomposed into four components (spherical harmonics) denoted W, X, Y, Z, which correspond to four coincident virtual microphones. The W component corresponds to an omnidirectional capture of the sound field while the X, Y and Z components, more directional, are similar to pressure gradient microphones oriented along the three orthogonal axes of space.An Ambisonics system is flexible in that recording and playback are separate and decoupled. It allows decoding (in the acoustic sense) on any loudspeaker configuration (e.g., binaural, 5.1 surround sound, or 7.1.4 periphery (with elevation). The Ambisonics approach can be generalized to more than four channels in B-format, and this generalized representation is commonly called "HOA" (for "Higher-Order Ambisonics"). Decomposing the sound into more spherical harmonics improves the spatial accuracy of reproduction when rendered on loudspeakers.

[0012] An ambisonic signal of order M includes K=(M+1) 2components and, at order 1 (if M=1), we find the four components, W, X, Y, and Z, which is commonly called FOA (for First-Order Ambisonics). There is also a so-called "planar" variant of ambisonics (W, X, Y) which decomposes the sound defined in a plane which is generally the horizontal plane. In this case, the number of components is K = 2M + 1 channels. First-order ambisonics (4 channels: W, X, Y, Z), planar first-order ambisonics (3 channels: W, X, Y), and higher-order ambisonics are all referred to hereinafter as "ambisonics" indiscriminately to facilitate reading, the treatments presented being applicable regardless of the planar type or not and the number of ambisonic components. From now on, we will call an "ambisonic signal" a B-format signal in a predetermined order with a certain number of ambisonic components.This also includes hybrid cases, where for example at order 2 we only have 8 channels (instead of 9) - more precisely, at order 2 we have the 4 channels of order 1 (W, X, Y, Z) to which we normally add 5 channels (usually denoted R, S, T, U, V), and we can for example ignore one of the higher order channels (for example R). This also includes cases where an ambisonic signal has undergone preprocessing to transform it into preprocessed channels before encoding.

[0013] The signals to be processed by the coder / decoder are presented as successions of blocks of sound samples called “frames” or “sub-frames” below.

[0014] Furthermore, hereinafter, mathematical notations follow the following convention:

[0015] - Scalar: s or N (lower case for variables or upper case for constants)

[0016] - the operator Re(.) denotes the real part of a complex number

[0017] - Vector: u (lowercase, bold)

[0018] - Matrix: A (capital letter, bold)

[0019] A ratings T and A H indicates respectively the transposition and the Hermitian transposition (transposed and conjugate) of A.

[0020] - A one-dimensional discrete-time signal, s(i), defined over a time interval i=0, ..., L- 1 of length L is represented by a row vector

[0021] S=[s(0), ...,s(L-1)]

[0022] We can also write: s = [s0,..., s L-1 ] to avoid the use of parentheses.

[0023] - A multidimensional discrete-time signal, b(i), defined over a time interval i=0, ..., L-1 of length L and with K dimensions is represented by a matrix of size LxK:

[0024] We can also note: B = [B ij ], i=0,...K-1, j=0...L-1, to avoid the use of parentheses.

[0025] Furthermore, we do not recall here the known conventions of the state of the art in Ambisonics concerning the order of Ambisonic components (including ACN for "Ambisonic Channel Number", SID for "Single Index Designation", FuMA for "Furse-Malham") and the normalization of Ambisonic components (SN3D, N3D, maxN). More details can be found for example in the resource available online:

[0026] By convention, the first component of an ambisonic signal generally corresponds to the omnidirectional component W.

[0027] The simplest approach to encoding an Ambisonics signal is to use a mono encoder and apply it separately to each of the individual channels, possibly with different bit allocations for each channel. This approach is referred to here as "multi-mono". The multi-mono approach can be extended to multi-stereo encoding (where pairs of channels are encoded separately by a stereo codec) or more generally to the use of multiple parallel instances of the same core codec. The input signal is divided into channels (one mono channel or multiple channels). These channels are encoded separately according to a predetermined bit distribution and allocation. During decoding, the decoded channels are recombined according to the convention of the input signal.

[0028] The quality of multi-mono or multi-stereo coding varies depending on the coding and decoding core used, and is generally only satisfactory at very high bitrates. For example, in the multi-mono case, EVS coding can be considered quasi-transparent (from a perceptual point of view) at a bitrate of at least 48 kbit / s per channel (mono); thus for an ambisonic signal of order 1 we obtain a minimum bitrate of 4x48 = 192 kbit / s. Since the multi-mono coding approach does not take into account the correlation between channels, it produces spatial distortions with the addition of various artifacts such as the appearance of phantom sound sources, diffuse noises or displacements of the sound source trajectories. Thus, coding an ambisonic signal using this approach causes degradations in spatialization.

[0029] An alternative approach to separate channel coding is given by parametric coding such as DiRAC coding described for example in the article V. Pulkki, Spatial sound reproduction with directional audio coding, Journal of the Audio Engineering Society, vol. 55, no. 6, pp. 503-516, 2007. In this document, a directional analysis of the ambisonic signal is performed by frame and sub-bands to determine source directions (DoA). The DoA are completed by diffuseness parameters, which gives a parametric description of the sound scene. The multi-channel input signal is coded in the form of downmix channels (typically a mono or stereo signal obtained by reduction of multiple captured channels) and spatial metadata (DoA and diffuseness by sub-bands). The invention focuses on another particular approach to ambisonic coding, described in the following publications:

[0030] - P. Mahé, S. Ragot, S. Marchand, "First-order ambisonic coding with quaternion-based interpolation of PCA rotation matrices," Proc. EAA Spatial Audio Signal Processing Symposium, Paris, France, Sept. 2019, pp. 7-12

[0031] - P. Mahé, S. Ragot, S. Marchand, "First-Order Ambisonic Coding with PCA Matrixing and Quaternion-Based Interpolation," Proc. DAFx, Birmingham, UK, Sept. 2019.

[0032] This approach, subsequently called principal component analysis coding or simply PCA coding (for Principal Component Analysis), uses the quantization and interpolation of rotation matrices associated with the eigenvectors of a PCA analysis, as also described in patent application WO2020177981. The strategy of this type of Ambisonic coding is to decorrelate the channels of the Ambisonic signal and then separately code the transformed channels with a core codec (e.g. multi-mono). This strategy makes it possible to limit spatial artifacts in the decoded Ambisonic signal.

[0033] In this approach, for a 1st order ambisonic signal, 4x4 3D rotation matrices (from a PCA / KLT analysis as described for example in the aforementioned patent application) are converted into parameters, for example 6 generalized Euler angles or two unit quaternions, which are encoded.

[0034] Without loss of generality, we retain here more particularly the domain of quaternions which allows to efficiently interpolate the transformation matrices calculated for the PCA / KLT analysis; the transformation matrices being rotation matrices, during decoding, the inverse matrixing operation is carried out simply by transposing the matrix applied to the coding.

[0035] Figure 1 illustrates this coding method in the case where the quaternion representation is used for both coding and interpolation of rotation matrices. The coding takes place in several stages.

[0036] The original multichannel signal A of dimension KxL (i.e. K components of L time or frequency samples) is at input. In block 100 a PCA analysis is carried out, divided into several steps:

[0037] The channel signals (e.g. W, Y, Z, X for the FOA case) are assumed to be in a matrix form A with an nx L matrix (for n ambisonic channels (here 4) and L samples per frame). These channels can optionally be pre-processed, for example, by a high-pass filter.

[0038] A covariance matrix of the multichannel signal A is obtained, for example, as follows: C = AA T to within a normalization factor (in the real case) or

[0039] C = Re(AA H ) to within a normalization factor (in the complex case)

[0040] Temporal smoothing operations of the covariance matrix can be used. In cases of a multichannel signal in the time domain, the covariance can be estimated recursively (sample by sample). Alternatively, the frame can be divided into subframes and a subframe covariance matrix determined, which is then smoothed.

[0041] In particular, we note the diagonal elements of C in the form C ii , they represent the energy of the i-th input channel of the PCA processing.

[0042] We apply a principal component analysis PCA or equivalently a Karhunen-Loeve transform (KLT), with an eigenvalue decomposition of the covariance matrix C, to obtain eigenvalues ​​A and an eigenvector matrix U such that C = UAU T

[0043] The initial eigenvector matrix U, obtained for the current frame t, undergoes signed permutations so that it is as aligned as possible with the matrix of the same nature V of the previous frame t - 1, in order to ensure maximum consistency between the transformation matrices between two frames. We also ensure that the eigenvector matrix of the current frame t, thus corrected by signed permutations, properly represents the application of a rotation.

[0044] In block 110, the new eigenvector matrix V for the current frame t (which is a rotation matrix) is converted into an appropriate domain of quantization parameters. The corresponding eigenvalue matrix is ​​here denoted A = diag(λ 1; ..., λ n ). Here we take the case of a conversion into 2 unit quaternions for a 4x4 matrix; we would have a single unit quaternion for a 3x3 matrix in the planar ambisonic case.

[0045] In dimension 4 (n = 4), a rotation matrix V can be parametrized by the product of two unit quaternions q1 and q2 in matrix form: where the quaternions are with par example :

[0046] And

[0047] Conversely, given a 4x4 rotation matrix, it is possible to find an associated double quaternion (q1, q2) and the corresponding matrices. In other words, we can factorize this matrix into a product of matrices in the form, for example, with the method called "Cayley factorization". This generally involves calculating an intermediate matrix called the "associated matrix" (or "tetragonal transform") and deducing the quaternions up to an indeterminacy on the sign of the two quaternions.

[0048] These parameters q1, q2 are coded using a state-of-the-art coding method (block 120) on a number of bits allocated to parameter quantization. For example, we could use 19 bits for q1 and 18 bits for q2, which gives a budget N Q =37 bits per frame.

[0049] The current frame is divided into subframes, the number of which is assumed to be fixed here. The representation by coded quaternions is interpolated (block 130) by successive subframes of index t' from the end of the previous frame t - 1 to the end of the current frame t, in order to smooth over time the difference between inter-frame matrixing. The quaternions interpolated in each subframe are converted into rotation matrices (block 140) and then applies the resulting decoded and interpolated rotation matrices in each subframe (block 150).

[0050] At the output of block 150, we obtain a matrix representing each of the subframes of the signals of the ambisonic channels to decorrelate these signals and obtain the transformed signal B. A binary allocation to the separate channels is also carried out (block 160) from the overall number of bits from which we subtract the N Q bits used in block 120.

[0051] Figure 2 illustrates the corresponding decoding. The quantization indices of the quantization parameters of the rotation matrix in the current frame are demultiplexed (block 200) and decoded in block 230 according to a decoding method corresponding to the coding (block 120). The transformed channels are also decoded (block 220), from the binary allocation (block 210) identical to the coder (block 160).

[0052] The conversion and interpolation steps (blocks 240, 250) of the decoder are identical to those performed at the encoder (blocks 130 and 140).

[0053] Block 260 applies the inverse matrixing from block 250 to the decoded signals of the ambisonic channels per subframe, recalling that the inverse of a rotation matrix is ​​its transpose. It should be noted that the algorithmic delay linked to the coding-decoding (blocks 170 and 220) must be compensated by adequately storing the inverse matrixing values.

[0054] Ambisonic coding as performed in Figures 1 and 2 assumes that the input channels are (sufficiently) correlated. In particular, it assumes that decorrelation by block 150 provides a coding gain; furthermore, it assumes that the matrixing is stable from one frame to another so as not to generate audio artifacts at the level of the transformed signal B. We also note that the coding of the metadata (block 120) uses a bit rate typically of the order of 2 kbit / s (for example 1.85 kbit / s when N Q =37 bits per 20 ms frame) which is taken from the channel coding budget (blocks 160 and 170).

[0055] However, for some signals such as applause recordings where the sound field is relatively diffuse, the decorrelation gain may be low. For spatially unstable signals, for example percussive sounds whose location alternates rapidly at each frame in the sound space, the PCA analysis (block 100) may lead to a very large variation in the matrixing by In both cases, constant use of metadata to represent the PCA transformation is not very relevant.

[0056] The present invention improves this situation.

[0057] To this end, it proposes a method for coding sound signals forming a succession in time of frames (t-1, t) of samples, in each of n channels in ambisonic representation of order greater than 0, the method comprising:

[0058] - determine for the current frame to be coded, a binary value indicating an active (ON) or inactive (OFF) mode of a decorrelation processing to be applied to the signals of the current frame and code this value in the binary train;

[0059] - in the case where the mode is determined as active, encode decorrelation processing information in the binary stream;

[0060] - generate an output signal to be coded in the binary train, according to the mode determined for the current frame and that of the previous frame.

[0061] Thus, the present invention makes it possible to adapt the use of a decorrelation between the n channels according to the characteristics of the input signal.

[0062] In one embodiment, the determination of the binary value indicating an active or inactive mode is carried out according to at least one signal coding gain criterion before and after decorrelation processing.

[0063] This criterion thus makes it possible to ensure that the decorrelation processing provides sufficient gain to be activated. According to a particular embodiment, the coding gain is defined by the following logarithmic value: with al the energies of the input channels of the decorrelation processing and the eigenvalues ​​of the input channels, the mode being determined as inactive for a predefined gain value G.

[0064] In one embodiment, the determination of the binary value indicating an active or inactive mode is carried out according to an inter-frame distance criterion between rotation matrices applying the decorrelation processing.

[0065] Thus, depending on the value of this distance, the generation of the signal to be coded is adapted to avoid excessive variations in the transformation matrix applying the decorrelation processing.

[0066] According to a particular embodiment, in which the rotation matrices are represented in double quaternion, the inter-frame distance between rotation matrices being expressed from a scalar product between the quaternions in the current frame and those of the previous frame.

[0067] In one embodiment, the determination of the binary value indicating an active or inactive mode is carried out according to a distance criterion between a rotation matrix, applying the decorrelation processing, of the current frame and the identity matrix.

[0068] Thus, here too, depending on the value of this distance, we adapt the generation of the signal to be coded to avoid too strong variations in the transformation matrix applying the decorrelation processing compared to direct coding of the input.

[0069] In a particular embodiment, in which the rotation matrices are represented in double quaternion, the distance between the rotation matrix of the current frame and the identity matrix is ​​expressed in the form of a scalar product between the quaternions at the current frame and unit quaternions.

[0070] The invention applies to a method for decoding sound signals forming a succession in time of frames (t-1, t) of samples, in each of n channels in ambisonic representation of order greater than 0, the method comprising:

[0071] - receive, for a current frame (t), in addition to the signals of the n channels of this current frame, a binary value indicating an active or inactive mode of a decorrelation processing applied to the signals of the current frame;

[0072] - in the case where the mode is determined as active, decoding decorrelation processing information received in the bitstream; - generating an output signal, depending on said mode determined for the current frame and that of the previous frame.

[0073] The decoding method has the same advantages as the corresponding encoding method.

[0074] The present invention also relates to a coding device comprising a processing circuit for implementing the coding method presented previously.

[0075] It also relates to a decoding device comprising a processing circuit for implementing the above decoding method.

[0076] It also relates to a computer program comprising instructions for implementing the above method, when these instructions are executed by a processor of a processing circuit.

[0077] It also aims at a non-transitory memory medium storing the instructions of such a computer program.

[0078] Other advantages and characteristics of the invention will appear on reading the exemplary embodiments presented in the detailed description below, and on examining the appended drawings in which:

[0079] [Fig 1] illustrates an embodiment of an encoder and an encoding method according to a state-of-the-art method;

[0080] [Fig 2] illustrates an embodiment of a decoder and a decoding method according to a state-of-the-art method;

[0081] [Fig 3] illustrates an embodiment of an encoder and an encoding method according to an embodiment of the invention;

[0082] [Fig 4] illustrates an embodiment of a decoder and a decoding method according to an embodiment of the invention;

[0083] [Fig 5] illustrates examples of structural embodiments of an encoder and a decoder within the meaning of the invention.

[0084] Without loss of generality, the input signal is assumed to be an ambisonic signal of order 1 FOA in ACN format and according to SN3D standardization. In variants, the input signal may have undergone pre-processing, to obtain 4 channels derived from an original ambisonic signal (FOA). Figure 3 illustrates a coding method according to the invention where the decorrelation by PCA is applied adaptively (PCA active or not) in each frame assumed here to be 20 ms long (for example L = 960 samples at 48 kHz). It is assumed that R bits are allocated to the current frame for coding, for example at 256 kbit / s we have R = 5120 bits. In variants, this budget R may be reduced according to bits already used by possible pre-processing operations on the signal before coding by PCA.

[0085] The adaptation decision is given by block 300 which determines the value of an indication of activation or not of a decorrelation treatment (PCA) to be applied to the current frame, this indication corresponds to mode =ON (PCA active or activated) or OFF (PCA inactive or deactivated). The decision criteria will be described later.

[0086] Thus, a binary value indicating an active (ON) or inactive (OFF) mode of a PCA type decorrelation processing is determined by block 300. The operation of the encoder depends on the mode in the current frame of index t (mode) and that of the previous frame of index t- 1 (prev_modë). Without loss of generality, it is assumed that at the start of the encoder, the initial state of the previous frame is prev_mode=OFF.

[0087] There are 4 possible combinations:

[0088] • If mode = ON and prev_mode = ON, we find an operation identical to that of figure 1, that is to say that blocks 100, 110, 120, 320, 140, 150 and 170 of figure 3 respectively apply the same processing as blocks 100, 110, 120, 130, 140, 150 and 170 described in figure 1. Branch 2 for which the input signal undergoes a transformation is selected by block 330. In allocation block 340, we take into account the fact that PCA coding uses 1 bit to indicate the mode (ON) of the current frame, N Q bits to encode metadata and R-1-N Q bits to encode the channels (block 170).

[0089] • If mode = OFF and prev_mode = OFF, the decorrelation by PCA is “short-circuited”, branch 1 is selected by the selection block 330. The signal to be coded B is identical to the input signal A. The allocation block 340 takes into account the fact that 1 bit is used to indicate the mode (OFF) of the current frame and R1 bits to code the channels (block 170). By default, the eigenvectors are fixed V to the identity matrix and the quaternions encoded at (1,0,0,0) in the current frame - to the next frame, these values ​​will be used to set the state of the previous frame.

[0090] • If mode = OFF and prev_mode = ON, we fix the eigenvectors to the identity matrix and the quaternions coded at (1,0,0,0), an interpolation is carried out (block 320) between these values ​​and the values ​​of the previous frame. The details of this interpolation are specified below. The selection module activates branch 2, in which the input signals are transformed in 150 by a matrix resulting from the interpolation carried out in 320. It is not necessary here to code information on the transformation matrix of the current frame since the indication has a negative value for this current frame. The module for inserting this data coded in 310 is therefore disconnected. The allocation block 340 takes into account the fact that 1 bit is used to indicate the mode of the current frame and R-1 bits to code the channels (block 170).

[0091] • If mode = ON and prev_mode = OFF, an interpolation is performed (block 320) between the current values ​​of the coded quaternions and the (default) values ​​at (1.0, 0.0) of the previous frame. The details of this interpolation are specified below. The selection module activates branch 2, in which the input signals are transformed in 150 by a matrix resulting from the interpolation carried out in 320. The module for inserting the transformation data of the current frame coded, in 310 is connected here because a PCA transformation and therefore a decorrelation treatment is indeed applied for this current frame. These data are coded on N Q bits. Allocation block 340 takes into account the fact that 1 bit is used to indicate the mode of the current frame and R-1-N Q bits to encode the channels (block 170).

[0092] After processing the previous frame, the eigenvectors V and the coded quaternions to the current frame t are stored to set the new state of the previous frame t-1 before processing the next frame. The mode decision (mode) of the current frame t is also stored to set the new state (prev_mode) of the previous frame t-1 before processing the next frame.

[0093] The multiplexing module 350 thus inserts the coded data into the bit stream according to the allocation defined in block 340 and as a function of the activation indication determined for the current frame and for the previous frame.

[0094] Implementations of the interpolation are now described (block 320). This block relies on the coded quaternions a current frame t and the coded quaternions and to the previous frame t-1. Interpolation can be implemented according to the pseudo following code where we use N subframes, for example N=40:

[0095] - Determination of the shortest path for quaternions:

[0096] - Subframe interpolation of quaternions over N subframes using the NLERP method (for Normalized linear interpolation):

[0097] For t' = 0 to N-1:

[0098] Where norm is the normalization operation (to the unit norm) of quaternion which corresponds to a normalization on the unit sphere in dimension 4. In variants the definition of α(t') can be modified, for example by taking α(t') = t' / (N-1). The use of the function 1-0.5 (1-cos(x)) allows here to ensure a slower interpolation on the ends.

[0099] Alternatively, other quaternion interpolations (e.g., SLERP for Spherical Linear Interpolation or others) are possible. The NLERP method is used here because it is less complex and more stable for limited numerical precision.

[0100] - Conversion of (which corresponds to block 140 and is known in the state of the art)

[0101] Thus this interpolation leads to piecewise constant matrix values ​​with a regular division into N sub-frames.

[0102] In variants, other interpolation methods will be possible. For example, it will be possible to apply sample-by-sample interpolation at the beginning of the frame (on the first N samples) before keeping a constant value on the rest of the frame (i.e. LN samples); in this case the index t' above corresponds to the index of the sample, the sub-frame comprises a single sample.

[0103] In variants other realizations of the interpolation are possible with a different division into sub-frames.

[0104] Examples of embodiments of the decision block 300 are now described.

[0105] In an exemplary embodiment, the determination of the indication of activation of the decorrelation processing uses several criteria with threshold decisions: 1. PCA coding gain

[0106] 2. Inter-frame distance between rotation matrices (which can be seen as a measure of spatial stability)

[0107] 3. Distance between the rotation matrix in the current frame and an identity matrix The motivations for the different criteria are respectively as follows:

[0108] 1. Ensure that decorrelation by PCA provides sufficient gain to justify its activation compared to direct coding

[0109] 2. Ensure that the interpolation between the rotation matrix in the current frame and that of the previous frame does not cause very strong variations in the matrixing over time

[0110] 3. Ensure that the interpolation between the rotation matrix in the current frame and a possible deactivation of the PCA does not cause very strong variations in the matrixing over time.

[0111] In a possible embodiment, another criterion may be based on the correlation matrix of the input signals, and its values ​​outside the diagonal values. The correlation matrix is ​​equivalent to the covariance matrix, except that the ambisonic components are respectively normalized by their standard deviation before calculating the cross-correlation. The criterion may thus be defined independently of the signal of the input signal, with a predetermined threshold applied for example to the maximum or average (in absolute value) off-diagonal value in the correlation matrix.

[0112] This allows to verify that there is minimal correlation between the input signals and that decorrelation processing is useful.

[0113] Examples of how these decision criteria are implemented are given below.

[0114] Here we take up the definition of the coding gain for a PCA / KLT transformation in the case of a Gaussian source with n channels:

[0115] This corresponds to the ratio between the arithmetic mean and the geometric mean of the variances (energies) of the components to be coded (in the Gaussian case).

[0116] According to the invention, we rather take the ratio of the geometric means between the energies C ii = input channels A to the numerator and the eigenvalues ​​λ i in the denominator. The gain expressed in logarithmic value becomes:

[0117] The eigenvalues ​​are assumed to be in descending order and positive. The term ε is for example set to ε = 10 -8 to condition the calculation of the logarithm.

[0118] The gain G actually corresponds to the sum (in the logarithmic domain) of the coding (or decorrelation) gains between the individual (separate) channels taken before and after PCA.

[0119] In variants, the eigenvalues ​​can also be normalized, in which case the normalization factor is also applied to the energy values

[0120] We now define an example of a distance criterion between matrices. In the preferred embodiment, we rely on the double quaternion representation, and we determine the angular distance by a scalar product between the quaternions at the current frame t and those of the previous frame t-1:

[0121] P1 = q1(t - 1). q1(t)

[0122] P2 = q2(t — 1). q2(t)

[0123] The distance between the rotation matrices associated with frames t and t-1 is evaluated as: minP=min(Pl, P2)

[0124] Similarly, we can define the distance between the rotation matrix at frame t and an identity matrix (represented by the unit quaternions q1= q2= (1,0, 0,0)) as:

[0125] (1,0,0,0). q1(t) = a1

[0126] (l,0,0,0). q2(t) = a2

[0127] And minP2 = min(a 1; a2)

[0128] Then the decision (block 300) can be made in an example as follows: when the activation indication is by default positive, that is to say such that mode = ON (active mode), the indication switches to negative mode such that mode = OFF (inactive mode) if the coding gain G is less than a predefined threshold, for example 6 or if the inter-frame distance between rotation matrix is ​​less than a threshold for example 0.8 or if the distance between the rotation matrix of the current frame and an identity matrix is ​​less than a threshold, for example 0.

[0129] We then have:

[0130] If G < 6 mode = OFF If minP < 0.8 mode = OFF

[0131] If minP2 < 0 mode = OFF

[0132] In variants, the threshold values ​​(respectively 6, 0.8, 0) can be different. In variants the activation indication is by default negative, i.e. such that mode = OFF, and the indication switches to positive mode such as mode = ON if all the criteria verify the conditions opposite to those defined above. This therefore amounts to simply reversing the decision logic for the same result.

[0133] In other variants, at least one of the three criteria defined according to the invention is used, the others may not be used or replaced by other criteria.

[0134] Variants of the decision criteria are now described.

[0135] Alternatively, other definitions of the criteria could be taken, for example the coding gain could be:

[0136] Where the constant term logQ) can be omitted

[0137] Alternatively, other distance measures between rotation matrices may be defined, for example a Frobenius distance or some other distance between rotation matrices, such as the Frobenius norm of where I is the n-dimensional identity matrix

[0138] In the case of a criterion based on the correlation matrix of the input signals, the correlation matrix corresponds to the covariance matrix applied to normalized components of the signals A. Such a covariance matrix, as described with reference to Figure 1, is obtained, for example, as follows:

[0139] C = AA T within a normalization factor (in the real case)

[0140] OR

[0141] C = Re(AA H) up to a normalization factor (in the complex case). In particular, we denote the diagonal elements of C in the form C ii , they represent the energy of the i-th input channel.

[0142] The elements of the covariance matrix considered here are the terms C ij , etc ji of the matrix with i #j. The maximum value of these terms is determined and compared to a threshold, for example of value 0.1. In the case where the value is greater than this threshold, then the mode of the current frame is determined as active, ON mode. Conversely, the mode is determined as inactive, OFF mode.

[0143] In variants, the decision block may perform a "closed loop" decision, this amounts to applying PCA processing for blocks 100 to 150 in order to obtain an initial version of the transformed signal B before confirming that the decision mode in the current frame of index t is ON. In this case, the eigenvalues ​​can be replaced by the respective energies of the channels of the initial version B. If the decision is finally OFF, it is then necessary to replace the initial version of the transformed signal B.

[0144] In variants, where closed-loop decision is used, additional activation decision criteria may be added, such as detecting the maximum absolute value in each individual channel before and after PCA (in A and B), if this absolute value (in the current frame) in one of the channels of B exceeds that of the corresponding channel in the input signal A the mode is set to OFF in the current frame.

[0145] Figure 4 illustrates a decoder implementing the decoding method according to one embodiment of the invention.

[0146] The binary train is demultiplexed at 400 and the decoder 220 receives the channels of the multichannel signal to be decoded according to a binary allocation determined at 420.

[0147] The module 410 receives the indication of activation of a decorrelation processing for the current frame and applies the decoding and transformation processing adapted to this indication, in the same way as those carried out during coding.

[0148] Depending on the value of the indication of the previous frame, we distinguish the following 4 combinations:

[0149] • If mode = ON and prev_mode = ON, we find an operation identical to that of figure 2, that is to say that the blocks 220, 230, 430, 250 and 260 of figure 4 respectively apply the same processing as the blocks 220, 230, 240, 250 and 260 described in figure 2. Branch 2 for which the decoded signal undergoes a transformation by block 260 is selected by block 440. The decoding module of the transformation information received in the bitstream is implemented by the connection block 430. The allocation block 420 uses the R- 1-NQ bits to decode the channels in 220. By default, the eigenvectors V are fixed to the identity matrix and the quaternions coded to (1,0, 0,0) in the current frame - in the next frame, these values ​​will be used to set the state of the previous frame.

[0150] • If mode = OFF and prev_mode = OFF, the decorrelation by PCA is “short-circuited”, branch 1 is selected by the selection block 440. The decoded signal is identical to the decoded output signal . Allocation block 420 uses the R-1 bits to decode the channels into 220.

[0151] • If mode = OFF and prev_mode = ON, we fix the eigenvectors to the identity matrix and the quaternions coded at (1,0, 0,0) in the current frame, we perform a interpolation (block 430) between these values ​​and the values ​​of the previous frame, in the same way as block 320. The selection module 440 activates branch 2, in which the decoded signals are transformed into 260 by a matrix resulting from the interpolation carried out in 430. It is not necessary here to decode information on the transformation matrix of the current frame since the indication has a negative value for this current frame. The decoding module of this data is therefore disconnected by block 430. The allocation block 420 uses the R-1 bits to decode the channels into 220.

[0152] • If mode = ON and prev_mode = OFF, an interpolation (block 430) is carried out between the current values ​​of the coded quaternions and the (default) values ​​of the previous frame in a manner identical to block 320. The selection module 440 activates branch 2, in which the decoded signals are transformed in 260 by a matrix resulting from the interpolation carried out in 430. The decoding module 230 of the coded transformation data of the current frame is here connected by the module 430, because a PCA transformation and therefore a decorrelation processing is indeed applied for this current frame. The allocation block 420 uses the R-1-N Q bits to decode channels in 220.

[0153] Note that the algorithmic delay linked to the coding-decoding (blocks 170 and 220) must be compensated by memorizing the inverse matrix values ​​V for each subframe in the current frame as well as in the previous frame. For example, when the coding-decoding (blocks 170-220) is a multi-mono EVS coding, there will typically be a delay of 12 ms to compensate. If the interpolation uses N=40 subframes in a 20 ms frame (i.e. 0.5 ms subframes), it will therefore be necessary to use a memory of 40+24=64 matrix values ​​(4x4 matrices) and block 260 will apply the matrix shifted in time by 24 subframes in the past. At the end of the processing of the previous frame, the eigenvectors and coded quaternions to the current frame t are memorized to define the new state of the previous frame t-1 before processing the next frame. The mode decision (mode) of the current frame t is also memorized to define the new state (prev_modë) of the previous frame t-1 before processing the next frame.

[0154] Figure 5 illustrates a DCOD coding device and a DDEC decoding device, within the meaning of the invention, these devices being dual to each other (in the sense of "reversible") and connected to each other by a RES communication network.

[0155] The DCOD coding device comprises a processing circuit typically including:

[0156] - a memory MEM1 for storing instruction data of a computer program within the meaning of the invention (these instructions can be distributed between the coder DCOD and the decoder DDEC);

[0157] - an INT1 interface for receiving ambisonic signals distributed over different channels (for example four channels W, Y, Z, X in order 1) with a view to their compression coding within the meaning of the invention;

[0158] - a processor PROC1 for receiving these signals and processing them by executing the computer program instructions stored in the memory MEM1, with a view to encoding them; and

[0159] - a COM 1 communication interface for transmitting coded signals via the network.

[0160] The DDEC decoding device has its own processing circuitry, typically including:

[0161] - a memory MEM2 for storing instruction data of a computer program within the meaning of the invention (these instructions can be distributed between the coder DCOD and the decoder DDEC as indicated previously);

[0162] - a COM2 interface for receiving coded signals from the RES network with a view to their compression decoding within the meaning of the invention;

[0163] - a PROC2 processor for processing these signals by executing the computer program instructions stored in the MEM2 memory, with a view to decoding them; and

[0164] - an output interface INT2 for delivering the decoded signals in the form of ambisonic channels W', Y', Z', X', for example for their restitution. Of course, this figure 5 illustrates an example of a structural embodiment of a codec (coder or decoder) within the meaning of the invention. Figures 3 to 4 describe in detail rather functional embodiments of these codecs.

Claims

CLAIMS A method for encoding sound signals forming a succession in time of frames (t-1, t) of samples, in each of n channels in ambisonic representation of order greater than 0, the method comprising: - determine (300) for the current frame to be coded, a binary value indicating an active (ON) or inactive (OFF) mode of a decorrelation treatment to be applied to the signals of the current frame and encode this value in the binary stream; - in the case where the mode is determined to be active, encode (310) in the binary stream of decorrelation processing information; - generate (330) an output signal to be encoded in the binary stream, according to the mode determined for the current frame and that of the previous frame. Method according to claim 1, wherein the determination of the binary value indicating an active or inactive mode is carried out according to at least one criterion of signal coding gain before and after decorrelation processing. Method according to claim 2, wherein the coding gain is defined by the following logarithmic value: with the energies of the input channels of the decorrelation treatment and The eigenvalues ​​of the input channels, the mode being determined as inactive for a predefined gain value G. A method according to claim 1, wherein the determination of the binary value indicating an active or inactive mode is performed according to an inter-frame distance criterion between rotation matrices applying the decorrelation treatment. A method according to claim 4, wherein the rotation matrices are represented as double quaternions, the inter-frame distance between rotation matrices being expressed from a dot product between the quaternions of the current frame and those of the previous frame. A method according to claim 1, wherein the determination of the binary value indicating an active or inactive mode is performed according to a distance criterion between a rotation matrix, applying the decorrelation treatment, of the current frame and the identity matrix.A method according to claim 6, wherein the rotation matrices are represented in double quaternion, the distance between the rotation matrix of the. The current frame and the identity matrix are expressed as a scalar product between the quaternions in the current frame and unit quaternions. A method for decoding sound signals forming a succession in time of frames (t-1, t) of samples, in each of n channels in ambisonic representation of order greater than 0, the method comprising: - receive (410), for a current frame (t), in addition to the signals of the n channels of this current frame, a binary value indicating an active (ON) or inactive (OFF) mode of a decorrelation processing applied to the signals of the current frame; - in the case where the mode is determined to be active, decode (430) decorrelation processing information received in the binary stream; - generate (440) an output signal, according to said mode determined for the current frame and that of the previous frame. Encoding device comprising a processing circuit for implementing the steps of the encoding process according to any one of claims 1 to 7. Decoding device comprising a processing circuit for implementing the steps of the decoding process according to claim 8. Storage medium, readable by a processor, storing a computer program comprising instructions for executing the encoding process according to any one of claims 1 to 7 or the decoding process according to claim 8.

Images