Spatial audio coding with decorrelation processing operation configuration

Abstract

The invention relates to a method for encoding audio signals that temporally form a series of sample frames (t-1, t) in each of n channels of an ambisonics representation with an order higher than 0, the method comprising:-for a current frame to be encoded,-encoding the current frame (t-1, t) in a sequence of sample frames (t-1, t) in each of the n channels of the ambisonics representation with an order higher than 0; determining a binary value indicating an active or inactive mode of a decorrelation processing operation to be applied to the signal of the current frame, and encoding the value into a bitstream; encoding decorrelation processing information into the bitstream in the event that the mode is determined to be active; -generating an output signal to encode it into the bitstream according to the mode determined for the current frame and the mode determined for the previous frame. The invention also relates to a corresponding decoding method, and to an encoding device and a decoding device implementing the corresponding encoding method and decoding method.

Description

Technical Field

[0001] The present invention relates to encoding / decoding of spatialized sound data, in particular in the context of ambisonics (hereinafter also denoted "ambisonic"). Background Art

[0002] The coders / decoders (hereinafter referred to as "codecs") currently used in mobile phones are monophonic (single signal channel for presentation on a single loudspeaker). The 3GPP EVS (standing for "Enhanced Voice Services") codec allows the provision of "Ultra-HD" (also known as "High Definition Plus" or HD+ Voice) sound quality, wherein a SWB (standing for "Ultra-Wideband") audio band is provided for signals sampled at 32 kHz or 48 kHz or a FB (standing for "Full Band") for signals sampled at 48 kHz; the audio bandwidth ranges from 14.4 kHz to 16 kHz in SWB mode (9.6 kbit / s to 128 kbit / s) and is above 20 kHz in FB mode (16.4 kbit / s to 128 kbit / s).

[0003] The next quality evolution of conversational services provided by operators should consist of immersive services that use terminals such as smartphones equipped with multiple microphones, or spatialized audio or video conferencing equipment of the telepresence or 360° video type, or even "real-time" audio content sharing equipment with 3D spatialized sound rendering that is more immersive than simple 2D stereo rendering. With the increasing use of audio headsets on mobile phones and the emergence of advanced audio equipment such as 3D microphones, voice assistants with acoustic antennas, virtual reality headsets and other accessories, it is now common enough to capture and render spatialized sound scenes to provide an immersive communication experience.

[0004] In this regard, the future standard 3GPP "IVAS" (standing for "Immersive Voice and Audio Services") includes extending the EVS codec to be immersive by accepting at least the following listed spatialized sound formats (and combinations thereof) as input formats to the codec:

[0005] - channel-based formats of the stereo or 5.1 type, where each channel supplies one speaker (e.g. L and R in stereo or L, R, Ls, Rs and C in 5.1);

[0006] - object-based formats, where a sound object is described as an audio signal (usually mono) associated with metadata describing the properties of that object (position in space, source spatial width, etc.),

[0007] - A scene-based format that describes the sound field at a given point, typically captured by a spherical microphone or synthesized in the spherical harmonics domain.

[0008] By way of example embodiments, the focus hereinafter is generally on encoding sound in a scene-based (or ambisonics) format (wherein at least some aspects presented hereinafter with respect to the invention may also be applicable to formats other than scene-based formats).

[0009] Ambisonics is a method of recording ("encoding" in the acoustic sense) spatialized sound and reproducing ("decoding" in the acoustic sense). An Ambisonics microphone (1st order) comprises at least four capsules (typically cardioid or subcardioid) arranged on a spherical grid (e.g., the vertices of a regular tetrahedron). The audio channels associated with these capsules are referred to as "A-format". This format is converted to "B-format", in which the sound field is decomposed into four components (spherical harmonics) represented by W, X, Y, Z, which correspond to four coincident virtual microphones. Component W corresponds to omnidirectional capture of the sound field, while the more directional components X, Y and Z can be considered as pressure gradient microphones oriented along three spatial orthogonal axes. The Ambisonics system is a flexible system in the sense that recording and rendering are separate and distinct. It allows (acoustically) decoding of any given configuration of speakers (e.g., binaural, 5.1-type "surround" sound, or 7.1.4-type omnidirectional sound (with elevation angles)). The Ambisonics approach can be generalized to more than four B-format channels, and this generalized representation is often referred to as "HOA" (standing for "Higher Order Ambisonics"). Decomposing the sound into more spherical harmonics improves the spatial accuracy when rendered on loudspeakers.

[0010] An M-order Ambisonics signal comprises K=(M+1)2 components, and at 1st order (if M=1), usually referred to as FOA (standing for first-order Ambisonics), four components W, X, Y and Z are recovered. There is also an Ambisonics variant called "planar" (W, X, Y), which decomposes the sound defined on a plane, usually a horizontal plane. In this case, the number of components is K=2M+1 channels. For ease of reading, 1st order Ambisonics (4 channels: W, X, Y, Z), 1st order planar Ambisonics (3 channels: W, X, Y), and higher order Ambisonics are hereinafter referred to as "Ambisonics", the processing operations presented being applicable regardless of the type, planar or otherwise, and the number of Ambisonics components.

[0011] In the following, an "Ambisonics signal" will refer to a B-format signal having a predetermined order of a certain number of Ambisonics components. This also includes the hybrid case where, for example, at 2nd order, there are only 8 channels (instead of 9) - more precisely, at 2nd order, there are 4 1st order channels (W, X, Y, Z), usually plus 5 channels (usually denoted R, S, T, U, V), and one of the higher order channels (e.g. R) may for example be ignored. This also includes the case where the Ambisonics signal has been pre-processed in order to transform it into pre-processed channels before encoding.

[0012] The signal to be processed by the encoder / decoder is in the form of consecutive blocks of sound samples, hereinafter referred to as "frames" or "sub-frames".

[0013] In addition, in the following text, mathematical symbols follow the following conventions:

[0014] - Scalar: s or N (lowercase for variables or uppercase for constants)

[0015] -The operator Re(.) represents the real part of a complex number

[0016] - Vector: u (lowercase, bold)

[0017] -Matrix: A (uppercase, bold)

[0018] symbol and Separate instructions The transpose and Hermitian transpose (transpose and conjugate) of .

[0019] The one-dimensional discrete-time signal s(i) defined in the time interval i=0,…,L-1 of length L is represented by the following row vector

[0020]

[0021] This can also be written as: to avoid using parentheses.

[0022] - a multidimensional discrete-time signal defined in a time interval i=0,…,L-1 of length L and having K dimensions It is represented by the following matrix of size L×K:

[0023]

[0024] This can also be written as: to avoid using parentheses.

[0025] Furthermore, conventions known from prior art Ambisonics technology regarding the order of Ambisonics components, including ACN (standing for "Ambisonics Channel Number"), SID (standing for "Single Index Designation"), FuMA (standing for "Furse-Malham"), and the normalization of Ambisonics components (SN3D, N3D, maxN) are not mentioned here. More details can be found in resources available online, for example:

[0026] https: / / en.wikipedia.org / wiki / Ambisonic_data_exchange_formats

[0027] By convention, the first component of an Hi-Fi stereo signal usually corresponds to the omnidirectional component W.

[0028] The simplest method for encoding a high-fidelity stereo signal is to use a mono encoder and apply the mono encoder to each of the individual channels individually, while potentially allocating different bits according to the channels. This method is referred to as "multi-mono" here. The multi-mono method can be extended to multi-stereo encoding (wherein, channel pairs are encoded individually by a stereo codec) or more generally to multiple similar instances using the same core codec. The input signal is divided into channels (one mono or several channels). These channels are individually encoded according to a predetermined distribution and binary allocation. During decoding, the decoded channels are reassembled according to the convention of the input signal.

[0029] The quality of multi-mono or multi-stereo coding varies depending on the core encoding and decoding used and is usually satisfactory only at very high rates. For example, in the case of multi-mono, EVS coding can be judged to be quasi-transparent (from a perceptual point of view) at a rate of at least 48 kbit / s per channel (mono); thus, for a 1st-order high-fidelity stereo signal, the minimum rate is 4×48=192 kbit / s. Since the multi-mono coding method does not take into account the inter-channel correlation, this method produces spatial distortions and various artifacts, such as the appearance of phantom sources, diffuse noise or source path displacements. Therefore, encoding of high-fidelity stereo signals according to this method leads to a degradation of spatialization.

[0030] An alternative to the individual coding of the channels is parametric coding, such as DiRAC coding, which is described, for example, in the following article: V. Pulkki, Spatial sound reproduction with directional audio coding, Journal of the Audio Engineering Society, Vol. 55, No. 6, pp. 503-516, 2007. In this document, a directional analysis of the high-fidelity stereo signal is performed per frame and subband to determine the source direction (DoA). The DoA is completed by a "diffuseness" parameter, which gives a parametric description of the sound scene. The multichannel input signal is encoded in the form of a downmix channel (typically a mono or stereo signal obtained by reducing a number of pickup channels) and spatial metadata (DoA and "diffuseness" for each subband).

[0031] The invention also relates to another specific high-fidelity stereo coding method described in the following publication:

[0032] - P. Mahé, S. Ragot, S. Marchand, "First-order ambisonic coding with quaternion-based interpolation of PCA rotation matrices," Proc. EAA Spatial Audio Signal Processing Symposium, Paris, France, September 2019, pp. 7-12

[0033] - P. Mahé, S. Ragot, S. Marchand, "First-Order Ambisonic Coding with PCA Matrixing and Quaternion-Based Interpolation", Proc. DAFx [Proceedings of the Digital Audio Effects], Birmingham, UK, September 2019.

[0034] This method (hereinafter referred to as principal component analysis coding or PCA coding for short) uses quantization and interpolation of rotation matrices associated with the eigenvectors of the PCA analysis, as also described in patent application WO 2020177981. The strategy of this type of Ambisonics coding is to decorrelate the channels of the Ambisonics signal and then encode these transformed channels separately using a core (e.g. multi-mono) codec. This strategy allows limiting spatial artifacts in the decoded Ambisonics signal.

[0035] In this method, for a 1st order ambisonics signal, a rotation matrix of size 4x4 in 3D (from a PCA / KLT analysis, as described, for example, in the above patent application) is converted into encoded parameters, such as 6 generalized Euler angles or two unit quaternions.

[0036] Without loss of generality, the domain of quaternions is more specifically retained here, which allows efficient interpolation of the transformation matrix calculated for PCA / KLT analysis; since the transformation matrix is ​​a rotation matrix, when decoding, the inverse matrixization operation is simply performed by transposing the matrix applied when encoding.

[0037] Figure 1 shows this encoding method when quaternions are used for both encoding and interpolation of the rotation matrix. The encoding is performed in several steps.

[0038] The original multi-channel signal of size K×L (i.e. K components of L time or frequency samples) Located at the input. In block 100, the PCA analysis is performed in several steps:

[0039] Assume that the channels (e.g. W, Y, Z, X for the case of FOA) are signaled in a matrix with n×L matrix (n ambisonics channels (here 4) and L samples per frame) Optionally, the channels may be pre-processed, for example by a high-pass filter.

[0040] Get multi-channel signal The covariance matrix of is, for example, as follows:

[0041] to within the normalization factor (in the case of real numbers)

[0042] or

[0043] to within the normalization factor (in the case of complex numbers)

[0044] An operation for temporal smoothing of the covariance matrix may be used. In the case of a multi-channel signal in the time domain, the covariance may be estimated in a recursive manner (sample by sample). A frame may also be divided into subframes and a covariance matrix is ​​determined per subframe, which is then smoothed.

[0045] The diagonal elements of C ii It is expressed in the form of, which represents the energy of the i-th input channel processed by PCA

[0046] - Apply principal component analysis PCA, or equivalently apply the Karhunen-Loeve transform (KLT), where the covariance matrix Decompose into eigenvalues ​​so as to obtain the eigenvalues and the eigenvector matrix Make

[0047] -The initial eigenvalue matrix obtained for the current frame t Go through the signed permutations to make it as consistent as possible with the properties The same matrix as the previous frame t-1 is aligned to ensure maximum coherence between the transformation matrices between the two frames. It is also ensured that the eigenvalue matrix of the current frame t, thus corrected by the signed permutation, does represent the application of the rotation.

[0048] In block 110, the new eigenvalue matrix of the current frame t is (which is a rotation matrix) is converted into an appropriate quantization parameter domain. Here, the corresponding eigenvalue matrix is ​​expressed as Here, the case of conversion to 2 unit quaternions for a 4x4 matrix is ​​considered; in the case of planar ambisonics, there will be a single unit quaternion for a 3x3 matrix.

[0049] In the case of dimension 4 (n=4), the rotation matrix Can be obtained through two unit quaternions and The product of is parameterized in matrix form:

[0050]

[0051] Among them, the quaternion is and Among them, for example:

[0052]

[0053] and

[0054]

[0055] Conversely, given a 4×4 rotation matrix, one can find the associated dual quaternion And the corresponding matrix. In other words, this matrix can be factored, for example using a method known as "Caley factorization," into a form This usually involves computing an intermediate matrix called the "association matrix" (or "quad transform") and inferring the quaternion from it, although the signs of the two quaternions are not determined.

[0056] According to the prior art encoding method (block 120), these parameters are quantized by a number of bits allocated to the parameters. For example, for 19 bits can be used, and for Using 18 bits, this gives N per frame. Q = 37 bits of budget.

[0057] The current frame is divided into subframes, assuming here that the number of subframes is fixed. The representation of the encoded quaternion is interpolated (block 130) from the end of the previous frame t-1 to the end of the current frame t by consecutive subframes with index t', so as to smooth the difference between the matrixing between frames in time. The quaternion interpolated in each subframe is converted into a rotation matrix (Block 140), and then apply the resulting rotation matrix decoded and interpolated within each subframe (Block 150).

[0058] At the output of block 150, a matrix is ​​obtained for each subframe representing the signals of the high-fidelity stereo channels, used to decorrelate these signals and obtain the transformed signal It is also based on subtracting the N used in block 120 from Q bits to perform binary allocation to the individual channels (block 160).

[0059] The corresponding decoding is shown in Figure 2. The quantization indices of the quantization parameters of the rotation matrix in the current frame are demultiplexed (block 200) and decoded in block 230 according to the decoding method corresponding to the encoding (block 120). The transformed channels are also decoded (block 220) based on the same binary allocation (block 210) as the encoder (block 160).

[0060] The conversion and interpolation steps at the decoder (blocks 240, 250) are identical to those performed at the encoder (blocks 130 and 140).

[0061] Block 260 applies the inverse matrixing from block 250 to the decoded signals of the high-fidelity stereo channels per subframe, recalling that the inverse of a rotation matrix is ​​its transposed matrix. It should be noted that the algorithmic delays associated with encoding-decoding (blocks 170 and 220) must be compensated by storing the inverse matrixing values ​​in memory in an appropriate manner.

[0062] The high-fidelity stereo coding as implemented in Figures 1 and 2 assumes that the input channels are (sufficiently) correlated. In particular, it is assumed that the decorrelation of block 150 provides coding gain; furthermore, it is assumed that the matrixing is stable from one frame to another, so that no distortions are caused in the transformed signal. It should also be noted that the encoding of the metadata (block 120) uses a rate of typically about 2 kbit / s (e.g., when N is 20 ms per frame). Q =37 bits, which is 1.85 kbit / s), which is taken from the coding budget of the channel (blocks 160 and 170).

[0063] However, for some signals, such as a recording of applause where the sound field is relatively diffuse, the decorrelation gain may be low. For spatially unstable signals, such as percussion sounds whose localization in each frame in the sound space rapidly alternates, the PCA analysis (block 100) may result in matrixing In both cases, the constant use of metadata to indicate the PCA transform is not very relevant.

[0064] The present invention will improve this situation. Summary of the invention

[0065] To this end, the invention provides a method for encoding audio signals forming a series of sample frames (t-1, t) in time in each of n channels in an hi-fi stereo representation of order greater than 0, the method comprising:

[0066] - determining, for a current frame to be encoded, a binary value indicating an active mode (ON) or an inactive mode (OFF) of a decorrelation process to be applied to the signal of the current frame, and encoding this value into the bitstream;

[0067] - in case the mode is determined to be active, encoding decorrelation processing information into the bitstream;

[0068] - generating an output signal for encoding into the bitstream according to the mode determined for the current frame and the mode determined for the previous frame.

[0069] The invention thus allows the use of decorrelation between the n channels to be adapted according to the characteristics of the input signal.

[0070] In one embodiment, the determination of the binary value indicating the active or inactive mode is performed according to at least one gain criterion used for encoding the signal before and after the decorrelation process.

[0071] This criterion therefore allows ensuring that the decorrelation process provides sufficient gain to be activated.

[0072] According to a specific embodiment, the coding gain is defined by the following logarithmic value:

[0073]

[0074] in, is the energy of the input channel of the decorrelation process, and λ i are the characteristic values ​​of the input channels, and for a predefined value of the gain G, the mode is determined to be inactive.

[0075] In one embodiment, the determination of the binary value indicating active or inactive mode is performed based on a criterion of an inter-frame distance between rotation matrices to which the decorrelation process is applied.

[0076] Therefore, depending on the value of this distance, the generation of the signal to be encoded is adjusted to avoid that the transformation matrix to which the decorrelation process is applied varies too much.

[0077] According to one particular embodiment in which the rotation matrices are represented as dual quaternions, the inter-frame distance between rotation matrices is represented using the scalar product between the quaternion of the current frame and the quaternion of the previous frame.

[0078] In one embodiment, the determination of the binary value indicating the active or inactive mode is performed based on a distance criterion between a rotation matrix of the current frame to which the decorrelation process is applied and an identity matrix.

[0079] Therefore, here again, as a function of the value of this distance, the generation of the signal to be encoded is adjusted in order to avoid that the transformation matrix to which the decorrelation process is applied varies too much with respect to the direct encoding of the input.

[0080] In one particular embodiment where the rotation matrix is ​​represented as a dual quaternion, the distance between the rotation matrix of the current frame and the identity matrix is ​​represented in the form of a scalar product between the quaternion of the current frame and the unit quaternion.

[0081] The invention applies to a method for decoding audio signals forming in time a series of sample frames (t-1, t) in each of n channels as an hi-fi stereo representation of order higher than 0, the method comprising:

[0082] - receiving, for a current frame (t), a binary value indicating an active or inactive mode of a decorrelation process applied to the signals of the current frame, in addition to the signals of the n channels of the current frame;

[0083] - in case the mode is determined to be active, decoding the decorrelation processing information received in the bitstream;

[0084] - generating an output signal based on said mode determined for the current frame and the mode determined for the previous frame.

[0085] The decoding method has the same advantages as the corresponding encoding method.

[0086] The invention is also directed to a coding device comprising a processing circuit for implementing the previously proposed coding method.

[0087] The invention is also directed to a decoding device comprising a processing circuit for implementing the above-mentioned decoding method.

[0088] The invention is also directed to a computer program comprising instructions for implementing the above method when these instructions are executed by a processor of a processing circuit.

[0089] The invention is also directed to a non-volatile memory medium storing instructions of such a computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Other advantages and features and characteristics of the present invention will become apparent by reading the exemplary embodiments presented in the following detailed description and examining the accompanying drawings, in which:

[0091] [FIG. 1] shows an embodiment of an encoder and an encoding method according to a method of the prior art;

[0092] [FIG. 2] shows an embodiment of a decoder and a decoding method according to a method of the prior art;

[0093] [ Figure 3 ] shows an embodiment of an encoder and an encoding method according to an embodiment of the present invention;

[0094] [ Figure 4 ] shows an embodiment of a decoder and a decoding method according to an embodiment of the present invention;

[0095] [ Figure 5 ] show examples of structural embodiments of encoders and decoders in the sense of the present invention. DETAILED DESCRIPTION

[0096] Without loss of generality, it is assumed that the input signal is a 1st order Ambisonics signal FOA in ACN format and normalized according to SN3D. In a variant, the input signal may have been subjected to pre-processing operations in order to obtain 4 channels derived from the original Ambisonics signal (FOA).

[0097] Figure 3 A coding method according to the invention is presented, wherein PCA decorrelation is applied in an adaptive manner (PCA active or inactive) within each frame, assumed here to be 20 ms long (e.g. L=960 samples at 48 kHz). It is assumed that R bits are allocated to the current frame for coding, e.g. R=5120 bits at 256 kbit / s. In a variant, this budget R can be reduced according to the bits already used by potential pre-processing operations on the signal before PCA coding.

[0098] The adaptive decision is made by block 300, which determines the value of an indication of the activation or otherwise of a decorrelation process (PCA) to be applied to the current frame, the indication corresponding to mode = ON (PCA active or activated) or OFF (PCA inactive or disabled). The decision criteria are described below.

[0099] Thus, a binary value indicating the active mode (ON) or the inactive mode (OFF) of the decorrelation process of the PCA type is determined by block 300. The operation of the encoder depends on the mode (mode) in the current frame indexed t and the mode (prev_mode) in the previous frame indexed t-1. Without loss of generality, it is assumed that at the start of the encoder, the initial state of the previous frame is prev_mode=OFF.

[0100] The following four possible combinations can be distinguished:

[0101] If mode = ON and prev_mode = ON, the same operation as in FIG. 1 is obtained, in other words, Figure 3 Blocks 100, 110, 120, 320, 140, 150 and 170 in FIG. 1 apply the same processing as blocks 100, 110, 120, 130, 140, 150 and 170 described in FIG. 1, respectively. Block 330 selects branch 2 to subject the input signal to transformation. In allocation block 340, considering that PCA encoding uses 1 bit to indicate the mode (ON) of the current frame, N Q bits are used to encode metadata and to convert R-1-N Q bits are used to encode the channels (block 170).

[0102] If mode = OFF and prev_mode = OFF, then PCA decorrelation is "short-circuited" and selection block 330 selects branch 1. Signal to be encoded With input signal The allocation block 340 takes into account the fact that 1 bit is used to indicate the mode of the current frame (OFF), and R-1 bits are used to encode the channel (block 170). By default, the feature value is fixed on the identity matrix, and the encoded quaternion and Fixed to (1,0,0,0) in the current frame - in the next frame, these values ​​will be used to define the state of the previous frame.

[0103] If mode = OFF and prev_mode = ON, the characteristic value is fixed on the identity matrix, and the encoded quaternion and is fixed to (1,0,0,0), then interpolation is performed between these values ​​and the values ​​of the previous frame (block 320). The details of this interpolation are described below. The selection module activates branch 2, in which the input signal is transformed at 150 by the matrix from the interpolation performed at 320. Here, there is no need to encode the information of the transformation matrix of the current frame, because the indication has a negative value for the current frame. Therefore, the module for inserting these encoded data is disconnected at 310. The allocation block 340 takes into account the fact that 1 bit is used to indicate the mode of the current frame and R-1 bits are used to encode the channels (block 170).

[0104] If mode = ON and prev_mode = OFF, then in the encoded quaternion and An interpolation is performed between the current value of and the value of the previous frame (1,0,0,0) (default) (block 320). The details of this interpolation are described below. The selection module activates branch 2, in which the input signal is transformed at 150 by the matrix from the interpolation performed at 320. Here, a module for inserting the coded transformed data of the current frame is connected at 310, because the PCA transformation and therefore the decorrelation process are indeed applied to this current frame. These data are transmitted through N Q The allocation block 340 considers 1 bit to indicate the mode of the current frame, and R-1-N Q bits are used to encode the channels (block 170).

[0105] After processing the previous frame, the feature value of the current frame t is and the encoded quaternion and The decision about the mode (mode) of the current frame t is also stored in the memory to define the new state (prev_mode) of the previous frame t-1 before processing the next frame.

[0106] Thus, the multiplexing module 350 inserts the encoded data into the bitstream according to the allocation defined at block 340 and according to the activation indications determined for the current frame and the previous frame.

[0107] An embodiment of interpolation (block 320) is now described. This block relies on the encoded quaternion of the current frame t and and the encoded quaternion of the previous frame t-1 and The interpolation can be implemented according to the following pseudo code, where N subframes are used, for example N=40:

[0108] -Determine the shortest path of a quaternion:

[0109]

[0110] If d1<0 and d2<0 or (if d0<0 and if -d0>d1) or (if d1<0 and if -d1>d0):

[0111]

[0112] - Interpolate the quaternion subframe by subframe over N subframes according to the NLERP (Normalized Linear Interpolation) method:

[0113] For t'=0 to N-1:

[0114] α(t′)=1–0.5(1-cos(π(t′ / N-1))

[0115]

[0116] where norm is the quaternion normalization operation (to unit norm), which corresponds to normalization on the 4-dimensional unit sphere. In a variant, the definition of α(t′) can be modified, for example by taking α(t′)=t' / (N-1). Here, the use of the function 1-0.5(1-cos(x)) allows slowing down the interpolation at both ends.

[0117] In variants, other quaternion interpolations are possible, such as SLERP (standing for Spherical Linear Interpolation) or others. The NLERP method is used here because it is less complex and more stable for limited numerical precision.

[0118] -Will and Convert to (which corresponds to block 140 and is known in the art)

[0119] This interpolation therefore results in piecewise matrixed constant values ​​through regular division into N subframes.

[0120] In variations, other interpolation methods would be possible. For example, interpolation could be applied sample by sample at the beginning of a frame (on the first N samples) and then kept constant over the rest of the frame (i.e., LN samples). and In this case, the index t' above corresponds to the index of the sample, and the subframe includes a single sample.

[0121] In variants, other embodiments of interpolation are possible, in which different subframe divisions are employed.

[0122] An exemplary embodiment of decision block 300 is now described.

[0123] In an exemplary embodiment, the determination of the activation indication of the decorrelation process uses several criteria, where the decision is made by thresholds:

[0124] 1. PCA Coding Gain

[0125] 2. The inter-frame distance between rotation matrices (which can be considered a measure of spatial stability)

[0126] 3. The distance between the rotation matrix in the current frame and the identity matrix

[0127] The motivations for the various standards are as follows:

[0128] 1. Ensure that PCA decorrelation provides enough gain to justify its activation relative to direct encoding 2. Ensure that the interpolation between the rotation matrix in the current frame and the rotation matrix of the previous frame does not cause the matrixization to change very drastically over time

[0129] 3. Ensure that interpolation between the rotation matrix in the current frame and potential disabling of PCA does not result in very large changes in the matrixization over time.

[0130] In one possible embodiment, another criterion can be based on the correlation matrix of the input signal and on its values ​​outside the diagonal values. The correlation matrix is ​​equivalent to the covariance matrix, except that the high-fidelity stereo components are each normalized by their standard deviations before calculating the cross-correlation. Thus, the criterion can be defined independently of the signal of the input signal, wherein a predetermined threshold is applied, for example, to the maximum value or the off-diagonal mean value (in absolute value) in the correlation matrix.

[0131] This allows verifying whether a minimum correlation exists between the input signals and whether the decorrelation process is useful.

[0132] In the following, exemplary embodiments of these decision criteria are given.

[0133] Here we recall the definition of the coding gain of the PCA / KLT transform in the case of a Gaussian source with n channels:

[0134]

[0135] This corresponds to the ratio between the arithmetic mean and the geometric mean (in the Gaussian case) of the variance (energy) of the components to be encoded.

[0136] According to the present invention, the energy of the input channel A in the numerator is used. and the eigenvalue λ in the denominator i The gain expressed as a logarithm becomes:

[0137]

[0138] Assume that the eigenvalues ​​are in decreasing order and are positive. The term ε is fixed to ε = 10, for example. -8 , to adjust the calculation of the logarithm.

[0139] The gain G actually corresponds to the sum (in the logarithmic domain) of the coding (or decorrelation) gains between the various (individual) channels taken before and after PCA.

[0140] In a variant, the eigenvalues ​​can also be normalized; in this case, the normalization factor is also applied to the energy value

[0141] An example of a distance criterion between matrices is now defined. The preferred embodiment relies on a representation as a dual quaternion, and the angular distance is determined by the scalar product between the quaternion of the current frame t and the quaternion of the previous frame t-1:

[0142]

[0143]

[0144] The distance between the rotation matrices associated with frames t and t-1 is evaluated as:

[0145] minP=min(P1,P2)

[0146] In a similar way, the rotation matrix of frame t can be defined as the identity matrix (given by the unit quaternion represents the distance between them, such that:

[0147]

[0148] And minP2 = min(a1, a2)

[0149] Thus, in one example, the decision (block 300) may be implemented as follows:

[0150] When the activation indication is positive by default (in other words, making mode = ON (active mode)), the indication switches to negative mode, so that if the coding gain G is lower than a predefined threshold (e.g. 6), or if the inter-frame distance between the rotation matrices is less than a threshold (e.g. 0.8), or if the distance between the rotation matrix of the current frame and the identity matrix is ​​less than a threshold (e.g. 0), then mode = OFF (inactive mode).

[0151] This then means:

[0152] If G<6

[0153] mode=OFF

[0154] If minP<0.8

[0155] mode=OFF

[0156] If minP2<0

[0157] mode=OFF

[0158] In variants, the values ​​of the thresholds (6, 0.8, 0 respectively) may be different.

[0159] In a variant, the activation indication defaults to negative (in other words, making mode = OFF), and if all criteria verify the opposite of the conditions defined above, the indication switches to positive mode, making mode = ON. This therefore amounts to simply reversing the decision logic, but with the same result.

[0160] In other variations, at least one of the three criteria defined according to the present invention is used, and the other criteria may not be used or may be replaced by other criteria.

[0161] Variants of the decision criteria are now described.

[0162] In variants, other definitions of the standard may be adopted, for example, the coding gain may be:

[0163]

[0164] The constant term can be omitted

[0165] or

[0166]

[0167] In variants, other measures of distance between rotation matrices can be defined, such as the rotation matrix and Between distance or another distance, such as of norm, where I is the n-dimensional identity matrix.

[0168] In the case of a criterion based on the correlation matrix of the input signal, the correlation matrix corresponds to the covariance matrix applied to the normalized components of the signal A. Such a covariance matrix (as described with reference to FIG1 ) is obtained, for example, as follows:

[0169] to within the normalization factor (in the case of real numbers)

[0170] or

[0171] to within the normalization factor (in the complex case).

[0172] In particular, The diagonal elements of C ii It is expressed in the form of, which represents the energy of the i-th input channel

[0173] The elements of the covariance matrix considered here are the entries C of the matrix ij and C ji , where i≠j. The maximum of these terms is determined and compared with a threshold value (e.g., a value of 0.1). In the case where the value is greater than the threshold value, then the mode of the current frame is determined to be the active ON mode. In the opposite case, the mode is determined to be the inactive OFF mode.

[0174] In a variant, the decision block can make a decision "in closed-loop mode"; this is equivalent to applying PCA processing to blocks 100 to 150 to obtain a transformed signal before confirming that the decision mode in the current frame with index t is ON In this case, the eigenvalue λ i The initial version If the final decision is OFF, the transformed signal must be replaced Initial version of .

[0175] In the variant using closed-loop mode decision, further activation decision criteria can be added, such as before and after PCA (in and ) detects the maximum absolute value in each individual channel; if The absolute value of one of the channels (in the current frame) exceeds the input signal If the absolute value of the corresponding channel in is equal to , the mode is set to OFF in the current frame.

[0176] Figure 4 A decoder implementing a decoding method according to an embodiment of the present invention is shown.

[0177] The bitstream is demultiplexed at 400 and the decoder 220 receives the channels of the multi-channel signal to be decoded according to the binary allocation determined at 420 .

[0178] Module 410 receives an indication of activation of the decorrelation process for the current frame and applies decoding and transform processing operations appropriate to the indication in the same manner as performed at encoding time.

[0179] Depending on the value of the indication of the previous frame, the following 4 combinations are distinguished:

[0180] If mode=ON and prev_mode=ON, the same operation as in FIG. 2 is obtained again, in other words, Figure 4 2, blocks 220, 230, 430, 250 and 260 in FIG. 2 apply the same processing operations as blocks 220, 230, 240, 250 and 260 described in FIG. 2, respectively. Block 440 selects branch 2 which subjects the decoded signal to the transformation of block 260. The module for decoding the transformation information received in the bitstream is implemented by the connection block 430. The allocation block 420 uses R-1-N Q bits to decode the channel at 220. By default, the feature value is fixed on the identity matrix, and the encoded quaternion and Fixed to (1,0,0,0) in the current frame - in the next frame, these values ​​will be used to define the state of the previous frame.

[0181] If mode = OFF and prev_mode = OFF, then PCA decorrelation is "short-circuited" and selection block 440 selects branch 1. Decoded signal The decoded output signal The allocation block 420 uses R-1 bits to decode the channel at 220 .

[0182] If mode = OFF and prev_mode = ON, the feature value in the current frame Quaternions that are fixed and encoded on the identity matrix and The bits R-1 are fixed to (1,0,0,0), then an interpolation is performed between these values ​​and the values ​​of the previous frame in the same way as in block 320 (block 430). The selection module 440 activates branch 2 in which the decoded signal is transformed at 260 by the matrix from the interpolation performed at 430. Here, it is not necessary to decode the information of the transformation matrix of the current frame, since this indication has a negative value for this current frame. Therefore, the module for decoding these data is disconnected by block 430. The allocation block 420 uses R-1 bits to decode the channels at 220.

[0183] If mode=ON and prev_mode=OFF, an interpolation is performed between the current value of the encoded quaternion and the value of the previous frame (default) in the same way as block 320 (block 430). The selection module 440 activates branch 2, in which the decoded signal is transformed at 260 by the matrix from the interpolation performed at 430. Here, the module 230 for decoding the encoded transformed data of the current frame is connected by the module 430, because the PCA transform and therefore the decorrelation processing operation are indeed applied to this current frame. The allocation block 420 uses R-1-N Q bits to decode the channel at 220.

[0184] It should be noted that the algorithmic delay associated with encoding-decoding (blocks 170 and 220) must be mitigated by storing in memory the inverse matrixed values ​​for each subframe in the current frame and the previous frame. to compensate. For example, when the encoding-decoding (blocks 170 to 220) is multi-mono EVS encoding, there will typically be a 12 ms delay to compensate. If the interpolation uses N=40 subframes (i.e., 0.5 ms subframes) in a 20 ms frame, a memory of 40+24=64 matrix values ​​(4×4 matrices) will therefore be required, and block 260 will apply a time-shifted matrixing of the past 24 subframes.

[0185] After processing the previous frame, the feature value of the current frame t is and the encoded quaternion and The decision about the mode (mode) of the current frame t is also stored in the memory to define the new state (prev_mode) of the previous frame t-1 before processing the next frame.

[0186] Figure 5 A coding device DCOD and a decoding device DDEC are shown, which devices are paired with each other (in the sense of "reversible") in the sense of the invention and are connected together via a communication network RES.

[0187] The coding device DCOD comprises a processing circuit which typically comprises:

[0188] - a memory MEM1 for storing instruction data of a computer program in the sense of the invention (wherein these instructions may be distributed between a coder DCOD and a decoder DDEC);

[0189] - an interface INT1 for receiving a high-fidelity stereo signal distributed over various channels (for example four first-order channels W, Y, Z, X) in order to compress and encode it in the sense of the invention;

[0190] - a processor PROC1 for receiving these signals and processing them by executing computer program instructions stored in the memory MEM1 in order to encode these signals; and

[0191] - A communication interface COM 1 for transmitting the encoded signal via a network.

[0192] The decoding device DDEC comprises its own processing circuitry, which typically comprises:

[0193] - a memory MEM2 for storing instruction data of a computer program in the sense of the invention (wherein these instructions are possibly distributed between the encoder DCOD and the decoder DDEC, as previously indicated);

[0194] - an interface COM2 for receiving coded signals from the network RES in order to compress and decode them in the sense of the invention;

[0195] - a processor PROC2 for processing these signals by executing computer program instructions stored in the memory MEM2 in order to decode them; and

[0196] - an output interface INT2 for delivering the decoded signal in the form of high-fidelity stereo channels W', Y', Z', X', for example in order to reproduce it.

[0197] It goes without saying that Figure 5 An example of a structural embodiment of a codec (encoder or decoder) in the sense of the present invention is shown. Figure 3 to Figure 4 More functional embodiments of these codecs are described in detail.

Claims

1. A method for encoding audio signals forming a series of sample frames (t-1, t) in time in each of n channels as an hi-fi stereo representation of order higher than 0, the method comprising: - determining (300) for a current frame to be encoded a binary value indicating an active mode (ON) or an inactive mode (OFF) of a decorrelation process to be applied to the signal of the current frame, and encoding this value into the bitstream; - in case the mode is determined to be active, encoding (310) decorrelation processing information into the bitstream; - generating (330) an output signal for encoding into the bitstream according to the mode determined for the current frame and the mode determined for the previous frame.

2. The method of claim 1, wherein: The determination of this binary value indicating an active or inactive mode is performed according to at least one gain criterion used for encoding the signal before and after the decorrelation process.

3. The method of claim 2, wherein: The coding gain is defined by the following logarithmic value: in, is the energy of the input channel of the decorrelation process, and λ i are the characteristic values ​​of the input channels, and for a predefined value of the gain G, the mode is determined to be inactive.

4. The method of claim 1, wherein: The determination of the binary value indicating active or inactive mode is performed according to the criterion of the inter-frame distance between the rotation matrices to which the decorrelation process is applied.

5. The method of claim 4, wherein: These rotation matrices are represented as dual quaternions, and the inter-frame distance between rotation matrices is represented using the scalar product between the quaternion of the current frame and the quaternion of the previous frame.

6. The method of claim 1, wherein: The determination of the binary value indicating the active or inactive mode is performed according to a distance criterion between the rotation matrix of the current frame to which the decorrelation process is applied and the identity matrix.

7. The method of claim 6, wherein: These rotation matrices are represented as dual quaternions, and the distance between the rotation matrix of the current frame and the identity matrix is ​​expressed in the form of a scalar product between the quaternion of the current frame and the unit quaternion.

8. A method for decoding audio signals forming in time a series of sample frames (t-1, t) in each of n channels as an hi-fi stereo representation of order higher than 0, the method comprising: - receiving (410) a binary value indicating an active mode (ON) or an inactive mode (OFF) of a decorrelation process applied to the signals of the current frame (t), in addition to the signals of the n channels of the current frame; - in case the mode is determined to be active, decoding the decorrelation processing information received in the bitstream (430); - generating (440) an output signal based on said mode determined for the current frame and the mode determined for the previous frame.

9. An encoding device comprising a processing circuit for implementing the steps of the encoding method according to any one of claims 1 to 7.

10. A decoding device comprising a processing circuit for implementing the steps of the decoding method as claimed in claim 8.

11. A storage medium readable by a processor, the storage medium storing a computer program in a memory comprising instructions for executing the encoding method according to any one of claims 1 to 7 or the decoding method according to claim 8.

Images