Method and apparatus for compressing and decompressing higher order ambisonics representations for sound fields - Patents.com

Abstract

To provide a method and device for compressing and decompressing Higher Order Ambisonics (HOA) representation for sound field representation.SOLUTION: A method includes steps of analyzing HOA representation for presence of a dominant sound source, estimating its direction, and decomposing the HOA representation into multiple dominant directional signals XDIR(k-1) and ambient HOA component DA(k-2). The ambient HOA component is transformed into a discrete space domain to obtain a general plane wave function with a uniform sampling direction. This uniform sampling direction is predicted from a dominant directional signal. Finally, a prediction error is transformed again into an HOA domain to represent an ambient HOA component of residuals with low dimensioning. Perceptual coding of the dominant directional signal and the component of the residual is then performed.SELECTED DRAWING: Figure 1a

Description

[Technical field]

[0001] The present invention relates to a method and apparatus for compressing and decompressing a high-order Ambisonics representation for a sound field. [Background technology]

[0002] Higher-order Ambisonics representation, called HOA, is one way to represent three-dimensional sound. Other techniques are wave field synthesis (WFS) and channel-based methods such as 22.2. Compared to channel-based methods, the HOA representation has the advantage that it is independent of a specific loudspeaker setup. However, this flexibility requires a decoding process to play the HOA representation on a specific loudspeaker setup. HOA allows setups with only a very small number of loudspeakers compared to the WFS approach, which usually requires a much larger number of loudspeakers. An additional advantage of HOA is that the same representation can be used without modification for binaural rendering to headphones.

[0003] HOA is based on the representation of the spatial density of complex harmonic plane wave amplitudes through a truncated spherical harmonic (SH) expansion. Each expansion coefficient is a function of angular frequency, which can be equivalently represented by a time domain function. Thus, without loss of generality, the complete HOA sound field representation can actually be considered as being composed of “O” time domain functions, where O denotes the number of expansion coefficients. We refer to the following sequence of HOA coefficients as equivalent to these time domain functions:

[0004] The spatial resolution of the HOA representation improves with increasing maximum order of the expansion, N. Unfortunately, the number of expansion coefficients, O, grows quadratically with order N, especially for O=(N+1). 2 For example, a general HOA representation with order N=4 requires O=25 HOA (expansion) coefficients. Taking the above into account, the total bit rate for the transmission of the HOA representation is calculated as follows:s and the number of bits per sample, N b Given, Ο f s N b For each sample, N b = 16 bits are used to s Transmitting an HOA representation of order N=4 at a sampling rate of N=48 kHz results in a bit rate of 19.2 Mbit / s, which is too high for many practical applications, e.g., streaming. Compressing the HOA representation is therefore highly desirable. Summary of the Invention

[0005] There are few existing methods dealing with the compression of HOA representations higher than first order. The most straightforward approach, explored by E. Hellerud, I. Burnett, A. Solvang, and U S Pvensson, "Encoding Higher Order Ambisonics with AAC", 124th AES Convention, Amsterdam, 2008, involves direct encoding of the individual HOA coefficient sequences using the perceptual coding algorithm, Advanced Audio Coding (AAC). However, an inherent problem with this approach is the perceptual coding of a signal that is never heard at all. The reconstructed playback signal is usually obtained by a weighted sum of the HOA coefficient sequences, which has a high probability of unmasking the perceptual coding noise when the decompressed HOA representation is rendered for a particular loudspeaker setup. The main problem with unmasking the perceptual coding noise is the high cross-correlation between the individual HOA coefficient sequences. Since the coding noise signals in the individual HOA coefficient sequences are not correlated with each other, a structural superposition of the perceptual coding noise may occur, which at the same time cancels out the noise-free HOA coefficient sequences. Another problem is that these cross-correlations lead to a decrease in the efficiency of the perceptual coder.

[0006] To minimize the extent of both effects, EP2469742A2 proposes to transform the HOA representation into an equivalent representation in the discrete spatial domain before perceptual encoding. Formally, the discrete spatial domain is the time domain equivalent to the spatial density of complex harmonic plane wave amplitudes sampled in some discrete direction. The discrete spatial domain is therefore represented by "O" conventional time domain signals. This signal can be interpreted as a generic plane wave coming from the sampling direction and would correspond to a loudspeaker signal if the loudspeaker was located in exactly the same direction as assumed for the spatial domain transformation.

[0007] Transformation to the discrete spatial domain reduces the cross-correlations between the individual spatial domain signals, but does not completely eliminate them. An example of a relatively high cross-correlation is a directional signal whose orientation lies between multiple adjacent directions encompassed by the spatial domain signal.

[0008] The main drawback of both approaches is that the number of signals to be perceptually encoded is (N+1). 2 and the data rate of the compressed HOA representation grows as the square of the Ambisonics order N.

[0009] In order to reduce the number of signals to be perceptually coded, EP 2 665 208 A1 proposes decomposing the HOA representation into a given maximum number of dominant directional signals and a residual ambient component. The reduction in the number of signals to be perceptually coded can be achieved by reducing the order of the residual ambient component. The rationale behind this approach is to represent the residual with sufficient accuracy by a lower order HOA representation, while maintaining a high spatial resolution for the dominant directional signals.

[0010] This approach works very well as long as the assumptions about the sound field are met, namely that the sound field consists of a small number of dominant directional signals (which represent general plane wave functions coded with full order N) and a residual non-directional ambient component. However, if after the decomposition the residual ambient component still contains some dominant directional components, the reduction in order leads to noticeable perceptual errors when rendering after the decomposition. A common example of an HOA representation where the assumptions are not met is a general plane wave coded with an order lower than N. Such general plane waves with an order lower than N can arise as a result of artistic creation to make the sound source seem more widespread, and can also arise with the recording of HOA sound field representations with spherical microphones. In both examples, the sound field is represented by a large number of highly correlated spatial domain signals (see the entry on spatial resolution of higher order Ambisonics for an explanation).

[0011] The problem solved by the present invention is to avoid the above-mentioned disadvantages of other prior art techniques by eliminating the disadvantages resulting from the process described in EP-A-2 665 208. This problem is solved by the methods disclosed in claims 1 and 3. Corresponding devices utilizing these methods are disclosed in claims 2 and 4.

[0012] The present invention improves the HOA sound field representation compression process described in EP 2 665 208. First, similar to EP 2 665 208, the HOA representation is analyzed for the presence of dominant sound sources and their directions are estimated. Using the information of the directions of the dominant sound sources, the HOA representation is decomposed into a number of dominant directional signals representing general plane waves and residual components. However, instead of immediately reducing the order of the residual HOA components, the residual HOA components are transformed into the discrete spatial domain in order to obtain general plane wave functions in uniform sampling directions representing the residual HOA components. These plane wave functions are then predicted from the dominant directional signals. The reason for this process is that parts of the residual HOA components may be highly correlated with the dominant directional signals.

[0013] The prediction can be simple, yielding only a small amount of side information. In the simplest case, the prediction consists of appropriate scaling and delay. Finally, the prediction error is transformed back to the HOA domain and taken as the ambient HOA component of the residual, which is then reduced in dimension.

[0014] Advantageously, the effect of subtracting the predictable signal from the HOA component of the residual is to reduce its overall order and the residual amount of the dominant directional signal, thus reducing the decomposition error resulting from the reduction in dimension.

[0015] In principle, the compression method of the present invention is suitable for compressing a Higher Order Ambisonics representation of a sound field, called HOA. - estimating a dominant sound source direction from the current time frame of HOA coefficients; - decomposing the HOA representation into a dominant directional signal in the time domain and a residual HOA component depending on the HOA coefficients and the dominant sound source direction, the latter being transformed into the discrete spatial domain to obtain a plane wave function in a uniform sampling direction representing the residual HOA component, the plane wave function being predicted from the dominant directional signal, resulting in parameters describing the prediction, and the corresponding prediction error being transformed back into the HOA domain; - reducing the current order of the residual HOA components to a lower order, resulting in reduced order residual HOA components; - decorrelating the reduced residual HOA components to obtain corresponding residual HOA component time domain signals; - perceptually encoding said dominant directional signal and said residual HOA component time domain signal to provide a compressed dominant directional signal and a compressed residual component signal.

[0016] In principle, the compressor of the present invention is suitable for compressing a Higher Order Ambisonics representation of a sound field, called HOA. - means configured to estimate a dominant sound source direction from a current time frame of HOA coefficients; - means adapted to decompose the HOA representation into a dominant directional signal in the time domain and a residual HOA component depending on the HOA coefficients and the dominant sound source direction, the latter being transformed into the discrete spatial domain to obtain a plane wave function with uniform sampling directions representing the residual HOA component, the plane wave function being predicted from the dominant directional signal to provide parameters describing the prediction, and the corresponding prediction error being transformed back into the HOA domain; - means adapted to reduce the current order of the residual HOA component to a lower order, resulting in a reduced order residual HOA component; and - means adapted to decorrelate said reduced residual HOA components to obtain corresponding residual HOA component time domain signals; means configured for perceptually encoding said dominant directional signal and said residual HOA component time-domain signals to provide a compressed dominant directional signal and a compressed residual component signal.

[0017] In principle, the decompression method of the invention is suitable for decompressing higher order Ambisonics representations compressed according to the compression methods described above. - perceptually decoding said compressed dominant directional signal and said compressed residual component signals to provide decompressed time-domain signals representing said decompressed dominant directional signal and said residual HOA components in the spatial domain; - re-correlating said decompressed time domain signal to obtain corresponding reduced residual HOA components; - expanding the order of the reduced residual HOA components to their original order, providing corresponding decompressed residual HOA components; - synthesizing a corresponding decompressed and resynthesized frame of HOA coefficients using the decompressed dominant directional signal, the HOA components of the original order decompressed residual, the estimated dominant sound source direction and the parameters describing the prediction.

[0018] In principle, the decompression device according to the invention is suitable for decompressing higher order Ambisonics representations compressed according to the compression method described above. - means configured for perceptually decoding said compressed dominant directional signal and said compressed residual component signals to provide decompressed time-domain signals representative of said decompressed dominant directional signal and said residual HOA components in the spatial domain; - means configured to re-correlate the decompressed time domain signal to obtain corresponding reduced residual HOA components; - means adapted to extend the order of the reduced residual HOA components to their original order, said means providing corresponding decompressed residual HOA components; - means configured to synthesize a corresponding decompressed and resynthesized frame of HOA coefficients by using said decompressed dominant directional signal, the HOA components of the original order decompressed residual, said estimated dominant sound source direction and said parameters describing said prediction.

[0019] Advantageous further embodiments of the invention are disclosed in the respective dependent claims.

[0020] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings. [Brief description of the drawings]

[0021] [Figure 1a] Compression step 1: Decomposition of the HOA signal into multiple dominant directional signals, residual ambient HOA components, and side information. [Figure 1b] FIG. 13 Compression step 2: dimensional reduction and decorrelation of the ambient HOA component and perceptual coding of both components. [Figure 2a] FIG. 1 illustrates decompression step 1: perceptual decoding of the time domain signal, recorrelation of a signal representing the ambient HOA component of the residual, and order expansion. [Figure 2b] FIG. 13 shows decompression step 2: Composition of all HOA representations. [Diagram 3] FIG. 1 illustrates HOA decomposition. [Figure 4] FIG. 1 shows HOA synthesis. [Diagram 5] FIG. 1 illustrates a spherical coordinate system. [Figure 6]FIG. 13 shows a plot of the normalized function vN(θ) for several different values ​​of N. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Compression The compression process according to the invention comprises two successive steps, which are illustrated in Figures 1a and 1b, respectively. The exact definition of the individual signals is given in the section Detailed Description of HOA Decomposition and Recomposition. A frame-wise process for compression is used, using non-overlapping input frames D(k) of HOA coefficient sequences of length B, where k represents the frame index. The frames are defined in terms of the HOA coefficient sequences specified in equation (1) below.

number

[0023] In Figure 1a, a frame D(k) of HOA coefficient sequences is input to a dominant sound source direction estimation step or stage 11, in which the HOA representation is analysed for the presence of dominant directional signals and the direction is estimated. The direction estimation is performed and may be performed, for example, by the process described in EP 2 665 208 A1. The estimated direction is:

number

number

number

[0024] Implicitly, it is assumed that the direction estimates are properly ordered by assigning them to direction estimates from previous frames. The temporal sequence of individual direction estimates is thus assumed to describe the direction trajectory of the dominant sound source. In particular, if the d-th dominant sound source is assumed to be inactive, then

number

number

[0025] In FIG. 1b, the directional signal X DIR (k-1) perceptual coding and the residual ambient HOA component D A The perceptual coding of (k-2) is shown. DIR (k-1) is a conventional time domain signal, which can be individually compressed using any existing perceptual compression technique. A The (k-2) compression can be performed in two successive steps or stages. In the reduction step or stage 13, the Ambisonics order N RED Here, for example, N RED = 1. As a result, the ambient HOA component D A,RED (k-2) is obtained. This reduction in dimension is A In (k-2), N REDThis is done by keeping only the HOA coefficients and discarding the other coefficients. At the decoder side, the omitted values ​​are filled with corresponding zero values, as explained below.

[0026] It should be noted that compared to the approach of EP 2 665 208, the reduced order N RED may typically be chosen to be small, since this reduces the overall order and therefore the residual directional residual of the ambient HOA component of the residual. Hence, the reduction in order leads to smaller errors compared to the case of EP 2 665 208.

[0027] In the following decorrelation step or stage 14, the reduced-order ambient HOA components D A,RED The HOA coefficient sequence representing (k-2) is decorrelated and the time domain signal W A,RED (k-2) is obtained. This time domain signal is input to a (bank of) parallel perceptual coders or compressors 15 operating according to any perceptual compression technique. This decorrelation is done to avoid masking the perceptual coding noise when rendering the HOA representation after decompression (see EP 12305860 for an explanation). An approximate decorrelation can be achieved by applying a spherical harmonic transform to obtain the D A,RED (k-2) in the spatial domain RED This can be accomplished by converting it into an equivalent signal.

[0028] Alternatively, one can use the adaptive spherical harmonic transform proposed in EP 12305861, where the grid of sampling directions is rotated to obtain maximum decorrelation effect. Another alternative decorrelation technique is the Karhunen-Loeve Transform (KLT), described in EP 12305860. Note that for these last two types of decorrelation, some side information, denoted α(k−2), is provided to enable the reverse process of decorrelation at the HOA decompression stage.

[0029] In one embodiment, to improve coding efficiency, all the time domain signals X DIR (k-1) and W A,RED A (k-2) perceptual compression is performed together.

[0030] The output of perceptual coding is a compressed directional signal.

number

number

[0031] Decompression process The decompression process is illustrated in Figures 2a and 2b. As with the compression process, the decompression process consists of two successive steps. In Figure 2a, the directional signal

number

number

number

number

number

number

number

[0032] In Fig. 2b, all HOA representations are decompressed dominant directional signals

number

number

number

number

[0033] To improve coding efficiency, all the time-domain signals XDIR (k-1) and W A,RED When (k-2) perceptual compression is performed together, the compressed directional signal

number

number

[0034] A detailed explanation of recomposition is provided in the HOA Recomposition section.

[0035] HOA decomposition A block diagram illustrating the process performed for HOA decomposition is given in Figure 3. The process is summarized as follows: First, the smoothed dominant directional signal X DIR (k-1) is computed and output for perceptual compression. Next, the HOA representation of the dominant directional signal D DIR The residual between (k-1) and the original HOA representation D(k-1) is “O” number of directional signals.

number

number

[0036] Before going into details, it is noted that orientation changes between successive frames may cause discontinuities in all calculated signals during synthesis. Therefore, first, instantaneous estimates of the signals for each of the overlapping frames with a length of 2B are calculated. Secondly, the results of successive overlapping frames are smoothed using an appropriate window function. However, each smoothing involves a latency of one frame.

[0037] Calculation of instantaneous dominant directional signals For the current frame D(k) of the HOA coefficient sequence

number

[0038] Furthermore, without loss of generality, the inclination angle θ DOM,d (k)∈[0,π] and azimuth angle φ DOM,d The vector (k) ∈ [0,2π] (see Fig. 5) gives an estimate of the direction of the active dominant sound source.

number

number

[0039] First, the mode matrix based on the direction estimate of the active sound source is calculated according to the following formula:

number

number

number

[0040] Second, the matrix

number

number

number

number

number

number

number

[0041] temporal smoothing For step or stage 31, directional signal

number

number

number

number

number

number

number

number

number

[0042] Calculating the HOA representation of the smoothed dominant directional signal X DIR (k-1) and

number

number

number

number

[0043] Representing the residual HOA representation by directional signals on a uniform grid D DIR From D(k-1) and D(k-1) (i.e., D(k) delayed by a frame delay 381), a residual HOA representation in terms of directional signals on a uniform grid is calculated in step or stage 33. The objective of this process is to find the residual [D(k-2)D(k-1)]-[D DIR (k-2)D DIR (k-1)], we use some fixed, approximately uniformly distributed orientation

number

[0044] First, the mode matrix Ξ GRID is calculated as follows:

number

number

[0045] The directional signal on each grid is obtained by the following equation:

number

[0046] Predicting directional signals on uniform grids from dominant directional signals

number

number

number

number

[0047] First,

number

number

number

number

number

[0048] Next, each grid signal

number

number

number

number

number

[0049] If the order of the prediction error is larger than that of the grid signal itself, the prediction is assumed to be unsuccessful, and each prediction parameter can be set to an arbitrary invalid value.

[0050] However, other types of prediction are possible: for example, instead of calculating full-band scaling factors, it is reasonable to determine scaling factors for perceptually oriented frequency bands, but this process improves the prediction at the expense of an increased amount of side information.

[0051] All prediction parameters can be arranged in a parameter matrix as follows:

number

number

number

[0052] Computing the HOA representation of predicted directional signals on a uniform grid The HOA representation of the predicted grid signal is calculated in step or stage 35 according to the following equation:

number

number

[0053] Calculation of the HOA representation of the residual ambient sound field components

number

number

number

[0054] HOA resynthesis Before describing the process of each step or stage in FIG. 4 in detail, a brief overview is given.

number

number

number

number

number

number

number

[0055] Calculating the HOA representation of the dominant directional signal

number

number

number

number

number

number

number

number

number

number

[0056] Predicting directional signals on uniform grids from dominant directional signals

number

number

number

number

number

[0057] Computing the HOA representation of predicted directional signals on a uniform grid In the step or stage 44 of calculating the HOA representation of the predicted directional signal on a uniform grid, the HOA representation of the predicted grid directional signal is obtained by the following equation:

number

[0058] Synthesis of HOA sound field representation

number

number

number

number

number

number

[0059] Higher Order Ambisonics Fundamentals Higher order Ambisonics is based on the description of the sound field in a compact region of interest, where no sound sources are assumed to exist. The spatiotemporal behavior of the sound pressure p(t,x) at time t and position x in the region of interest is then completely physically given by the wave equation for homogeneous media. The following is based on the spherical coordinate system shown in Figure 5. The x-axis points to the front position, the y-axis points to the left and the z-axis points upwards. For a position in space x=(r,θ,φ) T is expressed in terms of a radius r>0 (i.e., the distance to the coordinate origin), a tilt angle θ ∈ [0,π] measured from the polar axis z, and an azimuth angle φ ∈ [0,2π] measured counterclockwise in the xy plane from the x-axis. (·) T represents a transposition.

[0060] F t The Fourier transform of the sound pressure versus time is represented by (·), i.e.,

number

number

number

number

[0061] If a sound field is represented by an infinite superposition of harmonic plane waves ω of different angular frequencies, coming from all possible directions specified by a pair of angles (θ,φ), it can be shown that each plane wave complex amplitude function D(ω,θ,φ) can be expressed by the following spherical harmonic expansion (see B. Rafaely, “Plane-wave Decomposition of the Sound Field on a Sphere by Spherical Convolution”, Journal of the Acoustical Society of America 4(116), pp. 2149-2157, 2004).

number

number

number

number

[0062] Individual Coefficients

number

number

number

number

number

[0063] The final Ambisonics format is s to yield a sampled version of d(t)

number

number

[0064] Definition of real-valued spherical harmonics Real-valued spherical harmonics

number

number

number

number

[0065] Spatial resolution of higher-order Ambisonics Direction Ω0=(θ0,φ0) T A general plane wave function x(t) coming from is expressed in the HOA by the following equation:

number

number

number

number

number

[0066] discrete spatial domain The spatial density of the plane wave amplitude is O spatial directions Ω o (When discretized in 1≦ο≦Ο, the spatial direction Ω o are distributed almost uniformly on the unit sphere, but O directional signals d(t,Ω o ) is obtained. These signals are collected into a vector, which is expressed as follows:

number

number

number

[0067] direction Ω o Since is almost uniformly distributed on the unit sphere,

number

[0068] On the encoding side, and also on the decoding side, the process of the invention can be carried out in a single processor or electronic circuit, or in several processors or electronic circuits working in parallel and / or operating on several different parts of the process of the invention.

[0069] The invention can be applied to processing corresponding audio signals that can be rendered and played on a loudspeaker configuration in a home environment or on a loudspeaker configuration in a theatre.

[0070] Several aspects will be described. [Aspect 1] 1. A method for compressing a Higher Order Ambisonics representation, referred to as HOA, of a sound field, comprising the steps of: - The dominant sound source direction (

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

Claims

1. 1. A method for decompressing a compressed Higher Order Ambisonics (HOA) representation, the method comprising: perceptually decoding the compressed HOA representation to determine a decompressed dominant directional signal and a decompressed time domain signal representing a residual HOA component in the spatial domain, the decompressed time domain signal corresponding to a reduced order residual HOA component; determining a predicted directional signal based on the decompressed dominant directional signal, the predicted directional signal being determined based on smoothing using a window function; determining a decompressed residual HOA component based on the decompressed time domain signal, the decompressed residual HOA component being based on extending an order of the reduced order residual HOA component, the extending including adding zero values ​​to the reduced order residual HOA component; determining a HOA sound field representation based on the predicted directional signal and the HOA component of the decompressed residual; Including, method.

2. The method of claim 1 , wherein the predicted directional signal is determined for a current frame of the compressed HOA representation.

3. An apparatus for decompressing a compressed Higher Order Ambisonics (HOA) representation, the apparatus comprising: a decoder for perceptually decoding the compressed HOA representation to determine a decompressed dominant directional signal and a decompressed time domain signal representing a residual HOA component in the spatial domain, the decompressed time domain signal corresponding to a reduced order residual HOA component; a first processor for determining a predicted directional signal based on the decompressed dominant directional signal, the first processor being configured to determine the predicted directional signal based on smoothing using a window function; a second processor for determining a decompressed residual HOA component based on the decompressed time domain signal, the decompressed residual HOA component being based on extending an order of the reduced order residual HOA component, the extending including adding zero values ​​to the reduced order residual HOA component; a third processor for determining an HOA sound field representation based on the predicted directional signal and an HOA component of the decompressed residual; having Device.

4. The apparatus of claim 3 , wherein the predicted directional signal is determined for a current frame of the compressed HOA representation.

Images