High-order ambient stereo encoding and decoding

Abstract

To reduce the bit rate, high-order ambient stereo (HOA) data is encoded and decoded. One approach is to use principal component analysis to generate spatial descriptors. Other approaches include various spatial descriptor quantization techniques.

Description

[0001] Cross-references

[0002] This patent application claims the benefit of the earlier filing date of U.S. Provisional Patent Application No. 63 / 083,673, filed on September 25, 2020. Technical Field

[0003] This disclosure relates to techniques in digital audio signal processing, and more specifically to bit rate reduction of high-order ambient stereo (HOA) data. Background Technology

[0004] A sound field can be represented by the sum of weighted spherical harmonic basis functions of increasing orders 0, 1, 2, ... . As the set of basis functions is extended to include higher-order elements (second and higher), the representation of the sound field becomes more detailed (higher resolution). The weights applied to the basis functions are called spherical harmonic coefficients. The term High-Order Ambient Stereo (HOA) data is generally used to refer to this representation of the sound field.

[0005] Digital audio content (where the sound field is represented by HOA data) can be transmitted from one location to another via a communication link for playback at the latter location via any audio output system. At the audio output system, the HOA data is transformed into speaker driver signals through digital signal processing. Examples include, for instance, speaker driver signals for a two-channel amplifier system or a 5.1 surround sound system, and driver signals for binaural left and right headphones. However, the communication link may not always have sufficient bandwidth to transmit the raw or uncompressed HOA data for real-time, pause-free playback. Several encoding and decoding techniques have been proposed to encode (and specifically, compress) the raw HOA data into a bit-rate-reduced coded bitstream for transmission over a limited bandwidth communication link, and then decode the raw HOA data at the destination audio output system (before transforming the decoded HOA data into speaker driver signals for playback). These techniques include the use of Singular Value Decomposition (SVD) and Eigenvalue Decomposition (EVD), which are matrix factorization techniques applied to an input H matrix containing spherical harmonics, a large portion of the HOA data. Matrix factorization is applied to extract components containing the foreground sound (also known as the direct or primary sound) and its associated "spatial components," which describe some spatial aspects of the foreground sound components. The extracted foreground sound components and their accompanying spatial components are then quantized before being transmitted over the communication link. On the decoding side, the received foreground and spatial components are processed by a reconstruction algorithm to synthesize the recovered H^ matrix. Summary of the Invention

[0006] For the purpose of reducing bit rate, several aspects of this disclosure relate to the encoding and decoding of HOA data. In a first aspect, principal component analysis (PCA) or any linear transformation is performed based on an input H matrix, which produces a spatial descriptor SD (also referred to as a Wi component in the Wi components), where i = 1, 2, ..., N_sc. The SD component Wi describes the spatial aspects of the associated or i-th salient audio component, such as its direction of arrival and its spread. PCA or a linear transformation can be performed directly on the zero-mean covariance matrix, which is calculated from the result of subtracting the column-wise mean vector from the input H matrix. The column-wise mean vector minus the H matrix can be referred to herein as the H~ matrix. A salient component (SC) extraction process is then performed using the SD and H~ matrices, which produces N salient audio components Xi = H~ * Wi, where i = 1, 2, ..., N_sc. The resulting Xi and Wi can then be quantized for transmission to the decoding side. It should be recognized here that in order to accurately synthesize (on the decoding side) the recovered H matrix (also known as the H^ matrix), a column-wise mean vector should also be provided on the decoding side, which is used by the reconstruction algorithm (e.g., adding the mean vector to the product of the recovered Xi and the recovered Wi) to generate the recovered (synthesized) HOA matrix.

[0007] In the second aspect, the PCA-based encoding technique of the first aspect is modified so that the column-wise mean vector does not need to be transmitted to the decoding side, which advantageously reduces the required encoding / decoding bandwidth. Specifically, when extracting salient components Xi, the salient component extraction is modified on the encoding side to directly use the input H matrix instead of using the H^ matrix with the mean subtracted from each column. Using this method, the accurate H^ matrix is ​​synthesized (performed on the decoding side), although the column-wise mean vector cannot be accessed.

[0008] In a third aspect, the encoding side can dynamically switch between PCA encoding with mean vector transmission (first aspect) and PCA encoding without mean vector transmission (second aspect) (e.g., when transmitting streaming audio content to the decoding side). The resulting transmission (e.g., the encoded audio content bitstream) contains a tag associated with the coded segment, indicating which encoding aspect was used to generate Xi and Wi in that segment. The dynamic switching decision between these two aspects can be based on the audio content, such as on metadata associated with the input HOA matrix. On the decoding side, the process looks up the received tag and, depending on whether the tag is set, decides whether to add the mean vector to the product of the recovered Xi and the recovered Wi.

[0009] Additional aspects of this disclosure for encoding and decoding HOA data include several spatial descriptor quantization techniques described in detail below. These aspects are not limited to any particular analytical operation, as they can be used not only with PCA but also with other linear transformation analysis algorithms such as SVD and EVD matrix factorization algorithms.

[0010] The above overview does not constitute an exhaustive list of all aspects of this disclosure. It is contemplated that this disclosure encompasses all systems and methods that can be practiced by all suitable combinations of the aspects outlined above and those disclosed in the detailed descriptions below and specifically pointed out in the claims section. Such combinations may have specific advantages not specifically set forth in the foregoing summary. Attached Figure Description

[0011] The aspects of this disclosure are illustrated by way of example and are not limited to the illustrations in the accompanying drawings, in which similar reference numerals indicate similar elements. It should be noted that references to “a” or “an” aspect in this disclosure do not necessarily refer to the same aspect, and each refers to at least one. Furthermore, for the sake of brevity and to reduce the total number of drawings, a given drawing may be used to illustrate more than one aspect of this disclosure, and for a given aspect, not all elements in that drawing may be necessary.

[0012] Figure 1 A block diagram of an encoding and decoding system using PCA with mean vector transmission and associated encoded audio content bitstreams.

[0013] Figure 2 An encoding and decoding system for PCA using meanless vector transmission in an associated bitstream is shown.

[0014] Figure 3 A system with dynamic decision-making for the analysis block and the resulting bit stream is shown.

[0015] Figure 4 The multi-subband encoder and the resulting bitstream are shown.

[0016] Figure 5 The shared space descriptor quantization technique is illustrated.

[0017] Figure 6 The graph shows Figure 5 The concept of a shared space descriptor.

[0018] Figure 7 A hybrid spatial descriptor estimation (generation) technique is described.

[0019] Figure 8 It shows that it can be used Figure 7A diagram illustrating an exemplary hybrid SD estimation technique implemented in a block diagram, and a diagram illustrating a technique in which each SD is estimated separately on a per-subband basis.

[0020] Figure 9 Another SD quantization coding technique is described, in which different numbers of SD components are generated for different subbands.

[0021] Figure 10 It shows Figure 9 A graph of the encoded audio content in the resulting bitstream from the SD group.

[0022] Figure 11 It shows the corresponding Figure 10 The example saliency component group of the SD group is shown in the example.

[0023] Figure 12 The SD quantization coding technique is described, in which different numbers of SD components and associated band-limited significant components (SC) are generated for different subbands.

[0024] Figure 13 An example of a bitstream using SD quantization is shown, where the SD component generated for a given subband is reused or copied for another subband (within the same SD group).

[0025] Figure 14 An example of a bitstream using SD quantization is shown, where spatial descriptors cover merged subbands.

[0026] Figure 15 An example of a bitstream using SD quantization is shown, where the subband bandwidth varies across SD groups.

[0027] Figure 16 A graphical view of the arrangement of SD components in an encoded audio bitstream, where each of two or more SD groups is represented by a different HOA order. Detailed Implementation

[0028] Various aspects of this disclosure will now be explained with reference to the accompanying drawings. Where the shape, relative position, and other aspects of the described components are not explicitly defined, the scope of the invention is not limited to the components shown, which are for illustrative purposes only. Furthermore, while numerous details have been set forth, it should be understood that some aspects of this disclosure may be practiced without these details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0029] PCA-based HOA encoding and decoding

[0030] Figure 1This is a block diagram of a high-order ambient stereo (HOA) data encoding and decoding system for reducing the bit rate of the resulting encoded audio content bitstream using principal component analysis (PCA) with mean vector transfer, while maintaining sound quality during bitstream playback. The components of these systems are digital electronic devices such as one or more processors (collectively referred to herein as “processors”), which are configured, for example, according to instructions stored in memory to perform certain digital signal processing operations described below. An encoder or encoding side generates an encoded audio content bitstream, which can be transmitted to a decoder or decoding side, for example, via the Internet or any communication link that may experience bandwidth fluctuations or may have limited bandwidth. For example, the encoding side may be part of a system with multiple microphones, through which a sound field is captured and then formatted as HOA data. The decoding side may be part of a playback system with sound output transducers or speaker drivers (e.g., amplifiers, headphones), through which the HOA data is decoded and converted into appropriate speaker driver signals, and then output as sound.

[0031] This encoding method involves subtracting the mean vector from the input HOA matrix H to compute a mean-subtracted HOA matrix H~. Here, H can be a matrix with N rows and M columns, where the number of columns represents the number of HOA coefficients, and the order of the HOA is sqrt(M)⁻¹ (a larger number of columns means a higher order). The width of the input HOA matrix depends on the order of the HOA representation (e.g., the number of column vectors in the matrix depends on the order of the HOA representation). If the matrix is ​​a time-domain representation, the number of elements in each column vector is determined by the sampling rate, or by the subband domain or frequency domain resolution (e.g., the total number of subbands covering the full audio bandwidth). The mean vector can be a row vector, where each element of the row vector can be the average of the corresponding column in the input HOA matrix. Note that the size of H~ may be the same as H.

[0032] Next, spatial descriptors SD are generated by performing principal component analysis (PCA) based on the HOA matrix after subtracting the mean. SD is represented by Wi in the figure, where i = 1, 2, ..., Nsc, and Nsc is the total number of salient components (SCs) to be extracted from the HOA matrix after subtracting the mean. SD(Wi) describes the spatial aspects of the corresponding or i-th salient component, such as its direction of arrival and its spread. In this case, the total number of SDs is equal to the total number of corresponding salient components. The salient components are audio signals and are represented by Xi in the figure; they can be extracted by solving the equation H ~ *Wi.

[0033] Finally, the encoding method includes associating the significant components Xi and the spatial descriptors Wi with a mean vector, for example by formatting them all into an output encoded audio content bitstream. Note here that the significant components (Xi vectors) are essentially audio signals and can thus be encoded separately from their associated SDs in order to use any suitable audio signal encoding technique (e.g., AAC) to reduce the bitrate when formatting into a bitstream. Similarly, considering the trade-off between quality and bitrate, the spatial descriptors can also be reduced in bitrate by any suitable quantization technique (when being formatted into a bitstream), for example by performing a coarse quantization in cases where a lower playback quality is allowed, and a fine quantization in cases where a higher quality is needed despite the higher bitrate requirement.

[0034] The analysis operation can be performed by determining a zero-mean covariance matrix using the mean-subtracted HOA matrix and then performing PCA on the zero-mean covariance matrix as shown. The zero-mean covariance matrix can be determined by multiplying the transpose of the mean-subtracted HOA matrix by the mean-subtracted HOA matrix as shown. The analysis operation produces the spatial descriptors Wi as described above. Then, for each SD, the significant components are extracted by multiplying the SD with the mean-subtracted HOA matrix as shown. This operation is repeated for Nsc spatial descriptors to extract Nsc significant components, where Nsc < M achieves bitrate reduction.

[0035] Figure 1 Also shown is the decoding side process, or a method for decoding the HOA data received in a bitstream. The received bitstream contains the significant components and the corresponding spatial descriptors SD, where the SD is produced by performing principal component analysis PCA based on the mean-subtracted HOA matrix. A mean vector (which is used to calculate the mean-subtracted HOA matrix on the encoding side) is also received in the bitstream. Now the HOA matrix is calculated by multiplying the significant components with the SD and adding the mean vector (depicted as mu^_H in the figure). In the context of vectors, the multiplication can be considered as the matrix multiplication of the significant components (vector) and the SD (vector).

[0036] In one aspect, when calculating the HOA matrix, the decoding side process interprets the presence of only the mean vector in the bitstream as an instruction to add the mean vector. In another aspect, the received bitstream contains a flag, where the flag controls whether the mean vector is used (on the decoding side) to calculate the HOA matrix.

[0037] Now turning to Figure 2 , Figure 2 shown is an HOA data encoding and decoding system that uses PCA but does not transmit the mean vector in its associated bitstream. And Figure 1Similarly, the encoding here uses PCA. First, the mean vector (e.g., column-wise mean vector) is subtracted from the input HOA matrix to compute the mean-subtracted HOA matrix. Then, principal component analysis (PCA) is performed based on the mean-subtracted HOA matrix to generate the spatial descriptor SD. The difference here is that SD is used to extract salient components directly from the input HOA matrix H, instead of extracting salient components from the mean-subtracted HOA matrix H^. Therefore, when generating the synthetic HOA matrix H^, the reconstruction algorithm (on the decoding side) does not need to use the mean vector, as shown in the figure. Therefore, the mean vector does not need to be transmitted in the bitstream (by the encoding side), thus reducing the bit rate.

[0038] Now for reference Figure 3 The coding system shown here makes dynamic decisions in the analysis block to produce SD, Xi between PCA without mean vector transfer (A) and PCA with mean vector transfer (B). In case B, the coding process then encodes the significant component X^i (which, in conjunction with the above) Figure 1 The method described above uses Wi extraction and its corresponding SD with a mean vector, along with a set tag, to associate with the encoded audio content bitstream. This tag will be interpreted by the decoding side process as to whether to use the mean vector to calculate (synthesize) the HOA matrix, depending on whether the tag is set. In case A, the encoding process is as described above. Figure 2 The process is as described above, and the mean vector marker in the bitstream is not set. If the marker is not set, the mean vector does not need to be transmitted in the bitstream.

[0039] Multi-band HOA encoding and decoding

[0040] Turn now Figure 4This block diagram illustrates a multi-subband encoder and the resulting bitstream. The encoding process transforms the wideband HOA matrix H into at least multiple (B>1) subband HOA matrices H_1, H_2, ..., H_B. The term "wideband" applied to the HOA matrix, spatial descriptor, or salient component means that the HOA matrix, spatial descriptor, or salient component is given in the frequency domain and covers at least two subbands, for example, the full band or all subbands defined for the full bandwidth of the audio content being encoded, or that the HOA matrix, SD, or salient component is given in the time domain. The transformation applied to the wideband HOA matrix can be a filter bank, short-time Fourier transform, discrete cosine transform, or other transformation from the time domain to the frequency domain, or it can be a subband split of the wideband HOA matrix into multiple smaller (narrower bandwidth) subbands. It should also be noted that although each subband HOA matrix still has the same column width M as the broadband HOA matrix H, the heights (number of rows, or N_1, N_2, ..., N_B) of the subband HOA matrices H_1, H_2, ..., H_B can be different from each other, or they can all have the same height. In this case, for the purpose of the analysis block, the input HOA matrix is ​​a subband HOA matrix that is restricted to a specific subband. Therefore, as shown in the figure, a separate analysis operation is performed on each subband HOA matrix, and the resulting SD and the corresponding salient components are restricted to the specific subband.

[0041] Spatial descriptor quantization technique

[0042] The following sections of this disclosure describe various techniques for reducing the number of bits required for quantization to be formatted as a spatial descriptor SD in a bitstream, thereby reducing the bit rate. From Figure 5 start, Figure 5 A quantization technique is shown, in which analysis blocks (e.g., operate on a single subband HOA matrix H_1) are used. Figure 1 The PCA technique generates a single set of SD components. This single set of SD components is then shared by a salient component extraction block, which generates salient components for all subbands (all subbands across the full bandwidth of the encoded audio content). Figure 6This concept is illustrated graphically using an example where the full bandwidth of encoded audio content is divided into four subbands SB1-SB4, though the concept is not limited to this example. It can be seen how, for each subband (spanning the full bandwidth), a single SD line generated by performing analysis operations on the subband HOA of a single subband (in this case, SB1) is reused. In other words, for each subband, a "shared" set of SD components for that specific subband is used, representing the set of significant components extracted for that subband. This reduction in complexity is reflected in a lower bitrate in the bitstream, as the set of SD components generated only for SB1 is formatted as a bitstream. The bitstream may also contain instructions for the reconstruction algorithm running in the decoder, i.e., the sets of SD components for SB2, SB3, and SB4 that are missing from the bitstream but are identical to the set of SD components in the bitstream for SB1.

[0043] according to Figure 5 and Figure 6 A method for encoding HOAs using a shared subband domain SD can be performed as follows. For multiple subbands, the wideband HOA matrix is ​​transformed into at least multiple subband HOA matrices, such as 1, 2, ..., B=4 as shown in the figure. A set of spatial descriptor SD components for the first subband is generated, wherein the set of SD components for the first subband is generated from the first subband HOA matrix among the multiple subband HOA matrices. This can be achieved by subband HOA matrices with the mean subtracted (e.g., based on...). Figure 1 or Figure 2 Principal Component Analysis (PCA) is performed to generate a set of SD components. There are N components in the set of SD components for the first subband, and N components in the corresponding set of each subband salient component, where N is two or more. The set of SD components can be the row N=4 at SB1 in the figure, or in other words, W_1, W_2, ..., W_4. This set of SD components for the first subband is used to extract the corresponding set of subband salient components for each of the multiple subbands. In the figure, the salient components in SB1 are X_1,j, and the salient components in SB2 are X_2,i, etc., which are extracted using the formula H*W. The corresponding set of salient components for a given subband (here, four salient components) is i) extracted using the set of SD components for the first subband and ii) extracted from the corresponding subband HOA matrix for the given subband from the multiple subband HOA matrices. For example, the salient component X_2,i of SB2 is extracted using the formula H_2*W~_i.

[0044] Next, the encoding process may continue to format i) the set of SD components of the first sub-band and ii) the corresponding set of sub-band significant components for each of the multiple sub-bands into an encoded audio content bitstream. Optionally, the encoding process may also quantize i) the set of SD components of the first sub-band and ii) the corresponding set of sub-band significant components for each of the multiple sub-bands to further reduce the bit rate in the bitstream.

[0045] A kind of Figure 5 The encoding process and Figure 6 The concept of shared SD in [the original text] allows for a method compatible with decoding HOA data using shared subband domain spatial descriptors, as follows. This method begins by receiving an encoded audio content bitstream, containing a set of one or more first subband spatial descriptor SD components for a first subband, and lacking a separate set of subband SD components for a second subband. Therefore, refer to [the original text]. Figure 6 For example, there will be four SD components in the bitstream associated with SB1, but these SD components will not exist for SB2 (and in this particular example, these SD components will not exist for the remaining sub-bands, i.e., SB3 and SB4). The method continues to extract from the encoded audio content bitstream i) a set of one or more first sub-band SD components, ii) a set of one or more first sub-band significant components, and iii) a set of one or more second sub-band significant components. Therefore, it continues... Figure 6 For example, four significant components are extracted for SB1 (which correspond to the four SD components associated with SB1 that can also be extracted from the bitstream), and four significant components (not shown) are extracted for SB2. In other words, although the four significant components assigned to SB2 are extracted, the bitstream does not contain a separate set of SD components assigned to SB2. The decoding method continues to use the reconstruction algorithm by calculating the first subband HOA matrix (a synthetic version of H_1, see [link to H_1]) using the first subband SD components and the first subband significant components. Figure 5 ); and calculate the second subband HOA matrix using the first subband SD component and the second subband significant component (the synthesized version of H_2, see...). Figure 5 ).

[0046] The decoding method can further continue its reconstruction algorithm by calculating the subband HOA matrix for all remaining subbands of the encoded audio content bitstream using the first subband SD component. For example, the synthesized version of H_3 (the subband HOA matrix for SB3) is calculated using the formula H_3 = summation(X_3, i * Wi_transpose, where i = 1, 2, ..., N_sc), where N_sc is... Figure 6 The total number of columns in the table.

[0047] Mixed-domain SD quantization for HOA encoding

[0048] Turn now Figure 7 and Figure 8 , Figure 7 and Figure 8 Another HOA data encoding technique is shown, in which multiple subband compressions (bit rate reduction) are present. In this SD quantization technique, at least one SD is generated through time-domain analysis operations, and at least one other SD is generated as a set of SD components, where each SD component is for a corresponding or individual subband. Therefore, refer to Figure 8 The mixed SD estimation graph in the figure shows that SD1 is a single SD (or a single SD component) that “covers” the entire set of subbands (e.g., the full bandwidth of the encoded audio content across the bitstream), rather than a set of SD components across all the individual subbands, resulting in a lower bitrate. This method is used when generating SD2 (a set of four SDs (or SD components) in this example) and when used to generate SD3 and SD4 groups. In contrast, the graph on the left side of the figure shows that if the SD1 group is generated in the same way as the other SD groups (based on individual subbands), then there will be three additional SD components in the SD1 group. Note that each SD group corresponds to a full-band SC. For example, the four SCs derived from the SD2 group can be concatenated into a full-band SC. One method for... Figure 7 and Figure 8 The hybrid domain SD estimation technique for encoding HOA data can be performed as follows. This method involves generating a single wideband spatial descriptor (SD) by analyzing the input HOA matrix. Figure 8 (SD1 in the example). Any of the techniques described above for linear transformation analysis (e.g., PCA, SVD, EVD) can be used, and specifically, a wideband SD can be generated by performing a time-domain analysis operation based on the input HOA matrix. Next, the wideband SD is used to extract wideband salient components from the input HOA matrix.

[0049] Then, for the first sub-band, such as SB1, a set of one or more first sub-band SD components is generated by performing frequency domain analysis operations based on the input HOA matrix. Figure 7 As can be seen, this may involve transforming the (wideband) input HOA matrix into at least a plurality of subband HOA matrices, wherein a set of one or more first subband SD components is generated by performing a frequency domain analysis operation on a subband HOA array constrained to a first subband. Figure 8 In the example, this would be the row of the SD component at SB1. Finally, for the first subband, the method involves extracting a set of one or more first subband salient components from the input HOA matrix using a set of one or more first subband SD components. This can be achieved, for example, by generating a set of one or more second subband SD components for subband SB2 (in... Figure 8 In this context, these components are the components of SD2, SD3, and SD4 in row SB2, and a similar process is performed for the additional subbands using a set of one or more second subband SD components to extract a set of one or more second subband significant components from the input HOA matrix. Alternatively, the encoding method may also include generating the resulting output bitstream by formatting the wideband spatial descriptor, wideband significant components, the set of first subband SD components, the set of first subband significant components, the set of second subband SD components, and the set of second subband significant components into an encoded audio bitstream.

[0050] In other words, still refer to Figure 8 Calculate the first SD (vertically oriented SD1) that "covers" all sub-bands. Figure 7 The first SD (SD1) is calculated based on W~_1, while the remaining three SDs (SD2-SD4 in this case, vertically oriented) are calculated based on each component and each subband. For example, SD2 consists of the following components: W~_1,2 in SB1, W~_2,2 in SB2, W~_3,2 in SB3, and W~_4,2 in SB4. SD3 consists of the following components: W~_1,3 in SB1, W~_2,3 in SB2, W~_3,3 in SB3, and W~_4,3 in SB4. From another perspective, in the multi-subband (SB) HOA compression method described here, at least one individual SD covering the entire bandwidth is calculated, and the other SDs are calculated based on each individual SB.

[0051] refer to Figure 7 This block diagram illustrates how to compute a single SD (a vector W ~_1 of height N rows) in the time domain based on the input HOA matrix H, and then remove its contribution from the target subband HOA_b to produce the residual subband HOA Hbar_b. As shown in the figure, subsequent SDs, W ~_b,i, are computed based on the residual HOA.

[0052] A kind of Figure 7 The encoding process and Figure 8 The conceptual diagram on the right illustrates a method for decoding HOA data using both wideband spatial descriptors and subband spatial descriptors, as follows. This method begins by receiving an encoded audio bitstream containing a temporal spatial descriptor, (corresponding) temporal significant components, a set of one or more first subband spatial descriptors (SDs), and components (also known as the first SD group, or...). Figure 8 The set of (corresponding) SD1 and one or more first subband salient components. Then, using the temporal spatial descriptor and temporal salient components, for example, according to Figure 1 or Figure 2The equation for the synthesized HOA matrix H^ in the reconstruction algorithm shown is used to calculate the contribution to the HOA matrix. A set of one or more first subband SD components and a (corresponding) set of one or more first subband significant components are also used, for example, based on... Figure 7 The equation H^_1 = X^_i * W^_1transpose is shown to calculate the HOA matrix of the first subband.

[0053] continue Figure 8 For example, the decoding method may further receive a set of one or more second subband spatial descriptor (SD) components for the second subband in the encoded audio bitstream (in this example, the row of the SD component at SB2 starts at SD2, then at SD3 and SD4). Additionally, the bitstream will contain a (corresponding) set of one or more second subband salient components for the second subband SB2. The method includes computing the second subband HOA matrix using the set of one or more second subband SD components and the set of one or more second subband salient components.

[0054] More generally, the decoding method involves receiving multiple sets of one or more subband SD components for multiple subbands in an encoded audio bitstream, wherein the multiple subbands together span the full bandwidth of the sound program represented by HOA data. Therefore, in Figure 8 In the example, there exists a set of subband SD components starting from the column at SD2 and moving along the row at SB2, another set of subband SD components starting from the column at SD2 but moving along the row at SB3, and so on up to the row at SB4. Additionally, the method includes receiving multiple sets of one or more subband salient components for multiple subbands in the encoded audio bitstream, or in other words, receiving a set of salient components corresponding to each SD component row (starting from SD2). Finally, the method includes calculating multiple subband HOA matrices using the multiple subband SD components and the multiple subband salient components, wherein the multiple subband HOA matrices together span the full bandwidth of the audio program.

[0055] In Figure 7 In another aspect of the layout-compatible decoding method, besides the N_SC SD groups (i = 1, 2, ..., N_SC), the received bitstream also contains a time-domain SD and a corresponding time-domain SC, and each SD group is divided into B subbands (b = 1, 2, ..., B). The decoding method obtains the "final" synthesized HOA (based on the following formula). Figure 7 (Compatibility concept in encoding methods):

[0056] X^hat_final = X^hat_1 + cascaded subbands (b = 1, 2, ..., B) as sum_{i = 1}^{N_SC}X^hat_{b,i}. Then, X^hat_final can be rendered as a loudspeaker or headphone driver signal for playback.

[0057] Subband correlation number for spatial descriptors used in HOA encoding

[0058] In another technique used to reduce the bit rate of space descriptors, instead of generating the same number of subband space descriptor SD components for each subband and formatting these SD components into a bitstream, as... Figure 10 As shown in the left-hand chart, the number of subband SD components generated and formatted as a bitstream varies with the subband index, such as... Figure 10 The diagram on the right is shown. Therefore, this encoding / decoding technique allows the number of SD components encoded associated with each subband to vary on a per-subband basis. This is in Figure 9 The subbands are represented by different subband indices i, j, ..., k. The first subband (which can be any subband) has index i and may have, for example, four SD components calculated for it through analysis operations, corresponding to i = 1, 2, 3, and 4 (N_sc, i = 4). The second subband (which can be any subband different from the others, such as SB4) has index j and has, for example, two SD components, corresponding to j = 1 and 2 (N_SC, j = 2).

[0059] As an example of a process for encoding and decoding SD based on at least two subbands, consider Figure 10 The diagram illustrates the arrangement of four subbands. When generating salient components (on the encoding side of this process), a different number of salient components are extracted for each subband. Therefore, in Figure 10 In the example, for the first subband, four SD components are generated (in four columns respectively), and therefore four significant components are extracted for the first subband, while for the second subband, only three SD components are generated (and therefore only three significant components are extracted). In other words, each subband is described by a different number of SD components and a corresponding different number of significant components. This means that while SD groups #1 and #2 are full-band (each SD group has components in all four subbands, which in this example can be assumed to span the full bandwidth of the audio program being encoded), SD group #3 is not full-band (it is missing components in subband 4), and SD group #4 is also not full-band (it is missing components in subbands 2 and 4). The missing SD components are essentially omitted from the encoded audio content bitstream, thus reducing the bitrate of the bitstream.

[0060] A method for encoding HOA data by generating a variable number of spatial descriptors for different subbands can be performed as follows (see also...). Figure 9 and Figure 10 (Example). The method includes transforming an input HOA matrix H (with N rows and M columns) into at least a plurality of subband HOA matrices H1, H2, ... . For example, the first subband HOA matrix is ​​analyzed using PCA, SVD, or EVD to produce a first number of one or more spatial descriptor SD components, for example, Figure 10 The SD component row at SB1 is used. Furthermore, a first number of SD components are used to extract a first number of one or more significant components. Additionally, the second subband HOA matrix is ​​analyzed to generate a second number of one or more SD components, for example... Figure 10 The SD component row at SB2. Use a second number of SD components to extract one or more corresponding second number of significant components. The second number differs from the first number, for example, in... Figure 10 In this method, there are 3 SDs for SB2 and 4 SDs for SB1. The method continues to format one or more SD components of a first number, one or more SD components of a second number, one or more significant components of the first number, and one or more significant components of the second number into an encoded audio content bitstream. Now, if the first number of SD components is greater than the second number, the method further includes inserting information into the bitstream that (to the decoding side) indicates that fewer SD components and fewer significant components were encoded for the second subband compared to the first subband. Figure 10 In the example, the absence of two SD components in SD group #4 and the absence of one SD component in SD group #3 results in a decrease in the bit rate in the bit stream because i) no bits in the bit stream are used to encode the missing SD components and missing significant components of the second subband SB2, and ii) no bits are used to encode the missing SD components of the fourth subband SB4.

[0061] It should be noted that the bit rate is further reduced due to the corresponding missing significant components (which do not need to be formatted as a bitstream). This is in Figure 11 The diagram on the right illustrates this, where in this example, group #4 is missing SD in SB3 and SB4, while group #3 is missing SD in SB4. This results in three significant missing components that do not need to be encoded into the bitstream (thus leading to a further reduction in bit rate).

[0062] In one respect, return to reference Figure 9 The first subband HOA matrix H_1 is constrained to the low-frequency band, and the second subband HOA matrix H_2 is constrained to the high-frequency band.

[0063] On the decoding side (not shown) of this encoding / decoding technique that uses a variable number of SD components for different subbands, the incoming bitstream is parsed to extract a first number (set) of SD components associated with a first subband index and a second number (different set) of SD components associated with a second subband index for a given audio program represented by HOA data, and so on for additional subbands. The second number is different from the first number. The reconstruction algorithm continues to use one or more of the first subband SD components of the first number to calculate the first subband HOA matrix, and uses one or more of the second subband SD components of the second number to calculate the second subband HOA matrix. Furthermore, one or more of the third subband SD components of a third number can be extracted from the bitstream (in... Figure 10 The exemplary diagram on the right is represented by two SD components in SB4, where the first quantity is greater than the second quantity, and the second quantity is greater than the third quantity. Similarly, the third subband HOA matrix is ​​calculated using one or more third subband SD components of the third quantity. This is similar to the case where a separate SD is generated for each combination of subband and SD (in... Figure 10 As shown in the diagram on the left, a first number of one or more first subband SD components (e.g., components in row SB1) are constrained to the first subband (e.g., SB1), and a second number of one or more second subband SD components (e.g., components in row SB2) are constrained to a second subband (e.g., SB2) that is different from the first subband.

[0064] Continue with Figure 10 A decoding method compatible with the encoding concept in [the original text] includes a way to calculate the second subband HOA matrix involving vector multiplication, where multiple vector elements corresponding to missing second subband SD components are padded with zeros. Since the second number of SD components is less than the first number, the second subband SD component is missing in the encoded audio content bitstream. This reduces the complexity of the decoding method.

[0065] Recall that for the reconstruction algorithm, it is also necessary to extract a first number of one or more first subband salient components and a second number of one or more second subband salient components from the encoded audio content bitstream. This approach further reduces complexity by multiplying the second number of second subband SD components by the second number of salient components, while padding multiple vector elements corresponding to the missing second subband SD components (which are missing because the second number of second subband salient components is less than the first number of first subband salient components) with zeros to compute the second subband HOA matrix.

[0066] Now for reference Figure 12 , Figure 12 In order to combine Figure 7 and Figure 8A block diagram of an encoding process based on the idea of ​​generating at least one SD in the full bandwidth, which can generate different numbers of significant components for different subbands, such as Figure 10 The diagram on the right is shown. In other words, this method produces both a wideband spatial descriptor and a subband spatial descriptor. Recall that when calculating the salient component X using the following equation, as combined with... Figure 10 The missing SD component W described leads to the corresponding missing significant component X:

[0067] X_B,k=H_B*W~_B,k

[0068] The encoding process now begins with a so-called “wideband analysis” operation performed on the wideband input HOA matrix (matrix H), which can cover all subbands (e.g., all subbands spanning the full bandwidth of the encoded audio content in the bitstream). This produces a wideband spatial descriptor W_1,1, which is then used to extract the wideband (e.g., full bandwidth) salient components X_1,1. This analysis can be performed in the frequency domain over the entire set of defined subbands spanning the full bandwidth of the audio program, or in the time domain, where the wideband input matrix is ​​given in time-domain format. The resulting salient components X_1,1 are represented in the figure by vertical bars that span the entire set of subbands 1, 2, ..., B, or the full bandwidth of the audio program (represented by the HOA data).

[0069] Additionally, for example, after transforming the wideband HOA matrix H into at least several subband HOA matrices H_2, H_3, ..., H_B, another analysis operation is performed on a per-subband basis. It should be noted again that the heights N_2, N_3, ..., N_B of the subband HOA matrices can be different from each other. Next, it is determined whether some subband space descriptors and their corresponding significant components can be omitted from the encoded bitstream. When such processing is completed for all desired subbands, for example, the following is generated: Figure 11 The table on the right shows that the analysis has produced the first spatial descriptor group SD group #1, which has four components in each of the four subbands. This results in the corresponding full-band salient component SC group #1, which also has four components in each of the four subbands (as shown in the column for SC group #1). Similarly, the broadband analysis also produces SC group #2. Each of SC group #1 and SC group #2 can be considered to cover the full bandwidth of the audio program (in this example, it is defined by four subbands, although more generally by two or more subbands). However, the subband analysis for SB3 and SB4 does not produce the complete set of spatial descriptor components (here, four). Specifically, the analysis of SB3 does not produce the components in SD group #4, and the analysis of SB4 does not produce the components in SD group #3 and SD group #4. Therefore, the equations described above for extracting the salient component X do not produce three salient components, as shown in the column for SD group #1. Figure 12As shown, these three significant components are herein referred to as "empty subbands". The SD components and significant components without empty subbands are added to the encoded audio content bitstream, thereby reducing the bit rate.

[0070] On the decoding side of this codec technique (not shown), the processor executes the following method for decoding HOA data that has been encoded using a variable number of spatial descriptors for different subbands. The method may begin with receiving an encoded audio content bitstream including a sequence of audio content frames, where each frame includes encoded HOA data. The processor extracts a first number of one or more first subband spatial descriptors and a second number of one or more subband spatial descriptors from each frame. For example, in Figure 10 , 4 SDs in SB3 and 2 SDs in SB4. Additionally, the process extracts a first number of one or more corresponding first subband significant components and a second number of one or more corresponding subband significant components from each frame. For example, 4 significant components in SB3 and 2 significant components in SB4. Then, for each frame, the processor uses i) the first subband spatial descriptors and the corresponding first subband significant components in the frame, and ii) the second subband spatial descriptors and the corresponding second subband significant components in the frame to calculate the HOA matrix. In each frame, the first number of first subband spatial descriptors may be different from the second number of second subband spatial descriptors. Additionally, the first number of first subband spatial descriptors or the second number of second subband spatial descriptors may vary on a per-frame basis.

[0071] Subband partitioning for changes in each HOA spatial descriptor (SD) group

[0072] Another aspect of the spatial descriptor SD quantization disclosed herein is a multi-subband (SB) HOA data compression technique, where the SB bandwidth partitioning is a function of both the SD index and the SB index. Figure 13 This technique is illustrated in the diagram of Figure 13 , where the number of SDs in each SD group is different and each SD may cover a different SB bandwidth. More specifically, if the i-th SD group has M SDs that together cover N SBs, where M < N, then these SDs transmitted in the bitstream will leave one or more empty SBs. For example, if three SDs in a group are supposed to cover 4 SBs, then in order to fill a single empty SB slot, adjacent SDs may be assigned to cover both their normal SB slots and the empty SB slot. This can be seen in the

[0073] As Figure 13An example of a method for encoding HOA data by effectively changing the width of subband partitions is illustrated below. The method includes analyzing a first subband HOA matrix among multiple subband HOA matrices to generate multiple first subband spatial descriptor (SD) components, such as rows of three SD components at SB2 (which are part of SD groups #2, #3, and #4). Additionally, a second subband HOA matrix among the multiple subband HOA matrices is analyzed to generate multiple one or more second subband SD components, such as rows of two SD components at SB3 (which are part of SD groups #2 and #3). Instructions are then set in the encoded audio content bitstream to indicate which first subband SD component, assigned to a given SD group, will be copied as a second subband SD component assigned to that given SD group. Figure 3 In the example, the instruction indicates that the SD component in SB2, which is part of SD group #4, will be copied as the SD component in SB3, which is assigned to the same SD group #4.

[0074] continue Figure 3 For example, there could be further instructions (set in the bitstream) to indicate that the same SD component (i.e., the SD component in SB2 as part of SD group #4) would be copied as the SD component in SB4 assigned to SD group #4. The method could then proceed to format multiple first subband SD components into an encoded audio content bitstream, and format at least one of multiple second subband SD components into an encoded audio content bitstream, wherein the number of second subband SD components formatted into the encoded audio content bitstream is less than the number of first subband SD components formatted into the encoded audio content bitstream. This results in “empty subband slots” in the spatial descriptor of the bitstream, which the decoding side can then fill in in response to instructions received in the bitstream. Since no bits are used to actually encode the individual SD components of the empty subbands, a bitrate reduction is achieved in the bitstream.

[0075] In this respect, when referring to Figure 13 At that time, the effective width, bandwidth, or vertical extension of SB2 is larger in SD group #4 than in SD group #2 and SD group #3. Additionally, the width of SB3 is larger in SD group #3 than in SD group #2. Regarding the SB2 component of SD group #4, this specific component is generated on the encoding side by analyzing only the second subband HOA. Furthermore, when synthesizing the subband HOA matrices of SB2, SB3, and SB4, this SB2 component of SD group #4 is then used on the decoding side not only as a component of SB2, but also as a component of SB3 and SB4.

[0076] In addition, in this regard, the codec technology effectively variably bandwidth divides (e.g., Bark-level band divides) the combined band of SB3 - SB4 in SD group #3 into two smaller bands SB3 and SB4 in SD group #2 (in Figure 13 the exemplary diagram of). In addition, the combined band of SB2 - SB4 in SD group #4 is divided into three smaller bands SB2, SB3, and SB4 in SD group #2.

[0077] Figure 13 Examples of can also be used to illustrate the following general aspect of this codec technology. If SD groups A and B have M SBs and N SBs respectively (M < N), then some SBs in SD group B are said to have been "merged" to generate SBs in SD group A. For example, if SD group A has 2 SBs and SD group B has 4 SBs, then the first SB and the second SB in SD group B can be merged to generate the first SB in SD group A; the third SB and the fourth SB in SD group B can be merged to generate the second SB in SD group A. Thus, in Figure 13 , SB2 - SB4 are merged to become the second sub-band in SD group #4 (and the other sub-band in SD group #4 is SB1).

[0078] In another aspect, if SD groups A and B have M SBs and N SBs respectively (M < N), then each SD group can be divided into M Bark-level sub-bands and N Bark-level sub-bands respectively.

[0079] In another aspect of the codec technology, referring to Figure 12 and Figure 13 the exemplary diagrams of, the encoding process can generate a time-domain SD for SD group #1, which is the result of a single time-domain analysis operation performed on the wideband input HOA matrix H. This can also be referred to as analyzing the wideband input HOA matrix to generate a wideband spatial descriptor SD. The method also includes using the wideband SD to extract wideband significant components and formatting the wideband SD and the wideband significant components into an encoded audio content bitstream.

[0080] In this case, the method also includes transforming the wideband input HOA matrix into at least multiple sub-band HOA matrices, e.g., corresponding to sub-bands SB1 - SB4. Thus, four separate frequency-domain analysis operations are performed on these four sub-band HOA matrices to generate four components of SD group #2. These same four frequency-domain analysis operations also generate four components for SD group #3; however, for SD group #3, only three of these four components are formatted into an encoded audio content bitstream because the component for SB4 will be copied from the component for SB3 by the decoding side. Similarly, only two SD components of the generated SD components are formatted into a bitstream for SD group #4 because the SD components for SB3 and SB4 will be copied from the SD component for SB2 by the decoding side.

[0081] A method for encoding variable-width data that has been partitioned into subbands based on spatial descriptor groups and then... Figure 13 The example of encoding compatible HOA data can be performed as follows. The process extracts multiple first subband SD components (e.g., in line SB2) and at least one second subband SD component (e.g., in line SB3) from the encoded audio content bitstream, wherein the number of second subband SD components in the bitstream is less than the number of first subband SD components (e.g., SB3 has two SD components in the bitstream, while SB2 has four). At least one second subband SD component is assigned to a first SD group (e.g., SD group #2). Next, the processor uses the multiple first subband SD components to compute a first subband HOA matrix and copies one of the multiple first subband SD components assigned to a second SD group (e.g., SD group #3) according to instructions in the encoded audio content bitstream. Now, the processor also uses i) the at least one second subband SD component assigned to the first SD group (group #2) and ii) the copied first subband SD component assigned to the second SD group (group #3) to compute a second subband HOA matrix (for SB3).

[0082] Additionally, the processor extracts at least one third subband SD component (in line SB4) allocated to the first SD group (group #2) from the encoded audio content bitstream, and uses i) the at least one third subband SD component allocated to the first SD group, and ii) a copy of the first subband SD component allocated to the second SD group (group #3) according to instructions in the encoded audio content bitstream, to calculate the third subband HOA matrix. Furthermore, the processor can also extract wideband SD (e.g., in SD group #1) and corresponding wideband salient components from the encoded audio content bitstream, and use temporal spatial descriptors and temporal salient components to calculate their contribution to the HOA matrix.

[0083] Turn now Figure 14 The diagram uses an example to illustrate a method for encoding higher-order ambient stereo HOA data by combining and sub-banding according to spatial descriptors. References in parentheses below are... Figure 14 The elements in the diagram are for illustrative purposes only. The following methods can be executed to generate a single SD component that covers the merged subband and is assigned to the second SD group (SD group #3), a single SD component that covers only the first subband (SB2) and is assigned to the first SD group (SD group #2), and a single SD component that covers only the first subband (SB2) and is assigned to the second SD group (SD group #3). Figure 14As can be seen, the SD and SC of SD group #1 are calculated based on the full-band HOA input matrix, and can be referred to here as SD_1 and SC_1. Next, the residual HOA matrix is ​​calculated by subtracting the contribution SC_1*SD_1^T from the full-band HOA input matrix, and then divided into four residual sub-band HOAs in SB1-SB4 respectively. Next, the SD and SC of SD group #2 are calculated based on these residual sub-band HOAs. Then, another residual HOA matrix is ​​obtained by subtracting the contribution from SD group #2, and then analyzed to obtain the SD and SC of SD group #3, where in this case, the residual HOA matrix is ​​divided into 3 sub-bands, such as SB1, SB2 and the merged SB3-SB4. Finally, another residual HOA matrix is ​​obtained by removing the contribution from SD group #3, and then analyzed to obtain the SD and SC of SD group #4, where in this case, the residual HOA matrix is ​​divided into 2 sub-bands, such as the merged SB1-SB2 and the merged SB3-SB4. The processor sets instructions in the encoded audio content bitstream to instruct the merged subbands to cover the second subband (SB3) and the third subband (SB4). The bitrate reduction is achieved because, in SD group #3, a single SD component covers the merged subband (instead of two SD components each covering a separate subband).

[0084] It should be noted that in order to generate Figure 14 The SD arrangement in the data may require the following analysis operations: a single broadband or time-domain analysis to generate SD1 (SD group #1); four separate frequency-domain analysis operations to generate the SD components of SD group #2 in four subbands, which also generate the SD components of SD group #3 in subbands SB1 and SB2; a single frequency-domain analysis operation to generate two SD components in SD group #3 and SD group #4 in the merged subbands; and a single frequency-domain analysis operation to generate two SD components in SD group #4, which are in two different merged subbands.

[0085] An encoding and overlay method for subbands that have been merged based on spatial descriptor groups Figure 14 The method for decoding example HOA data can be performed as follows. The method includes extracting from the received encoded audio content bitstream: a single SD component and its corresponding significant component covering only the first subband and assigned to the first SD group; a single SD component and its corresponding significant component covering only the first subband and assigned to the second SD group; and a single SD component and its corresponding significant component covering the merged subband and assigned to the second SD group. Then, according to the instruction in the encoded audio content bitstream indicating that the merged subband covers the second and third subbands, the processor uses the single SD component and its corresponding significant component covering the merged subband to calculate the contribution to the HOA matrix covering the second and third subbands.

[0086] Turn now Figure 15 , Figure 15 An example is used to illustrate the SD quantization technique (of the HOA data codec), where a variable number of SD components may exist in each SD group. Figure 15 and Figure 14 Similarly, the difference lies in the SB bandwidth of SD group #3 and SD group #4. Figure 14 The process differs from the decoding process. On the decoding side, the processor extracts several SD groups and their corresponding significant components in the frequency domain from the received bitstream. The frequency domain spans at least several subbands, for example, SB1-SB4. The encoded audio content bitstream supports a format where the total number of one or more SD components in each SD group can vary depending on the SD group. Furthermore, the bandwidth of each SD component in one or more SD components in the first SD group differs from the bandwidth of each SD component in one or more SD components in the second SD group. Figure 15 In this case, it can be seen that the total number of SDs in SD group #2 is 4, while in SD group #3 it is 3, and in SD group #4 it is only 2. This also means that the bandwidth of each SD component in group #2 is different from the bandwidth of each SD component in group #3. The decoding process continues to use the SD groups extracted from the bitstream and the corresponding significant components to calculate (synthesize) the HOA matrix. Since the number of SD components in groups #3 and #4 is smaller (relative to the number of SD components in group #2), a bit rate reduction is achieved here.

[0087] In order to generate Figure 15 The arrangement of the SD components shown requires the following number of analysis operations: a single broadband or time-domain analysis to generate SD group #1; four frequency-domain analysis operations in SB1-SB4 to generate SD group #2; three frequency-domain analysis operations in three sub-bands divided differently from SB1-SB4 to generate SD group #3; and two frequency-domain analysis operations in two sub-bands divided differently from SB1-SB4 and from the three sub-bands of SD group #3 to generate SD group #4.

[0088] Turn now Figure 16 , Figure 16A graphical view showing an exemplary arrangement of SD components (in an encoded audio bitstream generated by a multi-SB HOA data compression technique), where each SD group in two or more SD groups is represented by a different number of HOA coefficients. If the number of HOA coefficients is M, the corresponding HOA order is sqrt(M)-1. The number of HOA coefficients can be represented by the number of elements in a given SD, or by the width dimension of the HOA matrix H. Generally speaking, for the number of HOA coefficients M (e.g., an input HOA matrix H with N rows and M columns), some of the SD groups generated by an analysis operation performed on the input HOA matrix can be represented by the number of HOA coefficients L, where L < M. For illustration, consider Figure 16 the left diagram, where each SD group has the same number of HOA coefficients, 25, compared to the right diagram (where each of two or more SD groups is represented by a different HOA order), each SD in SD group #3 and SD group #4 has 16 HOA coefficients, while each SD in group #2 has 25 HOA coefficients.

[0089] In addition, in this particular example, SD group #1 has a single wideband SD spanning the full bandwidth of the audio content. The wideband SD can be generated by a time-domain analysis of the input HOA matrix, and then its contribution is removed from the input HOA matrix, resulting in a residual HOA matrix. The remaining SD groups are generated by a frequency-domain analysis of the residual HOA matrix. It should also be noted the number of analysis operations required for each SD group in the diagram: SD group #1 requires a single time-domain analysis operation; SD group #2 has four subbands and thus requires four frequency-domain analysis operations; SD group #3 has three subbands and thus requires three frequency-domain analysis operations; and finally, SD group #4 requires two frequency-domain analysis operations.

[0090] Although certain aspects have been described and shown in the drawings, it should be understood that these aspects are merely illustrative of the invention and not limiting, and the invention is not limited to the specific structures and arrangements shown and described, as various other modified forms can be conceived by those of ordinary skill in the art. Therefore, the description should be regarded as exemplary rather than restrictive.

Claims

1. A method for encoding high-order ambient stereo (HOA) data using principal component analysis or any linear transformation, the method comprising: Subtract the mean vector from the input HOA matrix to calculate the HOA matrix with the mean subtracted; The spatial descriptor SD is generated by performing principal component analysis (PCA) or any linear transformation based on the HOA matrix with the mean subtracted; Extract significant components from the HOA matrix after subtracting the mean; as well as The significant components, the SD, and the mean vector are formatted into an encoded audio content bitstream.

2. The method according to claim 1, wherein the mean vector is a row vector, and each element of the row vector is the average value of the corresponding column in the input HOA matrix.

3. The method according to any one of the preceding claims, wherein performing PCA or any linear transformation comprises: The zero-mean covariance matrix is ​​determined using the HOA matrix minus the mean, and the PCA or linear transformation is performed on the zero-mean covariance matrix.

4. The method of claim 3, wherein determining the zero-mean covariance matrix comprises multiplying the transpose of the HOA matrix minus the mean by the HOA matrix minus the mean.

5. The method according to any one of claims 1 to 2, wherein extracting the significant component comprises multiplying the SD with the HOA matrix minus the mean.

6. The method according to any one of claims 1 to 2, the method further comprising transmitting the encoded audio content bitstream, wherein the encoded audio content bitstream is interpreted by a decoding-side process as adding the mean vector when calculating the HOA matrix.

7. The method of claim 6, wherein the significant component comprises an audio signal, and the method further comprises encoding the audio signal separately from the SD to reduce the bit rate.

8. The method according to any one of claims 1 to 2, further comprising: The broadband HOA matrix is ​​transformed into at least a plurality of subband HOA matrices, wherein the input HOA matrix is ​​a subband HOA matrix that is constrained to a subband, and the SD and the significant component are constrained to the subband.

9. A method for decoding High-Order Environmental Stereo (HOA) data, the method comprising: Receive significant components and spatial descriptors SD, wherein the SD is generated by performing principal component analysis (PCA) or any linear transformation based on the HOA matrix with the mean subtracted; Receive the mean vector; as well as The HOA matrix is ​​calculated by multiplying the significant components by the SD and adding the mean vector.

10. The method of claim 9, wherein the mean vector is a row vector, and each element of the row vector is the mean of the corresponding column in the input HOA matrix.

11. The method according to any one of claims 9 to 10, wherein the significant component and the SD are associated with the mean vector in the encoded audio content bitstream.

12. The method according to any one of claims 9 to 10, wherein the SD is generated by performing principal component analysis (PCA) or any linear transformation on the HOA matrix after subtracting the mean, and the significant components are extracted from the HOA matrix after subtracting the mean.

13. The method according to any one of claims 9 to 10, further comprising: A flag is received, wherein the flag controls whether the mean vector is used to calculate the HOA matrix.

14. The method according to any one of claims 9 to 10, wherein the HOA matrix is ​​a sub-band HOA matrix.

15. A method for encoding high-order environmental stereo (HOA) data using principal component analysis, the method comprising: Subtract the mean vector from the input HOA matrix to calculate the HOA matrix with the mean subtracted; The spatial descriptor SD is generated by performing principal component analysis (PCA) or any linear transformation based on the HOA matrix with the mean subtracted; The SD is used to directly extract significant components from the input HOA matrix; as well as The significant components and the SD are formatted into an encoded audio content bitstream.

16. The method of claim 15, wherein the mean vector is a row vector, and each element of the row vector is the mean of the corresponding column in the input HOA matrix.

17. The method according to any one of claims 15 to 16, further comprising: The significant components and the SD, as well as the mean vector and the marker, are associated with the encoded audio content bitstream, wherein the marker will be interpreted by the decoding-side process as to whether the mean vector is used to calculate the HOA matrix.

18. The method according to any one of claims 15 to 16, further comprising: The broadband HOA matrix is ​​transformed into at least a plurality of subband HOA matrices, wherein the input HOA matrix is ​​one of the subband HOA matrices.

Images