Internal channel processing method and device for realizing low-complexity format conversion

Abstract

A method for processing an audio signal according to an embodiment of the present invention comprises the following steps: receiving an audio bit stream encoded by MPEG Surround 212 (MPS212); generating an internal channel signal for a dual channel unit (CPE) based on the received audio bit stream and an equalization (EQ) value and a gain value in a rendering parameter for an MPS212 output channel defined in a format converter; and generating a stereo output signal based on the generated internal channel signal.

Description

[0001] This application is a divisional application of a Chinese patent application with an application date of June 17, 2016, application number 201680035415X, and invention name “Internal channel processing method and device for realizing low-complexity format conversion”. Technical Field

[0002] The present invention relates to an intra-channel (IC) processing method and apparatus for realizing low-complexity format conversion, and more particularly, to a method and apparatus for reducing the number of ICs in a format converter by performing IC processing on input channels in a stereo output layout environment, thereby reducing the number of covariance operations performed in the format converter. Background Art

[0003] According to MPEG-H 3D Audio, various types of signals can be processed and the types of input / output can be easily controlled. Therefore, MPEG-H 3D Audio can be used as a solution for next-generation audio signal processing. In addition, according to the trend of device miniaturization, the percentage of audio reproduction with the help of mobile devices in a stereo reproduction environment has increased.

[0004] When an immersive audio signal implemented through a multi-channel system such as 22.2 channels is transmitted to a stereo reproduction system, all input channels are decoded and the immersive audio signal is downmixed to be converted into a stereo format.

[0005] As the number of input channels increases and the number of output channels decreases, the complexity of the decoder required for covariance analysis and phase alignment in the above process increases. This increase in complexity not only affects the operating speed of the mobile device, but also affects the battery consumption of the mobile device. Summary of the invention

[0006] Technical issues

[0007] As mentioned above, the number of input channels is increased to provide immersive audio, while the number of output channels is reduced to achieve portability. In this environment, the complexity of format conversion during decoding becomes a problem.

[0008] To address this problem, the present invention provides a reduction in the complexity of format conversion in the decoder.

[0009] Technical Solution

[0010] In order to achieve the above-mentioned objects, representative features of the present invention are as follows.

[0011] According to one aspect of the present invention, a method for processing an audio signal is provided, the method comprising: receiving an audio bit stream encoded by MPEG Surround 212 (MPS212); generating an internal channel (IC) signal for a single two-channel unit (CPE) based on the received audio bit stream, an equalization (EQ) value of an MPS212 output channel defined in a format converter, and a gain value of the MPS212 output channel; and generating a stereo output signal based on the generated IC signal.

[0012] The generation of the IC signal may include: upmixing the received audio bitstream into a signal for a channel pair contained in the single CPE based on a channel level difference (CLD) contained in the MPS212 payload; scaling the upmixed bitstream based on the EQ value and the gain value; and mixing the scaled bitstream.

[0013] The generating of the IC signal may further include determining whether the IC signal for the single CPE is generated.

[0014] Whether to generate the IC signal for the single CPE may be determined based on whether the channel pairs included in the single CPE belong to the same IC group.

[0015] When the channel pairs included in a single CPE are all included in the left IC group, the IC signal can be output only through the left output channel of the stereo output channel. When the channel pairs included in a single CPE are all included in the right IC group, the IC signal can be output only through the right output channel of the stereo output channel.

[0016] When channel pairs included in a single CPE are all included in a central IC group or channel pairs included in a single CPE are all included in a low frequency effect (LFE) IC group, the IC signals may be uniformly output through left and right output channels among stereo output channels.

[0017] The audio signal may be an immersive audio signal.

[0018] The generating of the IC signal may further include: calculating an IC gain (ICG); and applying the ICG.

[0019] According to another aspect of the present invention, a device for processing an audio signal is provided, the device comprising: a receiver configured to receive an audio bit stream encoded by MPEG Surround 212 (MPS212); an internal channel (IC) signal generator configured to generate an IC signal for a single dual channel unit (CPE) based on the received audio bit stream, an equalization (EQ) value of an MPS212 output channel defined in a format converter, and a gain value of the MPS212 output channel; and a stereo output signal generator configured to generate a stereo output signal based on the generated IC signal.

[0020] The IC signal generator may be configured to: upmix the received audio bitstream to a signal for a channel pair contained in the single CPE based on a channel level difference (CLD) contained in an MPS212 payload; scale the upmixed bitstream based on the EQ value and the gain value; and mix the scaled bitstream.

[0021] The IC signal generator may be configured to determine whether the IC signal for a single CPE is generated.

[0022] Whether to generate the IC signal may be determined based on whether the channel pairs included in a single CPE belong to the same IC group.

[0023] When the channel pairs included in a single CPE are all included in the left IC group, the IC signal can be output only through the left output channel of the stereo output channel. When the channel pairs included in a single CPE are all included in the right IC group, the IC signal can be output only through the right output channel of the stereo output channel.

[0024] When channel pairs included in a single CPE are all included in a central IC group or channel pairs included in a single CPE are all included in a low frequency effect (LFE) IC group, the IC signals may be uniformly output through left and right output channels among stereo output channels.

[0025] The audio signal may be an immersive audio signal.

[0026] The IC signal generator may be configured to calculate an IC gain (ICG) and apply the ICG.

[0027] According to another aspect of the present invention, a computer-readable recording medium is provided, on which a computer program for executing the aforementioned method is recorded.

[0028] According to other embodiments of the present invention, other methods, other systems, and computer-readable recording media having recorded thereon a computer program for executing the methods are provided.

[0029] Beneficial Effects

[0030] According to the present invention, the number of channels input to the format converter is reduced by using an internal channel (IC), so the complexity of the format converter can be reduced. In more detail, since the number of channels input to the format converter is reduced, the covariance analysis to be performed in the format converter is simplified, thus reducing the complexity of the format converter. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a block diagram of a decoding structure for converting a 24 input channel format into stereo output channels according to one embodiment.

[0032] Figure 2 is a block diagram of a decoding structure that converts a 22.2 channel immersive audio signal format into stereo output channels using 13 internal channels (ICs) according to one embodiment.

[0033] Figure 3 An embodiment is illustrated in which a single IC is generated from a single dual channel unit (CPE).

[0034] Figure 4 is a detailed block diagram of an IC gain (ICG) application unit of a decoder according to an embodiment of the present invention applying ICG to an IC signal.

[0035] Figure 5 is a block diagram showing decoding when an encoder pre-processes ICG according to an embodiment of the present invention.

[0036] Figure 6 The flowchart shows an IC processing method in a structure that performs mono band replication (SBR) decoding and then performs MPEG surround (MPS) decoding when CPE is output through a stereo reproduction layout according to an embodiment of the present invention.

[0037] Figure 7 The flowchart shows an IC processing method in a structure in which MPS decoding is performed and then stereo SBR decoding is performed when CPE is output through a stereo reproduction layout according to an embodiment of the present invention.

[0038] Figure 8 is a block diagram of an IC processing method in a structure using a stereo SBR when a quad channel unit (QCE) is output through a stereo reproduction layout according to an embodiment of the present invention.

[0039] Fig. 9 is a block diagram of an IC processing method in a structure using stereo SBR when outputting QCE through a stereo reproduction layout according to another embodiment of the present invention.

[0040] Fig.10a An embodiment is shown in which the temporal envelope grid is determined when the start boundaries of the first envelope are the same and the stop boundaries of the last envelope are the same.

[0041] Fig.10b An embodiment of determining the temporal envelope grid when the start boundaries of the first envelope are different and the stop boundaries of the last envelope are the same is shown.

[0042] Fig.10c An embodiment of determining a temporal envelope grid when the start boundaries of the first envelopes are the same and the stop boundaries of the last envelopes are different is shown.

[0043] Fig.10d An embodiment of determining a temporal envelope grid when the start boundary of the first envelope is different and the stop boundary of the last envelope is different is shown.

[0044] Table 1 shows an embodiment of a mixing matrix of a format converter for rendering a 22.2 channel immersive audio signal into a stereo signal.

[0045] Table 2 shows an embodiment of a mixing matrix of a format converter for rendering a 22.2 channel immersive audio signal into a stereo signal using an IC.

[0046] Table 3 shows a CPE structure for configuring 22.2 channels using IC according to an embodiment of the present invention.

[0047] Table 4 shows the types of ICs corresponding to decoder input channels according to an embodiment of the present invention.

[0048] Table 5 shows the positions of channels additionally defined according to IC types according to an embodiment of the present invention.

[0049] Table 6 shows format converter output channels corresponding to IC types, and gains and EQ indexes to be applied to each format converter output channel according to an embodiment of the present invention.

[0050] Table 7 shows the syntax of ICGConfig according to an embodiment of the present invention.

[0051] Table 8 shows the syntax of mpegh3daExtElementConfig() according to an embodiment of the present invention.

[0052] Table 9 shows the syntax of usacExtElementType according to an embodiment of the present invention.

[0053] Table 10 shows the syntax of speakerLayoutType according to an embodiment of the present invention.

[0054] Table 11 shows the syntax of SpeakerConfig3d() according to an embodiment of the present invention.

[0055] Table 12 shows the syntax of immersiveDownmixFlag according to an embodiment of the present invention.

[0056] Table 13 shows the syntax of SAOC3DgetNumChannels() according to an embodiment of the present invention.

[0057] Table 14 shows the syntax of the channel allocation order according to an embodiment of the present invention.

[0058] Table 15 shows the syntax of mpegh3daChannelPairElementConfig() according to an embodiment of the present invention.

[0059] Table 16 shows a decoding scenario of MPS and SBR determined based on a channel unit and a reproduction layout according to an embodiment of the present invention.

[0060] Best Mode for Carrying Out the Invention

[0061] In order to achieve the above-mentioned objects, representative features of the present invention are as follows.

[0062] The method for processing an audio signal includes: receiving an audio bit stream encoded by MPEG Surround 212 (MPS212); generating an internal channel (IC) signal for a single two-channel unit (CPE) based on the received audio bit stream, an equalization (EQ) value of an MPS212 output channel defined in a format converter, and a gain value of the MPS212 output channel; and generating a stereo output signal based on the generated IC signal. DETAILED DESCRIPTION

[0063] The present invention will now be described in detail with reference to the accompanying drawings showing specific embodiments of the present invention. These embodiments are provided to make the present disclosure thorough and complete, and to fully convey the concept of the present invention to those of ordinary skill in the art. It should be understood that the various embodiments of the present invention are different from each other, but are not mutually exclusive.

[0064] For example, without departing from the spirit and scope of the present invention, the specific shapes, specific structures and specific features described in the specification may be different in one embodiment from another embodiment. It is also understood that the position or layout of each element in each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the following detailed description should be considered to be only descriptive, not for the purpose of limitation, and the scope of the present invention should be defined by the appended claims and their equivalents.

[0065] Throughout the specification, like reference numerals in the drawings represent the same or similar elements. In the drawings, in order to simplify the description, parts not related to the description are omitted, and the same reference numerals always represent the same elements.

[0066] Hereinafter, the present invention will be described in detail by explaining exemplary embodiments of the present invention with reference to the accompanying drawings. However, the present invention may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein.

[0067] Throughout the specification, when an element is referred to as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or may be electrically connected or coupled to the other element with an intervening element interposed therebetween. In addition, when used in this specification, the term "includes" or "comprising" indicates the presence of the element, but does not exclude the presence or addition of one or more other elements.

[0068] The terms used herein are defined as follows.

[0069] The internal channel (IC) is a virtual intermediate channel used in format conversion, and stereo output is considered in order to remove unnecessary operations generated during MPS212 (MPEG Surround Sound) upmixing and format converter (FC) downmixing.

[0070] The IC signal is a monaural signal mixed in the format converter to provide a stereo signal, and is generated using an IC gain (ICG).

[0071] The IC process means a process in which an IC signal is generated by the MPS 212 decoding block and is performed in the IC processing block.

[0072] ICG represents the gain calculated based on the channel level difference (CLD) value and the format conversion parameters and applied to the IC signal.

[0073] The IC group indicates the type of IC determined based on the core codec output channel position, and the core codec output channel position and the IC group are defined in Table 4 which will be described later.

[0074] The present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown.

[0075] Figure 1 is a block diagram of a decoding structure for converting a 24 input channel format into stereo output channels according to one embodiment.

[0076] When a multi-channel input bitstream is transmitted to a decoder, the decoder downmixes the input channel layout according to the output channel layout of the reproduction system. For example, when a 22.2 channel input signal conforming to the MPEG standard is composed of Figure 1 When reproducing the stereo channel output system shown, the format converter 130 included in the decoder downmixes the 24 input channel layout into a 2 output channel layout according to the format converter rules specified in the format converter 130.

[0077] The 22.2 channel input signal input to the decoder includes a two-channel element (CPE) bit stream 110 obtained by downmixing signals for two channels included in a single CPE. Since the CPE bit stream is encoded by MPS212 (MPEG-based stereo), the CPE bit stream is decoded by MPS212 120. In this case, the LFE channel, that is, the woofer channel, is not included in the CPE bit stream. Therefore, the 22.2 channel input signal input to the decoder includes bit streams for 11 CPEs and bit streams for two woofer channels.

[0078] When MPS212 decoding is performed on the CPE bitstream constituting the 22.2 channel input signal, two MPS212 output channels, 121 and 122, are generated for each CPE, and these two MPS212 output channels 121 and 122 become input channels of the format converter 130. Figure 1 In the case of two woofer channels, the number N of input channels of the format converter 130 is in is 24. Therefore, the format converter 130 should perform 24*2 downmixing.

[0079] The format converter 130 performs phase alignment according to the covariance analysis to prevent the timbre distortion from occurring due to the difference between the phases of the multi-channel signals. In this case, since the covariance matrix has N in ×N in Dimension, theoretically should be implemented (N in ×(N in -1) / 2+N in )×71 frequency bands×2×16× (48000 / 2048) complex multiplications are performed to analyze the covariance matrix.

[0080] When the number of input channels N in When φ is 24, four operations should be performed for one complex multiplication, and a performance of about 64 million operations per second (MOPS) is required.

[0081] Table 1 shows an embodiment of a mixing matrix of a format converter for rendering a 22.2 channel immersive audio signal into a stereo signal.

[0082] In the mixing matrix of Table 1, the 24 numbered input channels are represented on the horizontal axis 140 and the vertical axis 150. The order of the numbered 24 input channels does not have any particular relevance in the covariance analysis. In the embodiment shown in Table 1, when the value of each element of the mixing matrix is ​​1 (as shown by reference numeral 160), the covariance analysis is necessary, but when the value of each element of the mixing matrix is ​​0 (as shown by reference numeral 170), the covariance analysis can be omitted.

[0083] For example, in the case where input channels (e.g., channels CM_M_L030 and CH_M_R030) are not mixed with each other during format conversion to a stereo output layout, the value of the element in the mixing matrix corresponding to the unmixed input channel is 0, and thus the covariance analysis between the unmixed channels CM_M_L030 and CH_M_R030 can be omitted.

[0084] Therefore, 128 covariance analyses of input channels that are not mixed with each other can be excluded from the 24*24 covariance analyses.

[0085] In addition, since the mixing matrix is ​​configured to be symmetrical according to the input channels, the mixing matrix of Table 1 is divided into a lower part 190 and an upper part 180 about the diagonal line, and the covariance analysis of the area corresponding to the lower part 190 can be omitted. In [Table 1]:

[0086]

[0087] Furthermore, since the covariance analysis was performed only on the bold portion of the region corresponding to the upper portion 180 , 236 covariance analyses were finally performed.

[0088] When the value of the mixing matrix is ​​0 (channels are not mixed with each other) and unnecessary covariance analysis is removed based on the symmetry of the mixing matrix, 236×71 bands×2×16×(48000 / 2048) complex multiplications should be performed for the covariance analysis.

[0089] Therefore, in this case, a performance of 50 MOPS is required, and thus the system load due to the covariance analysis is reduced compared to the case where the covariance analysis is performed on the entire portion of the mixing matrix.

[0090] Figure 2 is a block diagram of a decoding structure that converts a 22.2-channel immersive audio signal format into stereo output channels using 13 ICs according to one embodiment.

[0091] MPEG-H 3D Audio uses CPE to more efficiently deliver multi-channel audio signals in constrained transmission environments. When the two channels corresponding to a single channel pair are mixed into a stereo layout, the IC correlation (ICC) is set to 1 so that the decorrelator is not applied. Therefore, the two channels have the same phase information.

[0092] In other words, when a channel pair included in each CPE is determined by considering a stereo output, the upmixed channel pair has the same panning coefficient, which will be described later.

[0093] A single IC is generated by mixing two in-phase channels contained in a single CPE. When two input channels contained in the IC are converted to stereo output channels, the single IC signal is downmixed according to a mixing gain and an equalization (EQ) value based on a conversion rule of a format converter. In this case, since the two channels contained in the single CPE are in-phase channels, a process of aligning phases between channels after downmixing is not required.

[0094] The stereo output signals of the MPS212 upmixer are not out of phase with each other. Figure 1 This is not taken into account in the embodiment of , and therefore the complexity is unnecessarily increased. When the reproduction layout is a stereo layout, the number of input channels of the format converter can be reduced by using a single IC instead of the upmixed CPE channel pairs as the input of the format converter.

[0095] according to Figure 2 In the illustrated embodiment, instead of each CPE bitstream 210 undergoing upmixing by MPS 212 to produce two channels, each CPE bitstream 210 undergoes IC processing 220 to generate a single IC 221. In this case, since the subwoofer channel does not form a CPE, each subwoofer channel signal becomes an IC signal.

[0096] according to Figure 2In the embodiment, in the case of 22.2 channels, 13 ICs (i.e., N ICs) including 11 CPE ICs for general channels and ICs for 2 subwoofer channels are included. in =13), which theoretically becomes the input channel of the format converter 230. Therefore, the format converter 230 performs 13*2 downmixing.

[0097] In the case of such a stereo reproduction arrangement, utilizing the IC further eliminates unnecessary processing generated in the process of upmixing through the MPS 212 and then downmixing through format conversion, thereby further reducing the complexity of the decoder.

[0098] When the mixing matrix M is used for two output channels i and j of a single CPE Mix When the value of (i,j) is 1, the inter-channel correlation (ICC) ICC l，m Set to 1, and decorrelation and residual processing can be omitted.

[0099] IC is defined as a virtual intermediate channel corresponding to the input of the format converter. Figure 2 As shown, each IC processing block 220 generates an IC signal by using an MPS 212 payload (e.g., CLD) and rendering parameters (e.g., EQ value and gain value). The EQ value and gain value represent rendering parameters for output channels of the MPS 212 block defined in the conversion rule table of the format converter.

[0100] Table 2 shows one embodiment of a mixing matrix of a format converter for rendering a 22.2 channel immersive audio signal into a stereo signal using an IC.

[0101]

Table 2

[0102] A B C D E F G H I J K L M A 1 1 1 1 1 1 1 1 1 1 1 1 1 B 1 1 1 1 1 1 1 1 1 1 1 1 1 C 1 1 1 1 1 1 1 1 1 1 1 1 1 D 1 1 1 1 1 1 1 1 1 1 1 1 1 E 1 1 1 1 1 1 1 1 1 1 1 1 1 F 1 1 1 1 1 1 1 1 1 0 0 0 0 G 1 1 1 1 1 1 1 1 1 0 0 0 0 H 1 1 1 1 1 1 1 1 1 0 0 0 0 I 1 1 1 1 1 1 1 1 1 0 0 0 0 J 1 1 1 1 1 0 0 0 0 1 1 1 1 K 1 1 1 1 1 0 0 0 0 1 1 1 1 L 1 1 1 1 1 0 0 0 0 1 1 1 1 M 1 1 1 1 1 0 0 0 0 1 1 1 1

[0103] Similar to Table 1, the horizontal and vertical axes of the mixing matrix of Table 2 represent the indexes of the input channels, and the order of the indexes is not very important in the covariance analysis.

[0104] As described above, since a general mixing matrix has symmetry based on a diagonal line, the mixing matrix of Table 2 is also divided into an upper part and a lower part based on a diagonal line, and thus the covariance analysis for a selected part of the two parts can be omitted. The covariance analysis of the input channels that are not mixed during the format conversion to the stereo output channel layout can also be omitted.

[0105] However, compared with the embodiment of Table 1, according to the embodiment of Table 2, 13 channels including 11 ICs consisting of general channels and 2 woofer channels are downmixed into stereo output channels, and the number of input channels N of the format converter is in It's 13.

[0106] Therefore, according to the embodiment using IC as shown in Table 2, performing 75 covariance analyses theoretically requires a performance of 19 MOPS. Therefore, compared with the case where IC is not used, the load of the format converter caused by the covariance analysis can be greatly reduced.

[0107] The downmix matrix M for downmixing is defined in the format converter Dmx , and using M Dmx The calculated mixing matrix M Mix ,as follows:

[0108]

[0109] Each OTT decoding block outputs two channels corresponding to channel numbers i and j, and the mixing matrix M Mix If it is 1, it is set to ICC l,m =1, thus calculating the upmix matrix of and In this way, each OTT decoding block does not use a decorrelator.

[0110] Table 3 shows a CPE structure for configuring 22.2 channels using IC according to an embodiment of the present invention.

[0111]

Table 3

[0112]

[0113] When the 22.2-channel bitstream has a structure as shown in Table 3, 13 ICs may be defined as ICH_A to ICH_M, and a mixing matrix of the 13 ICs may be determined as in Table 2.

[0114] The first column of Table 3 indicates the index of the input channel, and the first row thereof indicates whether the input channel constitutes a CPE, a mixing gain applied to the stereo channel, and the index of the IC.

[0115] For example, when CM_M_000 and CM_L_000 are ICH_A ICs included in a single CPE, the value of the mixing gain to be applied to the left and right output channels, respectively, to upmix the CPE to the stereo output channel is 0.707. In other words, the signals upmixed to the left and right output channels are reproduced at the same size.

[0116] As another example, when CH_M_L135 and CH_U_L135 are ICH_F ICs included in a single CPE, the value of the mixing gain to be applied to the left output channel is 1, and the value of the mixing gain to be applied to the right output channel is 0, so that the CPE is upmixed to the stereo output channel. In other words, all signals are reproduced only through the left output channel and not through the right output channel.

[0117] On the other hand, when CH_M_R135 and CH_U_R135 are ICH_F IC included in a single CPE, the value of the mixing gain to be applied to the left output channel is 0 and the value of the mixing gain to be applied to the right output channel is 1 to upmix the CPE to the stereo output channel. In other words, all signals are reproduced only through the right output channel and not through the left output channel.

[0118] Figure 3 is a block diagram of an apparatus for generating a single IC from a single CPE according to one embodiment.

[0119] The IC for a single CPE may be generated by applying format conversion parameters (eg, CLD, gain, and EQ) in the quadrature mirror filter (QMF) domain to the downmixed mono signal.

[0120] Figure 3 The IC generating device includes an upmixer 310 , a scaler 320 , and a mixer 330 .

[0121] In the case where the CPE signal 340 obtained by downmixing the signals for the pair of channels CH_M_000 and CH_L_000 is input, the upmixer 310 upmixes the CPE signal 340 by using the CLD parameter. The CPE signal 340 may be upmixed into a signal 351 for CH_M_000 and a signal 352 for CH_L_000 by the upmixer 310, and the upmixed signals 351 and 352 may maintain the same phase and may be mixed together in the format converter.

[0122] The CH_M_000 channel signal 351 and the CH_L_000 channel signal 352 as the upmixing result are scaled in units of subbands using gain and EQ values ​​corresponding to a conversion rule defined in the format converter using a scaler 320 and a scaler 321, respectively.

[0123] When scaled signals 361 and 362 are generated as a result of scaling the pair of channels CH_M_000 and CH_L_000, the mixer 330 mixes the scaled signals 361 and 362 and power-normalizes the mixed result to generate an IC signal ICH_A 370, which is an intermediate channel signal for format conversion.

[0124] In this case, the ICs for the mono unit (SCE) and subwoofer channels that do not utilize CLD upmixing are the same as the original input channels.

[0125] Since the core codec output using IC is performed in the hybrid QMF domain, the processing of ISOIEC23308-3 10.3.5.2 is not performed. In order to allocate each channel of the core codec, additional channel allocation rules and downmixing rules are defined as shown in Table 4-6.

[0126] Table 4 shows the types of ICs corresponding to decoder input channels according to an embodiment of the present invention.

[0127]

Table 4

[0128]

[0129] The IC corresponds to an intermediate channel between the format converter and the input channel of the core encoder, and includes four types of ICs, ie, a woofer channel, a center channel, a left channel, and a right channel.

[0130] When different types of channels represented by CPE have the same IC type, the format converter has the same panning coefficient and the same mixing matrix, so IC can be used. In other words, when two channels included in CPE have the same IC type, IC processing is possible, and therefore it is necessary to configure CPE with channels having the same IC type.

[0131] When the decoder input channel corresponds to a subwoofer channel, ie, CH_LFE1, CH_LFE2, or CH_LFE3, the IC type of the decoder input channel is determined as the subwoofer channel CH_I_LFE.

[0132] When the decoder input channel corresponds to a center channel, ie, CH_M_000, CH_L_000, CH_U_000, CH_T_000, CH_M_180, or CH_U_180, the IC type of the decoder input channel is determined as the center channel CH_I_CNTR.

[0133] When the decoder input channel corresponds to the left channel, i.e., CH_M_L022, CH_M_L030, CH_M_L045, CH_M_L060, CH_M_L090, CH_M_L110, CH_M_L135, CH_M_L150, CH_L_L045, CH_U_L045, CH_U_L030, CH_U_L045, CH_U_L090, CH_U_L110, CH_U_L135, CH_M_LSCR or CH_M_LSCH, the IC type of the decoder input channel is determined as the left channel CH_I_LEFT.

[0134] When the decoder input channel corresponds to the right channel, i.e., CH_M_R022, CH_M_R030, CH_M_R045, CH_M_R060, CH_M_R090, CH_M_R110, CH_M_R135, CH_M_R150, CH_L_R045, CH_U_R045, CH_U_R030, CH_U_R045, CH_U_R090, CH_U_R110, CH_U_R135, CH_M_RSCR or CH_M_RSCH, the IC type of the decoder input channel is determined as the right channel CH_I_RIGHT.

[0135] Table 5 shows the positions of channels additionally defined according to IC types according to an embodiment of the present invention.

[0136]

Table 5

[0137]

[0138] CH_I_LFE is the woofer channel and is located at an elevation angle of 0 degrees, and CH_I_CNTR corresponds to the channel with both elevation and azimuth angles of 0 degrees. CH_I_LFET corresponds to the channel with an elevation angle of 0 degrees and an azimuth angle of 30 degrees to 60 degrees to the left, and CH_I_RIGHT corresponds to the channel with an elevation angle of 0 degrees and an azimuth angle of 30 degrees to 60 degrees to the right.

[0139] In this case, the position of the redefined IC is not the relative position between the channels but the absolute position relative to a reference point.

[0140] The IC can even be applied to a quadruple channel element (QCE) composed of a CPE pair, which will be described later.

[0141] Two methods can be used to produce ICs.

[0142] The first method is pre-processing in the MPEG-H 3D Audio encoder, while the second method is post-processing in the MPEG-H 3D Audio decoder.

[0143] When IC is used in MPEG, Table 5 may be added as a new row to ISO / IEC 23008-3 Table 90.

[0144] Table 6 shows format converter output channels corresponding to IC types, and gain and EQ indicators to be applied to each format converter output channel according to an embodiment of the present invention.

[0145]

Table 6

[0146] source destination Gain EQ Index CH_I_CNTR CH_M_L030, CH_M_R030 1.0 0(off) CH_I_LFE CH_M_L030, CH_M_R030 1.0 0(off) CH_I_LEFT CH_M_L030 1.0 0(off) CH_I_RIGHT CH_M_L030 1.0 0(off)

[0147] In order to use IC, additional rules such as Table 6 should be added in the format converter.

[0148] The IC signal is generated by taking into account the gain value and the EQ value of the format converter. Therefore, an additional conversion rule with a gain value of 1 and an EQ index of 0 as shown in Table 6 may be used to generate the IC signal.

[0149] When the IC type is CH_I_CNTR corresponding to the center channel or CH_I_LFE corresponding to the woofer channel, the output channels are CH_M_L030 and CH_M_R030. At this time, because the gain value is determined to be 1, the EQ index is determined to be 0, and the two stereo output channels are fully used, each output channel signal should be multiplied by 1 / √2 in order to maintain the power of the output signal.

[0150] When the IC type is CH_I_LEFT corresponding to the left channel, the output channel is CH_M_L030. At this time, since the gain value is determined to be 1, the EQ index is determined to be 0, and only the left output channel is used, a gain of 1 is applied to CH_M_L030 and a gain of 0 is applied to CH_M_R030.

[0151] When the IC type is CH_I_RIGHT corresponding to the right channel, the output channel is CH_M_R030. At this time, since the gain value is determined to be 1, the EQ index is determined to be 0, and only the right output channel is used, a gain of 1 is applied to CH_M_R030, and a gain of 0 is applied to CH_M_L030.

[0152] In this case, the general format conversion rules are applied to the SCE channels whose IC and input channels are the same.

[0153] When IC is used in MPEG, Table 6 may be added as a new row to ISO / IEC 23008-3 Table 96.

[0154] Tables 7-15 show a portion of the existing standards that needs to be modified in order to utilize IC in MPEG.

[0155] Table 7 shows the syntax of ICGConfig according to an embodiment of the present invention.

[0156]

Table 7

[0157]

[0158] The ICGconfig shown in Table 7 defines the type of processing to be performed in the IC processing block.

[0159] ICGDisabledPresent indicates whether at least one IC process for CPE is disabled due to channel allocation. In other words, ICGDisabledPresent is an indicator indicating whether at least one ICGDisabledCPE has a value of 1.

[0160] ICGDisabledCPE indicates whether IC processing for each CPE is disabled due to channel allocation. In other words, ICGDisabledCPE is an indicator indicating whether each CPE uses IC.

[0161] ICGPreAppliedPresent indicates whether the ICG has been considered to encode at least one CPE.

[0162] ICGPreAppliedCPE is an indicator showing whether the ICG has been considered to encode each CPE (ie, whether the ICG has been pre-processed in the encoder).

[0163] When ICGAppliedPresent is set to 1 for each CPE, ICGPreAppliedCPE (which is a 1-bit flag of ICGPreAppliedCPE) is read out. In other words, it is determined whether the ICG should be applied to each CPE, and when it is determined that the ICG should be applied to each CPE, it is determined whether the ICG has been preprocessed in the encoder. If it is determined that the ICG has been preprocessed in the encoder, the decoder does not apply the ICG. On the other hand, if it is determined that the ICG has not been preprocessed in the encoder, the decoder applies the ICG.

[0164] When the immersive audio input signal is encoded by MPS212 using CPE or QCE and the output layout is a stereo layout, the core codec decoder generates an IC signal in order to reduce the number of input channels of the format converter. In this case, for the CPE whose ICGDisabledCPE is set to 1, the IC signal generation is omitted. The IC process corresponds to the process of multiplying the decoded mono signal by the ICG, and the ICG is calculated based on the CLD and the format conversion parameters.

[0165] ICGDisabledCPE[n] indicates whether the nth CPE may be subjected to IC processing. When two channels included in the nth CPE belong to the same channel group defined in Table 4, the nth CPE can be subjected to IC processing and ICGDisabledCPE[n] is set to 0.

[0166] For example, when CH_M_L060 and CH_T_L045 in the input channels constitute a single CPE, since the two channels belong to the same channel group, ICGDisabledCPE[n] can be set to 0, and IC of CH_I_LEFT can be generated. On the other hand, when CH_M_L060 and CH_M_000 in the input channels constitute a single CPE, since the two channels belong to different channel groups, ICGDisabledCPE[n] is set to 1, and IC processing is not performed.

[0167] For a QCE including a CPE pair, in the case where the QCE is configured with four channels belonging to one group (1), or in the case where the QCE is configured with two channels belonging to one group and two channels belonging to another group (2), IC processing is possible and ICGDisableCPE[n] and ICGDisableCPE[n+1] are both set to 0.

[0168] As an example in case (1), when QCE is configured with four channels of CH_M_000, CH_L_000, CH_U_000, and CH_T_000, IC processing is possible, and the IC type of QCE is CH_I_CNTR. As an example in case (2), when QCE is configured with four channels of CH_M_L060, CH_U_L045, CH_M_R060, and CH_U_R045, IC processing is possible, and the IC type of QCE is CH_I_LEFT and CH_I_RIGHT.

[0169] Except for cases (1) and (2), ICGDisableCPE[n] and ICGDisableCPE[n+1] of the CPE pair constituting the corresponding QCE should both be set to 1.

[0170] When the encoder applies ICG, the complexity required for the decoder can be reduced compared to when the decoder applies ICG.

[0171] ICGPreAppliedCPE[n] of ICGConfig indicates whether ICG has been applied to the nth CPE in the encoder. If ICGPreAppliedCPE[n] is true, the IC processing block of the decoder bypasses the downmix signal for stereo reproduction of the nth CPE. On the other hand, if ICGPreAppliedCPE[n] is false, the IC processing block of the decoder applies ICG to the downmix signal.

[0172] If ICGDisableCPE[n] is 1, it is not possible to compute the ICG for the corresponding QCE or CPE, and therefore ICGPreApplied[n] is set to 0. For a QCE containing a CPE pair, the indicators ICGPreApplied[n] and ICGPreApplied[n+1] of the two CPEs contained in the QCE shall have the same value.

[0173] The bitstream structure and bitstream syntax to be changed or added for IC processing will now be described using Tables 8 to 16.

[0174] Table 8 shows the syntax of mpegh3daExtElementConfig() according to an embodiment of the present invention.

[0175]

Table 8

[0176]

[0177] As shown in mpegh3daExtElementConfig() of Table 8, ICGConfig() may be called during the configuration process to obtain information on using IC or not using IC and applying ICG or not applying ICG as shown in Table 7.

[0178] Table 9 shows the syntax of usacExtElementType according to an embodiment of the present invention.

[0179]

Table 9

[0180]

[0181] As shown in Table 9, in usacExtElementType, ID_EXT_ELE_ICG may be added for IC processing, and the value of ID_EXT_ELE_ICG may be 9.

[0182] Table 10 shows the syntax of speakerLayoutType according to an embodiment of the present invention.

[0183]

Table 10

[0184]

[0185] For IC processing, the speaker layout type speakerLayoutType for IC should be defined. Table 10 shows the meaning of each value of speakerLayoutType.

[0186] When speakerLayoutType is 3, the speaker layout is represented by the index LCChannelConfiguration. The index LCChannelConfiguration has the same layout as ChannelConfiguration, but the channel allocation order of LCChannelConfiguration can achieve the best IC structure using CPE.

[0187] Table 11 shows the syntax of SpeakerConfig3d() according to an embodiment of the present invention.

[0188]

Table 11

[0189]

[0190] When speakerLayoutType is 3 as described above, the embodiment uses the same layout as CICPspeakerLayoutIdx, but the embodiment differs from CICPspeakerLayoutIdx in terms of the optimal channel allocation order.

[0191] When speakerLayoutType is 3 and the output layout is a stereo layout, the number of input channels N in The number of ICs after becoming the core codec.

[0192] Table 12 shows the syntax of immersiveDownmixFlag according to an embodiment of the present invention.

[0193]

Table 12

[0194]

[0195] By newly defining the speaker layout type for IC, immersiveDownmixFlag should also be corrected. When immersiveDownmixFlag is 1, a sentence for handling the case where speakerLayoutType is 3 should be added as shown in Table 12.

[0196] Target expansion should meet the following requirements:

[0197] - Local cloud speaker settings are notified by LoudspeakerRendering(),

[0198] -speakerLayoutType should be 0 or 3,

[0199] -CICPspeakerLayoutIdx has a value of 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, or 18.

[0200] Table 13 shows the syntax of SAOC3DgetNumChannels() according to an embodiment of the present invention.

[0201] As shown in Table 13, SAOC3DgetNumChannels should be corrected to include the case where speakerLayoutType is 3.

[0202]

Table 13

[0203]

[0204] Table 14 shows the syntax of the channel allocation order according to an embodiment of the present invention.

[0205] Table 14 indicates the number of channels, channel order, and possible IC type as a newly defined channel allocation order for IC according to the speaker layout or LCChannelConfiguration.

[0206]

Table 14

[0207]

[0208]

[0209] Table 15 shows the syntax of mpegh3daChannelPairElementConfig() according to an embodiment of the present invention.

[0210] For IC processing, as shown in Table 15, when stereoConfigIndex is greater than 0, mpegh3daChannelPairElementConfig() should be corrected so that Mps212Config() processing is followed by isInternalChangeProcessed().

[0211]

Table 15

[0212]

[0213] Figure 4 4 is a detailed block diagram of an ICG application unit of a decoder according to an embodiment of the present invention applying ICG to an IC signal.

[0214] When speakerLayout is 3, isInternalProcessed is 0, and the conditions that the reproduction layout is a stereo layout are met and therefore the decoder applies ICG, the following is performed: Figure 4 IC processing in.

[0215] Figure 4 The ICG application unit shown in FIG. 4 includes an ICG acquirer 410 and a multiplier 420 .

[0216] Assuming that the input CPE includes a pair of channels CH_M_000 and CH_L_000, when the mono QMF subband sample 430 of the input CPE is input, the ICG acquirer 410 acquires ICG by using CLD. The multiplier 420 acquires the IC signal ICH_A 440 by multiplying the received mono QMF subband sample 430 by the acquired ICG.

[0217] The mono QMF subband samples of CPE can be compared with ICG The IC signal is simply reconstructed by multiplying , where l indicates the time index and m indicates the frequency index.

[0218] ICG Defined in Equation 1:

[0219] [Equation 1]

[0220]

[0221] in, and Indicates the translation coefficient of CLD, G left and G right indicates the gain defined in the format conversion rule, and and Indicates the gain of the mth frequency band of the EQ value defined in the format conversion rule.

[0222] Figure 5 is a block diagram illustrating a decoding process when an encoder pre-processes ICG according to an embodiment of the present invention.

[0223] When speakerLayout is 3, isInternalProcessed is 1, and the conditions that the reproduction layout is a stereo layout are met and therefore the encoder applies ICG and sends ICG, the following is performed: Figure 5 IC processing in.

[0224] When the output layout is a stereo layout, the MPEG-H 3D Audio encoder pre-processes the ICG corresponding to the CPE so that the decoder bypasses the MPS 212, thus reducing the complexity of the decoder.

[0225] However, when the output layout is not a stereo layout, the MPEG-H 3D Audio encoder does not perform IC processing, so the decoder needs to perform the inverse of the multiplication ICG. And run the MPS212 processing to achieve decoding, such as Figure 5 shown.

[0226] Similar to Figure 3 and Figure 4 , assuming that the input CPE includes a pair of channels CH_M_000 and CH_L_000. When a mono QMF subband sample 540 having an ICG pre-processed in the encoder is input, the decoder determines whether the output layout is a stereo layout, as indicated by reference numeral 510.

[0227] When the output layout is a stereo layout, IC is used, and thus the decoder outputs the received mono QMF subband sample 540 as an IC signal of IC ICH_A 550. On the other hand, when the output layout is not a stereo layout, IC is not used during IC processing, and thus the decoder performs inverse ICG processing 520 to restore the IC-processed signal as indicated by reference numeral 560, and the signal restored by upmixing the MPS 212 as indicated by reference numeral 530, thereby outputting a signal 571 of CH_M_000 and a signal 572 of CH_L_000.

[0228] MPEG-H Audio has the greatest decoding complexity when the output layout is a stereo layout because the load caused by covariance analysis in the format converter becomes a problem when the number of input channels is large and the number of output channels is small.

[0229] On the other hand, when the output layout is not a stereoscopic layout, with two sets of CLDs per frame, the number of operations added to multiply the inverse of the ICG is (5 multiplications, 2 additions, 1 division, 1 square root ≈ 55 operations) × (71 frequency bands) × (2 parameter sets) × (48000 / 2048) × (13 ICs), so it becomes approximately 2.4 MOPS and does not act as a large load on the system.

[0230] After the ICs are generated, the QMF subband samples of the ICs, the number of ICs, and the type of ICs are sent to a format converter, and the size of the covariance matrix in the format converter depends on the number of ICs.

[0231] Table 16 shows decoding scenarios of MPEG Surround (MPS) and Spectral Band Replication (SBR) determined based on channel units and reproduction layout according to an embodiment of the present invention.

[0232]

Table 16

[0233] Reproduce layout unit Order of MPS and SBR stereo CPE MPS after mono SBR stereo CPE MPS before Stereo SBR stereo QCE Two MPS before two stereo SBRs Monaural CPE / QCE No matter the order

[0234] MPS is a technique for encoding a multi-channel audio signal by using auxiliary data containing spatial cue parameters representing a downmix to a minimum channel (mono or stereo) and human perception characteristics of the multi-channel audio signal.

[0235] The MPS encoder receives N multi-channel audio signals and extracts spatial parameters as auxiliary data. For example, the spatial parameters can be expressed as the difference between the volume of the two ears based on the binaural effect and the correlation between the channels. Since the amount of information of the extracted spatial parameters is very small (no more than 4kbps per channel), high-quality multi-channel audio can be provided even in a bandwidth that can only provide mono or stereo audio services.

[0236] The MPS encoder also generates a down-mix signal from the received N multi-channel audio signals, and the generated down-mix signal is encoded by, for example, MPEG USAC, which is an audio compression technology, and transmitted together with the spatial parameters.

[0237] At this time, the N multi-channel audio signals received by the MPS encoder are divided into multiple frequency bands by the analysis filter group. Representative methods for dividing the frequency domain into sub-bands include discrete Fourier transform (DFT) or using QMF. In MPEG surround, QMF is used to divide the frequency domain into sub-bands with low complexity. When QMF is used, its compatibility with SBR can be ensured, and therefore more efficient encoding can be performed.

[0238] SBR is a technique that copies and pastes a low-frequency band to a high-frequency band that is relatively imperceptible to humans, and parameterizes and transmits information about the high-frequency band signal. Therefore, according to SBR, a wide bandwidth can be achieved at a low bit rate. SBR is mainly used in codecs with high compression ratios and low bit rates, and it is difficult to express harmonics due to the loss of some high-frequency band information. However, SBR provides a high restoration rate within audible frequencies.

[0239] The SBR used in IC processing is the same as ISO / IEC 23003-3:2012, except that the domains are different. The SBR of ISO / IEC 23003-3:2012 is defined in the QMF domain, but IC is processed in the hybrid QMF domain. Therefore, when the index number of the QMF domain is k, the frequency index number of the overall SBR processing for IC is k+7.

[0240] An embodiment of a decoding scenario in which mono SBR decoding and then MPS decoding is performed when the CPE is output via a stereo reproduction layout is described in Figure 6 Shown in.

[0241] An embodiment of a decoding scenario in which MPS decoding and then stereo SBR decoding is performed when the CPE is output to a stereo reproduction layout is described in Figure 7 Shown in.

[0242] Figure 8 and Fig. 9 An embodiment of a decoding scenario is shown in which MPS decoding is performed on a CPE pair and then stereo SBR decoding is performed on each decoded signal when the QCE is output through a stereo reproduction layout.

[0243] When the reproduction layout used to output the CPE or QCE is not a stereo layout, the order in which MPS decoding and SBR decoding are performed is not important.

[0244] The MPS212 encoded CPE signal processed by the decoder is defined as follows:

[0245] cplx_out_dmx[] is the CPE downmix signal obtained by complex predictive stereo decoding.

[0246] cplx_out_dmx_preICG[] is the mono signal to which ICG has been applied in the encoder, which is to be decoded by complex predictive stereo decoding and hybrid QMF analysis filter bank decoding in the hybrid QMF domain.

[0247] cplx_out_dmx_postICG[] is a mono signal that has undergone complex predictive stereo decoding and IC processing in the hybrid QMF domain and to which ICG is to be applied in the decoder.

[0248] cplx_out_dmx_ICG[] is the full-band IC signal in the hybrid QMF domain.

[0249] The MPS212 encoded QCE signal processed by the decoder is defined as follows:

[0250] cplx_out_dmx_L[] is the first channel signal of the first CPE that has been subjected to composite predictive stereo decoding.

[0251] cplx_out_dmx_R[] is the second channel signal of the first CPE that has undergone composite predictive stereo decoding.

[0252] cplx_out_dmx_L_preICG[] is the first pre-ICG applied IC signal in the hybrid QMF domain.

[0253] cplx_out_dmx_R_preICG[] is the second pre-ICG applied IC signal in the hybrid QMF domain.

[0254] cplx_out_dmx_L_postICG[] is the IC signal after the first application of ICG in the hybrid QMF domain.

[0255] cplx_out_dmx_R_postICG[] is the second post-ICG applied IC signal in the hybrid QMF domain.

[0256] cplx_out_dmx_L_ICG_SBR is a first full-band decoded IC signal including a downmix parameter for 22.2 to 2 format conversion and a high frequency component generated by SBR.

[0257] cplx_out_dmx_R_ICG_SBR is a second full-band decoded IC signal including a downmix parameter for 22.2 to 2 format conversion and a high frequency component generated by SBR.

[0258] Figure 6 The flowchart shows an IC processing method in a structure for performing mono SBR decoding and then performing MPS decoding when the CPE is output through a stereo reproduction layout according to an embodiment of the present invention.

[0259] When a CPE bitstream is received, in operation 610, it is first determined whether to use CPE through the ICGDisabledCPE[n] flag.

[0260] When ICGDisabledCPE[n] is true, the CPE bitstream is decoded as defined in ISO / IEC 23008-3 in operation 620. On the other hand, when ICGDisabledCPE[n] is false, in operation 630, if SBR is necessary, mono SBR is performed on the CPE bitstream, and stereo decoding is performed thereon to generate a downmix signal cplx_out_dmx.

[0261] In operation 640, it is determined through ICGPreAppliedCPE whether ICG has been applied at the encoder end.

[0262] When ICGPreAppliedCPE[n] is false, in operation 650, the downmix signal cplx_out_dmx is subjected to IC processing in the hybrid QMF domain, thereby generating a downmix signal cplx_out_dmx_postICG to which ICG is applied post-ICG. In operation 650, ICG is calculated using the MPS parameters. The linear CLD value dequantized for CPE is calculated by ISO / IEC23008-3, and ICG is calculated using Equation 2.

[0263] The downmix signal cplx_out_dmx_postICG with the post-ICG applied is generated by multiplying the downmix signal cplx_out_dmx by the ICG calculated using Equation 2:

[0264] [Equation 2]

[0265]

[0266] in and indicates the dequantized linear CLD value for the 1st time slot and the mth hybrid QMF band of the CPE signal, G left and G right indicates the value of the Gain column of the output channel defined in ISO / IEC 23008-3 Table 96 (i.e., in the Format Conversion Rules table), and and Indicates the gain of the mth frequency band of the EQ value of the output channel defined in the format conversion rule table.

[0267] When ICGPreAppliedCPE[n] is true, in operation 660 , the down-mix signal cplx_out_dmx is analyzed to obtain a down-mix signal cplx_out_dmx_preICG to which ICG is pre-applied.

[0268] Depending on the setting of ICGPreAppliedCPE[n], the signal cplx_out_dmx_preICG or cplx_out_dmx_postICG becomes the final IC-processed output signal cplx_out_dmx_ICG.

[0269] Figure 7 The flowchart shows an IC processing method of performing MPS decoding and then stereo SBR decoding when the CPE is output through a stereo reproduction layout according to an embodiment of the present invention.

[0270] according to Figure 7 An embodiment of Figure 6 Compared to the embodiment of , since MPS decoding is followed by SBR decoding, stereo SBR decoding is performed when IC is not used. On the other hand, when IC is used, mono SBR is performed, and for this purpose, the parameters used for stereo SBR are downmixed.

[0271] Therefore, with Figure 6 compared to, Figure 7 The method further includes: an operation 780 of downmixing SBR parameters for two channels to generate SBR parameters for one channel; and an operation 770 of performing mono SBR by using the generated SBR parameters, and cplx_out_dmx_ICG that has undergone mono SBR becomes a final IC-processed output signal cplx_out_dmx_ICG.

[0272] In such Figure 7 In the illustrated operational arrangement, the signal cplx_out_dmx_preICG or the signal cplx_out_dmx_postICG corresponds to a band-limited signal since the execution of SBR after IC processing results in an extension of high frequency components. The SBR parameter pair for upmixing a stereo signal should be downmixed in the parameter domain to extend the bandwidth of the band-limited IC signal cplx_out_dmx_preICG or cplx_out_dmx_postICG.

[0273] The SBR parameter downmixer should include a process of multiplying the high frequency band extended due to SBR by the EQ value and gain parameter of the format converter. The method of downmixing the SBR parameters will be described in detail below.

[0274] Figure 8is a block diagram of an IC processing method in a structure using stereo SBR when QCE is output through a stereo reproduction layout according to an embodiment of the present invention.

[0275] Figure 8 An embodiment is a case where ICGPreApplied[n] and ICGPreApplied[n+1] are both 0, that is, an embodiment of a method of applying ICG in a decoder.

[0276] refer to Figure 8 The overall decoding process is performed in the order of bit stream decoding 810, stereo decoding 820, hybrid QMF analysis 830, IC processing 840 and stereo SBR 850.

[0277] When the bitstreams of the two CPEs included in the QCE are subjected to bitstream decoding 811 and bitstream decoding 812, respectively, the SBR payload, the MPS212 payload, and the CplxPred payload are extracted from the decoded signals corresponding to the results of the bitstream decoding.

[0278] Stereo decoding 821 is performed using the CplxPred payload, and stereo decoded signals cplx_dmx_L and cplx_dmx_R, which have undergone hybrid QMF analysis 831 and hybrid QMF analysis 832 , respectively, are transmitted as input signals of an IC processing unit 841 and an IC processing unit 842 , respectively.

[0279] At this time, the generated IC signals cplx_dmx_L_PostICG and cplx_dmx_R_PostICG are band-limited signals. Therefore, the two IC signals are subjected to stereo SBR 851 by using downmix SBR parameters obtained by downmixing the SBR payload extracted from the bitstreams of the two CPEs. The high frequency of the band-limited IC signal is extended by stereo SBR 851, thereby generating full-band IC-processed output signals cplx_dmx_L_ICG and cplx_dmx_R_ICG.

[0280] The downmix SBR parameters are used to expand the frequency band of the band-limited IC signal to generate a full-band IC signal.

[0281] Therefore, when IC is used for QCE, only one stereo decoding block is used and only one stereo SBR block is used, so the stereo decoding block 822 and the stereo SBR block 852 can be omitted. In other words, compared with the case where each CPE is processed, Figure 7 The simple decoding structure is implemented by using QCE.

[0282] Fig. 9is a block diagram of an IC processing method in a structure using stereo SBR when outputting QCE through a stereo reproduction layout according to another embodiment of the present invention.

[0283] Fig. 9 An embodiment is a case where ICGPreApplied[n] and ICGPreApplied[n+1] are both 1, that is, an embodiment of a method of applying ICG in an encoder.

[0284] refer to Fig. 9 The entire decoding process is performed in the order of bit stream decoding 910, stereo decoding 920, hybrid QMF analysis 930 and stereo SBR 950.

[0285] When the encoder has applied ICG, the decoder does not perform IC processing, and thus Fig. 9 The method is omitted Figure 8 IC processing block 841 and IC processing block 842. Fig. 9 Other processes and Figure 8 The repeated description of these processes is omitted here.

[0286] The stereo decoded signals cplx_dmx_L and cplx_dmx_R are respectively subjected to hybrid QMF analysis 931 and hybrid QMF analysis 932 and then transmitted as input signals of the stereo SBR block 951. After the stereo decoded signals cplx_dmx_L and cplx_dmx_R pass through the stereo SBR block 951, full-band IC-processed output signals cplx_dmx_L_ICG and cplx_dmx_R_ICG are generated.

[0287] When the output channels are not stereo channels, using IC may not be appropriate. Therefore, when the encoder has applied ICG, the decoder should apply the inverse of ICG if the output channels are not stereo channels.

[0288] In this case, as shown in Table 8, the decoding order of MPS and SBR is not important, but for ease of explanation, a scenario in which mono SBR decoding is performed and then MPS212 decoding is performed will be described.

[0289] As shown in Equation 3, the inverse IG of ICG is calculated using the MPS parameters and the format conversion parameters:

[0290] [Equation 3]

[0291]

[0292] and Indicates the dequantized linear CLD value of the 1st time slot and the mth hybrid QMF band of the CPE signal, G left and G right indicates the value of the Gain column of the output channel defined in ISO / IEC 23008-3 Table 96 (i.e. in the Format Conversion Rules table), and and Indicates the gain of the mth frequency band of the EQ value of the output channel defined in the format conversion rule table.

[0293] If ICGPreAppliedCPE[n] is true, the nth cplx_dmx shall be multiplied with the inverse of ICG before passing through the MPS block and the remaining decoding process shall follow ISO / IEC 23008-3.

[0294] When the decoder uses an IC processing block or the encoder pre-processes ICG and the output layout is a stereo layout, for CPE / QCE, not a stereo / quadraphonic signal upmixed by MPS, but a band-limited IC signal is generated at the end before the SBR block.

[0295] Since the SBR payload has been encoded by stereo SBR for the MPS upmixed stereo / quadraphonic signal, the stereo SBR payload should be downmixed by multiplying with the gain and EQ values ​​of the format converter in the parameter domain in order to achieve IC processing.

[0296] The method of parametric downmix stereo SBR will now be described in detail.

[0297] (1) Inverse filtering

[0298] The inverse filtering mode is selected by allowing the stereo SBR parameter to have a maximum value in each noise floor band.

[0299] It can be obtained using equation 4:

[0300] [Equation 4]

[0301] for(i=0;i<N Q ; i++)

[0302] bs_invf_mode Downmixed (i) = MAX(bs_invf_mode ch1 (i), bs_invf_mode ch2 (i))

[0303]

[0304] 2) Additional harmonics

[0305] The sound wave including the fundamental frequency f and odd harmonics 3f, 5f, 7f, ... of the fundamental frequency f has half-wave symmetry. However, the sound wave including even harmonics 0f, 2f, ... of the fundamental frequency f does not have symmetry. Instead, the nonlinear system that causes the waveform of the sound source to change rather than simply scale or shift generates additional harmonics, thereby causing harmonic distortion.

[0306] The additional harmonics are combinations of additional sine waves and can be expressed as in Equation 5:

[0307] [Equation 5]

[0308] for(i=0;i<N High ; i++)

[0309] bs_add_harmonic Downmixcd (i) = OR (bs_add_harmonic ch1 (i), bs_add_harmonic ch2 (i))

[0310] (3) Envelope time boundary

[0311] Fig.10a , Fig.10b , Fig.10c and Fig.10d A method for determining a time boundary according to an embodiment of the present invention is shown, where the time boundary is an SBR parameter.

[0312] Fig.10a The temporal envelope grid is shown when the start boundaries of the first envelope are the same and the stop boundaries of the last envelope are the same.

[0313] Fig.10b The temporal envelope grid is shown when the start boundaries of the first envelope are different and the stop boundaries of the last envelope are the same.

[0314] Fig.10c The temporal envelope grid is shown when the start boundaries of the first envelope are the same and the stop boundaries of the last envelope are different.

[0315] Fig.10d The temporal envelope grid is shown when the start boundary of the first envelope is different and the stop boundary of the last envelope is different.

[0316] The temporal envelope grid t of the SBR for IC E_Merged It is generated by splitting the stereo SBR time grid into the smallest segments with the highest resolution.

[0317] t E_MergedThe start boundary value of is set to the maximum start boundary value of the stereo channel. The envelope between time grid 0 and the start boundary has been processed in the previous frame. The stop boundary with the maximum value among the stop boundaries of the last envelopes of the two channels is used as the stop boundary of the last envelope.

[0318] As in Figures 10a-10d As shown in , by finding the intersection between the time boundaries of the two channels, the start / stop boundaries of the first and last envelopes are determined with the most segment resolution. If there are at least 5 envelopes, the reverse search starts from t E_Merged The stop point is t E_Merged The starting point of the 4 envelopes is found, and the starting boundaries of the 4 envelopes are removed to reduce the number of envelopes. This process continues until there are 5 envelopes left.

[0319] (4) Noise time boundary

[0320] The noise time boundary L of the downmix is ​​determined by adopting the noise time boundary having a larger value among the noise time boundaries of the two channels. Q_Merged By taking the envelope time boundary t E_Merged The first and last grids of the t are used to determine the first and merged noise time boundaries t Q_Merged .

[0321] If the noise time boundary of the downmix is ​​L Q_Merged If greater than 1, select t Q_Merged (1), where the noise time boundary L Q The t of the channel greater than 1 Q (1). If both channels have a noise time margin L greater than 1 Q , then t Q The minimum value of (1) is selected as t Q_Merged (1).

[0322] (5) Envelope data

[0323] Select the frequency resolution r of the merged envelope time boundaries Merged . Frequency resolution r Merged The frequency resolution of each part is r ch1 and frequency resolution r ch2 The maximum value between is chosen as r Merged .

[0324] Using Equation 6, the format conversion parameters are considered to calculate the envelope data E Orig Calculate the envelope data E of all envelopes Orig_Merged :

[0325] [Equation 6]

[0326]

[0327] in,

[0328]

[0329] h ch1 (l) is defined as:

[0330] t E_ch1 (h ch1 (l))≤t E_Merged (l)<t E_ch1 (h ch1 (l)+1),

[0331] h ch2 (l) is defined as:

[0332] t E_ch2 (h ch2 (l))≤t E_Merged (l)<t E_ch2 (h ch2 (l)+1),

[0333] g ch1 (k) is defined as:

[0334] F(g ch1 (k), r ch1 (h ch1 (l)))≤F(k,r Merged (l))<F(g ch1 (k)+1,r ch1 (h ch1 (l))),

[0335] And g ch2 (k) is defined as:

[0336] F(g ch2 (k), r ch2 (h ch2 (l)))≤F(k,r Merged (l))<F(g ch2 (k)+1,r ch2 (h ch2 (l))).

[0337] (6) Background noise data

[0338] The combined noise floor data is determined as the sum of the two channel data according to Equation 7:

[0339] [Equation 7]

[0340] QOrigMerged (k, l) = Q Origch1 (k,h ch1 (l))+Q Origch2 (k,h ch2 (l)),

[0341] 0≤k<N Q , 0≤l<L Q_Merged

[0342] where h ch1 (l) is defined as t Q_ch1 (h ch1 (l))≤t Q_Merged (l)<t Q_ch1 (h ch1 (l)+1), and h ch2 (l) is defined as t Q_ch2 (h ch2 (l))≤t Q_Merged (l)<t Q_ch2 (h ch2 (l)+1).

[0343] The above-mentioned embodiment of the present invention can be implemented as a program command that can be executed by various computer configuration elements and can be recorded on a computer-readable recording medium. The computer-readable recording medium may include a program command, a data file, a data structure, etc., which may be singular or in combination. The program command to be recorded on the computer-readable recording medium may be specially designed and configured for the embodiment of the present invention, or may be well known and available to those of ordinary skill in the field of computer software. Examples of computer-readable recording media include magnetic media (e.g., hard disks, floppy disks, or tapes), optical media (e.g., CD-ROMs or digital versatile disks (DVDs), magneto-optical media (e.g., floppy disks), and hardware devices (e.g., ROMs, random access memories (RAMs), or flash memory, etc.) that are specially configured to store and execute program commands. Examples of computer programs include high-level language codes that can be executed by a computer using an interpreter, etc., and machine language codes made by a compiler. Hardware devices may be configured to be used as one or more software modules to perform the operation of the present invention, or vice versa.

[0344] While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the appended claims.

[0345] Therefore, the scope of the invention is defined not by the detailed description but by the appended claims, and all differences within the scope will be construed as being included in the present invention.

Claims

1. A method for decoding a four-channel unit including a first two-channel unit and a second two-channel unit, the method comprising: Obtaining a first spectral band replication payload and a first MPEG surround 212 payload for a stereo output layout by decoding the first two-channel unit; Obtaining a second band replication payload and a second MPEG surround 212 payload for a stereo output layout by decoding the second two-channel unit; generating a pair of band-limited inner channel signals based on the first MPEG surround 212 payload, the second MPEG surround 212 payload, and a pair of inner channel gains; downmixing the first frequency band copy payload and the second frequency band copy payload into downmixed frequency band copy parameters using rendering parameters of a format converter; and A pair of full-band interior channel signals is generated based on the generated pair of band-limited interior channel signals and the downmix band copy parameters.

2. The method of claim 1, wherein: The generation of the pair of band-limited interior channel signals includes determining whether interior channel processing is enabled for the first two-channel unit and the second two-channel unit. 3 . The method of claim 1 , further comprising generating a stereo output channel signal based on the generated pair of full-band internal channel signals.

4. The method of claim 1, wherein: The generating of the pair of band-limited inner channel signals comprises: The pair of internal channel gains for the four-channel unit are calculated based on the rendering parameters of the format converter.

5. The method of claim 1, wherein: The internal channel gain of the pair of internal channel gains is through the use of To calculate, l represents the time slot index, m represents the frequency band index, and is the channel level difference of the first two-channel unit, and is the gain value defined in the format converter according to the MPEG Surround 212 output channel, and and is the equalization gain value defined in the format converter according to the MPEG surround 212 output channel.

6. A device for processing an audio signal, the device comprising: a receiver configured to receive a four-channel unit including a first two-channel unit and a second two-channel unit; and An internal channel signal generator, wherein the internal channel signal generator is configured to: Obtaining a first spectral band replication payload and a first MPEG surround 212 payload for a stereo output layout by decoding the first two-channel unit; Obtaining a second band replication payload and a second MPEG surround 212 payload for a stereo output layout by decoding the second two-channel unit; generating a pair of band-limited inner channel signals based on the first MPEG surround 212 payload, the second MPEG surround 212 payload, and a pair of inner channel gains; downmixing the first frequency band copy payload and the second frequency band copy payload into downmixed frequency band copy parameters using rendering parameters of a format converter; and A pair of full-band interior channel signals is generated based on the generated pair of band-limited interior channel signals and the downmix band copy parameters.

7. The device according to claim 6, wherein: The internal channel signal generator is further configured to determine whether internal channel processing can be performed on the first two-channel unit and the second two-channel unit. 8 . The apparatus of claim 6 , further comprising a stereo output signal generator configured to generate a stereo output channel signal based on the generated pair of full-band internal channel signals.

9. The device as claimed in claim 6, wherein: The internal channel signal generator is further configured to calculate the pair of internal channel gains for the four-channel unit based on the rendering parameters of the format converter.

10. The device according to claim 6, wherein: The internal channel gain of the pair of internal channel gains is through the use of To calculate, l represents the time slot index, m represents the frequency band index, and is the channel level difference of the first two-channel unit, and is the gain value defined in the format converter according to the MPEG Surround 212 output channel, and and is the equalization gain value defined in the format converter according to the MPEG surround 212 output channel.

Images