Information processing device, method, and program

Abstract

To reduce a decoding calculation amount of an audio signal.SOLUTION: A priority information acquisition unit acquires priority information on each channel from a bit stream and supplies it to an output selection unit. A channel audio signal decoding unit decodes encoding data of an audio signal of each channel and supplies an acquired MDCT coefficient to the output selection unit. The output selection unit supplies the MDCT coefficient to an IMDCT unit when a priority degree indicated by the priority information is equal to or higher than a prescribed degree, and supplies 0 as the MDCT coefficient to a 0-value output unit when the priority degree is less than the prescribed degree. The 0-value output unit generates an audio signal on the basis of 0 supplied as the MDCT coefficient. Also, the IMDCT unit performs IMDCT on the basis of the MDCT coefficient and generates an audio signal. This feature can be applied to an encoding device and a decoding device.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present technology relates to an information processing device, method, and program, and in particular to an information processing device, method, and program that are capable of reducing the amount of calculation required for decoding an audio signal. [Background technology]

[0002] For example, known methods for encoding audio signals include multi-channel encoding according to the international standards MPEG (Moving Picture Experts Group)-2 AAC (Advanced Audio Coding) standard, MPEG-4 AAC standard, and MPEG-D USAC (Unified Speech and Audio Coding) standard (see, for example, Non-Patent Documents 1 and 2). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] INTERNATIONAL STANDARD ISO / IEC 14496-3 Fourth edition 2009-09-01 Information technology-coding of audio-visual objects-part3:Audio [Non-Patent Document 2] INTERNATIONAL STANDARD ISO / IEC 23003-3 Frist edition 2012-04-01 Information technology-coding of audio-visual objects-part3:Unified speech and audio coding Summary of the Invention [Problem to be solved by the invention]

[0004] However, in order to achieve a more realistic reproduction that goes beyond the conventional 5.1 channel surround reproduction, or to transmit multiple sound materials (objects), coding techniques using more audio channels are required.

[0005] For example, consider a case where a 24-channel audio signal and an audio signal of multiple objects are encoded and decoded, and a case where a 2-channel audio signal is encoded and decoded. In such a case, a mobile device or the like with poor computing power may be able to decode the 2-channel audio signal in real time, but may have difficulty in decoding the 24-channel audio signal and the audio signal of multiple objects in real time.

[0006] With current audio codecs such as MPEG-D USAC, it is necessary to decode the audio signals of all channels and all objects, so it is difficult to reduce the amount of calculation required for decoding. As a result, depending on the decoding device, it may not be possible to play back the audio signals in real time.

[0007] The present technology has been made in consideration of such circumstances, and is intended to make it possible to reduce the amount of decoding calculations. [Means for solving the problem]

[0008] A decoding device according to a first aspect of the present technology includes an acquisition unit that acquires encoded audio signals of a plurality of channels or objects and priority information for each of the audio signals at a predetermined time, and an audio signal decoding unit that decodes the encoded audio signals of a predetermined number of channels or objects corresponding to the priority information based on the priority information.

[0009] A decoding method or program of a first aspect of the present technology includes a step of acquiring encoded audio signals of multiple channels or multiple objects and priority information for each of the audio signals at a predetermined time, and decoding the encoded audio signals of a predetermined number of channels or objects corresponding to the priority information based on the priority information.

[0010] In a first aspect of the present technology, encoded audio signals of multiple channels or multiple objects and priority information for each of the audio signals at a predetermined time are obtained, and based on the priority information, the encoded audio signals of a predetermined number of channels or objects corresponding to the priority information are decoded.

[0011] An encoding device according to a second aspect of the present technology includes a priority information generation unit that generates priority information for an audio signal of multiple channels or multiple objects at a predetermined time, and a packing unit that stores the priority information in a bitstream.

[0012] An encoding method or program according to a second aspect of the present technology includes a step of generating priority information for an audio signal of multiple channels or multiple objects at a predetermined time, and storing the priority information in a bitstream.

[0013] In a second aspect of the present technology, priority information for a plurality of channels or a plurality of objects of an audio signal at a predetermined time is generated, and the priority information is stored in a bitstream. [Brief description of the drawings]

[0014] [Figure 1] FIG. 2 is a diagram illustrating a bit stream. [Diagram 2] FIG. 1 is a diagram illustrating encoding. [Diagram 3] FIG. 11 is a diagram illustrating priority information. [Figure 4]11 is a diagram illustrating the meaning of values ​​of priority information. FIG. [Diagram 5] FIG. 1 illustrates an example of the configuration of an encoding device. [Figure 6] 13 is a diagram illustrating an example of the configuration of a channel audio encoding unit. [Figure 7] 2 is a diagram illustrating an example of the configuration of an object audio encoding unit. [Figure 8] 11 is a flowchart illustrating an encoding process. [Figure 9] FIG. 2 is a diagram illustrating an example of the configuration of a decoding device. [Figure 10] FIG. 13 is a diagram illustrating an example of the configuration of an unpacking / decoding unit. [Figure 11] 13 is a flowchart illustrating a decoding process. [Figure 12] 13 is a flowchart illustrating a selective decoding process. [Figure 13] FIG. 13 is a diagram illustrating another example of the configuration of the unpacking / decoding unit. [Figure 14] 13 is a flowchart illustrating a selective decoding process. [Figure 15] FIG. 11 is a diagram illustrating an example of object metadata syntax. [Figure 16] FIG. 2 is a diagram illustrating generation of an audio signal. [Figure 17] FIG. 2 is a diagram illustrating generation of an audio signal. [Figure 18] FIG. 13 is a diagram illustrating selection of an output destination of MDCT coefficients. [Figure 19] 11A and 11B are diagrams illustrating gain adjustment of an audio signal and a high-frequency power value. [Figure 20] 11A and 11B are diagrams illustrating gain adjustment of an audio signal and a high-frequency power value. [Figure 21] FIG. 13 is a diagram illustrating another example of the configuration of the unpacking / decoding unit. [Figure 22] 13 is a flowchart illustrating a selective decoding process. [Diagram 23] FIG. 1 is a diagram illustrating gain adjustment of an audio signal. [Figure 24] FIG. 1 is a diagram illustrating gain adjustment of an audio signal. [Diagram 25] FIG. 13 is a diagram illustrating another example of the configuration of the unpacking / decoding unit. [Figure 26] 13 is a flowchart illustrating a selective decoding process. [Figure 27] FIG. 13 is a diagram illustrating a VBAP gain. [Figure 28] FIG. 13 is a diagram illustrating a VBAP gain. [Figure 29] FIG. 13 is a diagram illustrating another example of the configuration of the unpacking / decoding unit. [Diagram 30] 13 is a flowchart illustrating a decoding process. [Diagram 31] 13 is a flowchart illustrating a selective decoding process. [Diagram 32] FIG. 1 illustrates an example of the configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

[0016] First embodiment Overview of this technology The present technology is intended to reduce the amount of calculation required for decoding by transmitting priority information for the audio signals of each channel constituting a multi-channel signal and the audio signals of objects.

[0017] In addition, on the decoding side, the present technology performs frequency-time conversion when the priority indicated in the priority information of each channel or object is equal to or higher than a predetermined level, and does not perform frequency-time conversion when the priority indicated in the priority information is lower than the predetermined level, and sets the result of the frequency-time conversion to 0, thereby making it possible to reduce the amount of calculation required for decoding an audio signal.

[0018] In the following, a case will be described in which multi-channel audio signals and object audio signals are coded according to the AAC standard, but similar processing is performed when the signals are coded according to other methods.

[0019] For example, when a multi-channel audio signal and an audio signal of a plurality of objects are encoded and transmitted in accordance with the AAC standard, the audio signal of each channel or each object is encoded and transmitted for each frame.

[0020] Specifically, as shown in Figure 1, the encoded audio signal and information required for decoding the audio signal are stored in multiple elements (bitstream elements), and a bitstream consisting of these elements is transmitted.

[0021] In this example, t elements EL1 to ELt are arranged in order from the beginning of a bit stream for one frame, and an identifier TERM indicating that this is the end position of the information for that frame is arranged at the end.

[0022] For example, the element EL1 located at the beginning is an ancillary data area called a DSE (Data Stream Element), and information related to each of a plurality of channels, such as information related to downmixing of audio signals and identification information, is described in the DSE.

[0023] The elements EL2 to ELt following the element EL1 store an encoded audio signal.

[0024] In particular, an element that stores a single-channel audio signal is called an SCE, and an element that stores a pair of two-channel audio signals is called a CPE. The audio signals of each object are stored in the SCE.

[0025] In the present technology, priority information of audio signals of each channel constituting a multi-channel signal and priority information of audio signals of each object are generated and stored in the DSE.

[0026] For example, assume that an audio signal of successive frames F11 to F13 as shown in FIG. 2 is to be encoded.

[0027] In such a case, the encoding device (encoder) analyzes the degree of priority of the audio signal of each channel for each of those frames, and generates priority information for each channel, for example, as shown in Fig. 3. Similarly, the encoding device also generates priority information for the audio signal of each object.

[0028] For example, the encoding device analyzes the degree of priority of an audio signal based on the sound pressure and spectrum shape of the audio signal, as well as correlations of the spectrum shapes between channels and between objects.

[0029] 3 shows an example of priority information for each channel when the total number of channels is M. That is, for each channel from channel number 0 to channel number M-1, a numerical value indicating the priority of the audio signal of each channel is shown as priority information.

[0030] For example, the priority information of a channel having a channel number of 0 is 3, and the priority information of a channel having a channel number of 1 is 0. Note that hereinafter, a channel having a predetermined channel number m (m=0, 1, . . . , M-1) will also be referred to as channel m.

[0031] The value of the priority information shown in Figure 3 is set to any value from 0 to 7 as shown in Figure 4, and the higher the value of the priority information, the higher the priority, or importance, of the audio signal when it is played back.

[0032] Therefore, an audio signal with a priority information value of 0 has the lowest priority, and an audio signal with a priority information value of 7 has the highest priority.

[0033] When multi-channel audio signals or audio signals of multiple objects are reproduced simultaneously, the sounds reproduced by those audio signals usually include sounds that are less important than other sounds. In other words, there are sounds that are not reproduced enough to cause discomfort to the listener even if a certain sound is not reproduced from the entire sound.

[0034] Therefore, by not decoding audio signals with low priority as necessary, it is possible to reduce the amount of calculation required for decoding while suppressing deterioration in sound quality. Therefore, in the encoding device, priority information indicating the degree of importance of each audio signal during playback, i.e., the degree to which decoding should be prioritized, is assigned to each audio signal for each frame so that audio signals not to be decoded can be appropriately selected.

[0035] When the priority information of each audio signal is determined as described above, the priority information is stored in the DSE of element EL1 shown in Fig. 1. In particular, in the example of Fig. 3, since the number of channels constituting the multi-channel audio signal is M, the priority information of each of the M channels from channel 0 to channel M-1 is stored in the DSE.

[0036] Similarly, priority information for each object is also stored in the DSE of element EL1. For example, if there are N objects with object numbers from 0 to N-1, priority information is defined for each of the N objects and stored in the DSE.

[0037] In the following description, an object with a predetermined object number n (n=0, 1, . . . , N-1) will also be referred to as object n.

[0038] In this way, by defining priority information for each audio signal, it is possible on the playback side, i.e., the audio signal decoding side, to easily determine which audio signal is important during playback and should be decoded with priority, i.e., should be used for playback.

[0039] Returning to the explanation of FIG. 2, for example, assume that the priority information of the audio signals of frames F11 and F13 of a given channel is 7, and the priority information of the audio signal of frame F12 of the given channel is 0.

[0040] Also, it is assumed that the audio signal decoding side, that is, the decoding device (decoder), is configured not to decode an audio signal having a priority level lower than a predetermined level.

[0041] Here, for example, if the specified priority level is called a threshold and the threshold level is 4, in the above example, decoding is performed on the audio signals of frames F11 and F13 of a specified channel, whose priority information is 7.

[0042] On the other hand, the audio signal of frame F12 of the predetermined channel, whose priority information is 0, is not decoded.

[0043] Therefore, in this example, the audio signal of frame F12 is made a silence signal, and the audio signals of frames F11 and F13 are synthesized to form a final audio signal of a predetermined channel.

[0044] More specifically, for example, when encoding each audio signal, a time-frequency transform is performed on the audio signal, information obtained by the time-frequency transform is encoded, and the resulting encoded data is stored in an element.

[0045] Although any processing may be performed as the time-frequency transformation, the following description will be given on the assumption that the time-frequency transformation is a modified discrete cosine transform (MDCT).

[0046] In addition, in the decoding device, the encoded data is decoded, and the resulting MDCT coefficients are subjected to an IMDCT (Inverse Modified Discrete Cosine Transform) to generate an audio signal. That is, the IMDCT is performed here as an inverse transform (frequency-time transform) of the time-frequency transform.

[0047] Therefore, more specifically, for frames F11 and F13 whose priority information is equal to or greater than the threshold value 4, IMDCT is performed to generate audio signals.

[0048] Furthermore, for frame F12, whose priority information is less than the threshold value 4, the IMDCT is not performed, and an audio signal is generated with the IMDCT result set to 0. As a result, the audio signal for frame F12 is a silent signal, that is, 0 data.

[0049] As yet another example, in the example shown in Figure 3, when the threshold value is 4, among the audio signals of channels 0 to M-1, the audio signals of channels 0, 1, and M-2 whose priority information has a value less than the threshold value of 4 will not be decoded.

[0050] As described above, by not decoding audio signals with a low priority level as indicated by the priority information depending on the result of the comparison with the threshold value, it is possible to reduce the amount of calculation required for decoding while minimizing deterioration in sound quality.

[0051] <Example of the configuration of the encoding device> Next, specific embodiments of an encoding device and a decoding device to which the present technology is applied will be described. First, the encoding device will be described.

[0052] FIG. 5 is a diagram showing an example of the configuration of an encoding device to which the present technology is applied.

[0053] The encoding device 11 in FIG. 5 includes a channel audio encoding unit 21, an object audio encoding unit 22, a metadata input unit 23, and a packing unit 24.

[0054] The channel audio encoding unit 21 is supplied with audio signals of each of the M multi-channels. For example, the audio signals of each channel are supplied from a microphone corresponding to each channel. In Fig. 5, the characters "#0" to "#M-1" represent the channel numbers of each channel.

[0055] The channel audio encoding unit 21 encodes the audio signal of each channel supplied thereto, generates priority information based on the audio signal, and supplies the encoded data obtained by encoding and the priority information to a packing unit 24.

[0056] The audio signals of the N objects are supplied to the object audio encoding unit 22. For example, the audio signals of the objects are supplied from microphones attached to the objects. In Fig. 5, the characters "#0" to "#N-1" represent the object numbers of the objects.

[0057] The object audio encoding unit 22 encodes the audio signal of each object supplied thereto, generates priority information based on the audio signal, and supplies the encoded data obtained by encoding and the priority information to a packing unit 24.

[0058] The metadata input unit 23 supplies metadata of each object to the packing unit 24. For example, the metadata of an object is spatial position information indicating the position of the object in space. More specifically, for example, the spatial position information is three-dimensional coordinate information indicating the coordinates of the position of the object in three-dimensional space.

[0059] The packing unit 24 packs the encoded data and priority information supplied from the channel audio encoding unit 21, the encoded data and priority information supplied from the object audio encoding unit 22, and the metadata supplied from the metadata input unit 23, to generate a bitstream, and outputs it.

[0060] The bitstream obtained in this way contains, for each frame, the encoded data for each channel, the priority information for each channel, the encoded data for each object, the priority information for each object, and the metadata for each object.

[0061] Here, the audio signals of the M channels and the audio signals of the N objects stored in the bitstream for one frame are audio signals of the same frame that should be reproduced simultaneously.

[0062] Note that, here, an example will be described in which priority information is generated for each audio signal for each channel or object for each frame, but it is also possible to generate one piece of priority information for audio signals for several frames, for example, in any predetermined unit of time.

[0063] <Configuration example of channel audio encoding unit> More specifically, the channel audio encoding unit 21 in FIG. 5 is configured as shown in FIG.

[0064] The channel audio encoding unit 21 shown in FIG.

[0065] The encoding unit 51 includes an MDCT unit 61, and encodes the audio signals of each channel supplied from the outside.

[0066] That is, the MDCT unit 61 performs MDCT on the audio signal of each channel supplied from the outside. The encoding unit 51 encodes the MDCT coefficients of each channel obtained by the MDCT, and supplies the encoded data of each channel obtained as a result, that is, the encoded audio signal, to the packing unit 24.

[0067] Furthermore, the priority information generating unit 52 analyzes the audio signals of each channel supplied from the outside, generates priority information of the audio signals of each channel, and supplies the priority information to the packing unit 24 .

[0068] <Configuration example of object audio encoding unit> Further, in more detail, the object audio encoding unit 22 in FIG. 5 is configured as shown in FIG.

[0069] The object audio encoding unit 22 shown in FIG.

[0070] The encoding unit 91 includes an MDCT unit 101, and encodes the audio signal of each object supplied from the outside.

[0071] That is, the MDCT unit 101 performs MDCT on the audio signal of each object supplied from the outside. The encoding unit 91 encodes the MDCT coefficient of each object obtained by the MDCT, and supplies the encoded data of each object obtained as a result, that is, the encoded audio signal, to the packing unit 24.

[0072] Furthermore, the priority information generating unit 92 analyzes the audio signal of each object supplied from the outside, generates priority information of the audio signal of each object, and supplies it to the packing unit 24 .

[0073] <Description of Encoding Process> Next, the process performed by the encoding device 11 will be described.

[0074] When the encoding device 11 is supplied with one frame of audio signals of multiple channels and audio signals of multiple objects to be played simultaneously, it performs an encoding process and outputs a bitstream including the encoded audio signals.

[0075] The encoding process by the encoding device 11 will be described below with reference to the flowchart in Fig. 8. Note that this encoding process is performed for each frame of the audio signal.

[0076] In step S11, the priority information generating unit 52 of the channel audio encoding unit 21 generates priority information of the audio signal of each supplied channel, and supplies the priority information to the packing unit 24. For example, the priority information generating unit 52 analyzes the audio signal for each channel, and generates priority information based on the sound pressure and spectral shape of the audio signal, correlation of spectral shapes between channels, and the like.

[0077] In step S12, the packing unit 24 stores, in the DSE of the bitstream, the priority information of the audio signal of each channel supplied from the priority information generating unit 52. That is, the priority information is stored in the first element of the bitstream.

[0078] In step S13, the priority information generation unit 92 of the object audio encoding unit 22 generates priority information of the audio signal of each supplied object, and supplies it to the packing unit 24. For example, the priority information generation unit 92 analyzes the audio signal for each object, and generates priority information based on the sound pressure and spectral shape of the audio signal, correlation of spectral shapes between objects, etc.

[0079] In addition, when generating priority information for audio signals of each channel or each object, the number of audio signals to which each priority level, which is the value of the priority information, is assigned may be predetermined with respect to the number of channels or the number of objects.

[0080] For example, in the example of Figure 3, the number of audio signals whose priority information is set to "7", i.e., the number of channels, may be predetermined to be 5, and the number of audio signals whose priority information is set to "6" may be predetermined to be 3.

[0081] In step S14, the packing unit 24 stores the priority information of the audio signal of each object supplied from the priority information generating unit 92 in the DSE of the bitstream.

[0082] In step S15, the packing unit 24 stores the metadata of each object in the DSE of the bitstream.

[0083] For example, the metadata input unit 23 receives an input operation from a user, communicates with the outside, or reads from an external recording area to obtain metadata of each object and supplies the metadata to the packing unit 24. The packing unit 24 stores the metadata thus supplied from the metadata input unit 23 in the DSE.

[0084] Through the above processing, the DSE of the bitstream stores priority information of audio signals of all channels, priority information of audio signals of all objects, and metadata of all objects.

[0085] In step S16, the encoding unit 51 of the channel audio encoding unit 21 encodes the supplied audio signal of each channel.

[0086] More specifically, the MDCT unit 61 performs MDCT on the audio signal of each channel, and the encoding unit 51 encodes the MDCT coefficients of each channel obtained by the MDCT, and supplies the resulting encoded data of each channel to the packing unit 24.

[0087] In step S17, the packing unit 24 stores the encoded data of the audio signal of each channel supplied from the encoding unit 51 in the SCE or CPE of the bitstream. That is, the encoded data is stored in each element arranged following the DSE in the bitstream.

[0088] In step S18, the encoding section 91 of the object audio encoding unit 22 encodes the audio signal of each object supplied thereto.

[0089] More specifically, the MDCT unit 101 performs MDCT on the audio signal of each object, and the encoding unit 91 encodes the MDCT coefficients of each object obtained by the MDCT, and supplies the resulting encoded data of each object to the packing unit 24.

[0090] In step S19, the packing unit 24 stores the encoded data of the audio signal of each object supplied from the encoding unit 91 in the SCE of the bitstream. That is, the encoded data is stored in some elements arranged after the DSE in the bitstream.

[0091] Through the above processing, a bitstream is obtained that stores priority information and encoded data of audio signals for all channels, priority information and encoded data of audio signals for all objects, and metadata of all objects for the frame being processed.

[0092] In step S20, the packing unit 24 outputs the obtained bitstream, and the encoding process ends.

[0093] In this way, the encoding device 11 generates priority information for the audio signal of each channel and for each object, stores the information in a bitstream, and outputs the bitstream. This makes it easy to know which audio signal has a higher priority on the decoding side.

[0094] This allows the decoding side to selectively decode the encoded audio signal according to the priority information, thereby reducing the amount of calculation required for decoding while minimizing deterioration in the sound quality of the sound reproduced by the audio signal.

[0095] In particular, by storing priority information of the audio signal of each object in the bitstream, not only can the amount of calculation required for decoding be reduced on the decoding side, but the amount of calculation required for subsequent processing such as rendering can also be reduced.

[0096] <Configuration Example of Decoding Device> Next, a decoding device will be described which receives the bit stream output from the encoding device 11 described above and decodes the encoded data contained in the bit stream.

[0097] Such a decoding device may be configured, for example, as shown in FIG.

[0098] The decoding device 151 shown in FIG. 9 includes an unpacking / decoding unit 161, a rendering unit 162, and a mixing unit 163.

[0099] The unpacking / decoding unit 161 obtains the bitstream output from the encoding device 11, and unpacks and decodes the bitstream.

[0100] The unpacking / decoding unit 161 supplies the audio signal of each object obtained by unpacking and decoding and the metadata of each object to the rendering unit 162. At this time, the unpacking / decoding unit 161 decodes the encoded data of each object according to priority information included in the bitstream.

[0101] Furthermore, the unpacking / decoding unit 161 supplies the audio signals of each channel obtained by unpacking and decoding to the mixing unit 163. At this time, the unpacking / decoding unit 161 decodes the encoded data of each channel in accordance with the priority information included in the bitstream.

[0102] The rendering unit 162 generates M channel audio signals based on the audio signals of each object supplied from the unpacking / decoding unit 161 and the spatial position information of each object as metadata, and supplies the generated audio signals to the mixing unit 163. At this time, the rendering unit 162 generates the M channel audio signals such that the sound image of each object is localized at the position indicated by the spatial position information of the object.

[0103] The mixing unit 163 generates a final audio signal for each channel by performing weighted addition for each channel of the audio signal for each channel supplied from the unpacking / decoding unit 161 and the audio signal for each channel supplied from the rendering unit 162. The mixing unit 163 supplies the final audio signal for each channel thus obtained to an external speaker corresponding to each channel, and reproduces the sound.

[0104] <Example of the unpacking / decoding unit configuration> Moreover, the unpacking / decoding unit 161 of the decoding device 151 shown in FIG. 9 is configured in more detail as shown in FIG. 10, for example.

[0105] The unpacking / decoding unit 161 shown in FIG. 10 includes a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero-value output unit 195, an IMDCT unit 196, an object audio signal acquisition unit 197, an object audio signal decoding unit 198, an output selection unit 199, a zero-value output unit 200, and an IMDCT unit 201.

[0106] The priority information acquisition unit 191 acquires priority information of the audio signals of each channel from the supplied bitstream and supplies it to the output selection unit 194, and also acquires priority information of the audio signals of each object from the bitstream and supplies it to the output selection unit 199.

[0107] Furthermore, the priority information acquisition unit 191 acquires metadata of each object from the supplied bitstream and supplies the metadata to the rendering unit 162, and also supplies the bitstream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197.

[0108] The channel audio signal acquisition unit 192 acquires encoded data of each channel from the bit stream supplied from the priority information acquisition unit 191, and supplies the data to a channel audio signal decoding unit 193. The channel audio signal decoding unit 193 decodes the encoded data of each channel supplied from the channel audio signal acquisition unit 192, and supplies the MDCT coefficients obtained as a result to an output selection unit 194.

[0109] The output selection unit 194 selectively switches the output destination of the MDCT coefficients of each channel supplied from the channel audio signal decoding unit 193 based on the priority information of each channel supplied from the priority information acquisition unit 191 .

[0110] That is, when the priority information for a predetermined channel is less than a predetermined threshold P, the output selection unit 194 sets the MDCT coefficient of the channel to 0 and supplies it to the zero value output unit 195. When the priority information for a predetermined channel is equal to or greater than the predetermined threshold P, the output selection unit 194 supplies the MDCT coefficient of the channel supplied from the channel audio signal decoding unit 193 to the IMDCT unit 196.

[0111] The zero value output unit 195 generates an audio signal based on the MDCT coefficients supplied from the output selection unit 194, and supplies the audio signal to the mixing unit 163. In this case, since the MDCT coefficients are zero, a silent audio signal is generated.

[0112] The IMDCT unit 196 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 194 to generate an audio signal, and supplies the audio signal to the mixing unit 163 .

[0113] The object audio signal acquisition unit 197 acquires encoded data of each object from the bitstream supplied from the priority information acquisition unit 191, and supplies the data to an object audio signal decoding unit 198. The object audio signal decoding unit 198 decodes the encoded data of each object supplied from the object audio signal acquisition unit 197, and supplies the MDCT coefficients obtained as a result to an output selection unit 199.

[0114] The output selection unit 199 selectively switches the output destination of the MDCT coefficient of each object supplied from the object audio signal decoding unit 198 based on the priority information of each object supplied from the priority information acquisition unit 191 .

[0115] That is, when the priority information for a predetermined object is less than a predetermined threshold Q, the output selection unit 199 sets the MDCT coefficient of the object to 0 and supplies it to the zero value output unit 200. When the priority information for a predetermined object is equal to or greater than the predetermined threshold Q, the output selection unit 199 supplies the MDCT coefficient of the object supplied from the object audio signal decoding unit 198 to the IMDCT unit 201.

[0116] The value of threshold Q may be the same as the value of threshold P, or may be different from the value of threshold P. By appropriately determining thresholds P and Q according to the computational capabilities of decoding device 151, the amount of computation required for decoding an audio signal can be reduced to a range within which decoding device 151 can decode in real time.

[0117] The zero value output unit 200 generates an audio signal based on the MDCT coefficients supplied from the output selection unit 199, and supplies the signal to the rendering unit 162. In this case, since the MDCT coefficients are zero, a silent audio signal is generated.

[0118] The IMDCT unit 201 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 199 to generate an audio signal, and supplies the audio signal to the rendering unit 162 .

[0119] Explanation of the Decryption Process Next, the operation of the decoding device 151 will be described.

[0120] When a bit stream for one frame is supplied from the encoding device 11, the decoding device 151 performs a decoding process to generate an audio signal, and outputs the audio signal to a speaker. The decoding process performed by the decoding device 151 will be described below with reference to the flowchart in FIG.

[0121] In step S51, the unpacking / decoding unit 161 obtains the bit stream transmitted from the encoding device 11. That is, the bit stream is received.

[0122] In step S52, the unpacking / decoding unit 161 performs a selective decoding process.

[0123] In the selective decoding process, the encoded data of each channel and the encoded data of each object are selectively decoded based on priority information, as will be described later in detail. The audio signals of each channel obtained as a result are then supplied to a mixing unit 163, and the audio signals of each object are supplied to a rendering unit 162. Metadata of each object obtained from the bitstream is also supplied to the rendering unit 162.

[0124] In step S53, the rendering unit 162 renders the audio signal of each object based on the audio signal of each object supplied from the unpacking / decoding unit 161 and the spatial position information of each object as metadata.

[0125] For example, the rendering unit 162 generates audio signals for each channel by vector based amplitude panning (VBAP) based on the spatial position information so that the sound image of each object is localized at the position indicated by the spatial position information, and supplies the generated audio signals to the mixing unit 163.

[0126] In step S54, the mixing unit 163 performs weighted addition for each channel of the audio signal of each channel supplied from the unpacking / decoding unit 161 and the audio signal of each channel supplied from the rendering unit 162, and supplies the result to the external speaker. As a result, the audio signal of the channel corresponding to each speaker is supplied to each speaker, and each speaker reproduces sound based on the supplied audio signal.

[0127] When the audio signals of each channel are supplied to the speakers, the decoding process is completed.

[0128] In this manner, the decoding device 151 obtains the priority information from the bitstream, and decodes the coded data of each channel and each object according to the priority information.

[0129] <Description of Selective Decoding Process> Next, the selective decoding process corresponding to the process in step S52 in FIG. 11 will be described with reference to the flowchart in FIG.

[0130] In step S81, the priority information acquisition unit 191 acquires priority information of the audio signal of each channel and priority information of the audio signal of each object from the supplied bitstream, and supplies them to the output selection unit 194 and the output selection unit 199, respectively.

[0131] Furthermore, the priority information acquisition unit 191 acquires metadata of each object from the bitstream and supplies the metadata to the rendering unit 162 , and also supplies the bitstream to the channel audio signal acquisition unit 192 and the object audio signal acquisition unit 197 .

[0132] In step S82, the channel audio signal acquisition unit 192 sets the channel number of the channel to be processed to 0 and holds it.

[0133] In step S83, the channel audio signal acquisition unit 192 determines whether the held channel number is less than the number M of channels.

[0134] If it is determined in step S83 that the channel number is less than M, in step S84, the channel audio signal decoding unit 193 decodes the encoded data of the audio signal of the channel to be processed.

[0135] That is, the channel audio signal acquisition unit 192 acquires the encoded data of the channel to be processed from the bit stream supplied from the priority information acquisition unit 191 , and supplies the acquired data to a channel audio signal decoding unit 193 .

[0136] Then, the channel audio signal decoding unit 193 decodes the encoded data supplied from the channel audio signal acquisition unit 192 , and supplies the MDCT coefficients obtained as a result to the output selection unit 194 .

[0137] In step S85, the output selection unit 194 judges whether or not the priority information of the channel to be processed supplied from the priority information acquisition unit 191 is equal to or greater than a threshold value P designated by a higher-level control device (not shown). Here, the threshold value P is determined according to, for example, the computational capability of the decoding device 151.

[0138] If it is determined in step S85 that the priority information is equal to or greater than the threshold value P, the output selection unit 194 supplies the MDCT coefficients of the channel to be processed supplied from the channel audio signal decoding unit 193 to the IMDCT unit 196, and the process proceeds to step S86. In this case, the priority of the audio signal of the channel to be processed is equal to or greater than a predetermined priority, so that decoding of the channel, more specifically, IMDCT is performed.

[0139] In step S86, the IMDCT unit 196 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 194 to generate an audio signal of the channel to be processed, and supplies the audio signal to the mixing unit 163. After the audio signal is generated, the process proceeds to step S87.

[0140] On the other hand, if it is determined in step S85 that the priority information is less than the threshold value P, the output selection unit 194 sets the MDCT coefficients to 0 and supplies them to the 0 value output unit 195.

[0141] The zero value output unit 195 generates an audio signal of the channel to be processed from the MDCT coefficients that are zero supplied from the output selection unit 194, and supplies the signal to the mixing unit 163. Therefore, the zero value output unit 195 does not actually perform any processing for generating an audio signal, such as IMDCT.

[0142] It should be noted that the audio signal generated by the zero value output unit 195 is a silent signal. After the audio signal is generated, the process proceeds to step S87.

[0143] If it is determined in step S85 that the priority information is less than the threshold value P, or if an audio signal is generated in step S86, in step S87, the channel audio signal acquisition unit 192 adds 1 to the channel number it holds, and updates the channel number of the channel to be processed.

[0144] Once the channel number is updated, the process returns to step S83, and the above-described process is repeated, i.e., an audio signal for a new channel to be processed is generated.

[0145] Also, if it is determined in step S83 that the channel number of the channel to be processed is not less than M, audio signals have been obtained for all channels, and the process proceeds to step S88.

[0146] In step S88, the object audio signal acquisition unit 197 sets the object number of the object to be processed to 0 and holds it.

[0147] In step S89, the object audio signal acquisition unit 197 determines whether the held object number is less than the number N of objects.

[0148] If it is determined in step S89 that the object number is less than N, then in step S90, the object audio signal decoding unit 198 decodes the encoded data of the audio signal of the object to be processed.

[0149] That is, the object audio signal acquisition unit 197 acquires the encoded data of the object to be processed from the bit stream supplied from the priority information acquisition unit 191 , and supplies the acquired data to an object audio signal decoding unit 198 .

[0150] Then, the object audio signal decoding unit 198 decodes the encoded data supplied from the object audio signal acquisition unit 197 , and supplies the MDCT coefficients obtained as a result to the output selection unit 199 .

[0151] In step S91, the output selection unit 199 judges whether or not the priority information of the object to be processed supplied from the priority information acquisition unit 191 is equal to or greater than a threshold Q designated by a higher-level control device (not shown). Here, the threshold Q is determined according to, for example, the computational capability of the decoding device 151.

[0152] If it is determined in step S91 that the priority information is equal to or greater than the threshold value Q, the output selection unit 199 supplies the MDCT coefficients of the object to be processed, supplied from the object audio signal decoding unit 198, to the IMDCT unit 201, and the process proceeds to step S92.

[0153] In step S92, the IMDCT unit 201 performs IMDCT based on the MDCT coefficients supplied from the output selection unit 199 to generate an audio signal of the object to be processed, and supplies the audio signal to the rendering unit 162. After the audio signal is generated, the process proceeds to step S93.

[0154] On the other hand, if it is determined in step S 91 that the priority information is less than the threshold value Q, the output selection unit 199 sets the MDCT coefficients to 0 and supplies them to the 0 value output unit 200 .

[0155] The zero value output unit 200 generates an audio signal of the object to be processed from the MDCT coefficients that are zero supplied from the output selection unit 199, and supplies the signal to the rendering unit 162. Therefore, the zero value output unit 200 does not actually perform any processing for generating an audio signal, such as IMDCT.

[0156] It should be noted that the audio signal generated by the zero value output unit 200 is a silent signal. After the audio signal is generated, the process proceeds to step S93.

[0157] If it is determined in step S91 that the priority information is less than the threshold value Q or if an audio signal is generated in step S92, then in step S93, the object audio signal acquisition unit 197 adds 1 to the object number it holds, and updates the object number of the object being processed.

[0158] Once the object number has been updated, the process returns to step S89, and the above-described process is repeated, i.e., an audio signal for a new object to be processed is generated.

[0159] Also, if it is determined in step S89 that the object number of the object being processed is not less than N, the selective decoding process ends since audio signals have been obtained for all channels and objects, and then the process proceeds to step S53 in FIG.

[0160] In this manner, the decoding device 151 decodes the encoded audio signal while comparing the priority information with a threshold value for each channel or object and determining whether or not to decode the encoded audio signal for each channel or object of the frame to be processed.

[0161] That is, in the decoding device 151, a predetermined number of encoded audio signals according to the priority information of each audio signal are decoded, and the remaining audio signals are not decoded.

[0162] This makes it possible to selectively decode only audio signals with high priority according to the playback environment, thereby reducing the amount of decoding calculations while minimizing degradation in the sound quality of the audio reproduced by the audio signals.

[0163] Furthermore, by decoding the encoded audio signal based on the priority information of the audio signal of each object, it is possible to reduce not only the amount of calculation required for decoding the audio signal, but also the amount of calculation required for subsequent processing, such as processing in the rendering unit 162, etc.

[0164] First Modification of the First Embodiment <About priority information> In the above description, one piece of priority information is generated for one audio signal of each channel or object. However, a plurality of pieces of priority information may be generated.

[0165] In such a case, for example, a plurality of pieces of priority information are generated for each computational capacity according to the amount of computation required for decoding, that is, the computational capacity of the decoding side.

[0166] Specifically, for example, priority information for a device having a computing capability equivalent to two channels is generated based on the amount of calculation required to decode an audio signal equivalent to two channels in real time.

[0167] In the case of such a device having two channels, the priority information is generated so that, for example, of all audio signals, more audio signals are assigned a lower priority, i.e., a value closer to 0, as priority information.

[0168] Furthermore, priority information for a device having a computing capability equivalent to 24 channels is also generated based on the amount of calculation required to decode an audio signal equivalent to 24 channels in real time. The priority information for a device equivalent to 24 channels is generated so that, for example, more audio signals among all audio signals are assigned a higher priority, i.e., a value closer to 7, as priority information.

[0169] In this case, for example, in step S11 of FIG. 8, the priority information generating unit 52 generates priority information for a device equivalent to two channels for the audio signals of each channel, adds an identifier to the priority information indicating that it is for a device equivalent to two channels, and supplies the priority information to the packing unit 24.

[0170] Furthermore, in step S11, the priority information generating unit 52 also generates priority information for a device equivalent to 24 channels for the audio signals of each channel, adds an identifier to the priority information indicating that it is for a device equivalent to 24 channels, and supplies the priority information to the packing unit 24.

[0171] 8, the priority information generating unit 92 also generates priority information for a device equivalent to 2 channels and priority information for a device equivalent to 24 channels, adds identifiers to the information, and supplies the information to the packing unit 24.

[0172] This makes it possible to obtain multiple pieces of priority information according to the computing capabilities of playback devices such as portable audio players, multi-function mobile phones, tablet computers, television receivers, personal computers, and high-quality audio equipment.

[0173] For example, playback devices such as portable audio players have relatively low computing power, so by decoding an audio signal encoded based on priority information for a device equivalent to two channels, such playback devices can play back the audio signal in real time.

[0174] As described above, when multiple pieces of priority information are generated for one audio signal, in the decoding device 151, for example, a higher-level control device instructs the priority information acquisition unit 191 etc. which of the multiple pieces of priority information should be used for decoding. The instruction as to which priority information to use is given by, for example, supplying an identifier.

[0175] It should be noted that it may be determined in advance for each decryption device 151 which identifier's priority information is to be used.

[0176] For example, in the case where it is determined in advance in the priority information acquisition unit 191 which identifier of the priority information to be used, or where an identifier is specified by a higher-level control device, the priority information acquisition unit 191 acquires priority information to which the determined identifier is added in step S81 in Fig. 12. Then, the acquired priority information is supplied from the priority information acquisition unit 191 to the output selection unit 194 and the output selection unit 199.

[0177] In other words, one appropriate piece of priority information is selected from among a plurality of pieces of priority information stored in the bitstream according to the computational capabilities of the decoding device 151, more specifically, the unpacking / decoding unit 161, and so on.

[0178] In this case, the priority information may be read from the bitstream using different identifiers for the priority information of each channel and the priority information of each object.

[0179] In this way, by selecting and acquiring specific priority information from among multiple pieces of priority information contained in the bitstream, it is possible to select appropriate priority information according to the computational capabilities, etc. of the decoding device 151 and perform decoding. This makes it possible for any of the decoding devices 151 to reproduce the audio signal in real time.

[0180] Second embodiment <Example of the unpacking / decoding unit configuration> In the above, an example has been described in which priority information is included in the bitstream output from the encoding device 11. However, depending on the encoding device, priority information may not be included in the bitstream.

[0181] Therefore, the priority information may be generated in the decoding device 151. For example, the priority information can be generated using information indicating the sound pressure of an audio signal or information indicating a spectral shape, which can be extracted from the encoded data of the audio signal included in the bitstream.

[0182] In this way, when generating priority information in the decoding device 151, the unpacking / decoding unit 161 of the decoding device 151 is configured as shown in Fig. 13, for example. Note that in Fig. 13, the same reference numerals are used to designate parts corresponding to those in Fig. 10, and the description thereof will be omitted as appropriate.

[0183] The unpacking / decoding unit 161 shown in FIG. 13 includes a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero-value output unit 195, an IMDCT unit 196, an object audio signal acquisition unit 197, an object audio signal decoding unit 198, an output selection unit 199, a zero-value output unit 200, an IMDCT unit 201, a priority information generation unit 231, and a priority information generation unit 232.

[0184] The configuration of the unpacking / decoding unit 161 shown in FIG. 13 differs from that of the unpacking / decoding unit 161 in FIG. 10 in that a priority information acquisition unit 191 is not provided and a priority information generation unit 231 and a priority information generation unit 232 are newly provided, but the other configurations are the same as those of the unpacking / decoding unit 161 in FIG. 10.

[0185] The channel audio signal acquisition unit 192 acquires the encoded data of each channel from the supplied bit stream, and supplies the data to a channel audio signal decoding unit 193 and a priority information generation unit 231 .

[0186] The priority information generating unit 231 generates priority information for each channel based on the encoded data for each channel supplied from the channel audio signal acquiring unit 192 , and supplies the priority information to the output selecting unit 194 .

[0187] The object audio signal acquisition unit 197 acquires encoded data of each object from the supplied bitstream, and supplies the data to the object audio signal decoding unit 198 and the priority information generation unit 232. The object audio signal acquisition unit 197 also acquires metadata of each object from the supplied bitstream, and supplies the metadata to the rendering unit 162.

[0188] The priority information generating unit 232 generates priority information of each object based on the encoded data of each object supplied from the object audio signal obtaining unit 197 , and supplies the priority information to the output selecting unit 199 .

[0189] <Description of Selective Decoding Process> When the unpacking / decoding unit 161 has the configuration shown in Fig. 13, the decoding device 151 performs selective decoding processing shown in Fig. 14 as processing corresponding to step S52 of the decoding processing shown in Fig. 11. Hereinafter, the selective decoding processing by the decoding device 151 will be described with reference to the flowchart in Fig. 14.

[0190] In step S131, the priority information generating unit 231 generates priority information for the audio signal of each channel.

[0191] For example, the channel audio signal acquisition unit 192 acquires the encoded data of each channel from the supplied bit stream, and supplies the data to the channel audio signal decoding unit 193 and the priority information generation unit 231 .

[0192] The priority information generating unit 231 generates priority information for each channel based on the encoded data for each channel supplied from the channel audio signal acquiring unit 192 , and supplies the priority information to the output selecting unit 194 .

[0193] For example, a bit stream contains, as coded data of an audio signal, scale factors for obtaining MDCT coefficients, side information, and a quantized spectrum, where the scale factors indicate the sound pressure of the audio signal, and the quantized spectrum indicates the spectral shape of the audio signal.

[0194] The priority information generating unit 231 generates priority information for the audio signal of each channel based on the scale factor and quantized spectrum included in the encoded data of each channel. By generating the priority information using the scale factor and quantized spectrum in this way, the priority information can be obtained immediately before decoding the encoded data, and the amount of calculation required for generating the priority information can be reduced.

[0195] Alternatively, the priority information may be generated based on the sound pressure of the audio signal obtained by calculating the mean square value of the MDCT coefficients, or the spectrum shape of the audio signal obtained from the peak envelope of the MDCT coefficients. In this case, the priority information generating unit 231 decodes the encoded data and obtains the MDCT coefficients from the channel audio signal decoding unit 193 as appropriate.

[0196] After the priority information of each channel is obtained, the processes of steps S132 to S137 are performed, but since these processes are similar to the processes of steps S82 to S87 in Fig. 12, the description thereof will be omitted. However, in this case, since the encoded data of each channel has already been obtained, only the decoding of the encoded data is performed in step S134.

[0197] Also, if it is determined in step S133 that the channel number is not less than M, the priority information generating unit 232 generates priority information of the audio signal of each object in step S138.

[0198] For example, the object audio signal acquisition unit 197 acquires encoded data of each object from the supplied bitstream, and supplies the data to the object audio signal decoding unit 198 and the priority information generation unit 232. The object audio signal acquisition unit 197 also acquires metadata of each object from the supplied bitstream, and supplies the metadata to the rendering unit 162.

[0199] The priority information generating unit 232 generates priority information of each object based on the encoded data of each object supplied from the object audio signal obtaining unit 197, and supplies the priority information to the output selecting unit 199. For example, the priority information is generated based on a scale factor or a quantized spectrum, like in the case of each channel.

[0200] Alternatively, the priority information may be generated based on the sound pressure or the spectrum shape obtained from the MDCT coefficients. In this case, the priority information generating unit 232 decodes the encoded data and obtains the MDCT coefficients from the object audio signal decoding unit 198 as appropriate.

[0201] After the priority information of each object is obtained, the processes of steps S139 to S144 are performed and the selective decoding process ends, but since these processes are similar to the processes of steps S88 to S93 in Fig. 12, their description will be omitted. However, in this case, since the encoded data of each object has already been obtained, only the decoding of the encoded data is performed in step S141.

[0202] After the selective decoding process is completed, the process proceeds to step S53 in FIG.

[0203] In this manner, the decoding device 151 generates priority information for audio signals of each channel and each object based on the encoded data included in the bitstream. By generating priority information in this manner in the decoding device 151, appropriate priority information for each audio signal can be obtained with a small amount of calculation, and the amount of calculation for decoding and rendering, etc. can be reduced. In addition, deterioration of the sound quality of the sound reproduced by the audio signal can be minimized.

[0204] Note that the priority information may be generated when the priority information acquisition unit 191 of the unpacking / decoding unit 161 shown in Fig. 10 attempts to acquire priority information of audio signals of each channel and each object from a supplied bitstream but is unable to acquire the priority information from the bitstream. In such a case, the priority information acquisition unit 191 performs the same process as the priority information generation unit 231 and the priority information generation unit 232, and generates priority information of audio signals of each channel and each object from encoded data.

[0205] Third embodiment <About priority information threshold> Furthermore, in the above description, the audio signal to be decoded, more specifically the MDCT coefficients to perform IMDCT, is selected for each channel and each object by comparing the priority information with the thresholds P and Q. However, these thresholds P and Q may be dynamically changed for each frame of the audio signal.

[0206] For example, the priority information acquisition unit 191 of the unpacking / decoding unit 161 shown in FIG. 10 can acquire the priority information of each channel and each object from the bitstream without requiring decoding.

[0207] Therefore, for example, if the priority information acquisition unit 191 reads out the priority information of the audio signals of all channels, it can obtain the distribution of the priority information in the frame to be processed. Also, the decoding device 151 knows in advance its own computational capability, for example, how many channels it can process simultaneously, i.e., in real time.

[0208] Therefore, the priority information acquisition unit 191 may determine a threshold value P of the priority information for the frame to be processed, based on the distribution of the priority information in the frame to be processed and the computational capacity of the decoding device 151.

[0209] For example, the threshold value P is determined so that the maximum amount of audio signals can be decoded within the range that the decoding device 151 can process in real time.

[0210] Furthermore, the priority information acquisition unit 191 can dynamically determine the threshold Q in the same manner as in the case of the threshold P. In this case, the priority information acquisition unit 191 obtains a distribution of the priority information of the audio signals of all objects based on the priority information of the objects, and determines the threshold Q of the priority information for the frame to be processed based on the obtained distribution and the computing capacity of the decoding device 151.

[0211] The threshold values ​​P and Q can be determined with a relatively small amount of calculation.

[0212] In this way, by dynamically changing the threshold value of the priority information, it is possible to minimize the deterioration of the sound quality of the sound reproduced by the audio signal while performing decoding in real time. In particular, in such a case, since it is not necessary to prepare multiple pieces of priority information, and it is not necessary to provide an identifier for the priority information, the amount of code of the bit stream can be reduced.

[0213] About object metadata Furthermore, in the first to third embodiments described above, it has been explained that metadata, priority information, and the like of objects for one frame are stored in the first element of a bit stream.

[0214] In this case, the syntax of the portion of the first element of the bitstream in which object metadata and priority information are stored is, for example, as shown in FIG.

[0215] In the example shown in FIG. 15, spatial position information and priority information of an object are stored in the metadata of the object for one frame only.

[0216] In this example, "num_objects" indicates the number of objects. Also, "object_priority[o]" indicates priority information of the Oth object. Here, the Oth object is an object identified by its object number.

[0217] "position_azimuth[o]" indicates the horizontal angle representing the three-dimensional spatial position of the Oth object as seen by the user who is the viewer, that is, as seen from a predetermined reference position. "position_elevation[o]" indicates the vertical angle representing the three-dimensional spatial position of the Oth object as seen by the user who is the viewer. "position_radius[o]" indicates the distance from the viewer to the Oth object.

[0218] Therefore, the position of an object in three-dimensional space is determined from "position_azimuth[o]," "position_elevation[o]," and "position_radius[o]," and this information is considered to be the spatial position information of the object.

[0219] Also, "gain_factor[o]" indicates the gain of the Oth object.

[0220] 15, the "object_priority[o]", "position_azimuth[o]", "position_elevation[o]", "position_radius[o]", and "gain_factor[o]" for one object are arranged in order as data for that object. In the metadata, the data for each object is arranged, for example, in the order of the object number of the object.

[0221] Fourth embodiment Perfect reconstruction of audio signals and noise caused by discontinuities In the above, an example has been described in which the amount of processing during decoding is reduced by omitting the decoding process such as IMDCT when the priority information of each frame (hereinafter, particularly referred to as a time frame) for each channel or object read from the bitstream in the decoding device 151 is less than a predetermined threshold. Specifically, it has been described that when the priority information is less than a threshold, a silent audio signal is output from the zero value output unit 195 or the zero value output unit 200, that is, zero data is output as an audio signal.

[0222] However, in such a case, the sound quality deteriorates when heard. Specifically, the sound quality deteriorates due to perfect reconstruction of the audio signal and due to noise caused by discontinuity of the signal, such as glitch noise.

[0223] (Sound quality degradation due to perfect reconstruction) For example, if 0 data is output as an audio signal when the priority information is less than a threshold value, deterioration in sound quality occurs when switching between output of 0 data and output of a normal audio signal other than 0 data.

[0224] As described above, in the unpacking / decoding unit 161, the IMDCT is performed on the MDCT coefficients for each time frame read from the bitstream in the IMDCT unit 196 or the IMDCT unit 201. More specifically, in the unpacking / decoding unit 161, an audio signal of the current time frame is generated from the IMDCT result or 0 data for the current time frame and the IMDCT result or 0 data for the time frame immediately preceding.

[0225] Here, the generation of an audio signal will be described with reference to Fig. 16. Note that, although the generation of an audio signal of an object will be described as an example here, the generation of an audio signal of each channel is similar. In addition, hereinafter, the audio signal output from the zero value output unit 200 and the audio signal output from the IMDCT unit 201 will also be particularly referred to as IMDCT signals. Similarly, the audio signal output from the zero value output unit 195 and the audio signal output from the IMDCT unit 196 will also be particularly referred to as IMDCT signals.

[0226] 16, the horizontal direction represents time, and the rectangles marked with the characters "data[n-1]" to "data[n+2]" represent the bit streams of a given object in time frame (n-1) to time frame (n+2), respectively. The numerical values ​​in the bit stream of each time frame represent the value of the priority information of the object in that time frame, and in this example, the value of the priority information of each time frame is "7."

[0227] Furthermore, in FIG. 16, rectangles marked with the letters "MDCT_coef[q]" (where q=n-1, n, . . . ) each represent an MDCT coefficient of a time frame (q).

[0228] Now, assuming that the threshold Q=4, the value "7" of the priority information for time frame (n-1) is equal to or greater than the threshold Q, so the IMDCT is performed on the MDCT coefficients for time frame (n-1). Similarly, the value "7" of the priority information for time frame (n) is also equal to or greater than the threshold Q, so the IMDCT is performed on the MDCT coefficients for time frame (n).

[0229] As a result, an IMDCT signal OPS11 of time frame (n-1) and an IMDCT signal OPS12 of time frame (n) are obtained.

[0230] In this case, the unpacking / decoding unit 161 adds the first half of the IMDCT signal OPS12 in the time frame (n) to the second half of the IMDCT signal OPS11 in the time frame (n-1) one time frame earlier to obtain the audio signal in the time frame (n), that is, the audio signal in the period FL(n). In other words, the portion of the IMDCT signal OPS11 in the period FL(n) and the portion of the IMDCT signal OPS12 in the period FL(n) are overlap-added to reproduce the audio signal in the time frame (n) before the encoding of the object to be processed.

[0231] Such processing is necessary to perfectly reconstruct the IMDCT signal into the signal before the MDCT.

[0232] However, in the above-described unpacking / decoding unit 161, the IMDCT signal is not perfectly reconstructed to the signal before MDCT at the timing of switching between the IMDCT signal of the IMDCT unit 201 and the IMDCT signal of the 0 value output unit 200 in accordance with the priority information of each time frame, as shown in Fig. 17 for example. In other words, if 0 data is used instead of the original signal during overlap addition, perfect reconstruction is not performed, so that the original audio signal cannot be reproduced and the audible sound quality of the audio signal deteriorates.

[0233] In FIG. 17, the same characters are used for the parts corresponding to those in FIG. 16, and the explanation thereof will be omitted.

[0234] In the example of FIG. 17, the priority information value of time frame (n-1) is "7", while the priority information of the other time frames (n) through (n+2) is the lowest "0".

[0235] Therefore, if the threshold value Q=4, then for time frame (n-1), the IMDCT section 201 performs IMDCT on the MDCT coefficients to obtain an IMDCT signal OPS21 for time frame (n-1). On the other hand, for time frame (n), the IMDCT is not performed on the MDCT coefficients, and the zero data output from the zero value output section 200 becomes the IMDCT signal OPS22 for time frame (n).

[0236] In this case, the first half of the 0 data, which is the IMDCT signal OPS22 of the time frame (n), is added to the second half of the IMDCT signal OPS21 of the time frame (n-1) one time frame earlier to generate the audio signal of the final time frame (n). That is, the IMDCT signal OPS22 and the IMDCT signal OPS21 of the period FL(n) are overlap-added to generate the audio signal of the final time frame (n) of the object to be processed.

[0237] In this way, when the output source of the IMDCT signal switches from the IMDCT unit 201 to the zero-value output unit 200, or from the zero-value output unit 200 to the IMDCT unit 201, the IMDCT signal from the IMDCT unit 201 is no longer perfectly reconstructed, resulting in a deterioration in the audible sound quality.

[0238] (Deterioration of sound quality due to noise caused by discontinuity) Furthermore, when the output source of the IMDCT signal switches from the IMDCT unit 201 to the 0 value output unit 200, or from the 0 value output unit 200 to the IMDCT unit 201, the signal is not completely reconstructed, and the signal may become discontinuous at the connection between the IMDCT signal obtained by the IMDCT and the IMDCT signal that has been set to 0 data. If this happens, glitch noise occurs at the discontinuous connection, degrading the audible sound quality of the audio signal.

[0239] Furthermore, in order to improve the sound quality in the unpacking / decoding unit 161, processing such as SBR (Spectral Band Replication) may be performed on the audio signal obtained by overlap-adding the IMDCT signals output from the IMDCT unit 201 and the zero-value output unit 200.

[0240] Although various processes can be considered as the processes subsequent to the IMDCT unit 201 and the zero-value output unit 200, the following description will be continued using SBR as an example.

[0241] In SBR, the high-frequency components of the original audio signal before encoding are generated from the audio signal obtained by overlap-adding, which is the low-frequency components, and the high-frequency power values ​​stored in the bitstream.

[0242] Specifically, an audio signal for one time frame is divided into several sections called time slots, and the audio signal for each time slot is band-divided into multiple low-frequency subband signals (hereinafter also referred to as low-frequency subband signals).

[0243] Then, based on the low frequency subband signal of each subband and the power value of each subband on the high frequency side, a signal of each subband in the high frequency range (hereinafter also referred to as a high frequency subband signal) is generated. For example, the low frequency subband signal of a specific subband is power-adjusted or frequency-shifted according to the power value of a target subband in the high frequency range, thereby generating a target high frequency subband signal.

[0244] Furthermore, the high-frequency subband signal and the low-frequency subband signal are synthesized to generate an audio signal containing high-frequency components, and the audio signals containing high-frequency components generated for each time slot are combined to generate an audio signal of one time frame containing high-frequency components.

[0245] When such SBR is performed in the subsequent stages of the IMDCT unit 201 and the zero value output unit 200, high frequency components are generated by SBR for the audio signal consisting of the IMDCT signal output from the IMDCT unit 201. However, since the IMDCT signal output from the zero value output unit 200 is zero data, the high frequency components obtained by SBR for the audio signal consisting of the IMDCT signal output from the zero value output unit 200 also become zero data.

[0246] As a result, when the output source of the IMDCT signal switches from the IMDCT unit 201 to the zero-value output unit 200, or from the zero-value output unit 200 to the IMDCT unit 201, the connection portion may become discontinuous even in the high frequency range. In such a case, glitch noise occurs, degrading the sound quality as perceived by the ear.

[0247] Therefore, in this technology, we have made it possible to suppress the above-mentioned deterioration in audible sound quality and improve sound quality by selecting the output destination of MDCT coefficients taking into account the previous and next time frames, and by performing fade-in and fade-out processing on the audio signal.

[0248] <Selection of output destination of MDCT coefficients considering previous and next time frames> First, the selection of the output destination of the MDCT coefficients taking into account the preceding and following time frames will be described. Note that, although the audio signal of an object will be described as an example here as well, the same applies to the audio signals of each channel. Furthermore, the processing described below is performed for each object and each channel.

[0249] For example, in the above embodiment, it has been explained that the output selection unit 199 selectively switches the output destination of the MDCT coefficients of each object based on the priority information of the current time frame. In contrast, in this embodiment, the output selection unit 199 switches the output destination of the MDCT coefficients based on the priority information of three temporally consecutive time frames, namely, the current time frame, the time frame immediately before the current time frame, and the time frame immediately after the current time frame. In other words, whether or not to decode the encoded data is selected based on the priority information of the three consecutive time frames.

[0250] Specifically, when the conditional expression shown in the following equation (1) is satisfied for an object to be processed, the output selection unit 199 supplies the MDCT coefficients of the time frame (n) of the object to the IMDCT unit 201.

[0251]

number

[0252] In the formula (1), object_priority[q] (where q=n−1, n, n+1) indicates priority information for each time frame (q), and thre indicates a threshold value Q.

[0253] Therefore, if there is at least one time frame whose priority information is equal to or greater than the threshold Q among a total of three consecutive time frames including the current time frame and the time frames before and after the current time frame, the IMDCT unit 201 is selected as the supply destination of the MDCT coefficients. In this case, decoding of the encoded data, more specifically, IMDCT of the MDCT coefficients is performed. On the other hand, if the priority information of all of the three time frames is less than the threshold Q, the MDCT coefficients are set to 0 and output to the zero value output unit 200. In this case, decoding of the encoded data, more specifically, IMDCT of the MDCT coefficients is not substantially performed.

[0254] As a result, the audio signal is perfectly reconstructed from the IMDCT signal, and deterioration of the audible sound quality is suppressed, as shown in Fig. 18. Note that in Fig. 18, the same characters and the like are used for parts corresponding to those in Fig. 16, and their explanation will be omitted.

[0255] In the example shown in the upper part of Fig. 18, the value of the priority information for each time frame is the same as the example shown in Fig. 17. For example, if the threshold Q=4, in the example shown in the upper part of the figure, the priority information for time frame (n-1) is equal to or greater than the threshold Q, but the priority information for time frames (n) to (n+2) is less than the threshold Q.

[0256] Therefore, from the conditional expression shown in equation (1), the IMDCT is performed on the MDCT coefficients of the time frame (n-1) and the time frame (n), and the IMDCT signals OPS31 and OPS32 are obtained, respectively. On the other hand, in the time frame (n+1) where the conditional expression is not satisfied, the IMDCT is not performed on the MDCT coefficients, and 0 data is used as the IMDCT signal OPS33.

[0257] Therefore, the audio signal of time frame (n) which was not completely reconstructed in the example of Fig. 17 is completely reconstructed in the example shown in the upper part of Fig. 18, and deterioration of the sound quality as perceived by the ear is suppressed. However, in this example, since the audio signal is not completely reconstructed in the next time frame (n+1), a fade-out process to be described later is performed in time frame (n) and time frame (n+1), and deterioration of the sound quality as perceived by the ear is suppressed.

[0258] In addition, in the example shown at the bottom of the figure, the priority information is less than threshold Q from time frame (n-1) to time frame (n+1), and the priority information is greater than or equal to threshold Q in time frame (n+2).

[0259] Therefore, in time frame (n) where the conditional expression shown in equation (1) is not satisfied, the IMDCT is not performed on the MDCT coefficients, and 0 data is output as the IMDCT signal OPS 41. In contrast, the IMDCT is performed on the MDCT coefficients of time frame (n+1) and time frame (n+2), resulting in the IMDCT signals OPS 42 and OPS 43, respectively.

[0260] In this example, the audio signal can be completely reconstructed in time frame (n+2) when the priority information switches from a value below threshold Q to a value equal to or greater than threshold Q, thereby suppressing deterioration in the audible sound quality. However, even in this case, the audio signal is not completely reconstructed in the immediately preceding time frame (n+1), so a fade-in process (described later) is performed in time frame (n+1) and time frame (n+2), thereby suppressing deterioration in the audible sound quality.

[0261] In this example, the priority information for one time frame is read ahead, and the output destination of the MDCT coefficients is selected based on the priority information for three consecutive time frames. Therefore, fade-out processing is performed in the time frame (n) and the time frame (n+1) in the example shown on the upper side of the figure, and fade-in processing is performed in the time frame (n+1) and the time frame (n+2) in the example shown on the lower side of the figure.

[0262] However, if it is possible to read ahead priority information for two time frames, fade-out processing may be performed in time frame (n+1) and time frame (n+2) in the example shown at the top of the figure, and fade-in processing may be performed in time frame (n) and time frame (n+1) in the example shown at the bottom of the figure.

[0263] <Fade-in and fade-out processing> Next, fade-in and fade-out processing for audio signals will be described. Note that, although the audio signal of an object will be described as an example here as well, the same applies to the audio signal of each channel. Furthermore, fade-in and fade-out processing is performed for each object and each channel.

[0264] In this technique, as in the example shown in FIG. 18, for example, fade-in processing or fade-out processing is performed in a time frame in which an IMDCT signal obtained by IMDCT and an IMDCT signal that is 0 data are overlap-added, and in a time frame before or after that.

[0265] In the fade-in process, gain adjustment is performed on the audio signal so that the amplitude (volume) of the audio signal in that time frame increases over time, whereas in the fade-out process, gain adjustment is performed on the audio signal so that the amplitude of the audio signal in that time frame decreases over time.

[0266] This makes it possible to suppress deterioration of the audible sound quality even when the connection portion between the IMDCT signal obtained by IMDCT and the IMDCT signal set to 0 data is discontinuous. Note that hereinafter, the gain value by which the audio signal is multiplied during such gain adjustment is also referred to as a fading signal gain.

[0267] Furthermore, in this technique, fade-in processing or fade-out processing is also performed in SBR on the connection portion between the IMDCT signal obtained by the IMDCT and the IMDCT signal which is 0 data.

[0268] That is, in SBR, the power value of each high frequency subband is used for each time slot, but in this technology, the power value of each high frequency subband is multiplied by a gain value determined for each time slot for fade-in processing or fade-out processing to perform SBR. In other words, the gain of the high frequency power value is adjusted.

[0269] In the following description, the gain value determined for each time slot and multiplied by the high frequency power value will be referred to as a fading SBR gain.

[0270] Specifically, the fading SBR gain for fade-in processing is determined so that its gain value increases with time, i.e., the fading SBR gain value increases for later time slots, whereas the fading SBR gain for fade-out processing is determined so that its gain value decreases for later time slots.

[0271] In this way, by performing fade-in and fade-out processing even during SBR, it is possible to suppress deterioration in sound quality as perceived even when high frequencies are discontinuous.

[0272] Specifically, gain adjustment such as fade-in and fade-out processing for such audio signals and high-frequency power values ​​is performed as shown in Figures 19 and 20. Note that in Figures 19 and 20, the same characters and symbols are used for parts corresponding to those in Figure 18, and their explanation will be omitted.

[0273] The example shown in Fig. 19 is the example shown in the upper part of Fig. 18. In this example, the audio signals of the time frame (n) and the time frame (n+1) are multiplied by the fading signal gain shown by the broken line GN11.

[0274] The value of the fading signal gain shown by the broken line GN11 changes linearly over time from "1" to "0" in the portion of time frame (n), and remains at "0" in the portion of time frame (n+1). Therefore, by adjusting the gain of the audio signal using the fading signal gain, the audio signal gradually changes to 0 data, thereby suppressing deterioration of the sound quality as perceived by the ear.

[0275] Also, in this example, the high frequency power value of each time slot of time frame (n) is multiplied by the fading SBR gain indicated by arrow GN12.

[0276] The fading SBR gain value indicated by the arrow GN12 varies from "1" to "0" so that it becomes smaller the further back the time slot. Therefore, by adjusting the high-frequency gain using the fading SBR gain, the high-frequency components of the audio signal gradually change to 0 data, suppressing deterioration of the sound quality as perceived by the ear.

[0277] In contrast, the example shown in Fig. 20 is an example of the case shown in the lower part of Fig. 18. In this example, the audio signals of the time frame (n+1) and the time frame (n+2) are multiplied by the fading signal gain shown by the broken line GN21.

[0278] The value of the fading signal gain shown by the broken line GN21 remains "0" in the time frame (n+1), and changes linearly from "0" to "1" over time in the time frame (n+2). Therefore, by adjusting the gain of the audio signal using the fading signal gain, the audio signal gradually changes from 0 data to the original signal, thereby suppressing deterioration of the audible sound quality.

[0279] Also, in this example, the high frequency power value of each time slot of the time frame (n+2) is multiplied by the fading SBR gain indicated by the arrow GN22.

[0280] The value of the fading SBR gain indicated by the arrow GN22 changes from "0" to "1" so that it becomes larger the further back the time slot. Therefore, by adjusting the high-frequency gain using the fading SBR gain, the high-frequency components of the audio signal gradually change from 0 data to the original signal, thereby suppressing deterioration of the sound quality as perceived by the ear.

[0281] <Example of the unpacking / decoding unit configuration> When the above-described selection of the output destination of the MDCT coefficients and gain adjustment such as fade-in processing and fade-out processing are performed, the unpacking / decoding unit 161 is configured as shown in Fig. 21. In Fig. 21, the same reference numerals are used to designate parts corresponding to those in Fig. 10, and the description thereof will be omitted as appropriate.

[0282] The unpacking / decoding unit 161 shown in FIG. 21 is made up of a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, an overlap addition unit 271, a gain adjustment unit 272, an SBR processing unit 273, an object audio signal acquisition unit 197, an object audio signal decoding unit 198, an output selection unit 199, a zero value output unit 200, an IMDCT unit 201, an overlap addition unit 274, a gain adjustment unit 275, and an SBR processing unit 276.

[0283] The configuration of the unpacking / decoding unit 161 shown in FIG. 21 is configured by further providing an overlap addition unit 271 through an SBR processing unit 276 in addition to the configuration of the unpacking / decoding unit 161 shown in FIG.

[0284] The overlap addition unit 271 performs overlap addition on the IMDCT signal (audio signal) supplied from the zero value output unit 195 or the IMDCT unit 196 to generate an audio signal for each time frame, and supplies the audio signal to the gain adjustment unit 272 .

[0285] The gain adjustment unit 272 adjusts the gain of the audio signal supplied from the overlap addition unit 271 based on the priority information supplied from the priority information acquisition unit 191 , and supplies the adjusted gain to the SBR processing unit 273 .

[0286] The SBR processing unit 273 acquires the power value of each subband in the high frequency band for each time slot from the priority information acquisition unit 191, and adjusts the gain of the power value of the high frequency band based on the priority information supplied from the priority information acquisition unit 191. The SBR processing unit 273 also performs SBR on the audio signal supplied from the gain adjustment unit 272 using the gain-adjusted power value of the high frequency band, and supplies the audio signal obtained as a result to the mixing unit 163.

[0287] The overlap addition unit 274 performs overlap addition on the IMDCT signal (audio signal) supplied from the zero value output unit 200 or the IMDCT unit 201 to generate an audio signal for each time frame, and supplies the audio signal to the gain adjustment unit 275.

[0288] The gain adjustment unit 275 adjusts the gain of the audio signal supplied from the overlap addition unit 274 based on the priority information supplied from the priority information acquisition unit 191 , and supplies the adjusted signal to the SBR processing unit 276 .

[0289] The SBR processing unit 276 acquires the power value of each subband in the high frequency band for each time slot from the priority information acquisition unit 191, and adjusts the gain of the power value of the high frequency band based on the priority information supplied from the priority information acquisition unit 191. The SBR processing unit 276 also performs SBR on the audio signal supplied from the gain adjustment unit 275 using the gain-adjusted power value of the high frequency band, and supplies the audio signal obtained as a result to the rendering unit 162.

[0290] <Description of Selective Decoding Process> Next, an operation of the decoding device 151 in the case where the unpacking / decoding unit 161 has the configuration shown in Fig. 21 will be described. In this case, the decoding device 151 performs the decoding process described with reference to Fig. 11. However, as the selective decoding process in step S52, the process shown in Fig. 22 is performed.

[0291] The selective decoding process corresponding to the process in step S52 in FIG. 11 will now be described with reference to the flowchart in FIG.

[0292] In step S181, the priority information acquisition unit 191 acquires the high-frequency power values ​​of the audio signal of each channel from the supplied bitstream and supplies them to the SBR processing unit 273, and also acquires the high-frequency power values ​​of the audio signal of each object from the bitstream and supplies them to the SBR processing unit 276.

[0293] After the high-frequency power value is obtained, the processes of steps S182 to S187 are performed to generate an audio signal (IMDCT signal) of the channel to be processed. Since these processes are similar to the processes of steps S81 to S86 in FIG. 12, their description is omitted.

[0294] However, in step S186, if a conditional expression similar to the above-mentioned expression (1) is satisfied, that is, if any one of the priority information of the current time frame of the channel to be processed and the priority information of each of the time frames immediately before and after the current time frame is equal to or greater than the threshold value P, it is determined that the priority information is equal to or greater than the threshold value P. In addition, the IMDCT signal generated in the zero value output unit 195 or the IMDCT unit 196 is output to the overlap addition unit 271.

[0295] If it is not determined in step S186 that the priority information is equal to or greater than the threshold value P, or if an IMDCT signal is generated in step S187, the process goes to step S188.

[0296] In step S 188 , the overlap addition unit 271 performs overlap addition on the IMDCT signal supplied from the zero value output unit 195 or the IMDCT unit 196 , and supplies the audio signal of the current time frame obtained as a result to the gain adjustment unit 272 .

[0297] Specifically, for example, as described with reference to FIG. 18, the first half of the IMDCT signal of the current time frame and the second half of the IMDCT signal of the immediately preceding time frame are added together to generate the audio signal of the current time frame.

[0298] In step S189, the gain adjustment unit 272 adjusts the gain of the audio signal supplied from the overlap addition unit 271 based on the priority information of the channel to be processed supplied from the priority information acquisition unit 191, and supplies the adjusted gain to the SBR processing unit 273.

[0299] Specifically, when the priority information of the time frame immediately preceding the current time frame is equal to or greater than the threshold P, and the priority information of the current time frame and the priority information of the time frame immediately following the current time frame are less than the threshold P, the gain adjustment unit 272 adjusts the gain of the audio signal with the fading signal gain shown by the broken line GN11 in Fig. 19. In this case, the time frame (n) in Fig. 19 corresponds to the current time frame, and in the time frame immediately following the current time frame, gain adjustment is performed with the fading signal gain = 0, as shown by the broken line GN11.

[0300] Furthermore, when the priority information of the current time frame is equal to or greater than the threshold P and the priority information of the two time frames immediately preceding the current time frame is both less than the threshold P, the gain adjustment unit 272 adjusts the gain of the audio signal with the fading signal gain shown by the broken line GN21 in Fig. 20. In this case, the time frame (n+2) in Fig. 20 corresponds to the current time frame, and in the time frame immediately preceding the current time frame, gain adjustment is performed with the fading signal gain=0, as shown by the broken line GN21.

[0301] It should be noted that the gain adjustment unit 272 performs gain adjustment only in these two cases, and in other cases, does not perform gain adjustment and supplies the audio signal to the SBR processing unit 273 as is.

[0302] In step S190, the SBR processing unit 273 performs SBR on the audio signal supplied from the gain adjustment unit 272, based on the high-frequency power value and priority information of the channel to be processed supplied from the priority information acquisition unit 191.

[0303] Specifically, when the priority information of the time frame immediately preceding the current time frame is equal to or greater than the threshold P, and the priority information of the current time frame and the priority information of the time frame immediately following the current time frame are less than the threshold P, the SBR processing unit 273 gain-adjusts the high-frequency power value with the fading SBR gain indicated by the arrow GN12 in Fig. 19. That is, the high-frequency power value is multiplied by the fading SBR gain.

[0304] Then, the SBR processing unit 273 performs SBR using the gain-adjusted high-frequency power value, and supplies the audio signal obtained as a result to the mixing unit 163. In this case, the time frame (n) in Fig. 19 corresponds to the current time frame.

[0305] Furthermore, when the priority information of the current time frame is equal to or greater than the threshold P and the priority information of the two time frames immediately preceding the current time frame is both less than the threshold P, the SBR processing unit 273 performs gain adjustment on the high-frequency power value with a fading SBR gain indicated by an arrow GN22 in Fig. 20. Then, the SBR processing unit 273 performs SBR using the gain-adjusted high-frequency power value, and supplies the audio signal obtained as a result to the mixing unit 163. In this case, the time frame (n+2) in Fig. 20 corresponds to the current time frame.

[0306] It should be noted that the SBR processing unit 273 performs gain adjustment of the high-frequency power value only in these two cases, and in other cases, does not perform gain adjustment, performs SBR using the acquired high-frequency power value as is, and supplies the resulting audio signal to the mixing unit 163.

[0307] When the audio signal of the current time frame is obtained by performing SBR, the processes of steps S191 to S196 are then performed. However, since these processes are similar to the processes of steps S87 to S92 in FIG. 12, their description will be omitted.

[0308] However, in step S195, if the conditional expression of the above-mentioned equation (1) is satisfied, it is determined that the priority information is equal to or greater than the threshold value Q. In addition, the IMDCT signal (audio signal) generated by the zero value output unit 200 or the IMDCT unit 201 is output to the overlap addition unit 274.

[0309] Once the IMDCT signal of the current time frame is obtained in this manner, the processes of steps S197 to S199 are carried out to generate an audio signal of the current time frame. Since these processes are similar to the processes of steps S188 to S190, their description will be omitted.

[0310] In step S200, when the object audio signal acquisition unit 197 adds 1 to the object number, the process returns to step S193. Then, if it is determined in step S193 that the object number is not less than N, the selection decoding process ends, and then the process proceeds to step S53 in FIG.

[0311] In this manner, the unpacking / decoding unit 161 selects the output destination of the MDCT coefficients according to the priority information of the current time frame and the time frames before and after it. This allows the audio signal to be completely reconstructed at the transition portion between a time frame whose priority information is equal to or greater than the threshold and a time frame whose priority information is less than the threshold, thereby suppressing deterioration of the audible sound quality.

[0312] Furthermore, the unpacking / decoding unit 161 adjusts the gain of the audio signal after overlap addition and the power value of the high frequency band based on the priority information of three consecutive time frames. That is, fade-in processing and fade-out processing are performed as appropriate. This makes it possible to suppress the occurrence of glitch noise and suppress deterioration of the sound quality as perceived by the ear.

[0313] Fifth embodiment <Fade-in and fade-out processing> In the fourth embodiment, gain adjustment is performed on the audio signal after overlap addition, and further gain adjustment is performed on the power value of the high frequency range during SBR. In this case, gain adjustment is performed separately for the low frequency component and the high frequency component of the final audio signal, that is, fade-in processing and fade-out processing are performed.

[0314] Therefore, in order to realize these fade-in and fade-out processes with less processing, gain adjustment may not be performed immediately after overlap addition or during SBR, but rather gain adjustment may be performed on the audio signal obtained by SBR.

[0315] In such a case, gain adjustment is performed, for example, as shown in Figures 23 and 24. In Figures 23 and 24, the same characters and the like are used for parts corresponding to those in Figures 19 and 20, and the description thereof will be omitted.

[0316] The example shown in Fig. 23 is an example in which the change in priority information is the same as that shown in Fig. 19. In this example, if the threshold Q=4, the priority information of time frame (n-1) is equal to or greater than the threshold Q, but the priority information of time frame (n) to time frame (n+2) is less than the threshold Q.

[0317] In such a case, the audio signals obtained by SBR in time frame (n) and time frame (n+1) are multiplied by the fading signal gain indicated by broken line GN31 for gain adjustment.

[0318] The fading signal gain indicated by this broken line GN31 is the same as the fading signal gain indicated by the broken line GN11 in Fig. 19. However, in the example of Fig. 23, the audio signal to be subjected to gain adjustment contains both low-frequency components and high-frequency components, so that the gain adjustment of these low-frequency components and high-frequency components can be performed with one fading signal gain.

[0319] By adjusting the gain of the audio signal using this fading signal gain, the audio signal gradually changes to 0 data in the portion where the IMDCT signal obtained by IMDCT and the IMDCT signal set to 0 data are overlap-added and in the portion immediately before that. This makes it possible to suppress deterioration of the sound quality as perceived by the ear.

[0320] In contrast, the example shown in Fig. 24 is an example in which the change in priority information is the same as that shown in Fig. 20. In this example, if the threshold Q=4, the priority information in time frame (n) and time frame (n+1) is less than threshold Q, but the priority information in time frame (n+2) is equal to or greater than threshold Q.

[0321] In such a case, the audio signals obtained by SBR in the time frame (n+1) and the time frame (n+2) are multiplied by the fading signal gain shown by the broken line GN41 for gain adjustment.

[0322] The fading signal gain indicated by this broken line GN41 is the same as the fading signal gain indicated by the broken line GN21 in Fig. 20. However, in the example of Fig. 24, the audio signal to be the target of gain adjustment contains both low-frequency components and high-frequency components, so that the gain adjustment of these low-frequency components and high-frequency components can be performed with one fading signal gain.

[0323] By adjusting the gain of the audio signal using this fading signal gain, the audio signal gradually changes from 0 data to the original signal in the portion where the IMDCT signal obtained by IMDCT and the IMDCT signal set to 0 data are overlap-added and in the portion immediately following that. This makes it possible to suppress deterioration of the sound quality as perceived by the ear.

[0324] <Example of the unpacking / decoding unit configuration> When gain adjustment is performed by the fade-in process or fade-out process described with reference to Fig. 23 and Fig. 24, the unpacking / decoding unit 161 is configured as shown in Fig. 25, for example. In Fig. 25, the same reference numerals are used to designate parts corresponding to those in Fig. 21, and description thereof will be omitted as appropriate.

[0325] The unpacking / decoding unit 161 shown in FIG. 25 is made up of a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, an overlap addition unit 271, an SBR processing unit 273, a gain adjustment unit 272, an object audio signal acquisition unit 197, an object audio signal decoding unit 198, an output selection unit 199, a zero value output unit 200, an IMDCT unit 201, an overlap addition unit 274, an SBR processing unit 276, and a gain adjustment unit 275.

[0326] The configuration of the unpacking / decoding unit 161 shown in FIG. 25 differs from the configuration of the unpacking / decoding unit 161 shown in FIG. 21 in that the gain adjustment unit 272 and the gain adjustment unit 275 are arranged subsequent to the SBR processing unit 273 and the SBR processing unit 276, respectively.

[0327] 25, the SBR processing unit 273 performs SBR on the audio signal supplied from the overlap addition unit 271 based on the high-frequency power value supplied from the priority information acquisition unit 191, and supplies the audio signal obtained as a result to the gain adjustment unit 272. In this case, the SBR processing unit 273 does not perform gain adjustment of the high-frequency power value.

[0328] The gain adjustment unit 272 adjusts the gain of the audio signal supplied from the SBR processing unit 273 based on the priority information supplied from the priority information acquisition unit 191 , and supplies the adjusted signal to the mixing unit 163 .

[0329] The SBR processing unit 276 performs SBR on the audio signal supplied from the overlap addition unit 274 based on the high-frequency power value supplied from the priority information acquisition unit 191, and supplies the audio signal obtained as a result to the gain adjustment unit 275. In this case, the SBR processing unit 276 does not perform gain adjustment of the high-frequency power value.

[0330] The gain adjustment unit 275 adjusts the gain of the audio signal supplied from the SBR processing unit 276 based on the priority information supplied from the priority information acquisition unit 191 , and supplies the adjusted signal to the rendering unit 162 .

[0331] <Description of Selective Decoding Process> Next, an operation of the decoding device 151 in the case where the unpacking / decoding unit 161 has the configuration shown in Fig. 25 will be described. In this case, the decoding device 151 performs the decoding process described with reference to Fig. 11. However, as the selective decoding process in step S52, the process shown in Fig. 26 is performed.

[0332] Hereinafter, the selective decoding process corresponding to the process of step S52 in Fig. 11 will be described with reference to the flowchart in Fig. 26. Note that the processes of steps S231 to S238 are similar to the processes of steps S181 to S188 in Fig. 22, and therefore their description will be omitted. However, in step S232, priority information is not supplied to the SBR processing unit 273 and the SBR processing unit 276.

[0333] In step S239, the SBR processing unit 273 performs SBR on the audio signal supplied from the overlap addition unit 271 based on the high-frequency power value supplied from the priority information acquisition unit 191, and supplies the resulting audio signal to the gain adjustment unit 272.

[0334] In step S240, the gain adjustment unit 272 adjusts the gain of the audio signal supplied from the SBR processing unit 273 based on the priority information of the channel to be processed supplied from the priority information acquisition unit 191, and supplies the adjusted signal to the mixing unit 163.

[0335] Specifically, when the priority information of the time frame immediately preceding the current time frame is equal to or greater than the threshold P, and the priority information of the current time frame and the priority information of the time frame immediately following the current time frame are less than the threshold P, the gain adjustment unit 272 adjusts the gain of the audio signal with the fading signal gain shown by the broken line GN31 in Fig. 23. In this case, the time frame (n) in Fig. 23 corresponds to the current time frame, and in the time frame immediately following the current time frame, gain adjustment is performed with the fading signal gain = 0, as shown by the broken line GN31.

[0336] Furthermore, when the priority information of the current time frame is equal to or greater than the threshold P and the priority information of the two time frames immediately preceding the current time frame is both less than the threshold P, the gain adjustment unit 272 adjusts the gain of the audio signal with the fading signal gain shown by the broken line GN41 in Fig. 24. In this case, the time frame (n+2) in Fig. 24 corresponds to the current time frame, and in the time frame immediately preceding the current time frame, gain adjustment is performed with the fading signal gain=0, as shown by the broken line GN41.

[0337] It should be noted that the gain adjustment unit 272 performs gain adjustment only in these two cases, and in other cases, supplies the audio signal to the mixing unit 163 as is without performing gain adjustment.

[0338] After the gain adjustment of the audio signal is performed, the processes of steps S241 to S247 are performed. However, since these processes are similar to the processes of steps S191 to S197 in FIG. 22, the description thereof will be omitted.

[0339] Once the audio signal of the current time frame of the object being processed is obtained in this manner, processing of steps S248 and S249 is carried out to generate the final audio signal of the current time frame; however, as these processing steps are similar to the processing steps S239 and S240, their description will be omitted.

[0340] In step S250, the object audio signal obtainer 197 adds 1 to the object number, and the process returns to step S243. If it is determined in step S243 that the object number is not less than N, the selection decoding process ends, and the process then proceeds to step S53 in FIG.

[0341] In this way, the unpacking / decoding unit 161 adjusts the gain of the audio signal obtained by SBR based on the priority information of three consecutive time frames. This makes it easier to suppress the occurrence of glitch noise and to suppress deterioration of sound quality as perceived by the ear.

[0342] In this embodiment, an example has been described in which the output destination of the MDCT coefficients is selected using priority information for three time frames, and the gain adjustment is performed based on the fading signal gain. However, it is also possible to perform only the gain adjustment based on the fading signal gain.

[0343] In such a case, the output selection unit 194 and the output selection unit 199 select the output destination of the MDCT coefficients by the same process as in the first embodiment. Then, when the priority information of the current time frame is less than the threshold, the gain adjustment unit 272 and the gain adjustment unit 275 perform fade-in processing or fade-out processing by linearly increasing or decreasing the fading signal gain of the current time frame. Here, whether to perform fade-in processing or fade-out processing can be determined based on the priority information of the current time frame and the priority information of the time frames before and after it.

[0344] Sixth embodiment <Fade-in and fade-out processing> Meanwhile, in the rendering unit 162, for example, VBAP is performed to generate audio signals of each channel for reproducing the sound of each object from the audio signal of each object.

[0345] Specifically, in VBAP, for each channel, that is, for each speaker that outputs sound, a gain value of an audio signal (hereinafter also referred to as VBAP gain) is calculated for each object for each time frame. Then, the sum of the audio signals of each object multiplied by the VBAP gain for the same channel (speaker) is set as the audio signal of that channel. In other words, for each object, the audio signal of the object is assigned to each of those channels with the VBAP gain calculated for each channel.

[0346] Therefore, for the audio signal of an object, rather than adjusting the gain of the audio signal of the object or the power value of the high frequency range, the VBAP gain may be appropriately adjusted to suppress the occurrence of glitch noise and thereby suppress deterioration of the audible sound quality.

[0347] In such a case, for example, linear interpolation or the like is performed on the VBAP gain of each time frame, the VBAP gain for each sample of the audio signal in each time frame is calculated, and the audio signal of each channel is generated using the obtained VBAP gain.

[0348] For example, the VBAP gain value of the first sample of the time frame to be processed is set to the VBAP gain value of the last sample of the time frame immediately preceding the time frame to be processed, and the VBAP gain value of the last sample of the time frame to be processed is set to the VBAP gain value calculated by the normal VBAP for the time frame to be processed.

[0349] Then, in the time frame to be processed, the VBAP gain value of each sample between the first sample and the last sample is determined so that the VBAP gain changes linearly from the first sample to the last sample.

[0350] However, if the priority information of the time frame to be processed is less than the threshold value, the VBAP calculation is not performed, and the VBAP gain value of the last sample of the time frame to be processed is set to 0. Then, the VBAP gain of each sample is determined so that the VBAP gain changes linearly from the first sample to the last sample of the time frame to be processed.

[0351] In this way, by adjusting the gain of the audio signal of each object using the VBAP gain, it is possible to adjust the gain of low-frequency components and high-frequency components at the same time, thereby suppressing the occurrence of glitch noise with less processing power and reducing deterioration in the audible sound quality.

[0352] When the VBAP gain is determined for each sample in this manner, the VBAP gain for each sample in each time frame is as shown in FIG. 27 or FIG. 28, for example.

[0353] 27 and 28, the same characters are used for parts corresponding to those in Fig. 19 and 20, and the explanation thereof will be omitted. Also, in Fig. 27 and 28, "VBAP_gain[q][s]" (where q=n-1, n, n+1, n+2) indicates the VBAP gain of the object to be processed in time frame (q) where the speaker index specifying the speaker corresponding to a specified channel is s.

[0354] The example shown in Fig. 27 is an example in which the change in priority information is the same as that shown in Fig. 19. In this example, if the threshold Q=4, the priority information of time frame (n-1) is equal to or greater than the threshold Q, but the priority information of time frame (n) to time frame (n+2) is less than the threshold Q.

[0355] In such a case, the VBAP gain from time frame (n-1) to time frame (n+1) is set to, for example, the gain indicated by broken line GN51.

[0356] In this example, since the priority information of the time frame (n-1) is equal to or greater than the threshold Q, the VBAP gain of each sample is determined based on the VBAP gain calculated by the normal VBAP.

[0357] That is, the VBAP gain value of the first sample in time frame (n-1) is the same as the VBAP gain value of the last sample in time frame (n-2). The VBAP gain value of the last sample in time frame (n-1) is the VBAP gain value of the channel corresponding to speaker s calculated by normal VBAP for time frame (n-1) for the object being processed. The VBAP gain value of each sample in time frame (n-1) is determined to change linearly from the first sample to the last sample.

[0358] Furthermore, since the priority information of time frame (n) is less than the threshold Q, the VBAP gain value of the last sample of time frame (n) is set to 0.

[0359] That is, the VBAP gain value of the first sample of time frame (n) is set to be the same as the VBAP gain value of the last sample of time frame (n-1), and the VBAP gain value of the last sample of time frame (n) is set to 0. The VBAP gain value of each sample in time frame (n) is determined to change linearly from the first sample to the last sample.

[0360] Furthermore, since the priority information for time frame (n+1) is less than threshold Q, the VBAP gain value of the last sample in time frame (n+1) is set to 0, and as a result, the VBAP gain values ​​of all samples in time frame (n+1) are 0.

[0361] In this way, by setting the value of the VBAP gain of the sample at the end of the time frame in which the priority information is less than the threshold value Q to 0, fade-out processing equivalent to the example in FIG. 23 is possible.

[0362] In contrast, the example shown in Fig. 28 is an example in which the change in priority information is the same as that shown in Fig. 24. In this example, if the threshold Q=4, the priority information is less than threshold Q from time frame (n-1) to time frame (n+1), but the priority information of time frame (n+2) is equal to or greater than threshold Q.

[0363] In such a case, the VBAP gain from time frame (n-1) to time frame (n+2) is set to, for example, the gain indicated by broken line GN61.

[0364] In this example, since both the priority information of time frame (n) and the priority information of time frame (n+1) are less than the threshold Q, the VBAP gain of all samples in time frame (n+1) is 0.

[0365] Furthermore, since the priority information for time frame (n+2) is greater than or equal to threshold value Q, the VBAP gain for each sample of the object being processed is determined based on the VBAP gain of the channel corresponding to speaker s calculated by normal VBAP.

[0366] That is, the VBAP gain value of the first sample of time frame (n+2) is set to 0, which is the VBAP gain value of the last sample of time frame (n+1), and the VBAP gain value of the last sample of time frame (n+2) is set to the VBAP gain value calculated by the normal VBAP for time frame (n+2).The VBAP gain values ​​of each sample in time frame (n+2) are determined to change linearly from the first sample to the last sample.

[0367] In this way, by setting the value of the VBAP gain of the sample at the end of the time frame in which the priority information is less than the threshold value Q to 0, fade-in processing equivalent to the example in FIG. 24 is possible.

[0368] <Example of the unpacking / decoding unit configuration> When gain adjustment is performed by the fade-in process or fade-out process described with reference to Fig. 27 and Fig. 28, the unpacking / decoding unit 161 is configured as shown in Fig. 29, for example. Note that in Fig. 29, the same reference numerals are used to designate parts corresponding to those in Fig. 25, and descriptions thereof will be omitted as appropriate.

[0369] The unpacking / decoding unit 161 shown in FIG. 29 is made up of a priority information acquisition unit 191, a channel audio signal acquisition unit 192, a channel audio signal decoding unit 193, an output selection unit 194, a zero value output unit 195, an IMDCT unit 196, an overlap addition unit 271, an SBR processing unit 273, a gain adjustment unit 272, an object audio signal acquisition unit 197, an object audio signal decoding unit 198, an output selection unit 199, a zero value output unit 200, an IMDCT unit 201, an overlap addition unit 274, and an SBR processing unit 276.

[0370] The configuration of the unpacking / decoding unit 161 shown in FIG. 29 differs from the configuration of the unpacking / decoding unit 161 shown in FIG. 25 in that a gain adjustment unit 275 is not provided, but has the same configuration in other respects.

[0371] In the unpacking / decoding unit 161 shown in FIG. 29, the SBR processing unit 276 performs SBR on the audio signal supplied from the overlap addition unit 274 based on the high-frequency power value supplied from the priority information acquisition unit 191, and supplies the resulting audio signal to the rendering unit 162.

[0372] Furthermore, the priority information acquisition unit 191 acquires metadata and priority information of each object from the supplied bitstream, and supplies the metadata and priority information to the rendering unit 162. The priority information of each object is also supplied to an output selection unit 199.

[0373] Explanation of the Decryption Process Next, the operation of the decoding device 151 in the case where the unpacking / decoding unit 161 has the configuration shown in FIG. 29 will be described.

[0374] In this case, the decoding device 151 performs the decoding process shown in Fig. 30. Hereinafter, the decoding process performed by the decoding device 151 will be described with reference to the flowchart in Fig. 30. However, in step S281, the same process as that in step S51 in Fig. 11 is performed, and therefore the description thereof will be omitted.

[0375] In step S282, the unpacking / decoding unit 161 performs a selective decoding process.

[0376] Now, with reference to the flowchart in FIG. 31, the selective decoding process corresponding to the process in step S282 in FIG. 30 will be described.

[0377] Note that the processing from step S311 to step S328 is similar to the processing from step S231 to step S248 in Fig. 26, and therefore description thereof will be omitted. However, in step S312, the priority information acquisition unit 191 also supplies the priority information acquired from the bitstream to the rendering unit 162.

[0378] In step S329, the object audio signal obtainer 197 adds 1 to the object number, and the process returns to step S323. If it is determined in step S323 that the object number is not less than N, the selection decoding process ends, and then the process proceeds to step S283 in FIG.

[0379] Therefore, in the selective decoding process shown in FIG. 31, gain adjustment is performed on the audio signals of each channel using the fading signal gain as in the fifth embodiment, but gain adjustment is not performed on each object, and the audio signals obtained by SBR are output as is to the rendering unit 162.

[0380] Returning to the explanation of the decoding process in Figure 30, in step S283, the rendering unit 162 renders the audio signal of each object based on the audio signal of each object supplied from the SBR processing unit 276, the position information of each object as metadata supplied from the priority information acquisition unit 191, and the priority information of each object in the current time frame.

[0381] For example, as described with reference to Fig. 27 and Fig. 28, the rendering unit 162 calculates the VBAP gain of each sample of the current time frame for each object based on the priority information of the current time frame for each channel and the VBAP gain of the last sample of the time frame immediately preceding the current time frame. At this time, the rendering unit 162 calculates the VBAP gain by VBAP based on the position information as appropriate.

[0382] Then, the rendering unit 162 generates an audio signal for each channel based on the VBAP gain for each sample of each channel calculated for each object and the audio signal of each object, and supplies the generated audio signal to the mixing unit 163.

[0383] Although an example has been described here in which the VBAP gain of each sample is calculated so that the VBAP gain of each sample in a time frame changes linearly, the VBAP gain may change nonlinearly. Also, although an example has been described in which an audio signal of each channel is generated by VBAP, even when an audio signal of each channel is generated by another method, it is possible to adjust the gain of the audio signal of each object by the same process as in VBAP.

[0384] After the audio signals of each channel are generated, the process proceeds to step S284 where the decoding process ends. However, since the process of step S284 is similar to the process of step S54 in FIG. 11, a description thereof will be omitted.

[0385] In this way, the decoding device 151 calculates the VBAP gain for each sample of each object based on the priority information, and adjusts the gain of the audio signal of the object using the VBAP gain when generating the audio signal of each channel. This makes it possible to suppress the occurrence of glitch noise with a smaller amount of processing and to suppress deterioration of the sound quality as perceived by the ear.

[0386] In the fourth to sixth embodiments, it has been described that priority information of the time frames immediately before and after the current time frame is used to select an output destination of MDCT coefficients and to adjust gain using fading signal gain, etc. However, the present invention is not limited to this, and priority information of the current time frame, priority information of a time frame a predetermined number of frames before the current time frame, or priority information of a time frame a predetermined number of frames after the current time frame may be used.

[0387] The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose computer, for example, capable of executing various functions by installing various programs.

[0388] FIG. 32 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0389] In the computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are interconnected via a bus 504.

[0390] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0391] The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, etc. The output unit 507 includes a display, a speaker, etc. The recording unit 508 includes a hard disk, a non-volatile memory, etc. The communication unit 509 includes a network interface, etc. The drive 510 drives removable media 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0392] In the computer configured as above, the CPU 501 loads, for example, a program recorded in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504, and executes the program, thereby performing the series of processes described above.

[0393] The program executed by the computer (CPU 501) can be provided by being recorded on a removable medium 511 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0394] In the computer, the program can be installed in the recording unit 508 via the input / output interface 505 by mounting the removable medium 511 in the drive 510. The program can also be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. Alternatively, the program can be installed in the ROM 502 or the recording unit 508 in advance.

[0395] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0396] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0397] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0398] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0399] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0400] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0401] Furthermore, the present technology can also be configured as follows.

[0402] (1) an acquisition unit for acquiring encoded audio signals of multiple channels or multiple objects and priority information of each of said audio signals at a given time; an audio signal decoding unit that decodes the encoded audio signals of a predetermined number of channels or objects according to the priority information based on the priority information; A decoding device comprising: (2) The audio signal decoding unit decodes the encoded audio signal, the priority of which is indicated by the priority information and is equal to or higher than a predetermined level. A decoding device according to (1). (3) The acquisition unit changes the predetermined degree based on the priority information of the audio signals of the plurality of channels or the plurality of objects at the predetermined time. A decoding device according to (2). (4) the acquisition unit acquires a plurality of pieces of priority information for each of the audio signals; The audio signal decoding unit decodes the encoded audio signal based on one piece of priority information selected from the plurality of pieces of priority information. A decoding device according to any one of (1) to (3). (5) The plurality of pieces of priority information are generated for each computational capability in accordance with a computational capability of a decoding side of the encoded audio signal. A decoding device according to (4). (6) The audio signal processing device further includes a priority information generating unit that generates the priority information based on the encoded audio signal. A decoding device according to any one of (1) to (5). (7) The priority information generating unit generates the priority information based on a sound pressure or a spectral shape of an audio signal obtained from the encoded audio signal. A decoding device according to (6). (8) The audio signal decoding unit selects whether to decode the encoded audio signal at the predetermined time based on the priority information at the predetermined time and the priority information at a time before or after the predetermined time for each channel or each object. A decoding device according to (1). (9) an adder that generates an audio signal for the predetermined time by adding, for each channel or each object, the output signal for the predetermined time and the output signal for a time before or after the predetermined time, while setting a signal obtained by the decoding as an output signal when the decoding is performed, and setting 0 data as an output signal when the decoding is not performed; a gain adjustment unit that performs gain adjustment of an audio signal at the predetermined time based on the priority information at the predetermined time and the priority information at a time before or after the predetermined time, for each channel or each object; The decoding device according to (1) further comprises: (10) The audio signal processing device further includes a high-frequency generating unit that performs gain adjustment of a high-frequency power value for each channel or object based on the priority information for the predetermined time and the priority information for a time before or after the predetermined time, and generates a high-frequency component of the audio signal for the predetermined time based on the gain-adjusted power value and the audio signal for the predetermined time. A decoding device according to (9). (11) A high-frequency generation unit is further provided for generating an audio signal for the predetermined time period including a high-frequency component based on a power value of a high frequency band and the audio signal for the predetermined time period for each channel or each object, The gain adjustment unit adjusts the gain of the audio signal for the predetermined time period, the audio signal including a high-frequency component. A decoding device according to (9). (12) a rendering unit that generates an audio signal for each of a plurality of channels by allocating a predetermined gain value to the audio signal of the object based on the priority information at the predetermined time. A decoding device according to (1). (13) Obtaining encoded audio signals of a plurality of channels or objects and priority information for each of said audio signals at a given time; Decoding the encoded audio signals for a predetermined number of channels or objects according to the priority information. A decryption method comprising the steps: (14) Obtaining encoded audio signals of a plurality of channels or objects and priority information for each of said audio signals at a given time; Decoding the encoded audio signal for a predetermined number of channels or objects according to the priority information. A program that causes a computer to execute a process including steps. (15) a priority information generating unit for generating priority information of audio signals of a plurality of channels or a plurality of objects at a predetermined time; a packing unit for storing the priority information in a bitstream; An encoding device comprising: (16) The priority information generating unit generates a plurality of pieces of priority information for each of the audio signals. An encoding device according to (15). (17) The priority information generating unit generates the priority information for each computational capability in accordance with a computational capability of a decoding side of the encoded audio signal. An encoding device according to (16). (18) The priority information generating unit generates the priority information based on a sound pressure or a spectral shape of the audio signal. An encoding device according to any one of (15) to (17). (19) An encoding unit that encodes the audio signals of the plurality of channels or the plurality of objects, The packing unit stores the priority information and the encoded audio signal in the bitstream. An encoding device according to any one of (15) to (18). (20) generating priority information for a multi-channel or multi-object audio signal at a given time; The priority information is stored in a bitstream. 2. An encoding method comprising the steps of: (twenty one) generating priority information for a multi-channel or multi-object audio signal at a given time; The priority information is stored in a bitstream. A program that causes a computer to execute a process including steps. [Explanation of symbols]

[0403] 11 encoding device, 21 channel audio encoding unit, 22 object audio encoding unit, 23 metadata input unit, 24 packing unit, 51 encoding unit, 52 priority information generation unit, 61 MDCT unit, 91 encoding unit, 92 priority information generation unit, 101 MDCT unit, 151 decoding device, 161 unpacking / decoding unit, 162 rendering unit, 163 mixing unit, 191 priority information acquisition unit, 193 channel audio signal decoding unit, 194 output selection unit, 196 IMDCT unit, 198 object audio signal decoding unit, 199 output selection unit, 201 IMDCT unit, 231 priority information generation unit, 232 priority information generation unit, 271 overlap addition unit, 272 gain adjustment unit, 273 SBR processing unit, 274 Overlap processing section, 275 Gain adjustment section, 276 SBR processing section

Claims

1. an acquisition unit for acquiring a bitstream including an encoded channel audio signal or an encoded object audio signal, and priority information relating to said encoded audio signal; a decoding unit for decoding the channel audio signal or the object audio signal; A rendering unit that calculates the gain of the decoded audio signal using VBAP; An information processing device comprising:

2. The object audio signal comprises metadata indicating spatial position information of the object. The information processing device according to claim 1 .

3. The spatial position information includes a horizontal angle of the audio object from a predetermined reference position, a vertical angle of the audio object from the predetermined reference position, and a distance of the audio object from the predetermined reference position. The information processing device according to claim 2 .

4. The encoded audio signal comprises metadata indicating gain values ​​of audio objects. The information processing device according to claim 1 .

5. The object audio signal comprises metadata indicating the number of audio objects. The information processing device according to claim 1 .

6. The value of the priority information is set to a value from 0 to 7, with 0 being the minimum priority and 7 being the maximum priority. The information processing device according to claim 1 .

7. the channel audio signal or the object audio signal for which the value indicating the priority information is equal to or greater than a predetermined value is decoded, and the channel audio signal or the object audio signal for which the value indicating the priority information is less than the predetermined value is not decoded. The information processing device according to claim 1 .

8. The predetermined value is 7. The information processing device according to claim 6.

9. The decoding unit decodes the channel audio signal or the object audio signal using an IMDCT process. The information processing device according to claim 1 .

10. and an adder that generates an audio signal for a predetermined time by adding, for each channel or for each audio object, the output signal obtained by the decoding as an output signal when the decoding is performed, and 0 data as an output signal when the decoding is not performed, and adding the output signal for a predetermined time to the output signal for a time before or after the predetermined time. The information processing device according to claim 1 .

11. The audio signal processing device further includes a gain adjustment unit that adjusts a gain of an audio signal at a predetermined time based on the priority information at the predetermined time and the priority information at a time before or after the predetermined time for each channel or audio object. The information processing device according to claim 1 .

12. An information processing device, obtaining a bitstream containing an encoded channel audio signal or an encoded object audio signal and priority information relating to said encoded audio signal; decoding said channel audio signal or said object audio signal; Calculating the gain of the decoded audio signal using VBAP; An information processing method comprising:

13. obtaining a bitstream containing an encoded channel audio signal or an encoded object audio signal and priority information relating to said encoded audio signal; decoding said channel audio signal or said object audio signal; Calculating the gain of the decoded audio signal using VBAP; A program that causes a computer to execute a process including the steps of:

Images