Information processing device, information processing method and program

Abstract

To reduce a processing load of a reception side when transmitting a plurality of types of audio data.SOLUTION: A container of a prescribed format having a prescribed number of audio streams including encoded data of a plurality of groups is transmitted. For example, either or both of channel encoded data and object encoded data are included in encoded data of the plurality of groups. Attribute information indicating respective attributes of encoded data of the plurality of groups is inserted into a layer of the container and / or a layer of the audio stream. For example, stream correspondence information indicating which audio stream contains which encoded data of the plurality of groups is further inserted.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a technique has been proposed as a stereoscopic (3D) audio technique in which encoded sample data is mapped to speakers located at arbitrary positions based on metadata and then rendered (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2014-520491 Summary of the Invention [Problem to be solved by the invention]

[0004] It is conceivable that object-coded data consisting of coded sample data and metadata can be transmitted together with channel-coded data such as 5.1 channels or 7.1 channels, thereby enabling audio reproduction with enhanced realism on the receiving side.

[0005] An object of the present technology is to reduce the processing load on the receiving side when multiple types of audio data are transmitted. [Means for solving the problem]

[0006] The concept of this technology is as follows: a transmitter for transmitting a container having a predetermined format and a predetermined number of audio streams including encoded data of a plurality of groups; an information inserting unit that inserts attribute information indicating attributes of each of the plurality of groups of encoded data into the container layer and / or the audio stream layer; Located in the transmitting device.

[0007] In the present technology, a transmitting unit transmits a container in a predetermined format having a predetermined number of audio streams including a plurality of groups of encoded data, for example, the plurality of groups of encoded data may include either or both of channel encoded data and object encoded data.

[0008] The information inserting unit inserts attribute information indicating attributes of each of the multiple groups of encoded data into the container layer and / or the audio stream layer. For example, the container may be a transport stream (MPEG-2 TS) adopted in the digital broadcasting standard. Also, for example, the container may be MP4 used for Internet distribution or a container of another format.

[0009] In this manner, in the present technology, attribute information indicating the attributes of each of the multiple groups of encoded data included in a predetermined number of audio streams is inserted into the container layer and / or the audio stream layer, so that the receiving side can easily recognize the attributes of each of the multiple groups of encoded data before decoding the encoded data, and can selectively decode and use only the encoded data of the necessary groups, thereby reducing the processing load.

[0010] In addition, in the present technology, for example, the information inserting unit may further insert stream correspondence relationship information indicating which audio stream contains each of the encoded data of the multiple groups into the container layer and / or the audio stream layer, which allows the receiving side to easily recognize the audio stream containing the encoded data of the required group, thereby reducing the processing load.

[0011] In this case, for example, the container may be MPEG2-TS, and when the information insertion unit inserts the attribute information and stream correspondence information into the container, the information insertion unit may be configured to insert the attribute information and stream correspondence information into an audio elementary stream loop corresponding to at least one of a predetermined number of audio streams present under the program map table.

[0012] In this case, for example, when the information insertion unit inserts attribute information and stream correspondence information into a layer of an audio stream, the information insertion unit may be configured to insert this attribute information and stream correspondence information into a PES payload of a PES packet of at least one or more audio streams among a predetermined number of audio streams.

[0013] For example, the stream correspondence information may be information indicating a correspondence between a group identifier that identifies each of the multiple groups of encoded data and a stream identifier that identifies each of the predetermined number of audio streams. In this case, for example, the information insertion unit may further insert stream identifier information indicating each of the stream identifiers of the predetermined number of audio streams into the container layer and / or the audio stream layer.

[0014] For example, the container may be an MPEG2-TS, and when inserting stream identifier information into the container, the information inserting unit may insert the stream identifier information into an audio elementary stream loop corresponding to each of a predetermined number of audio streams present under the program map table. Also, for example, when inserting stream identifier information into an audio stream, the information inserting unit may insert the stream identifier information into a PES payload of a PES packet of each of the predetermined number of audio streams.

[0015] Also, for example, the stream correspondence information may be information indicating a correspondence relationship between a group identifier identifying each of the multiple groups of encoded data and a packet identifier assigned when each of the predetermined number of audio streams is packetized. Also, for example, the stream correspondence information may be information indicating a correspondence relationship between a group identifier identifying each of the multiple groups of encoded data and type information indicating the stream type of each of the predetermined number of audio streams.

[0016] Another concept of the present technology is a receiving unit for receiving a container in a predetermined format having a predetermined number of audio streams including encoded data of a plurality of groups; attribute information indicating attributes of each of the plurality of groups of encoded data is inserted into a layer of the container and / or a layer of the audio stream; The audio stream processing device further includes a processing unit that processes the predetermined number of audio streams contained in the received container based on the attribute information. in the receiving device.

[0017] In the present technology, a receiving unit receives a container in a predetermined format having a predetermined number of audio streams including encoded data of a plurality of groups. For example, the encoded data of the plurality of groups may include either or both of channel encoded data and object encoded data. Attribute information indicating attributes of each of the encoded data of the plurality of groups is inserted in a layer of the container and / or a layer of the audio stream. A processing unit processes the predetermined number of audio streams included in the received container based on the attribute information.

[0018] In this way, in the present technology, a predetermined number of audio streams contained in a received container are processed based on attribute information indicating the attributes of each of the multiple groups of encoded data inserted in the container layer and / or the audio stream layer. Therefore, it is possible to selectively decode and use only the encoded data of the necessary groups, thereby reducing the processing load.

[0019] In the present technology, for example, stream correspondence information indicating which audio stream contains each of the encoded data of the multiple groups may be further inserted into the container layer and / or the audio stream layer, and the processing unit may process a predetermined number of audio streams based on the stream correspondence information in addition to the attribute information. In this case, it is possible to easily recognize the audio stream containing the encoded data of the required group, thereby reducing the processing load.

[0020] In addition, in the present technology, for example, the processing unit may be configured to selectively perform a decoding process on an audio stream including encoded data of a group having attributes that match the speaker configuration and user selection information, based on the attribute information and the stream correspondence information.

[0021] Further, another concept of the present technology is a receiving unit for receiving a container in a predetermined format having a predetermined number of audio streams including encoded data of a plurality of groups; attribute information indicating attributes of each of the plurality of groups of encoded data is inserted into a layer of the container and / or a layer of the audio stream; a processing unit that selectively acquires a predetermined group of encoded data from the predetermined number of audio streams contained in the received container based on the attribute information, and reconstructs an audio stream including the encoded data of the predetermined group; The audio signal processing device further includes a stream transmitting unit that transmits the audio stream reconstructed by the processing unit to an external device. in the receiving device.

[0022] In the present technology, a receiving unit receives a container in a predetermined format having a predetermined number of audio streams including encoded data of a plurality of groups. Attribute information indicating attributes of each of the encoded data of the plurality of groups is inserted into a layer of the container and / or a layer of the audio stream. A processing unit selectively acquires encoded data of the predetermined group from the predetermined number of audio streams based on the attribute information, and reconstructs an audio stream including the encoded data of the predetermined group. Then, a stream transmitting unit transmits the reconstructed audio stream to an external device.

[0023] In this way, in the present technology, a predetermined group of encoded data is selectively obtained from a predetermined number of audio streams based on attribute information indicating the attributes of each of the multiple groups of encoded data inserted in the container layer and / or the audio stream layer, and an audio stream to be transmitted to an external device is reconstructed. It is possible to easily obtain the encoded data of a required group, thereby reducing the processing load.

[0024] In the present technology, for example, stream correspondence information indicating which audio streams each contain encoded data of a plurality of groups may be further inserted into the container layer and / or the audio stream layer, and the processing unit may selectively acquire encoded data of a predetermined group from a predetermined number of audio streams based on the stream correspondence information in addition to the attribute information. In this case, audio streams containing encoded data of a predetermined group can be easily recognized, and the processing load can be reduced. Effect of the Invention

[0025] According to the present technology, it is possible to reduce the processing load on the receiving side when transmitting multiple types of audio data. Note that the effects described in this specification are merely examples and are not limiting, and additional effects may also be provided. [Brief description of the drawings]

[0026] [Figure 1] 1 is a block diagram showing an example of the configuration of a transmission / reception system according to an embodiment; [Diagram 2] FIG. 1 is a diagram showing the structure of an audio frame (1024 samples) in 3D audio transmission data. [Diagram 3] FIG. 11 is a diagram showing an example of the configuration of 3D audio transmission data. [Figure 4] 1A to 1C are diagrams illustrating schematic examples of audio frame configurations when 3D audio transmission data is transmitted in one stream and when it is transmitted in multiple streams. [Diagram 5] FIG. 11 is a diagram showing an example of group division when transmitting in two streams in an example configuration of 3D audio transmission data. [Figure 6] FIG. 13 is a diagram showing the correspondence between groups and streams in an example of group division (division into two). [Figure 7] FIG. 11 is a diagram showing an example of group division when transmitting in two streams in an example configuration of 3D audio transmission data. [Figure 8] FIG. 13 is a diagram showing the correspondence between groups and streams in an example of group division (division into two). [Figure 9] 4 is a block diagram showing an example of the configuration of a stream generating unit included in the service transmitter. FIG. [Figure 10] FIG. 13 is a diagram showing an example of the structure of a 3D audio stream config descriptor. [Figure 11] The following shows the main information contained in a sample 3D audio stream config descriptor structure: [Figure 12] FIG. 13 is a diagram showing the types of content defined in “contentKind.” [Figure 13] FIG. 13 is a diagram showing an example of the structure of a 3D audio stream ID descriptor and the contents of main information in the example structure. [Figure 14] FIG. 2 is a diagram illustrating an example of the configuration of a transport stream. [Figure 15] 1 is a block diagram showing a configuration example of a service receiver; [Figure 16] FIG. 2 illustrates an example of a received audio stream. [Figure 17] FIG. 2 shows a schematic diagram of a decoding process when no descriptor information is present in the audio stream. [Figure 18] FIG. 11 is a diagram showing an example of the structure of an audio access unit (audio frame) of an audio stream when descriptor information is not present in the audio stream. [Figure 19] FIG. 2 shows a schematic diagram of a decoding process when descriptor information is present in the audio stream. [Figure 20] FIG. 11 is a diagram showing an example of the structure of an audio access unit (audio frame) of an audio stream when descriptor information exists in the audio stream. [Figure 21] FIG. 11 is a diagram showing another example structure of an audio access unit (audio frame) of an audio stream when descriptor information exists in the audio stream. [Figure 22] 13 is a flowchart (1 / 2) showing an example of an audio decode control process by a CPU in a service receiver. [Figure 23] 13 is a flowchart (2 / 2) showing an example of an audio decode control process by a CPU in the service receiver. [Figure 24] FIG. 13 is a block diagram showing another example of the configuration of the service receiver. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, modes for carrying out the invention (hereinafter, referred to as "embodiments") will be described. The description will be made in the following order. 1. Embodiment 2. Variations

[0028] <1. Preferred embodiment> [Example of transmission and reception system configuration] 1 shows an example of the configuration of a transmission / reception system 10 according to an embodiment. This transmission / reception system 10 is composed of a service transmitter 100 and a service receiver 200. The service transmitter 100 transmits a transport stream TS via broadcast waves or network packets. This transport stream TS has a video stream and a predetermined number of audio streams, i.e., one or multiple audio streams, each of which includes multiple groups of encoded data.

[0029] Fig. 2 shows the structure of an audio frame (1024 samples) in the 3D audio transmission data handled in this embodiment. This audio frame is made up of multiple MPEG Audio Stream Packets. Each MPEG Audio Stream Packet is made up of a header and a payload.

[0030] The header has information such as packet type, packet label, and packet length. The payload contains information defined by the packet type in the header. This payload information contains "SYNC", which is the synchronization start code, "Frame", which is the actual data for the 3D audio transmission, and "Config", which indicates the configuration of this "Frame".

[0031] A "Frame" includes channel-encoded data and object-encoded data that constitute the transmission data of 3D audio. Here, the channel-encoded data is composed of encoded sample data such as SCE (Single Channel Element), CPE (Channel Pair Element), and LFE (Low Frequency Element). The object-encoded data is composed of encoded sample data of SCE (Single Channel Element) and metadata for mapping it to a speaker located at an arbitrary position and rendering it. This metadata is included as an extension element (Ext_element).

[0032] Figure 3 shows an example of the structure of 3D audio transmission data. This example consists of one channel encoded data and two object encoded data. The one channel encoded data is 5.1 channel encoded data (CD), and consists of encoded sample data for SCE1, CPE1.1, CPE1.2, and LFE1.

[0033] The two object coded data are coded data of an immersive audio object (IAO) and a speech dialog object (SDO). The immersive audio object coded data is object coded data for immersive sound, and is composed of coded sample data SCE2 and metadata EXE_El (Object metadata)2 for mapping the coded sample data to a speaker located at an arbitrary position and rendering it.

[0034] The speech dialogue object coded data is object coded data for a speech language. In this example, there are speech dialogue object coded data corresponding to each of a first and a second language. The speech dialogue object coded data corresponding to the first language is made up of coded sample data SCE3 and metadata EXE_El (Object metadata)3 for mapping the coded sample data SCE3 to a speaker located at an arbitrary position for rendering. The speech dialogue object coded data corresponding to the second language is made up of coded sample data SCE4 and metadata EXE_El (Object metadata)4 for mapping the coded sample data SCE4 to a speaker located at an arbitrary position for rendering.

[0035] The coded data are classified into groups according to their types. In the illustrated example, the coded channel data of 5.1 channels is group 1, the coded immersive audio object data is group 2, the coded speech dialogue object data of a first language is group 3, and the coded speech dialogue object data of a second language is group 4.

[0036] Moreover, on the receiving side, those that can be selected among the groups are registered in a switch group (SW Group) and encoded. Furthermore, groups are bundled together to form a preset group (preset group) to enable playback according to the use case. In the illustrated example, groups 1, 2, and 3 are bundled together to form preset group 1, and groups 1, 2, and 4 are bundled together to form preset group 2.

[0037] Returning to FIG. 1, the service transmitter 100 transmits 3D audio transmission data including multiple groups of encoded data as described above in one stream or multiple streams.

[0038] Fig. 4(a) shows an outline of an example of the configuration of an audio frame when transmitted in one stream (main stream) in the example of the configuration of the 3D audio transmission data in Fig. 3. In this case, this one stream includes channel coded data (CD), immersive audio object coded data (IAO), and speech dialogue object coded data (SDO) together with "SYNC" and "Config".

[0039] Fig. 4(b) shows an example of the configuration of an audio frame when multiple streams (two streams in this case) are transmitted in the example of the configuration of the 3D audio transmission data in Fig. 3. In this case, the main stream includes channel encoded data (CD) and immersive audio object encoded data (IAO) together with "SYNC" and "Config". Also, the substream includes speech dialogue object encoded data (SDO) together with "SYNC" and "Config".

[0040] Fig. 5 shows an example of group division in the example of the configuration of 3D audio transmission data in Fig. 3 when transmitting in two streams. In this case, the main stream includes channel coded data (CD) classified as group 1 and immersive audio object coded data (IAO) classified as group 2. The substream includes speech dialogue object coded data (SDO) in a first language classified as group 3 and speech dialogue object coded data (SDO) in a second language classified as group 4.

[0041] FIG. 6 shows the correspondence between groups and streams in the group division example (two division) of FIG. 5. Here, group ID is an identifier for identifying a group. attribute indicates the attribute of the encoded data of each group. switch group ID is an identifier for identifying a switching group. preset group ID is an identifier for identifying a preset group. stream ID (sub stream ID) is an identifier for identifying a sub stream. kind indicates the type of content of each group.

[0042] The illustrated correspondence relationship indicates that the coded data belonging to group 1 is channel coded data, does not constitute a switch group, and is included in stream 1. The illustrated correspondence relationship indicates that the coded data belonging to group 2 is object coded data for immersive sound (immersive audio object coded data), does not constitute a switch group, and is included in stream 1.

[0043] The illustrated correspondence relationship also indicates that the coded data belonging to group 3 is object coded data for a first speech language (speech dialogue object coded data), constitutes switch group 1, and is included in stream 2. The illustrated correspondence relationship also indicates that the coded data belonging to group 4 is object coded data for a second speech language (speech dialogue object coded data), constitutes switch group 1, and is included in stream 2.

[0044] The illustrated correspondence relationship also indicates that preset group 1 includes group 1, group 2, and group 3. The illustrated correspondence relationship also indicates that preset group 2 includes group 1, group 2, and group 4.

[0045] 7 shows an example of group division in a configuration example of 3D audio transmission data when transmitting in two streams. In this case, the main stream includes channel-encoded data (CD) classified as group 1 and immersive audio object-encoded data (IAO) classified as group 2.

[0046] This main stream also includes SAOC (Spatial Audio Object Coding) object-coded data classified as group 5, and HOA (Higher Order Ambisonics) object-coded data classified as group 6. SAOC object-coded data is data for achieving higher compression in object coding by utilizing the characteristics of object data. HOA object-coded data is a technology that treats 3D sound as the entire sound field, and is data that aims to reproduce the direction of sound from the direction of arrival of the sound at the microphone to the auditory position.

[0047] The substream includes coded speech dialogue object data (SDO) in a first language classified as group 3, and coded speech dialogue object data (SDO) in a second language classified as group 4. The substream also includes coded first audio description data classified as group 7, and coded second audio description data classified as group 8. The coded audio description data is data for providing audio commentary on content (mainly video), and is data for transmission separately from normal audio, mainly for visually impaired people.

[0048] Fig. 8 shows the correspondence between groups and substreams in the group division example (two division) of Fig. 7. The correspondence shown in the figure indicates that the coded data belonging to group 1 is channel coded data, does not constitute a switch group, and is included in stream 1. The correspondence shown in the figure also indicates that the coded data belonging to group 2 is object coded data for immersive sound (immersive audio object coded data), does not constitute a switch group, and is included in stream 1.

[0049] The illustrated correspondence relationship also indicates that the coded data belonging to group 3 is object coded data for a first speech language (speech dialogue object coded data), constitutes switch group 1, and is included in stream 2. The illustrated correspondence relationship also indicates that the coded data belonging to group 4 is object coded data for a second speech language (speech dialogue object coded data), constitutes switch group 1, and is included in stream 2.

[0050] The illustrated correspondence relationship also indicates that the coded data belonging to group 5 is SAOC object coded data, constitutes switch group 2, and is included in stream 1. The illustrated correspondence relationship also indicates that the coded data belonging to group 6 is HOA object coded data, constitutes switch group 2, and is included in stream 1.

[0051] The illustrated correspondence relationship also indicates that the coded data belonging to group 7 is object coded data of a first audio description, constitutes switch group 3, and is included in stream 2. The illustrated correspondence relationship also indicates that the coded data belonging to group 8 is object coded data of a second audio description, constitutes switch group 3, and is included in stream 2.

[0052] The illustrated correspondence relationship also indicates that preset group 1 includes group 1, group 2, group 3, and group 7. The illustrated correspondence relationship also indicates that preset group 2 includes group 1, group 2, group 4, and group 8.

[0053] Returning to Fig. 1, the service transmitter 100 inserts attribute information indicating attributes of each of the multiple groups of encoded data included in the 3D audio transmission data into the container layer. The service transmitter 100 also inserts stream correspondence information indicating which audio stream contains each of the multiple groups of encoded data into the container layer. In this embodiment, the stream correspondence information is, for example, information indicating the correspondence between a group ID and a stream identifier.

[0054] The service transmitter 100 inserts these attribute information and stream correspondence information, for example, as descriptors into an audio elementary stream loop corresponding to one or more audio streams among a predetermined number of audio streams present under a program map table (PMT).

[0055] The service transmitter 100 also inserts stream identifier information indicating the stream identifiers of each of the predetermined number of audio streams into the layer of the container. The service transmitter 100 inserts this stream identifier information as a descriptor into an audio elementary stream loop corresponding to each of the predetermined number of audio streams existing under a program map table (PMT).

[0056] The service transmitter 100 also inserts attribute information indicating attributes of each of the multiple groups of encoded data included in the 3D audio transmission data into the audio stream layer. The service transmitter 100 also inserts stream correspondence information indicating which audio stream includes each of the multiple groups of encoded data into the audio stream layer. The service transmitter 100 inserts the attribute information and stream correspondence information into the PES payload of the PES packet of one or more audio streams among the predetermined number of audio streams, for example.

[0057] The service transmitter 100 also inserts stream identifier information indicating the stream identifiers of each of the predetermined number of audio streams into the layer of the audio streams, for example, into the PES payload of the PES packet of each of the predetermined number of audio streams.

[0058] The service transmitter 100 inserts information into these audio stream layers by inserting "Desc", that is, descriptor information, between "SYNC" and "Config" as shown in FIGS. 4(a) and 4(b).

[0059] In this embodiment, an example has been shown in which the various information (attribute information, stream correspondence information, stream identifier information) is inserted into both the container layer and the audio stream layer as described above, but it is also possible to insert the various information into only the container layer or only the audio stream layer.

[0060] The service receiver 200 receives the transport stream TS transmitted by broadcast waves or network packets from the service transmitter 100. As described above, in addition to the video stream, this transport stream TS has a predetermined number of audio streams including multiple groups of encoded data constituting the transmission data of 3D audio.

[0061] Then, attribute information indicating the attributes of each of the multiple groups of encoded data contained in the 3D audio transmission data is inserted into the container layer and / or the audio stream layer, and stream correspondence information indicating in which audio stream each of the multiple groups of encoded data is contained is inserted.

[0062] Based on the attribute information and the stream correspondence information, the service receiver 200 selectively performs decoding processing on an audio stream including encoded data of a group having attributes that match the speaker configuration and user selection information to obtain 3D audio output.

[0063] [Stream generation part of the service transmitter] 9 shows an example of the configuration of the stream generating unit 110 included in the service transmitter 100. This stream generating unit 110 has a video encoder 112, an audio encoder 113, and a multiplexer 114. In this example, the audio transmission data is made up of one coded channel data and two coded object data, as shown in FIG.

[0064] The video encoder 112 receives the video data SV and encodes the video data SV to generate a video stream (video elementary stream). The audio encoder 113 receives the immersive audio and speech dialogue object data as audio data SA together with channel data.

[0065] The audio encoder 113 performs encoding on the audio data SA to obtain 3D audio transmission data. As shown in Fig. 3, the 3D audio transmission data includes channel encoded data (CD), immersive audio object encoded data (IAO), and speech dialogue object encoded data (SDO).

[0066] The audio encoder 113 generates one or more audio streams (audio elementary streams) including encoded data of multiple groups, four groups in this case (see Figs. 4(a) and (b)). At this time, the audio encoder 113 inserts, between "SYNC" and "Config", descriptor information ("Desc") including attribute information, stream correspondence information, and further stream identifier information, as described above.

[0067] The multiplexer 114 packetizes the video stream output from the video encoder 112 and a predetermined number of audio streams output from the audio encoder 113 into PES packets, and then packetizes and multiplexes the packets to obtain a transport stream TS as a multiplexed stream.

[0068] The multiplexer 114 also inserts attribute information indicating the attributes of each of the multiple groups of encoded data and stream correspondence information indicating which audio stream contains each of the multiple groups of encoded data under the program map table (PMT). The multiplexer 114 inserts this information into an audio elementary stream loop corresponding to at least one of the predetermined number of audio streams by using a 3D audio stream config descriptor (3Daudio_stream_config_descriptor). Details of this descriptor will be described later.

[0069] The multiplexer 114 also inserts stream identifier information indicating the stream identifiers of each of the predetermined number of audio streams under the program map table (PMT). The multiplexer 114 inserts this information into the audio elementary stream loops corresponding to each of the predetermined number of audio streams by using a 3D audio stream ID descriptor (3Daudio_substreamID_descriptor). Details of this descriptor will be described later.

[0070] The operation of the stream generation unit 110 shown in Fig. 9 will be briefly described. Video data is supplied to a video encoder 112. In this video encoder 112, the video data SV is encoded to generate a video stream including the encoded video data. This video stream is supplied to a multiplexer 114.

[0071] The audio data SA is supplied to the audio encoder 113. This audio data SA includes channel data and object data of immersive audio and speech dialogue. In the audio encoder 113, the audio data SA is encoded to obtain transmission data of 3D audio.

[0072] The 3D audio transmission data includes immersive audio object coded data (IAO) and speech dialogue object coded data (SDO) in addition to the channel coded data (CD) (see FIG. 3). The audio encoder 113 generates one or more audio streams including the four groups of coded data (see FIGS. 4(a) and (b)).

[0073] At this time, the audio encoder 113 inserts, between "SYNC" and "Config", descriptor information ("Desc") including the above-mentioned attribute information, stream correspondence information, stream identifier information, and the like.

[0074] The video stream generated by the video encoder 112 is supplied to the multiplexer 114. Also, the audio stream generated by the audio encoder 113 is supplied to the multiplexer 114. In the multiplexer 114, the streams supplied from the respective encoders are PES packetized, and further transport packetized and multiplexed, thereby obtaining a transport stream TS as a multiplexed stream.

[0075] Furthermore, the multiplexer 114 inserts a 3D audio stream config descriptor into an audio elementary stream loop corresponding to at least one of the predetermined audio streams, for example. This descriptor includes attribute information indicating the attributes of each of the multiple groups of encoded data, and stream correspondence information indicating in which audio stream each of the multiple groups of encoded data is included.

[0076] Furthermore, in the descriptor 114, a 3D audio stream ID descriptor is inserted into the audio elementary stream loop corresponding to each of the predetermined number of audio streams. This descriptor includes stream identifier information indicating the stream identifier of each of the predetermined number of audio streams.

[0077] [3D Audio Stream Config Descriptor Details] Figure 10 shows a structure example (Syntax) of a 3D audio stream config descriptor (3Daudio_stream_config_descriptor), and Figure 11 shows the contents (Semantics) of the main information in the structure example.

[0078] The 8-bit field "descriptor_tag" indicates the descriptor type. In this case, it indicates that it is a 3D audio stream config descriptor. The 8-bit field "descriptor_length" indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.

[0079] The 8-bit field "NumOfGroups, N" indicates the number of groups. The 8-bit field "NumOfPresetGroups, P" indicates the number of preset groups. The 8-bit field "groupID", the 8-bit field "attribute_of_groupID", the 8-bit field "SwitchGroupID", and the 8-bit field "audio_streamID" are repeated for the number of groups.

[0080] The "groupID" field indicates the group identifier. The "attribute_of_groupID" field indicates the attributes of the encoded data of the group. The "SwitchGroupID" field is an identifier indicating which switch group the group belongs to. "0" indicates that the group does not belong to any switch group. A value other than "0" indicates the switch group to which the group belongs. The "contentKind" 8-bit field indicates the type of content of the group. "audio_streamID" is an identifier indicating the audio stream to which the group belongs. Figure 12 shows the types of content defined in "contentKind".

[0081] Furthermore, the 8-bit field of "presetGroupID" and the 8-bit field of "NumOfGroups_in_preset, R" are repeated for the number of preset groups. The "presetGroupID" field is an identifier indicating a bundle of preset groups. The "NumOfGroups_in_preset, R" field indicates the number of groups that belong to the preset group. For each preset group, the 8-bit field of "groupID" is repeated for the number of groups that belong to it, indicating the groups that belong to the preset group. This descriptor may be placed under an extended descriptor.

[0082] [3D Audio Stream ID Descriptor Details] Fig. 13(a) shows a structure example (Syntax) of a 3D audio stream ID descriptor (3Daudio_substreamID_descriptor), and Fig. 13(b) shows the contents (Semantics) of the main information in the structure example.

[0083] The 8-bit field "descriptor_tag" indicates the descriptor type. In this case, it indicates that it is a 3D audio stream ID descriptor. The 8-bit field "descriptor_length" indicates the length (size) of the descriptor, indicating the number of subsequent bytes as the length of the descriptor. The 8-bit field "audio_streamID" indicates the identifier of the audio stream. This descriptor may be placed under an extended descriptor.

[0084] [Transport Stream TS Structure] FIG. 14 shows an example of the configuration of a transport stream TS. This example corresponds to the case where 3D audio transmission data is transmitted in two streams (see FIG. 5). In this example, there is a PES packet "video PES" for a video stream identified by PID1. In addition, there are PES packets "audio PES" for two audio streams identified by PID2 and PID3, respectively. A PES packet consists of a PES header (PES_header) and a PES payload (PES_payload). DTS and PTS timestamps are inserted in the PES header. By properly attaching the timestamps, such as matching the timestamps of PID2 and PID3 during multiplexing, it is possible to ensure synchronization between the two throughout the system.

[0085] Here, the PES packet "audio PES" of the audio stream identified by PID2 includes channel coded data (CD) classified as group 1 and immersive audio object coded data (IAO) classified as group 2. Also, the PES packet "audio PES" of the audio stream identified by PID3 includes speech dialogue object coded data (SDO) of a first language classified as group 3 and speech dialogue object coded data (SDO) of a second language classified as group 4.

[0086] The transport stream TS also contains a PMT (Program Map Table) as PSI (Program Specific Information). The PSI is information that describes which program each elementary stream in the transport stream belongs to. The PMT contains a program loop that describes information related to the entire program.

[0087] The PMT also contains elementary stream loops that contain information related to each elementary stream. In this configuration example, there is a video elementary stream loop (video ES loop) corresponding to the video stream, and there is also an audio elementary stream loop (audio ES loop) corresponding to the two audio streams.

[0088] In the video elementary stream loop (video ES loop), information such as stream type and PID (packet identifier) ​​corresponding to the video stream is placed, and a descriptor describing information related to the video stream is also placed. The value of "Stream_type" of this video stream is set to "0x24", and the PID information indicates PID1 assigned to the PES packet "video PES" of the video stream as described above. As one of the descriptors, an HEVC descriptor is placed.

[0089] In each audio elementary stream loop (audio ES loop), information such as stream type and PID (packet identifier) ​​corresponding to the audio stream is placed, and a descriptor describing information related to the audio stream is also placed. PID2 is the main audio stream, the value of "Stream_type" is set to "0x2C", and the PID information indicates the PID assigned to the PES packet "audio PES" of the audio stream as described above. PID3 is the sub audio stream, the value of "Stream_type" is set to "0x2D", and the PID information indicates the PID assigned to the PES packet "audio PES" of the audio stream as described above.

[0090] Furthermore, both the above-mentioned 3D audio stream config descriptor and 3D audio stream ID descriptor are placed in each audio elementary stream loop (audio ES loop).

[0091] In addition, descriptor information is inserted into the PES payload of the PES packet of each audio elementary stream. As mentioned above, this descriptor information is "Desc" inserted between "SYNC" and "Config" (see Figure 4). If the information contained in the 3D audio stream config descriptor is D1 and the information contained in the 3D audio stream ID descriptor is D2, then this descriptor information has "D1+D2" information.

[0092] [Example of service receiver configuration] 15 shows an example of the configuration of a service receiver 200. The service receiver 200 includes a receiving unit 201, a demultiplexer 202, a video decoder 203, a video processing circuit 204, a panel driving circuit 205, and a display panel 206. The service receiver 200 also includes multiplexing buffers 211-1 to 211-N, a combiner 212, a 3D audio decoder 213, an audio output processing circuit 214, and a speaker system 215. The service receiver 200 also includes a CPU 221, a flash ROM 222, a DRAM 223, an internal bus 224, a remote control receiving unit 225, and a remote control transmitter 226.

[0093] The CPU 221 controls the operation of each part of the service receiver 200. The flash ROM 222 stores control software and data. The DRAM 223 constitutes a work area for the CPU 221. The CPU 221 loads software and data read from the flash ROM 222 onto the DRAM 223, starts up the software, and controls each part of the service receiver 200.

[0094] The remote control receiving unit 225 receives a remote control signal (remote control code) transmitted from the remote control transmitter 226, and supplies it to the CPU 221. The CPU 221 controls each unit of the service receiver 200 based on the remote control code. The CPU 221, the flash ROM 222, and the DRAM 223 are connected to an internal bus 224.

[0095] The receiving unit 201 receives a transport stream TS transmitted by broadcast waves or network packets from the service transmitter 100. In addition to a video stream, this transport stream TS has a predetermined number of audio streams including multiple groups of encoded data constituting 3D audio transmission data.

[0096] Figure 16 shows an example of a received audio stream. Figure 16(a) shows an example of one stream (main stream). This stream contains channel coding data (CD), immersive audio object coding data (IAO), speech dialogue object coding data (SDO) along with "SYNC" and "Config". This stream is identified by PID2.

[0097] In addition, between "SYNC" and "Config", descriptor information ("Desc") is included. This descriptor information includes attribute information indicating the attributes of each of the multiple groups of encoded data, stream correspondence information indicating which audio stream each of the multiple groups of encoded data is included in, and stream identifier information indicating its own stream identifier.

[0098] Figure 16(b) shows an example of two streams. The main stream identified by PID2 contains channel coding data (CD) and immersive audio object coding data (IAO) together with "SYNC" and "Config". The substream identified by PID3 contains speech dialogue object coding data (SDO) together with "SYNC" and "Config".

[0099] In addition, in each stream, descriptor information ("Desc") is included between "SYNC" and "Config". This descriptor information includes attribute information indicating the attributes of each of the multiple groups of encoded data, stream correspondence information indicating which audio stream each of the multiple groups of encoded data is included in, and stream identifier information indicating its own stream identifier.

[0100] The demultiplexer 202 extracts video stream packets from the transport stream TS and sends them to the video decoder 203. The video decoder 203 reconstructs the video stream from the video packets extracted by the demultiplexer 202, and performs a decoding process to obtain uncompressed video data.

[0101] The video processing circuit 204 performs scaling processing, image quality adjustment processing, and the like on the video data obtained by the video decoder 203 to obtain video data for display. The panel driving circuit 205 drives the display panel 206 based on the display image data obtained by the video processing circuit 204. The display panel 206 is composed of, for example, an LCD (Liquid Crystal Display), an organic EL display (organic electroluminescence display), or the like.

[0102] Furthermore, the demultiplexer 202 extracts various types of information such as descriptor information from the transport stream TS, and sends it to the CPU 221. This various information includes the above-mentioned 3D audio stream config descriptor (3Daudio_stream_config_descriptor) and 3D audio stream ID descriptor (3Daudio_substreamID_descriptor) (see FIG. 14).

[0103] Based on the attribute information contained in these descriptor information indicating the attributes of the encoded data of each group, and the stream relationship information indicating which audio stream each group is included in, the CPU 221 recognizes the audio stream containing the encoded data of a group having attributes that match the speaker configuration and viewer (user) selection information.

[0104] In addition, under the control of the CPU 221, the demultiplexer 202 selectively extracts packets of one or more audio streams using a PID filter, from a predetermined number of audio streams contained in the transport stream TS, that contain encoded data of a group having attributes that match the speaker configuration and viewer (user) selection information.

[0105] The multiplexing buffers 211-1 to 211-N each take in one of the audio streams extracted by the demultiplexer 202. Here, the number N of the multiplexing buffers 211-1 to 211-N is set to a necessary and sufficient number, but in actual operation, the number of buffers used is equal to the number of audio streams extracted by the demultiplexer 202.

[0106] The combiner 212 reads out the audio stream for each audio frame from one of the multiplexing buffers 211-1 to 211-N into which each audio stream extracted by the demultiplexer 202 has been loaded, and sends the audio stream to the 3D audio decoder 213.

[0107] If the audio stream supplied from the combiner 212 includes descriptor information ("Desc"), the 3D audio decoder 213 sends the descriptor information to the CPU 221. Under the control of the CPU 221, the 3D audio decoder 213 selectively extracts encoded data from a group having attributes that match the speaker configuration and viewer (user) selection information, performs decoding processing, and obtains audio data for driving each speaker of the speaker system 215.

[0108] Here, the coded data to be decoded may include three types of data: a data set that includes only channel coded data, a data set that includes only object coded data, or a data set that includes both channel coded data and object coded data.

[0109] When decoding channel-encoded data, the 3D audio decoder 213 performs downmixing or upmixing to the speaker configuration of the speaker system 215 to obtain audio data for driving each speaker. When decoding object-encoded data, the 3D audio decoder 213 calculates speaker rendering (mixing ratio for each speaker) based on object information (metadata) and mixes the audio data of the object into audio data for driving each speaker according to the calculation result.

[0110] The audio output processing circuit 214 performs necessary processing such as D / A conversion and amplification on the audio data for driving each speaker obtained by the 3D audio decoder 213, and supplies the audio data to a speaker system 215. The speaker system 215 includes multiple channels, such as 2 channels, 5.1 channels, 7.1 channels, and 22.2 channels.

[0111] The operation of the service receiver 200 shown in Fig. 15 will be briefly described. The receiver 201 receives the transport stream TS transmitted from the service transmitter 100 on broadcast waves or network packets. In addition to the video stream, this transport stream TS has a predetermined number of audio streams including multiple groups of encoded data constituting the transmission data of 3D audio. This transport stream TS is supplied to a demultiplexer 202.

[0112] In the demultiplexer 202, packets of a video stream are extracted from the transport stream TS and supplied to a video decoder 203. In the video decoder 203, a video stream is reconstructed from the video packets extracted by the demultiplexer 202, and a decoding process is performed to obtain uncompressed video data. This video data is supplied to a video processing circuit 204.

[0113] The video processing circuit 204 performs scaling processing, image quality adjustment processing, etc. on the video data obtained by the video decoder 203 to obtain video data for display. This video data for display is supplied to a panel driving circuit 205. The panel driving circuit 205 drives a display panel 206 based on the video data for display. As a result, an image corresponding to the video data for display is displayed on the display panel 206.

[0114] The demultiplexer 202 also extracts various types of information, such as descriptor information, from the transport stream TS and sends it to the CPU 221. This information includes a 3D audio stream configuration descriptor and a 3D audio stream ID descriptor. The CPU 221 recognizes audio streams that include encoded data of groups with attributes that match the speaker configuration and viewer (user) selection information, based on the attribute information and stream-related information included in the descriptor information.

[0115] In addition, in the demultiplexer 202, under the control of the CPU 221, packets of one or more audio streams containing encoded data of a group having attributes that match the speaker configuration and viewer selection information are selectively extracted using a PID filter from a predetermined number of audio streams contained in the transport stream TS.

[0116] The audio stream extracted by the demultiplexer 202 is taken into a corresponding one of the multiplexing buffers 211-1 to 211-N. The combiner 212 reads out the audio stream for each audio frame from each multiplexing buffer into which the audio stream has been taken, and supplies the audio stream to the 3D audio decoder 213.

[0117] In the 3D audio decoder 213, if the audio stream supplied from the combiner 212 contains descriptor information ("Desc"), the descriptor information is extracted and sent to the CPU 221. Under the control of the CPU 221, the 3D audio decoder 213 selectively extracts coded data of a group having attributes that match the speaker configuration and viewer (user) selection information, and performs a decoding process to obtain audio data for driving each speaker of the speaker system 215.

[0118] Here, when the channel-encoded data is decoded, downmixing or upmixing processing is performed to obtain audio data for driving each speaker to the speaker configuration of the speaker system 215. Also, when the object-encoded data is decoded, speaker rendering (mixing ratio for each speaker) is calculated based on the object information (metadata), and the audio data of the object is mixed into audio data for driving each speaker according to the calculation result.

[0119] The audio data for driving each speaker obtained by the 3D audio decoder 213 is supplied to an audio output processing circuit 214. In this audio output processing circuit 214, necessary processing such as D / A conversion and amplification is performed on the audio data for driving each speaker. The processed audio data is then supplied to a speaker system 215. As a result, an audio output corresponding to the image displayed on the display panel 206 is obtained from the speaker system 215.

[0120] 17 shows an outline of the decoding process when the descriptor information is not present in the audio stream. A transport stream TS, which is a multiplexed stream, is input to the demultiplexer 202. The demultiplexer 202 analyzes the system layer and supplies the descriptor information 1 (information on the 3D audio stream config descriptor and 3D audio stream ID descriptor) to the CPU 221.

[0121] The CPU 221 recognizes an audio stream including encoded data of a group having attributes that match the speaker configuration and viewer (user) selection information based on the descriptor information 1. The demultiplexer 202 selects between streams under the control of the CPU 221.

[0122] That is, in the demultiplexer 202, packets of one or more audio streams including encoded data of a group having attributes matching the speaker configuration and viewer selection information are selectively extracted by a PID filter from a predetermined number of audio streams contained in the transport stream TS. The audio streams thus extracted are taken into the multiplexing buffer 211 (211-1 to 211-N).

[0123] The 3D audio decoder 213 performs packet type analysis on each audio stream taken into the multiplexing buffer 211. Then, the demultiplexer 202 performs selection within the stream under the control of the CPU 221 based on the above-mentioned descriptor information 1.

[0124] In other words, encoded data of a group having attributes that match the speaker configuration and viewer (user) selection information is selectively extracted from each audio stream as the target for decoding, subjected to a decoding process, and subjected to processes such as mixing and rendering to obtain audio data (uncompressed audio) for driving each speaker.

[0125] 18 shows an example of the configuration of an audio access unit (audio frame) of an audio stream when the descriptor information does not exist in the audio stream. Here, an example of two streams is shown.

[0126] For the audio stream identified by PID2, the information "FrWork #ch=2, #obj=1" included in "Config" indicates the presence of a "Frame" with two channels of channel-encoded data and one object-encoded data. The information "GroupID[0]=1, GroupID[1]=2" registered in order in "AudioSceneInfo()" included in "Config" indicates that a "Frame" with encoded data from Group 1 and a "Frame" with encoded data from Group 2 are arranged in that order. Note that the packet label (PL) value is the same for "Config" and each corresponding "Frame".

[0127] Here, the "Frame" having the coded data of Group 1 includes coded sample data of CPE (Channel Pair Element), and the "Frame" having the coded data of Group 2 is made up of a "Frame" including metadata as an extension element (Ext_element) and a "Frame" including coded sample data of SCE (Single Channel Element).

[0128] For the audio stream identified by PID3, the information "FrWork #ch=0, #obj=2" included in "Config" indicates the presence of a "Frame" with two object encoded data. The information "GroupID[2]=3, GroupID[3]=4, SW_GRPID[0]=1" registered in order in "AudioSceneInfo()" included in "Config" indicates that a "Frame" with encoded data of group 3 and a "Frame" with encoded data of group 4 are arranged in that order, and these groups constitute switch group 1. Note that the packet label (PL) value is the same for "Config" and each corresponding "Frame".

[0129] Here, a "Frame" having coded data of group 3 is made up of a "Frame" including metadata as an extension element (Ext_element) and a "Frame" including coded sample data of SCE (Single Channel Element). Similarly, a "Frame" having coded data of group 4 is made up of a "Frame" including metadata as an extension element (Ext_element) and a "Frame" including coded sample data of SCE (Single Channel Element).

[0130] 19 shows an outline of the decoding process when the descriptor information exists in the audio stream. A multiplexed stream, a transport stream TS, is input to the demultiplexer 202. The demultiplexer 202 analyzes the system layer and sends the descriptor information 1 (information on the 3D audio stream config descriptor and 3D audio stream ID descriptor) to the CPU 221.

[0131] The CPU 221 recognizes an audio stream including encoded data of a group having attributes that match the speaker configuration and viewer (user) selection information based on the descriptor information 1. The demultiplexer 202 selects between streams under the control of the CPU 221.

[0132] That is, in the demultiplexer 202, packets of one or more audio streams including encoded data of a group having attributes matching the speaker configuration and viewer selection information are selectively extracted by a PID filter from a predetermined number of audio streams contained in the transport stream TS. The audio streams thus extracted are taken into the multiplexing buffer 211 (211-1 to 211-N).

[0133] The 3D audio decoder 213 performs packet type analysis on each audio stream captured in the multiplexing buffer 211, and sends the descriptor information 2 present in the audio stream to the CPU 221. The presence of encoded data of a group having attributes matching the speaker configuration and viewer (user) selection information is recognized based on this descriptor information 2. Then, the demultiplexer 202 performs selection within the stream under the control of the CPU 221 based on this descriptor information 2.

[0134] In other words, encoded data of a group having attributes that match the speaker configuration and viewer (user) selection information is selectively extracted from each audio stream as the target for decoding, subjected to a decoding process, and subjected to processes such as mixing and rendering to obtain audio data (uncompressed audio) for driving each speaker.

[0135] Fig. 20 shows an example of the configuration of an audio access unit (audio frame) of an audio stream when descriptor information exists in the audio stream. Here, an example of two streams is shown. Fig. 20 is the same as Fig. 18, except that "Desc", that is, the descriptor information, is inserted between "SYNC" and "Config".

[0136] For the audio stream identified by PID2, the "GroupID[0]=1, channeldata" information included in "Desc" indicates that the coded data of group 1 is channel coded data. Also, the "GroupID[1]=2, object sound" information included in "Desc" indicates that the coded data of group 2 is object coded data for immersive sound. Furthermore, the "Stream_ID" information indicates the stream identifier of the audio stream.

[0137] For the audio stream identified by PID3, the information "GroupID[2]=3, object lang1" included in "Desc" indicates that the coded data of group 3 is object coded data for the first speech language. Furthermore, the information "GroupID[3]=4, object lang2" included in "Desc" indicates that the coded data of group 4 is object coded data for the second speech language. Furthermore, the information "SW_GRPID[0]=1" included in "Desc" indicates that groups 3 and 4 constitute switch group 1. Furthermore, the information "Stream_ID" indicates the stream identifier of the audio stream.

[0138] 21 shows an example of the structure of an audio access unit (audio frame) of an audio stream when the descriptor information exists in the audio stream, in which an example of one stream is shown.

[0139] The information "FrWork #ch=2, #obj=3" included in "Config" indicates the presence of a "Frame" with two channels of channel encoded data and three object encoded data. The information "GroupID[0]=1, GroupID[1]=2, GroupID[2]=3, GroupID[3]=4, SW_GRPID[0]=1" registered in order in "AudioSceneInfo()" included in "Config" indicates that a "Frame" with encoded data of Group 1, a "Frame" with encoded data of Group 2, a "Frame" with encoded data of Group 3, and a "Frame" with encoded data of Group 4 are arranged in this order, and Groups 3 and 4 constitute Switch Group 1. The packet label (PL) value is the same for "Config" and each corresponding "Frame".

[0140] Here, the "Frame" having the coded data of group 1 includes coded sample data of CPE (Channel Pair Element). Also, the "Frame" having the coded data of groups 2 to 4 is composed of a "Frame" including metadata as an extension element (Ext_element) and a "Frame" including coded sample data of SCE (Single Channel Element), respectively.

[0141] The information "GroupID[0]=1, channeldata" included in "Desc" indicates that the coded data of group 1 is channel coded data. Also, the information "GroupID[1]=2, object sound" included in "Desc" indicates that the coded data of group 2 is object coded data for immersive sound.

[0142] Furthermore, the information "GroupID[2]=3, object lang1" contained in "Desc" indicates that the coded data of group 3 is object coded data for the first speech language. Furthermore, the information "GroupID[3]=4, object lang2" contained in "Desc" indicates that the coded data of group 4 is object coded data for the second speech language. Furthermore, the information "SW_GRPID[0]=1" contained in "Desc" indicates that groups 3 and 4 constitute switch group 1. Furthermore, the information "Stream_ID" indicates the stream identifier of the audio stream.

[0143] The flowcharts of Fig. 22 and Fig. 23 show an example of audio decode control processing by the CPU 221 in the service receiver 200 shown in Fig. 15. The CPU 221 starts processing in step ST1. Then, in step ST2, the CPU 221 detects the receiver speaker configuration, that is, the speaker configuration of the speaker system 215. Next, in step ST3, the CPU 221 obtains selection information regarding audio output by a viewer (user).

[0144] Next, in step ST4, the CPU 221 reads the descriptor information related to the main stream in the PMT, selects an audio stream to which a group having attributes matching the speaker configuration and viewer selection information belongs, and takes it into the buffer. Then, in step ST5, the CPU 221 checks whether a descriptor type packet exists in the audio stream.

[0145] Next, in step ST6, the CPU 221 determines whether or not a descriptor type packet exists. If a descriptor type packet exists, in step ST7, the CPU 221 reads the descriptor information of the packet and detects the information "groupID", "attribute", "switchGroupID", and "presetGroupID", and then proceeds to the process of step ST9. On the other hand, if a descriptor type packet does not exist, in step ST8, the CPU 221 detects the information "groupID", "attribute", "switchGroupID", and "presetGroupID" from the descriptor information of the PMT, and then proceeds to the process of step ST9. It is also possible to perform a process of decoding the entire target audio stream without executing step ST8.

[0146] In step ST9, the CPU 221 judges whether or not to decode the coded object data. If the coded object data is to be decoded, the CPU 221 decodes the coded object data in step ST10, and then proceeds to the process of step ST11. On the other hand, if the coded object data is not to be decoded, the CPU 221 immediately proceeds to the process of step ST11.

[0147] In step ST11, the CPU 221 determines whether or not to decode the channel-encoded data. If the data is to be decoded, the CPU 221 decodes the channel-encoded data in step ST12, and performs downmixing or upmixing processing for the speaker configuration of the speaker system 215 as necessary to obtain audio data for driving each speaker. After that, the CPU 221 proceeds to processing in step ST13. On the other hand, if the data is not to be decoded, the CPU 221 immediately proceeds to processing in step ST13.

[0148] In step ST13, when the CPU 221 decodes the encoded object data, it calculates mixing with channel data or speaker rendering based on the information. In the calculation of speaker rendering, it calculates speaker rendering (mixing ratio to each speaker) based on azimuth (direction information) and elevation (elevation angle information), and mixes the audio data of the object with the channel data for driving each speaker according to the calculation result.

[0149] Next, in step ST14, the CPU 221 performs dynamic range control on the audio data for driving each speaker and outputs the audio data. After that, in step ST15, the CPU 221 ends the process.

[0150] As described above, in the transmission / reception system 10 shown in Fig. 1, the service transmitter 100 inserts attribute information indicating attributes of each of the multiple groups of encoded data included in a predetermined number of audio streams into a container layer and / or an audio stream layer. Therefore, on the receiving side, the attributes of each of the multiple groups of encoded data can be easily recognized before decoding the encoded data, and only the encoded data of the necessary groups can be selectively decoded and used, thereby reducing the processing load.

[0151] 1, the service transmitter 100 inserts stream correspondence information, which indicates which audio stream contains each of the encoded data of the multiple groups, into the container layer and / or the audio stream layer. This allows the receiving side to easily recognize the audio stream containing the encoded data of the required group, thereby reducing the processing load.

[0152] <2. Modifications> In the above-described embodiment, the service receiver 200 is configured to selectively extract, from the multiple audio streams transmitted from the service transmitter 100, audio streams that include encoded data of a group having attributes that match the speaker configuration and viewer selection information, and perform decoding processing to obtain audio data for driving a predetermined number of speakers.

[0153] However, as a service receiver, it is also possible to selectively extract one or more audio streams having encoded data of a group having attributes that match the speaker configuration and viewer selection information from the multiple audio streams transmitted from the service transmitter 100, reconstruct an audio stream having encoded data of a group having attributes that match the speaker configuration and viewer selection information, and distribute the reconstructed audio stream to devices (including DLNA devices) connected to the local network.

[0154] Fig. 24 shows a configuration example of a service receiver 200A that distributes the reconstructed audio stream to devices connected to an in-house network as described above. In Fig. 24, the same reference numerals are used for parts corresponding to those in Fig. 15, and detailed descriptions thereof will be omitted as appropriate.

[0155] In the demultiplexer 202, under the control of the CPU 221, packets of one or more audio streams containing encoded data of a group having attributes that match the speaker configuration and viewer selection information are selectively extracted using a PID filter from a predetermined number of audio streams contained in the transport stream TS.

[0156] The audio stream extracted by the demultiplexer 202 is loaded into a corresponding one of the multiplexing buffers 211-1 to 211-N. The combiner 212 reads out the audio stream for each audio frame from each multiplexing buffer into which the audio stream has been loaded, and supplies the read audio stream to the stream reconstruction unit 231.

[0157] In the stream reconstruction unit 231, if the audio stream supplied from the combiner 212 contains descriptor information ("Desc"), the descriptor information is extracted and sent to the CPU 221. Under the control of the CPU 221, the stream reconstruction unit 231 selectively acquires encoded data of a group having attributes that match the speaker configuration and viewer (user) selection information, and reconstructs an audio stream having the encoded data. This reconstructed audio stream is supplied to a distribution interface 232. Then, from this distribution interface 232, it is distributed (transmitted) to the device 300 connected to the local network.

[0158] The local network connection may include an Ethernet connection, or a wireless connection such as "WiFi" or "Bluetooth." "WiFi" and "Bluetooth" are registered trademarks.

[0159] The device 300 also includes surround speakers, a second display, and an audio output device attached to a network terminal. The device 200 receiving the reconstructed audio stream performs a decoding process similar to that of the 3D audio decoder 213 in the service receiver 200 in Fig. 15 to obtain audio data for driving a predetermined number of speakers.

[0160] The service receiver may also be configured to transmit the reconstructed audio stream to a device connected via a digital interface such as "HDMI (High-Definition Multimedia Interface)", "MHL (Mobile High definition Link)", or "DisplayPort". Note that "HDMI" and "MHL" are registered trademarks.

[0161] In the above-described embodiment, the stream correspondence information inserted into the container layer or the like is information indicating the correspondence between the group ID and the substream ID. That is, the substream ID is used to associate the group with the audio stream. However, it is also possible to use a packet identifier (PID: Packet ID) or a stream type (stream_type) to associate the group with the audio stream. Note that when the stream type is used, it is necessary to change the stream type of each audio stream.

[0162] In the above embodiment, an example was shown in which attribute information of encoded data of each group is transmitted by providing an "attribute_of_groupID" field (see FIG. 10). However, the present technology also includes a method in which a special meaning is defined for the value of the group ID itself between a transmitter and a receiver, so that the type (attribute) of encoded data can be recognized by recognizing a specific group ID. In this case, the group ID functions as an identifier for the group as well as attribute information for the encoded data of the group, making the "attribute_of_groupID" field unnecessary.

[0163] In the above embodiment, an example has been shown in which the coded data of the multiple groups includes both channel coded data and object coded data (see FIG. 3). However, the present technology can be similarly applied to a case in which the coded data of the multiple groups includes only channel coded data or only object coded data.

[0164] In the above embodiment, an example in which the container is a transport stream (MPEG-2 TS) is shown. However, the present technology can be similarly applied to a system in which data is delivered in a container of MP4 or other formats. For example, this is an MPEG-DASH-based stream delivery system, or a transmission / reception system that handles an MMT (MPEG Media Transport) structure transmission stream.

[0165] The present technology can also be configured as follows. (1) a transmitter for transmitting a container having a predetermined format and a predetermined number of audio streams including encoded data of a plurality of groups; an information inserting unit that inserts attribute information indicating attributes of each of the plurality of groups of encoded data into the container layer and / or the audio stream layer; Transmitting device. (2) The information insertion unit is and inserting, into the container layer and / or the audio stream layer, stream correspondence relationship information indicating which audio stream contains each of the plurality of groups of encoded data. The transmitting device according to (1) above. (3) The above-mentioned stream correspondence information is is information indicating a correspondence relationship between a group identifier for identifying each of the plurality of groups of encoded data and a stream identifier for identifying each of the predetermined number of audio streams. The transmitting device according to (2) above. (4) The information insertion unit is and further inserting stream identifier information indicating a stream identifier for each of the predetermined number of audio streams into the container layer and / or the audio stream layer. The transmitting device according to (3) above. (5) The container is an MPEG2-TS. The information insertion unit is When inserting the stream identifier information into the container, the stream identifier information is inserted into an audio elementary stream loop corresponding to each of the predetermined number of audio streams existing under a program map table. The transmitting device according to (4) above. (6) The information insertion unit is If the stream identifier information is inserted into the audio stream, the stream identifier information is inserted into a PES payload of a PES packet of each of the predetermined number of audio streams. The transmitting device according to (4) or (5). (7) The above-mentioned stream correspondence information is The information indicates a correspondence relationship between a group identifier for identifying each of the plurality of groups of encoded data and a packet identifier that is assigned when each of the predetermined number of audio streams is packetized. The transmitting device according to (2) above. (8) The above-mentioned stream correspondence information is is information indicating a correspondence relationship between a group identifier for identifying each of the plurality of groups of encoded data and type information indicating a stream type of each of the predetermined number of audio streams. The transmitting device according to (2) above. (9) The container is an MPEG2-TS. The information insertion unit is When inserting the attribute information and the stream correspondence relationship information into the container, the attribute information and the stream correspondence relationship information are inserted into an audio elementary stream loop corresponding to at least one of the predetermined number of audio streams existing under a program map table. A transmitting device according to any one of (2) to (8). (10) The information insertion unit is When inserting the attribute information and the stream correspondence information into the audio stream, the attribute information and the stream correspondence information are inserted into a PES payload of a PES packet of at least one of the predetermined number of audio streams. A transmitting device according to any one of (2) to (8). (11) The plurality of groups of coded data include one or both of channel coded data and object coded data. A transmitting device according to any one of (1) to (10). (12) a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format having a predetermined number of audio streams including encoded data of a plurality of groups; An information inserting step of inserting attribute information indicating attributes of each of the plurality of groups of encoded data into a layer of the container and / or a layer of the audio stream. Transmission method. (13) A receiving unit for receiving a container having a predetermined format and a predetermined number of audio streams including encoded data of a plurality of groups, attribute information indicating attributes of each of the plurality of groups of encoded data is inserted into a layer of the container and / or a layer of the audio stream; The audio stream processing device further includes a processing unit that processes the predetermined number of audio streams contained in the received container based on the attribute information. Receiving device. (14) Stream correspondence information indicating which audio stream contains each of the encoded data of the plurality of groups is further inserted into the container layer and / or the audio stream layer; The processing unit includes: The predetermined number of audio streams are processed based on the attribute information as well as the stream correspondence information. The receiving device according to (13) above. (15) The processing unit is Selectively performing a decoding process on an audio stream including encoded data of a group having an attribute matching the speaker configuration and the user selection information based on the attribute information and the stream correspondence relationship information. The receiving device according to (14) above. (16) The plurality of groups of coded data include one or both of channel coded data and object coded data. A receiving device according to any one of (13) to (15). (17) A receiving step of receiving, by a receiving unit, a container in a predetermined format having a predetermined number of audio streams including encoded data of a plurality of groups, attribute information indicating attributes of each of the plurality of groups of encoded data is inserted into a layer of the container and / or a layer of the audio stream; The method further includes a processing step of processing the predetermined number of audio streams contained in the received container based on the attribute information. Receiving method. (18) A receiving unit for receiving a container having a predetermined format and a predetermined number of audio streams including encoded data of a plurality of groups, attribute information indicating attributes of each of the plurality of groups of encoded data is inserted into a layer of the container and / or a layer of the audio stream; a processing unit that selectively acquires a predetermined group of encoded data from the predetermined number of audio streams contained in the received container based on the attribute information, and reconstructs an audio stream including the encoded data of the predetermined group; The audio signal processing device further includes a stream transmitting unit that transmits the audio stream reconstructed by the processing unit to an external device. Receiving device. (19) Stream correspondence information indicating which audio stream includes each of the groups of encoded data is further inserted into the container layer and / or the audio stream layer; The processing unit includes: Selectively acquiring the predetermined group of encoded data from the predetermined number of audio streams based on the stream correspondence information in addition to the attribute information. The receiving device according to (18) above. (20) A receiving step of receiving, by a receiving unit, a container in a predetermined format having a predetermined number of audio streams including encoded data of a plurality of groups, attribute information indicating attributes of each of the plurality of groups of encoded data is inserted into a layer of the container and / or a layer of the audio stream; a processing step of selectively acquiring a predetermined group of encoded data from the predetermined number of audio streams contained in the received container based on the attribute information, and reconstructing an audio stream including the encoded data of the predetermined group; The method further includes a stream transmission step of transmitting the audio stream reconstructed in the processing step to an external device. Receiving method.

[0166] The main feature of this technology is that it makes it possible to reduce the processing load on the receiving side by inserting attribute information indicating the attributes of each of multiple groups of encoded data contained in a predetermined number of audio streams and stream correspondence information indicating in which audio stream each of the multiple groups of encoded data is contained (see Figure 14). [Explanation of symbols]

[0167] 10. Transmitting and receiving system 100···Service transmitter 110 Stream generation unit 112 Video Encoder 113 Audio Encoder 114 Multiplexer 200,200A···Service receiver 201... Receiving unit 202 Demultiplexer 203 Video decoder 204...Image processing circuit 205 Panel drive circuit 206 Display panel 211-1 to 211-N Multiplexing buffer 212 Combiner 213...3D Audio Decoder 214... Audio output processing circuit 215···Speaker System 221···CPU 222...Flash ROM 223...DRAM 224... Internal bus 225 Remote control receiver 226···Remote control transmitter 231 Stream reconstruction unit 232···Distribution Interface 300··Devices

Claims

1. a receiving unit for receiving a transport stream; a demultiplexer for extracting a plurality of audio streams included in the transport stream; a combiner that combines the multiple audio streams and outputs a combined audio stream; a decoder for performing a decoding process on the composite audio stream to obtain audio data for driving one or more outputs. Information processing device.

2. The transport stream is an MPEG-2 TS. The information processing device according to claim 1 .

3. the audio stream includes either or both of channel-coded data and object-coded data; The information processing device according to claim 1 .

4. the object coding data includes audio objects and metadata for mapping and rendering the audio objects to the output; The information processing device according to claim 3 .

5. The metadata is included as an extension element (Ext_element), The information processing device according to claim 4.

6. The plurality of audio streams are composed of a main stream and a sub-stream. The information processing device according to claim 1 .

7. The encoded data included in the plurality of audio streams is classified into groups according to type. The information processing device according to claim 6.

8. The encoded data selectable between the groups is registered in a switch group and encoded. The information processing device according to claim 7.

9. The plurality of audio streams include a group ID which is an identifier for identifying the group, and a switch group ID which is an identifier for identifying the switch group. The information processing device according to claim 8.

10. The plurality of audio streams each include an identifier indicating a type of content of the group. The information processing device according to claim 7.

11. The identifier has a value from 0 to 15. The information processing device according to claim 10.

12. The plurality of audio streams include identifiers for identifying the sub-streams. The information processing device according to claim 6.

13. the object coding data includes coding data for a speech language; The information processing device according to claim 3 .

14. the plurality of audio streams each include a descriptor describing information associated with the audio stream; The information processing device according to claim 1 .

15. The descriptor has an 8-bit field "descriptor_tag" which is information indicating the descriptor type of the descriptor. The information processing device according to claim 14.

16. The descriptor has an 8-bit field "descriptor_length" which is information indicating the length or size of the descriptor. The information processing device according to claim 14.

17. an audio elementary stream loop (audio ES loop) corresponding to each of the plurality of audio streams includes a descriptor describing information related to each of the audio streams; The information processing device according to claim 1 .

18. receiving a transport stream; extracting a plurality of audio streams included in the transport stream; outputting a composite audio stream obtained by combining the plurality of audio streams; and performing a decoding process on the composite audio stream to obtain audio data for driving one or more outputs. Information processing methods.

19. receiving a transport stream; extracting a plurality of audio streams included in the transport stream; outputting a composite audio stream obtained by combining the plurality of audio streams; and performing a decoding process on the composite audio stream to obtain audio data for driving one or more outputs. A program for causing a computer to execute an information processing method.

Images