Transmitting device, transmitting method, receiving device and receiving method

Abstract

The present application relates to a transmitting device, a transmitting method, a receiving device and a receiving method. In order to reduce the processing load on the receiving side when transmitting multiple types of audio data. A metafile with meta information is transmitted, and the meta information is used to obtain a prescribed number of audio streams containing multiple groups of coded data at the receiving device. Attribute information representing respective attributes of the coded data of the multiple groups is inserted into the metafile. For example, stream correspondence information representing which audio stream the coded data of each of the multiple groups is included in is further inserted into the metafile.

Description

[0001] The present application is a divisional application of a Chinese national phase application whose international application date is September 7, 2015, whose international application number is PCT / JP2015 / 075318, and whose invention title is “Transmitting device, transmitting method, receiving device and receiving method”. The national phase entry date of the Chinese national phase application is March 3, 2017, whose application number is 201580047469.3, and whose invention title is “Transmitting device, transmitting method, receiving device and receiving method”. Technical Field

[0002] The present technology relates to a transmission device, a transmission method, a reception device, and a reception method, and more particularly, to a transmission device and the like of a technology for transmitting a plurality of types of audio data. Background Art

[0003] Conventionally, as a stereo (3D) sound effect technology, a technology for mapping encoded sample data with speakers existing at arbitrary positions based on metadata to be rendered has been proposed (for example, see Patent Document 1).

[0004] Citation List

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application National Publication (Publication) No. 2014-520491 Summary of the invention

[0007] Problems to be solved by the present invention

[0008] When object coded data consisting of coded sample data and metadata is transmitted together with channel coded data such as 5.1 channels and 7.1 channels, this can provide sound reproduction with a more realistic surround effect in the receiving side.

[0009] An object of the present technology is to reduce the processing load in the receiving side when multiple types of encoded data are transmitted.

[0010] The concept of the present technology is a sending device, which includes:

[0011] a transmitting unit configured to transmit a metafile having meta information for obtaining a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device; and

[0012] The information inserting unit is configured to insert attribute information indicating each attribute of the plurality of sets of encoded data into the metafile.

[0013] Solution to the problem

[0014] In the present technology, the transmitting unit transmits a metafile having meta information for obtaining a predetermined number of audio streams including a plurality of groups of coded data in the receiving device. For example, the plurality of groups of coded data may include one or both of channel coded data and object coded data.

[0015] The information insertion unit inserts attribute information representing each attribute of the coded data of the plurality of groups into a metafile. For example, the metafile may be a media presentation description (MPD) file. In this case, for example, the information insertion unit may insert the attribute information into the metafile by using a "supplemental descriptor".

[0016] In addition, for example, the transmission unit may transmit the metafile via an RF transmission path or a communication network transmission path. In addition, for example, the transmission unit may further transmit a container of a predetermined format having a predetermined number of audio streams including a plurality of groups of encoded data. For example, the container is MP4. According to this technical report, MP4 represents ISO Base Media File Format (ISOBMFF) (ISO / IEC14496-12:2012).

[0017] In this way, according to the present technology, attribute information indicating each attribute of a plurality of groups of encoded data is inserted into a metafile having meta information for obtaining a predetermined number of audio streams including the plurality of groups of encoded data in a receiving device. Therefore, the receiving side can easily recognize each attribute of the plurality of groups of encoded data before the relevant encoded data is decoded, so that the necessary group of encoded data can be selectively decoded and used, and the processing load can be reduced.

[0018] Herein, according to the present technology, for example, the information insertion unit may further insert stream correspondence information into the metafile, the stream correspondence information indicating in which audio stream the encoded data of the plurality of groups are respectively included. In this case, for example, the stream correspondence information may be information indicating a correspondence between group identifiers identifying the respective encoded data of the plurality of groups and identifiers identifying the corresponding streams of a predetermined number of audio streams. In this case, the receiving side can easily identify the audio stream including the encoded data of the necessary group, and this can reduce the processing load.

[0019] In addition, another concept of the present technology is a receiving device, the receiving device comprising:

[0020] a receiving unit configured to receive a metafile having meta information for obtaining a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device,

[0021] Attribute information indicating each attribute of the plurality of sets of encoded data is inserted into the metafile; and

[0022] The processing unit is configured to process a predetermined number of audio streams based on the attribute information.

[0023] According to the present technology, a receiving unit receives a metafile. The metafile includes meta information for obtaining a predetermined number of audio streams including a plurality of groups of coded data in a receiving device. For example, the plurality of groups of coded data may include one or both of channel coded data and object coded data. Attribute information representing each attribute of the plurality of groups of coded data is inserted into the metafile. A processing unit processes the predetermined number of audio streams based on the attribute information.

[0024] In this way, according to the present technology, based on the attribute information representing each attribute of multiple groups of encoded data inserted in the metafile, processing is performed on a predetermined number of audio streams. Therefore, only necessary groups of encoded data can be selectively decoded and used, and this can reduce the processing load.

[0025] Herein, according to the present technology, for example, the metafile may further include stream correspondence information indicating in which audio streams the plurality of groups of encoded data are respectively included, and the processing unit may process a predetermined number of audio streams based on the stream correspondence information and the attribute information. In this case, the audio stream including the necessary group of encoded data can be easily identified, and this can reduce the processing load.

[0026] Furthermore, according to the present technology, for example, the processing unit can selectively perform decoding processing on an audio stream including encoded data having a set of attributes compatible with the speaker configuration and the user selection information based on the attribute information and the stream correspondence information.

[0027] In addition, another concept of the present technology is a receiving device, the receiving device comprising:

[0028] a receiving unit configured to receive a metafile having meta information for obtaining a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device,

[0029] Attribute information representing each attribute of the plurality of groups of encoded data is inserted into the metafile;

[0030] a processing unit configured to selectively acquire a predetermined group of encoded data from a predetermined number of audio streams based on the attribute information, and reconfigure an audio stream including the predetermined group of encoded data; and

[0031] The stream sending unit is configured to send the reconfigured audio stream to an external device.

[0032] According to the present technology, a receiving unit receives a metafile. The metafile includes meta information for acquiring a predetermined number of audio streams including a plurality of sets of encoded data in a receiving device. Attribute information indicating each attribute of the plurality of sets of encoded data is inserted into the metafile.

[0033] The processing unit selectively acquires a predetermined set of encoded data from a predetermined number of audio streams based on the attribute information, and reconfigures the audio stream including the predetermined set of encoded data. Subsequently, the stream transmission unit transmits the reconfigured audio stream to an external device.

[0034] In this way, according to the present technology, based on the attribute information of each attribute representing the plurality of groups of encoded data inserted in the metafile, a predetermined group of encoded data is selectively acquired from a predetermined number of audio streams, and the audio stream to be transmitted to the external device is reconfigured. The necessary group of encoded data can be easily acquired, and this can reduce the processing load.

[0035] Herein, according to the present technology, for example, stream correspondence information indicating in which audio streams a plurality of groups of encoded data are respectively included is further inserted into the metafile, and the processing unit can selectively obtain a predetermined group of encoded data from a predetermined number of audio streams based on the stream correspondence information and the attribute information. In this case, the audio stream including the predetermined group of encoded data can be easily identified, and this can reduce the processing load.

[0036] Effects of the Invention

[0037] According to the present technology, when multiple types of encoded data are transmitted, the processing load in the receiving side can be reduced.Herein, the effects described in this specification are merely examples and no limitation is set, and there may be other effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 (a) and (b) are block diagrams showing exemplary configurations of a streaming system based on MPEG-DASH.

[0039] Figure 2 (a) to (d) are diagrams showing examples of the relationship between structures hierarchically arranged in an MPD file.

[0040] Figure 3 1 is a block diagram showing an exemplary configuration of a transceiver system as one embodiment.

[0041] Figure 4 is a diagram showing the structure of an audio frame (1024 samples) of 3D audio transmission data.

[0042] Figure 5 is a diagram illustrating an exemplary configuration of 3D audio transmission data.

[0043] Figure 6 (a) and (b) are diagrams schematically showing exemplary configurations of an audio frame in a case where 3D audio transmission data is transmitted using one track (one audio stream) and an audio frame in a case where 3D audio transmission data is transmitted using multiple tracks (more than one audio stream).

[0044] Figure 7 : is a diagram showing an example of group division in the case of transmitting using four tracks in an exemplary configuration of 3D audio transmission data.

[0045] Figure 8 It is a diagram showing the correspondence relationship between groups and tracks in a group division example (divided into four groups), etc.

[0046] Fig. 9 : is a diagram showing an example of group division in the case of transmission using two tracks in an exemplary configuration of 3D audio transmission data.

[0047] Fig.10 It is a diagram showing the correspondence relationship between groups and tracks in a group division example (divided into two groups), etc.

[0048] Fig.11 is a diagram showing a description example of an MPD file.

[0049] Fig.12 is a diagram showing another description example of an MPD file.

[0050] Fig.13 is a diagram showing an example of "schemeIdUri" defined by "SupplementaryDescriptor".

[0051] Fig.14 (a) to (c) are the descriptions of the <baseurl>” represents a diagram of the essence of the media file at the location destination.

[0052] Fig.15 (a) and (b) are diagrams illustrating descriptions of correspondence between track identifiers (track IDs) and level identifiers (level IDs) in a “moov” box.

[0053] Fig.16 (a) and (b) are diagrams showing a transmission example of each block in the case of a transmission system, etc.

[0054] Fig.17 is a block diagram showing an exemplary configuration of a DASH / MP4 generation unit included in the service delivery system.

[0055] Fig.18 is a block diagram illustrating an exemplary configuration of a service receiver.

[0056] Fig.19 is a flowchart showing an example of audio decoding control processing by a CPU in a service receiver.

[0057] Fig. 20 is a block diagram illustrating another exemplary configuration of a service receiver. DETAILED DESCRIPTION

[0058] Hereinafter, a mode for carrying out the present invention (hereinafter, referred to as “embodiment”) will be described. Note that the description will be given in the following order.

[0059] 1. Implementation Method

[0060] 2. Modifications

[0061] <1. Implementation Method>

[0062] [Overview of MPEG-DASH-based streaming system]

[0063] First, an overview of an MPEG-DASH-based streaming system to which the present technology can be applied will be described.

[0064] Figure 1 (a) shows an exemplary configuration of a streaming system 30A based on MPEG-DASH. In this exemplary configuration, the media stream and the MPD file are sent via a communication network transmission path. The streaming system 30A is configured so that N service receivers 33-1, 33-2, ..., and 33-N are connected to a DASH streaming file server 31 and a DASH MPD server 32 via a content delivery network (CDN) 34.

[0065] The DASH streaming file server 31 generates a streaming segment of the DASH specification (hereinafter, appropriately referred to as a "DASH segment") based on media data (video data, audio data, subtitle data, etc.) of predetermined content, and transmits the segment according to an HTTP request made by a service receiver. The DASH streaming file server 31 may be a server dedicated to streaming, and also used as a network server.

[0066] Furthermore, in response to a request for a predetermined stream segment transmitted from a service receiver 33 (33-1, 33-2, ..., and 33-N) via the CDN 34, the DASH streaming file server 31 transmits the stream segment to the receiver as the request source via the CDN 34. In this case, the service receiver 33 selects a stream of the optimal rate according to the state of the network environment in which the client is located with reference to the rate value described in the media presentation description (MPD) file, and makes a request.

[0067] The DASH MPD server 32 is a server that generates an MPD file for acquiring the DASH segment generated in the DASH stream file server 31. The MPD file is generated based on content metadata received from a content management server (not shown) and the address (url) of the segment generated in the DASH stream file server 31. Here, the DASH stream file server 31 and the DASH MPD server 32 may be physically the same.

[0068] In the MPD format, each attribute is described using an element such as a representation for each stream (such as a video stream or an audio stream). For example, the representation is divided for each multiple video data stream with different rates, and each rate thereof is described in the MPD file. The service receiver 33 can select the best stream according to the state of the network environment in which the service receiver 33 is located in consideration of the rate value as described above.

[0069] Figure 1 (b) shows an exemplary configuration of a stream delivery system 30B based on MPEG-DASH. In this exemplary configuration, the media stream and the MPD file are transmitted via an RF transmission path. The stream delivery system 30B is configured with a broadcast transmission system 36, which is connected to a DASH stream file server 31, a DASH MPD server 32, and M service receivers 35-1, 35-2, ..., and 35-M.

[0070] In the case of the streaming system 30B, the broadcast sending system 36 sends the stream segments (DASH segments) of the DASH specification generated by the DASH streaming file server 31 and the MPD file generated by the DASH MPD server 32 through broadcast waves.

[0071] Figure 2 (a) to (d) show examples of the relationship between structures arranged hierarchically in an MPD file. Figure 2 As shown in (a), the media presentation of the entire MPD file includes multiple periods defined by time intervals. For example, the first period starts from the 0th second, and the next period starts from the 100th second.

[0072] like Figure 2 As shown in (b), a period includes multiple representations. Among the multiple representations, there is a group of representations grouped according to an adaptation set and involving the same media stream with different stream attributes, such as different rates.

[0073] like Figure 2 As shown in (c) in FIG. 1 , it indicates that segment information (SegmentInfo) is included. In the segment information, Figure 2 As shown in (d), there are an initialization segment and a plurality of media segments, wherein information of the segments obtained by more finely defining the period is described. The media segments include information such as an address (url) for actually acquiring segment data such as video or audio.

[0074] In addition, among the multiple representations grouped according to the adaptation set, stream switching can be freely performed. Therefore, it is possible to select a stream with the best rate according to the state of the network environment where the service receiver is located, and perform seamless transmission.

[0075] [Exemplary Configuration of Transceiver System]

[0076] Figure 3 The exemplary configuration of the transceiver system 10 as an embodiment is shown. The transceiver system 10 is composed of a service transmission system 100 and a service receiver 200. In the transceiver system 10, the service transmission system 100 and the above-mentioned Figure 1 The DASH streaming file server 31 and the DASH MPD server 32 in the streaming system 30A in (a) correspond to each other. Figure 1 The DASH streaming file server 31, the DASH MPD server 32, and the broadcast sending system 36 in the streaming transmission system 30B in (b) correspond to each other.

[0077] In addition, in the transceiver system 10, the service receiver 200 and the above Figure 1 The service receiver 200 corresponds to the service receiver 33 (33-1, 33-2, ..., 33-N) in the streaming system 30A in (a). Figure 1 This corresponds to the service receivers 35 (35-1, 35-2, ..., 35-M) in the streaming system 30B in (b).

[0078] The service delivery system 100 is transmitted via an RF transmission path (see Figure 1 (b)) or communication network transmission path (see Figure 1 (a)) sends DASH / MP4, that is, an MPD file used as a metafile and MP4 including a media stream (media segment) such as video or audio.

[0079] Figure 4 The structure of an audio frame (1024 samples) in the transmission data of 3D audio (MPEGH) handled in this embodiment is shown. An audio frame is configured with a plurality of MPEG audio stream packets (mpeg Audio Stream Packets). Each MPEG audio stream packet is configured with a header (Header) and a payload (Payload).

[0080] The header includes information such as a packet type (Packet Type), a packet label (Packet Label), and a packet length (Packet Length). Information defined by the packet type of the header is arranged in the payload. The payload information includes "SYNC" information corresponding to a synchronization start code, "Frame" information serving as actual data of 3D audio transmission data, and "Config" information indicating the configuration of the "Frame" information.

[0081] The "frame" information includes channel coding data and object coding data configuring 3D audio transmission data. Here, the channel coding data is configured with coded sample data such as a single channel element (SCE), a channel pair element (CPE), and a low frequency element (LFE). In addition, the object coding data is configured with coded sample data of a single channel element (SCE) and metadata, which is used to map the coded sample data with a speaker located at an arbitrary position and render the coded sample data. The metadata is included as an extension element (Ext_element).

[0082] Figure 5 An exemplary configuration of 3D audio transmission data is shown. In this example, the 3D audio transmission data consists of one channel coded data and two object coded data. The channel coded data is channel coded data (CD) of 5.1 channels, and is configured with each coded sample data in SCE1, CPE1.1, CPE1.2, and LFE1.

[0083] The two pieces of object encoding data are encoding data of an immersive audio object (IAO) and a speech dialogue object (SDO). The immersive audio object encoding data is object encoding data for immersive sound, and consists of encoding sample data SCE2 and metadata EXE_El (object metadata) 2, which is used to map the encoding sample data SCE2 with a speaker located at an arbitrary position and render the encoding sample data SCE2.

[0084] The speech dialogue object coding data is object coding data for a spoken language. In this example, there are speech dialogue object coding data corresponding to a first language and a second language, respectively. The speech dialogue object coding data corresponding to the first language consists of coding sample data SCE3 and metadata EXE_E1 (object metadata) 3, which is used to map the coding sample data SCE3 with a speaker located at an arbitrary position and render the coding sample data SCE3. In addition, the speech dialogue object coding data corresponding to the second language consists of coding sample data SCE4 and metadata EXE_E1 (object metadata) 4, which is used to map the coding sample data SCE4 with a speaker located at an arbitrary position and render the coding sample data SCE4.

[0085] According to the data type, the encoded data is distinguished by the concept of group. In the example shown, the encoded channel data of the 5.1 channel is defined as Group 1, the immersive audio object encoded data is defined as Group 2, the voice dialogue object encoded data related to the first language is defined as Group 3, and the voice dialogue object encoded data related to the second language is defined as Group 4.

[0086] In addition, the groups switchable on the receiving side are registered in the switch group (SW Group) and encoded. In the example shown, Group 3 and Group 4 are registered in the switch group 1 (SW Group 1). In addition, some groups can be grouped into preset groups (preset Group) and reproduced according to the use situation. In the example shown, Group 1, Group 2, and Group 3 are grouped into preset Group 1, and Group 1, Group 2, and Group 4 are grouped into preset Group 2.

[0087] Return to reference Figure 3 The service transmission system 100 transmits the 3D audio transmission data including the plurality of groups of encoded data as described above using one track as one audio stream or using a plurality of tracks as a plurality of audio streams.

[0088] Figure 6 (a) schematically shows the Figure 5 An exemplary configuration of 3D audio transmission data in the exemplary configuration of an audio frame in the case of transmitting data using one track (one audio stream). In this case, Track 1 includes "SYNC" information and "Config" information as well as channel coding data (CD), immersive audio object coding data (IAO), and voice dialogue object coding data (SDO).

[0089] Figure 6 (b) schematically shows the Figure 5 An exemplary configuration of 3D audio transmission data in the exemplary configuration of an audio frame in the case where data is transmitted using multiple tracks (multiple audio streams), in this example, using three tracks. In this case, Track 1 includes "SYNC" information and "Config" information and channel coding data (CD). In addition, Track 2 includes "SYNC" information and "Config" information and immersive audio object coding data (IAO). In addition, Track 3 includes "SYNC" information and "Config" information and voice dialogue object coding data (SDO).

[0090] Figure 7 Show about Figure 5 An exemplary configuration of 3D audio transmission data in the example of group division in the case of transmitting data using four tracks. In this case, track 1 includes channel coded data (CD) defined as group 1. In addition, track 2 includes immersive audio object coded data (IAO) defined as group 2. In addition, track 3 includes speech dialogue object coded data (SDO) of a first language defined as group 3. In addition, track 4 includes speech dialogue object coded data (SDO) of a second language defined as group 4.

[0091] Figure 8 Show Figure 7 The correspondence between the groups and tracks, etc. in the group division instance (divided into four groups). Here, the group ID (groupID) is an identifier for distinguishing the groups. The attribute (attribute) represents the attribute of the encoded data of each group. The switch group ID (switchGroupID) is an identifier for distinguishing the switch group. The preset group ID (presetGroupID) is an identifier for distinguishing the preset group. The track ID (trackID) is an identifier for distinguishing the tracks.

[0092] The illustrated correspondence relationship indicates that the encoded data of group 1 is channel encoded data that does not constitute the switching group and is included in track 1. In addition, the illustrated correspondence relationship indicates that the encoded data of group 2 is object encoded data of immersive sound (immersive audio object encoded data) that does not constitute the switching group and is included in track 2.

[0093] Furthermore, the illustrated correspondence relationship indicates that the encoded data of group 3 is object encoded data (voice dialogue object encoded data) for the spoken language of the first language that constitutes switching group 1 and is included in track 3. Furthermore, the illustrated correspondence relationship indicates that the encoded data of group 4 is object encoded data (voice dialogue object encoded data) for the spoken language of the second language that constitutes switching group 1 and is included in track 4.

[0094] In addition, the corresponding relationship shown indicates that the preset group 1 includes group 1, group 2, and group 3. In addition, the corresponding relationship shown indicates that the preset group 2 includes group 1, group 2, and group 4.

[0095] Fig. 9 Show about Figure 5 An example of group division in the case of transmitting data using two tracks of an exemplary configuration of 3D audio transmission data. In this case, track 1 includes channel coded data (CD) defined as group 1 and immersive audio object coded data (IAO) defined as group 2. In addition, track 2 includes voice dialogue object coded data (SDO) of a first language defined as group 3 and voice dialogue object coded data (SDO) of a second language defined as group 4.

[0096] Fig.10 Show Fig. 9 The correspondence between the groups and the substreams of the group division example (divided into two groups) of FIG. 1 shows that the coded data of group 1 is channel coded data that does not constitute the switching group and is included in track 1. In addition, the correspondence shown shows that the coded data of group 2 is object coded data (immersive audio object coded data) for immersive sound that does not constitute the switching group and is included in track 1.

[0097] Furthermore, the illustrated correspondence relationship indicates that the encoded data of group 3 is object encoded data (voice dialogue object encoded data) for the spoken language of the first language that constitutes switching group 1 and is included in track 2. Furthermore, the illustrated correspondence relationship indicates that the encoded data of group 4 is object encoded data (voice dialogue object encoded data) for the spoken language of the second language that constitutes switching group 1 and is included in track 2.

[0098] In addition, the corresponding relationship shown indicates that the preset group 1 includes group 1, group 2, and group 3. In addition, the corresponding relationship shown indicates that the preset group 2 includes group 1, group 2, and group 4.

[0099] Return to reference Figure 3 , the service delivery system 100 inserts attribute information indicating each attribute of the coded data of the plurality of groups included in the 3D audio delivery data into the MPD file. In addition, the service delivery system 100 inserts stream correspondence information indicating in which tracks (audio streams) the coded data of the plurality of groups are respectively included into the MPD file. In the present embodiment, it is assumed that the stream correspondence information is, for example, information indicating the correspondence between the group ID and the track ID.

[0100] The service delivery system 100 inserts the attribute information and the stream correspondence information into the MPD file. Separated from the existing definition in the existing standard, in the present embodiment where "SupplementaryDescriptor" can newly define "schemeIdUri" as broadcast or any other application, the service delivery system 100 inserts the attribute information and the stream correspondence information into the MPD file by using "SupplementaryDescriptor".

[0101] Fig.11 Shown with Figure 7 A description example of the MPD file corresponding to the group division instance (divided into four groups). Fig.12 Shown with Fig. 9 Here, for the purpose of simplifying the description, an example of information related only to the audio stream is described; however, in reality, information related to other media streams such as a video stream is also described. Fig.13 is a diagram showing an example of "schemeIdUri" defined by "SupplementaryDescriptor".

[0102] First, we will explain Fig.11 An example description of an MPD file.<AdaptationSet mimeType="audio / mp4"group="1"> " indicates that there is an adaptation set (AdaptationSet) for an audio stream, the audio stream is provided in the MP4 file structure, and group 1 is allocated.

[0103] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:codecType"value="mpegh" / > " indicates that the codec of the audio stream is MPEGH (3D audio). Fig.13 As shown, "schemeIdUri="urn:brdcst:codecType"" indicates the type of codec. In this example, it indicates "mpegh".

[0104] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId"value="group1" / > " indicates that the encoded data of group 1 "group1" is included in the audio stream. Fig.13 As shown, "schemeIdUri="urn:brdcst:3dAudio:groupId"" indicates a group identifier.

[0105] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:attribute"value="channeldata" / > " indicates that the coded data of group 1 "group1" is channel coded data "channeldata". Fig.13 As shown, "schemeIdUri="urn:brdcst:3dAudio:attribute"" indicates the attributes of the encoded data of the corresponding group.

[0106] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId"value="0" / > " indicates that the coded data of group 1 "group1" does not belong to any switching group. Fig.13 As shown, "schemeIdUri="urn:brdcst:3dAudio:switchGroupId"" indicates the identifier of the switch group to which the corresponding group belongs. For example, when "value" is "0", this indicates that the corresponding group does not belong to any switch group. When "value" is a value other than "0", this indicates that it belongs to the switch group.

[0107] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset1" / > " indicates that the encoded data of group 1 "group1" belongs to preset group 1 "preset1". In addition,

[0108] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset2" / > " indicates that the coded data of group 1 "group1" belongs to preset group 2 "preset2". Fig.13 As shown, "schemeIdUri="urn:brdcst:3dAudio:presetGroupId"" represents the identifier of the preset group to which the corresponding group belongs.

[0109] "<Representation id="1"bandwidth="128000"> " indicates that there is an audio stream with a bit rate of 128 kbps, which includes the encoded data of group 1 "group1" in the adaptation set of group 1 as the representation identified by "Representation id="1"". Subsequently, " <baseurl> audio / jp1 / 128.mp4< / baseurl> " indicates that the location destination of the audio stream is "audio / jp1 / 128.mp4".

[0110] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:levelId"value="level1" / > " indicates that the audio stream is sent using the track corresponding to level 1 "Level1". Fig.13 As shown, "schemeIdUri="urn:brdcst:3dAudio:levelId" represents an identifier of a level corresponding to an identifier of a track of an audio stream that transmits encoded data including a corresponding group. Here, as described below, for example, the correspondence between a track identifier (track ID) and a level identifier (level ID) in a "moov" box is described.

[0111] also,"<AdaptationSet mimeType="audio / mp4"group="2"> " indicates that there is an adaptation set (AdaptationSet) of the audio stream, the audio stream is provided in the MP4 file structure, and group 2 is allocated.

[0112] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:codecType"value="mpegh" / > " indicates that the codec of the audio stream is "MPEGH (3D Audio)."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId"value="group2" / > " indicates that the encoded data of group 2 "group2" is included in the audio stream.

[0113] "<SupplementaryDescriptorschemeIdUri="urn:brdcst:3dAudio:attribute"value="objectSound" / > " indicates that the encoded data of group 2 "group2" is object encoded data "objectSound" for immersive sound.<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId"value="0" / > " indicates that the encoded data of group 2 "group2" does not belong to any switching group.

[0114] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset1" / > "The description indicates that the encoded data of group 2 "group2" belongs to preset group 1 "preset1".

[0115] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset2" / > "The description indicates that the encoded data of group 2 "group2" belongs to preset group 2 "preset2".

[0116] "<Representation id="1"bandwidth="128000"> " indicates that there is an audio stream with a bit rate of 128 kbps, which includes the encoded data of group 2 "group2" in the adaptation set of group 2 as the representation identified by "Representation id="2"". Subsequently, " <baseurl> audio / jp2 / 128.mp4< / baseurl> " indicates that the destination of the audio stream is "audio / jp2 / 128.mp4". Subsequently, "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:levelId"value="level2" / > " indicates that the audio stream is sent using the track corresponding to level 2 "level2".

[0117] also,"<AdaptationSet mimeType="audio / mp4"group="3"> The description of " indicates that there is an adaptation set (AdaptationSet) corresponding to the audio stream, the audio stream is provided in the MP4 file structure, and group 3 is allocated.

[0118] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:codecType"value="mpegh" / > " indicates that the codec of the audio stream is "MPEGH (3D Audio)."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId"value="group3" / > " indicates that the encoded data of group 3 "group3" is included in the audio stream."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:attribute"value="objectLang1" / > " indicates that the encoded data of group 3 "group3" is object encoded data "objectLang1" for the speaking language of the first language.

[0119] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId"value="1" / > "The description indicates that the encoded data of group 3 "group3" belongs to switch group 1 (switchgroup 1).

[0120] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset1" / > "The description indicates that the encoded data of group 3 "group3" belongs to preset group 1 "preset1".

[0121] The description of "<Representation id="3" bandwidth="128000">” indicates that there is an audio stream with a bit rate of 128 kbps, and this audio stream includes the encoded data of "group3” in the adaptive set of group 3 as the representation identified by "<Representation id="3". Subsequently, " <baseurl> audio / jp3 / 128.mp4< / baseurl> ” indicates that the location destination of the audio stream is "audio / jp3 / 128.mp4”. Subsequently, the description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:levelId" value="level3" / >” indicates that the audio stream is sent using the track corresponding to level 3 "level3”.

[0122] Furthermore, the description of "<AdaptationSet mimeType="audio / mp4" group="4">” indicates that there is an AdaptationSet corresponding to the audio stream, and the audio stream is provided in the MP4 file structure, and group 4 is assigned.

[0123] The description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:codecType" value="mpegh" / >” indicates that the codec of the audio stream is "MPEGH (3D audio)”. The description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId" value="group4" / >” indicates that the encoded data of group 4 "group4” is included in the audio stream. The description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:attribute" value="objectLang2" / >” indicates that the encoded data of group 4 "group4” is object-encoded data "objectLang 2” for the spoken language of the second language.

[0124] The description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId" value="1" / >” indicates that the encoded data of group 4 "group4” belongs to switch group 1 (switchgroup 1).

[0125] The description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId" value="preset2" / >" indicates that the encoded data of "group4" in group 4 belongs to preset group 2 "preset2".

[0126] The description of "<Representation id="4" bandwidth="128000">” indicates that there is an audio stream with a bitrate of 128 kbps, and this audio stream includes the encoded data of "group4" in the adaptive set of group 4 as the representation identified by "<Representation id="4". Subsequently, " <baseurl> audio / jp4 / 128.mp4< / baseurl> ” indicates that the location destination of the audio stream is "audio / jp4 / 128.mp4". Subsequently, the description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:levelId" value="level4" / >” indicates that the audio stream is sent using the track corresponding to level 4 "level4".

[0127] Next, the description example of Fig.12 the MPD file will be described. The description of "<AdaptationSet mimeType="audio / mp4" group="1">” indicates that there is an adaptation set (AdaptationSet) of the audio stream, which provides the audio stream in the MP4 file structure, and group 1 is assigned. Subsequently, the description of "<SupplementaryDescriptor schemeIdUri="urn:brdcst:codecType" value="mpegh" / >” indicates that the codec of the audio stream is "MPEGH (3D audio)".

[0128] The description of "<Representation id="1" bandwidth="128000">” indicates that there is an audio stream with a bitrate of 128 kbps in the adaptation set of group 1 as the representation identified by "Representation id="1". Subsequently, " <baseurl> audio / jp1 / 128.mp4< / baseurl> " indicates that the destination of the audio stream is "audio / jp1 / 128.mp4". In addition, "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:levelId"value="level1" / > " indicates that the audio stream is sent using the track corresponding to level 1 "level1".

[0129] "<SubRepresentation id="11"subgroupSet="1"> The description of " indicates that in the representation identified by "Representation id="1"", a subrepresentation identified by "SubRepresentation id="11"" exists, and subgroup 1 is allocated.

[0130] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId"value="group1" / > " indicates that the encoded data of group 1 "group1" is included in the audio stream."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:attribute"value="channeldata" / > "The description indicates that the coded data of group 1 "group1" is channel coded data "channeldata".

[0131] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId"value="0" / > " indicates that the coded data of group 1 "group1" does not belong to any switching group."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset1" / > " indicates that the encoded data of group 1 "group1" belongs to preset group 1 "preset1". In addition,

[0132] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset2" / > "The description indicates that the encoded data of group 1 "group1" belongs to preset group 2 "preset2".

[0133] "<SubRepresentation id="12"subgroupSet="2"> The description of " indicates that in the representation identified by "Representation id="1"", there is a subrepresentation identified by "SubRepresentation id="12"", and subset set 2 is allocated.

[0134] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId"value="group2" / > " indicates that the encoded data of group 2 "group2" is included in the audio stream."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:attribute"value="objectSound" / > " indicates that the encoded data of group 2 "group2" is object encoded data "objectSound" for immersive sound.

[0135] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId"value="0" / > " indicates that the coded data of group 2 "group2" does not belong to any switching group."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset1" / > "The description indicates that the encoded data of group 2 "group2" belongs to preset group 1 "preset1".

[0136] <SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset2" / > "The description indicates that the encoded data of group 2 "group2" belongs to preset group 2 "preset2".

[0137] further,"<AdaptationSet mimeType="audio / mp4"group="2"> "The description indicates that there is an adaptation set (AdaptationSet) corresponding to the audio stream, and the audio stream is provided in an MP4 file structure, and group 2 is allocated. Subsequently, "<SupplementaryDescriptor schemeIdUri="urn:brdcst:codecType"value="mpegh" / > " indicates that the codec of the audio stream is "MPEGH (3D Audio)".

[0138] "<Representation id="2"bandwidth="128000"> " indicates that there is an audio stream with a bit rate of 128 kbps in the adaptation set of group 1 as a representation identified by "Representation id="2"". Subsequently, " <baseurl> audio / jp2 / 128.mp4< / baseurl> " indicates that the destination of the audio stream is "audio / jp2 / 128.mp4". In addition, "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:levelId"value="level2" / > " indicates that the audio stream is sent using the track corresponding to level 2 "level2".

[0139] "<SubRepresentation id="21"subgroupSet="3"> The description of " indicates that in the representation identified by "Representation id="2"", there is a subrepresentation identified by "SubRepresentation id="21"", and subset set 3 is allocated.

[0140] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId"value="group3" / > " indicates that the encoded data of group 3 "group3" is included in the audio stream."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:attribute"value="objectLang1" / > " indicates that the encoded data of group 3 "group3" is object encoded data "objectLang1" for the speaking language of the first language.

[0141] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId"value="1" / > "The description indicates that the encoded data of group 3 "group3" belongs to switch group 1 (switchgroup 1).

[0142] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset1" / > "The description indicates that the encoded data of group 3 "group3" belongs to preset group 1 "preset1".

[0143] "<SubRepresentation id="22"subgroupSet="4"> The description of " indicates that in the representation identified by "Representation id="2"", there is a subrepresentation identified by "SubRepresentation id="22"", and the subset set 4 is allocated.

[0144] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:groupId"value="group4" / > " indicates that the encoded data of group 4 "group4" is included in the audio stream."<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:attribute"value="objectLang2" / > "The description indicates that the encoded data of group 4 "group4" is object encoded data "objectLang2" for the speaking language of the second language.

[0145] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:switchGroupId"value="1" / > "The description indicates that the encoded data of group 4 "group4" belongs to switch group 1 (switchgroup 1).

[0146] "<SupplementaryDescriptor schemeIdUri="urn:brdcst:3dAudio:presetGroupId"value="preset2" / > "The description indicates that the encoded data of group 4 "group4" belongs to preset group 2 "preset2".

[0147] In the following, the description will be given of the <baseurl>"(i.e., the files contained in each track) indicates the location of the media file substance of the destination. Fig.14 As shown in (a), in the case of non-fragmented MP4, for example, the media file substance is sometimes defined as "url1". In this case, the "ftyp" box describing the file type is first arranged. The "ftyp" box indicates that the file is a non-fragmented MP4 file. Subsequently, the "moov" box and the "mdat" box are arranged. The "moov" box includes all metadata, such as header information of each track, meta description of the content substance, time information, etc. The "mdat" box includes the media data body.

[0148] like Fig.14 As shown in (b), in the case of fragmented MP4, for example, the media file substance is sometimes defined as "url 2". In this case, the "styp" box describing the fragment type is first arranged. Subsequently, the "sidx" box describing the fragment index is arranged. Subsequently, a predetermined number of movie fragments are arranged. Here, the movie fragment is configured with a "moof" box including control information and an "mdat" box including a media data body. Since the fragment obtained by sending the media in segments is included in the "mdat" box of a movie fragment, the control information included in the "moof" box is control information related to the fragment. "styp", "sidx", "moof" and "mdat" are units that constitute a fragment.

[0149] In addition, the combination of the above "url 1" and "url 2" is also considered. In this case, for example, "url 1" can be set to the initialization segment, and "url 1" and "url 2" can be set to the MP4 of a service. Alternatively, as Fig.14 As shown in (c), "url 1" and "url 2" can be combined into one and defined as "url 3".

[0150] As described above, in the "moov" box, the correspondence between the track identifier (track ID) and the level identifier (level ID) is written. Fig.15 As shown in (a), the "ftyp" box and the "moov" box constitute an initialization segment. In the "moov" box there is an "mvex" box, and in the "mvex" box there is further a "leva" box.

[0151] like Fig.15 As shown in (b), in the "leva" box, the correspondence between the track identifier (track ID) and the level identifier (level ID) is defined. In the example shown, "level0" is associated with "track0", "level1" is associated with "track1", and "level2" is associated with "track2".

[0152] Fig.16 (a) shows an example of sending each box in the case of a broadcast system. A segment is configured with a first initialization segment (is), followed by a "styp", followed by a "sidx" box, and followed by a predetermined number of movie segments (configured with a "moof" box and an "mdat" box). In the example shown, the predetermined number is one.

[0153] As described above, in the "moov" box constituting the initialization fragment (is), the correspondence between the track identifier (track ID) and the level identifier (level ID) is written. Fig.16 As shown in (b), in the "sidx" box, each track is defined by level, and there is registration range information for each track. In other words, corresponding to each level, the reproduction time information and start position information of the track are recorded in the file. In the receiving side, regarding audio, the audio stream of the desired track is selectively extracted based on the range information.

[0154] Return to reference Figure 3 The service receiver 200 receives DASH / MP4 transmitted via an RF transmission path or a communication network transmission path from the service sending system 100, which is an MP4 including an MPD file as a metafile and media streams (media segments) such as video, audio, etc.

[0155] As described above, in addition to the video stream, MP4 includes a predetermined number of tracks (audio streams) including a plurality of groups of encoded data constituting the 3D audio transmission data. Subsequently, in the MPD file, attribute information indicating each attribute of the plurality of groups of encoded data included in the 3D audio transmission data and stream correspondence information indicating in which tracks (audio streams) the plurality of groups of encoded data are respectively included are inserted.

[0156] The service receiver 200 selectively performs a decoding process on an audio stream including encoded data having a set of attributes compatible with the speaker configuration and the user selection information based on the attribute information and the stream correspondence information, and obtains an audio output of a 3D audio.

[0157] [DASH / MP4 generation unit of service delivery system]

[0158] Fig.17 There is shown an exemplary configuration of the DASH / MP4 generation unit 110 included in the service transmission system 100. The DASH / MP4 generation unit 110 includes a control unit 111, a video encoder 112, an audio encoder 113, and a DASH / MP4 formatter 114.

[0159] The video encoder 112 inputs video data SV, performs encoding such as MPEG2, H.264 / AVC, and H.265 / HEVC on the video data SV, and generates a video stream (video elementary stream). The audio encoder 113 inputs object data of immersive audio and voice dialogue and channel data as audio data SA.

[0160] The audio encoder 113 performs MPEGH encoding on the audio data SA and obtains 3D audio transmission data. Figure 5 As shown, the 3D audio transmission data includes channel coded data (CD), immersive audio object coded data (IAO) and voice dialogue object coded data (SDO). The audio encoder 113 generates one or more audio streams (audio elementary streams) including a plurality of groups (ie, four groups in this example) of coded data (see Figure 6 (a) and Figure 6 (b)).

[0161] The DASH / MP4 formatter 114 generates MP4 including media streams (media segments) of video and audio as content based on the video stream generated in the video encoder 112 and a predetermined number of audio streams generated in the audio encoder 113. Here, each video stream or audio stream is stored in MP4 as an independent track, respectively.

[0162] In addition, the DASH / MP4 formatter 114 generates an MPD file by using content metadata, segment URL information, etc. In the present embodiment, the DASH / MP4 formatter 114 inserts attribute information indicating each attribute of the plurality of groups of encoded data included in the 3D audio transmission data into the MPD file, and also inserts stream correspondence information indicating in which tracks (audio streams) the plurality of groups of encoded data are respectively included (see Fig.11 and Fig.12 ).

[0163] Will briefly explain Fig.17 The operation of the DASH / MP4 generation unit 110 is shown. The video data SV is supplied to the video encoder 112. The video encoder 112 performs encoding of H.264 / AVC, H.265 / HEVC, etc. on the video data SV, and generates a video stream including the encoded video data. The video stream is supplied to the DASH / MP4 formatter 114.

[0164] The audio data SA is provided to the audio encoder 113. The audio data SA includes channel data and object data of immersive audio and voice dialog. The audio encoder 113 performs encoding of MPEGH on the audio data SA, and obtains 3D audio transmission data.

[0165] In addition to the channel coded data (CD), the 3D audio transmission data includes immersive audio object coded data (IAO) and speech dialogue object coded data (SDO) (see Figure 5 Then, the audio encoder 113 generates one or more audio streams including four sets of encoded data (see Figure 6 (a) and Figure 6 The audio stream is provided to the DASH / MP4 formatter 114.

[0166] The DASH / MP4 formatter 114 generates MP4 including media streams (media segments) of video, audio, etc. as content based on the video stream generated in the video encoder 112 and a predetermined number of audio streams generated in the audio encoder 113. Here, each video stream or audio stream is stored in MP4 as a separate track.

[0167] In addition, the DASH / MP4 formatter 114 generates an MPD file by using content metadata, segment URL information, etc. In the MPD file, attribute information indicating each attribute of a plurality of groups of encoded data included in the 3D audio transmission data is inserted, and stream correspondence information indicating in which tracks (audio streams) the plurality of groups of encoded data are respectively included is also inserted.

[0168] [Exemplary Configuration of Service Receiver]

[0169] Fig.18 2 shows an exemplary configuration of a service receiver 200. The service receiver 200 includes a receiving unit 201, a DASH / MP4 analysis unit 202, a video decoder 203, an image processing circuit 204, a panel driving circuit 205, and a display panel 206. In addition, the service receiver 200 includes container buffers 211-1 to 211-N, a combiner 212, a 3D audio decoder 213, an audio output processing circuit 214, and a speaker system 215. In addition, the service receiver 200 includes a CPU 221, a flash ROM 222, a DRAM 223, an internal bus 224, a remote control receiving unit 225, and a remote control transmitter 226.

[0170] The CPU 221 controls the operation of each unit in the service receiver 200. The flash ROM 222 stores control software and saves data. The DRAM 223 constitutes a work area of ​​the CPU 221. The CPU 221 activates software by developing software in the DRAM 223 or data read from the flash ROM 222, and controls each unit in the service receiver 200.

[0171] The remote control receiving unit 225 receives a remote control signal (remote control code) transmitted from the remote control transmitter 226 and supplies the signal to the CPU 221. The CPU 221 controls each unit in the service receiver 200 based on the remote control code. The CPU 221, the flash ROM 222, and the DRAM 223 are connected to the internal bus 224.

[0172] The receiving unit 201 receives DASH / MP4 transmitted from the service transmission system 100 via an RF transmission path or a communication network transmission path, which is an MPD file as a metafile and MP4 including a media stream (media segment) such as video and audio.

[0173] In addition to the video stream, MP4 includes a predetermined number of tracks (audio streams) including a plurality of sets of encoded data constituting the 3D audio transmission data. In addition, in the MPD file, attribute information indicating each attribute of the plurality of sets of encoded data included in the 3D audio transmission data is inserted, and stream correspondence information indicating in which tracks (audio streams) the plurality of sets of encoded data are respectively included is also inserted.

[0174] The DASH / MP4 analysis unit 202 analyzes the MPD file and MP4 received by the reception unit 201. The DASH / MP4 analysis unit 202 extracts a video stream from the MP4, and transmits the video stream to the video decoder 203. The video decoder 203 performs a decoding process on the video stream, and obtains uncompressed video data.

[0175] The image processing circuit 204 performs scaling processing and image quality adjustment processing on the video data obtained by the video decoder 203, and obtains video data for display. The panel drive circuit 205 drives the display panel 206 based on the video data to be displayed obtained by the image processing circuit 204. The display panel 206 is configured with, for example, a liquid crystal display (LCD), an organic electroluminescent display (organic EL display), or the like.

[0176] In addition, the DASH / MP4 analysis unit 202 extracts MPD information included in the MPD file and sends the MPD information to the CPU 221. The CPU 221 controls the acquisition process of the video stream or the audio stream based on the MPD information. In addition, the DASH / MP4 analysis unit 202 extracts metadata such as header information of each track, meta description of the content essence, and time information from MP4, for example, and sends the metadata to the CPU 221.

[0177] CPU 221 identifies the audio track (audio stream) of the encoded data of the group having attributes compatible with the speaker configuration and viewer (user) selection information based on the attribute information representing the attributes of the encoded data of each group and the stream correspondence information representing which audio track (audio stream) each group is included in the MPD file.

[0178] In addition, under the control of the CPU 221, in other words, the DASH / MP4 analysis unit 202 refers to the level ID (levelID), the track ID (track ID), and selectively extracts one or more audio streams including encoded data of a group having attributes compatible with the speaker configuration and viewer (user) selection information from a predetermined number of audio streams included in the MP4.

[0179] The container buffers 211-1 to 211-N respectively input each audio stream extracted by the DASH / MP4 analysis unit 202. Here, the number N of the container buffers 211-1 to 211-N is a necessary and sufficient number, and in actual operation, the number is equal to the number of audio streams extracted in the DASH / MP4 analysis unit 202.

[0180] The combiner 212 reads the audio stream of each audio frame from the container buffers 211-1 to 211-N to which each audio stream extracted by the DASH / MP4 analysis unit 202 is input, and provides the encoded data of the group having attributes compatible with the speaker configuration and the viewer (user) selection information to the 3D audio decoder 213.

[0181] The 3D audio decoder 213 performs a decoding process on the encoded data provided from the combiner 212, and obtains audio data to drive each speaker of the speaker system 215. Here, the encoded data on which the decoding process is performed may have three cases, which are a case including only channel encoded data, a case including only object encoded data, and a case including both channel encoded data and object encoded data.

[0182] When decoding the channel coded data, the 3D audio decoder 213 obtains audio data to drive each speaker by performing downmixing and upmixing for the speaker configuration of the speaker system 215. In addition, when decoding the object coded data, the 3D audio decoder 213 calculates speaker rendering (mixing ratio for each speaker) based on the object information (metadata), and according to the calculation result, mixes the audio data of the object to the audio data for driving each speaker.

[0183] The audio output processing circuit 214 performs necessary processing such as D / A conversion, amplification, etc. on the audio data for driving each speaker obtained from the 3D audio decoder 213, and provides the data to the speaker system 215. The speaker system 215 includes a plurality of speakers such as a plurality of channels (e.g., 2 channels, 5.1 channels, 7.1 channels, 22.2 channels, etc.).

[0184] Will explain Fig.18 The operation of the service receiver 200 shown in FIG. 1 is performed by the receiving unit 201. In the receiving unit 201, DASH / MP4, which is an MPD file as a metafile and MP4 including a media stream (media segment) such as video and audio, is received from the service delivery system 100 via an RF transmission path or a communication network transmission path. The MPD file and MP4 received in this way are provided to the DASH / MP4 analysis unit 202.

[0185] In the DASH / MP4 analysis unit 202, the MPD file and the MP4 received in the receiving unit 201 are analyzed. Subsequently, in the DASH / MP4 analysis unit 202, a video stream is extracted from the MP4, and the video stream is sent to the video decoder 203. In the video decoder 203, a decoding process is performed on the video stream, and uncompressed video data is obtained. The video data is supplied to the image processing circuit 204.

[0186] In the image processing circuit 204, scaling processing, image quality adjustment processing, etc. are performed on the video data obtained in the video decoder 203, and video data to be displayed is obtained. The video data to be displayed is supplied to the panel drive circuit 205. In the panel drive circuit 205, the display panel 206 is driven based on the video data to be displayed. With this configuration, on the display panel 206, an image corresponding to the video data to be displayed is displayed.

[0187] Furthermore, in the DASH / MP4 analysis unit 202, MPD information included in the MPD file is extracted and sent to the CPU 221. Furthermore, in the DASH / MP4 analysis unit 202, metadata such as header information of each track, meta description of the content essence, time information, etc. are extracted from MP4, and the metadata is sent to the CPU 221. In the CPU 221, based on the attribute information, stream correspondence information, etc. included in the MPD file, an audio track (audio stream) in which the encoded data of the group has an attribute compatible with the speaker configuration and the viewer (user) selection information is identified.

[0188] In addition, under the control of the CPU 221, in the DASH / MP4 analysis unit 202, one or more audio streams of encoded data including a group of attributes compatible with the speaker configuration and viewer (user) selection information are selectively extracted from a predetermined number of audio streams included in the MP4 by referring to the track ID (trackID).

[0189] The audio stream extracted in the DASH / MP4 analysis unit 202 is input to a corresponding container buffer among the container buffers 211-1 to 211-N. In the combiner 212, the audio stream is read from each audio frame of the container buffer to which the audio stream is input, and is provided to the 3D audio decoder 213 via the audio stream as encoded data of a set of attributes compatible with the speaker configuration and viewer selection information. In the 3D audio decoder 213, a decoding process is performed on the encoded data provided by the combiner 212, and audio data for driving each speaker of the speaker system 215 is obtained.

[0190] Here, when the channel coded data is decoded, down-mixing and up-mixing processing for the speaker configuration of the speaker system 215 is performed, and audio data for driving each speaker is obtained. In addition, when the object coded data is decoded, the speaker rendering (mixing ratio for each speaker) is calculated based on the object information (metadata), and according to the calculation result, the audio data of the object is mixed to the audio data for driving each speaker.

[0191] The audio data for driving each speaker obtained in the 3D audio decoder 213 is supplied to the audio output processing circuit 214. In the audio output processing circuit 214, necessary processing such as D / A conversion, amplification, etc. is performed on the audio data for driving each speaker. Then, the processed audio data is supplied to the speaker system 215. With this configuration, a sound output corresponding to the display image of the display panel 206 is obtained from the speaker system 215.

[0192] Fig.19 Show Fig.18 1 is an example of an audio decoding control process by the CPU 221 in the service receiver 200 shown. In step ST1, the CPU 221 starts the process. Then, in step ST2, the CPU 221 detects the speaker configuration of the receiver, that is, the speaker configuration of the speaker system 215. Next, in step ST3, the CPU 221 obtains selection information about audio output from the viewer (user).

[0193] Next, in step ST4, the CPU 221 reads information about each audio stream of the MPD information, which is "groupID", "attribute", "switchGroupID", "presetGroupID", and "levelID". Subsequently, in step ST5, the CPU 221 identifies the track ID (track ID) of the audio track to which the encoded data group having attributes compatible with the speaker configuration and viewer selection information belongs.

[0194] Next, in step ST6, the CPU 221 selects each track based on the recognition result and inputs the stored audio stream to the container buffer. Subsequently, in step ST7, the CPU 221 reads the audio stream for each audio frame from the container buffer and provides the necessary set of encoded data to the 3D audio decoder 213.

[0195] Next, in step ST8, the CPU 221 determines whether to decode the object coded data. In step ST9, when decoding the object coded data, the CPU 221 calculates the speaker rendering (mixing rate for each speaker) by using the azimuth (azimuth information) and the altitude (elevation information) based on the object information (metadata). Thereafter, the CPU 221 proceeds to the processing in step ST10. Here, when the object coded data is not decoded in step ST8, the CPU 221 immediately proceeds to the processing in step ST10.

[0196] In step ST10, the CPU 221 determines whether to decode the channel coded data. In step ST11, when the channel coded data is decoded, the CPU 221 performs the processing of downmixing and upmixing for the speaker configuration of the speaker system 215, and obtains the audio data for driving each speaker. Thereafter, the CPU 221 proceeds to the processing in step ST12. Here, when the object coded data is not decoded in step ST10, the CPU 221 immediately proceeds to the processing in step ST12.

[0197] In step ST12, when the object coded data is decoded, the CPU 221 mixes the audio data of the object to the audio data for driving each speaker according to the calculation result in step ST9, and then performs dynamic range control. Subsequently, in step ST13, the CPU 221 ends the processing. Herein, when the object coded data is not decoded, the CPU 221 skips the processing in step ST12.

[0198] As mentioned above, in Figure 3 In the transceiver system 10 shown, the service delivery system 100 inserts attribute information indicating each attribute of the plurality of groups of encoded data included in a predetermined number of audio streams into the MPD file. Therefore, the receiving side can easily identify each attribute of the plurality of groups of encoded data before decoding the encoded data, so that the necessary groups of encoded data can be selectively decoded and used, and this can reduce the processing load.

[0199] In addition, Figure 3 In the transceiver system 10 shown, the service delivery system 100 inserts stream correspondence information indicating which audio tracks (audio streams) the encoded data of the plurality of groups are respectively included in into the MPD file. Therefore, the receiving side can easily identify the audio tracks (audio streams) including the encoded data of the necessary groups, and the processing load can be reduced.

[0200] <2. Modifications>

[0201] Herein, in the above embodiment, the service receiver 200 is configured to selectively extract an audio stream of encoded data of a group having attributes compatible with the speaker configuration and viewer selection information from a plurality of audio streams sent by the service sending system 100, and obtain audio data for driving a predetermined number of speakers by performing decoding processing.

[0202] Herein, as a service receiver, it can be regarded as selectively extracting one or more audio streams including encoded data of a group having attributes compatible with the speaker configuration and viewer selection information from a plurality of audio streams transmitted by the service transmission system 100, reconfiguring the audio streams of the encoded data of the group having attributes compatible with the speaker configuration and viewer selection information, and distributing the reconfigured audio streams to devices (including DLNA devices) connected to the internal network.

[0203] Fig. 20 An exemplary configuration of the service receiver 200A that distributes the reconfigured audio stream to the devices connected to the internal network as described above is shown. Fig. 20 In the same reference numerals as in Fig.18 , and their detailed explanation will be omitted as needed.

[0204] Under the control of CPU 221, the DASH / MP4 analysis unit 202 refers to the level ID, i.e., the track ID, and selectively extracts one or more audio streams including encoded data of a group having attributes compatible with the speaker configuration and viewer (user) selection information from a predetermined number of audio streams included in MP4.

[0205] The audio stream extracted in the DASH / MP4 analysis unit 202 is input to a corresponding container buffer among the container buffers 211-1 to 211-N. In the combiner 212, the audio stream for each audio frame is read from each container buffer to which the audio stream is input, and the audio stream is provided to the stream reconfiguration unit 231.

[0206] In the stream reconfiguration unit 231, a predetermined set of encoded data having attributes compatible with the speaker configuration and the viewer selection information is selectively acquired, and an audio stream having the predetermined set of encoded data is reconfigured. The reconfigured audio stream is provided to the transmission interface 232. Subsequently, the reconfigured audio stream is transmitted (sent) from the transmission interface 232 to the device 300 connected to the internal network.

[0207] Internal network connections include Ethernet connections and wireless connections such as "WiFi" and "Bluetooth." As used herein, "WiFi" and "Bluetooth" are registered trademarks.

[0208] Furthermore, the apparatus 300 includes surround sound speakers, a second display, and an audio output device attached to the network terminal. The apparatus 300 receives the transmission of the reconfigured audio stream and performs the same Fig.18 The 3D audio decoder 213 in the service receiver 200 performs a decoding process similar to that in the 3D audio decoder 213 in the service receiver 200, and obtains audio data for driving a predetermined number of speakers.

[0209] In addition, the service receiver may have a configuration for transmitting the above-mentioned reconfigured audio stream to a device connected by a digital interface such as "High-Definition Multimedia Interface (HDMI)", "Mobile High-Definition Link (MHL)", "DisplayPort", etc. Herein, "HDMI" and "MHL" are registered trademarks.

[0210] Furthermore, the above embodiment describes an example in which a field of "attribute" is provided and attribute information of encoded data of each group is transmitted (see Figures 11 to 13 ). However, the present technology includes a specific method of defining the value of the group ID (GroupID) itself between the transmitter and the receiver so that the type (attribute) of the encoded data can be identified by identifying the specific group ID. In this case, in addition to being used as an identifier of the group, the group ID is used as attribute information of the encoded data of the group, and the "attribute" field is not required.

[0211] Furthermore, the above-mentioned embodiment describes an example in which channel coded data and object coded data are included in coded data of a plurality of groups (see Figure 5 ). However, the present technology can be applied in a similar manner to a case where the coded data of a plurality of groups includes only channel coded data or only object coded data.

[0212] Herein, the present technology may have the following configurations.

[0213] (1) A sending device, comprising:

[0214] a transmitting unit configured to transmit a metafile having meta information for obtaining a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device; and

[0215] The information inserting unit is configured to insert attribute information indicating each attribute of the plurality of sets of encoded data into the metafile.

[0216] (2) The transmission device according to (1), wherein the information inserting unit further inserts stream correspondence information into the metafile, the stream correspondence information indicating in which audio stream the plurality of groups of encoded data are respectively included.

[0217] (3) The transmission device according to (2), wherein the stream correspondence information is information indicating a correspondence relationship between a group identifier that respectively identifies each of the plurality of groups of encoded data and an identifier that respectively identifies each of a predetermined number of audio streams.

[0218] (4) The transmission device according to any one of (1) to (3), wherein the metafile is an MPD file.

[0219] (5) The transmission device according to (4), wherein the information inserting unit inserts the attribute information into the metafile by using a “supplemental descriptor”.

[0220] (6) The transmission device according to any one of (1) to (5), wherein the transmission unit transmits the metafile via an RF transmission path or a communication network transmission path.

[0221] (7) The transmission device according to any one of (1) to (6), wherein the transmission unit further transmits a container of a predetermined format having a predetermined number of audio streams including a plurality of groups of encoded data.

[0222] (8) The transmission device according to (7), wherein the container is MP4.

[0223] (9) The transmission device according to any one of (1) to (8), wherein the plurality of groups of coded data include one or both of channel coded data and object coded data.

[0224] (10) A sending method, comprising:

[0225] a sending step of sending, by a sending unit, a meta file having meta information for obtaining a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device; and

[0226] The information inserting step inserts attribute information indicating each attribute of the plurality of sets of encoded data into the metafile.

[0227] (11) A receiving device comprising:

[0228] a receiving unit configured to receive a metafile having meta information for obtaining a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device,

[0229] Attribute information indicating each attribute of the plurality of sets of encoded data is inserted into the metafile; and

[0230] The processing unit is configured to process a predetermined number of audio streams based on the attribute information.

[0231] (12) The receiving device according to (11),

[0232] in,

[0233] Stream correspondence information indicating in which audio stream the coded data of the plurality of groups are respectively included is further inserted into the metafile, and

[0234] The processing unit processes a predetermined number of audio streams based on the stream correspondence information and the attribute information.

[0235] (13) The receiving device according to (12), wherein the processing unit selectively performs decoding processing on the audio stream including the encoded data having a group of attributes compatible with the speaker configuration and the user selection information based on the attribute information and the stream correspondence information.

[0236] (14) The receiving device according to any one of (11) to (13), wherein the plurality of groups of coded data include one or both of channel coded data and object coded data.

[0237] (15) A receiving method, comprising:

[0238] a receiving step of receiving, by a receiving unit, a metafile having meta information, the meta information being used to obtain, in a receiving device, a predetermined number of audio streams including a plurality of groups of encoded data,

[0239] Attribute information indicating each attribute of the plurality of sets of encoded data is inserted into the metafile; and

[0240] A processing step is performed to process a predetermined number of audio streams based on the attribute information.

[0241] (16) A receiving device comprising:

[0242] a receiving unit configured to receive a metafile having meta information for obtaining a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device,

[0243] Attribute information representing each attribute of the plurality of groups of encoded data is inserted into the metafile;

[0244] a processing unit configured to selectively acquire a predetermined group of encoded data from a predetermined number of audio streams based on the attribute information, and reconfigure an audio stream including the predetermined group of encoded data; and

[0245] The stream sending unit is configured to send the reconfigured audio stream to an external device.

[0246] (17) The receiving device according to (16), wherein:

[0247] Stream correspondence information indicating in which audio stream the coded data of the plurality of groups are respectively included is further inserted into the metafile, and

[0248] The processing unit selectively acquires a predetermined group of encoded data from a predetermined number of audio streams based on the stream correspondence information and the attribute information.

[0249] (18) A receiving method, comprising:

[0250] a receiving step, in which a receiving unit receives a metafile having meta information, the meta information being used to obtain a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device,

[0251] inserting attribute information representing each attribute of the plurality of groups of encoded data into the metafile;

[0252] a processing step of selectively acquiring a predetermined group of encoded data from a predetermined number of audio streams based on the attribute information, and reconfiguring the audio stream including the predetermined group of encoded data; and

[0253] The stream sending step sends the reconfigured audio stream to an external device.

[0254] The main feature of the present technology is to insert attribute information indicating respective attributes of a plurality of groups of encoded data included in a predetermined number of audio streams and stream correspondence information indicating in which tracks (audio streams) the plurality of groups of encoded data are respectively included into an MPD file (see Fig.11 , Fig.12 and Fig.17 ) to reduce the processing load on the receiving side.

[0255] Explanation of symbols

[0256] 10 Transceiver System

[0257] 30A, 30B MPEG-DASH based streaming system

[0258] 31 DASH streaming file server

[0259] 32 DASH MPD Server

[0260] 33, 33-1 to 33-N) Service receiver

[0261] 34 CDN

[0262] 35, 35-1 to 35-M) Service Receiver

[0263] 36 Broadcasting System

[0264] 100 Service Delivery System

[0265] 110 DASH / MP4 generation unit

[0266] 112 Video Encoder

[0267] 113 Audio Codec

[0268] 114 DASH / MP4 formatter

[0269] 200 Service Receiver

[0270] 201 Receiving unit

[0271] 202 DASH / MP4 analysis unit

[0272] 203 Video Decoder

[0273] 204 Image Processing Circuit

[0274] 205 Panel drive circuit

[0275] 206 Display Panel

[0276] 211-1 to 211-N Container Buffer

[0277] 212 Combiner

[0278] 213 3D Audio Decoder

[0279] 214 Audio output processing circuit

[0280] 215 Speaker System

[0281] 221 CPU

[0282] 222 Flash ROM

[0283] 223 DRAM

[0284] 224 Internal bus

[0285] 225 Remote Control Receiver Unit

[0286] 226 Remote Control Transmitter

[0287] 231 Stream Reconfiguration Unit

[0288] 232 Transmission Interface

[0289] 300 devices.< / baseurl> < / baseurl>

Claims

1. A sending device, comprising: a sending unit configured to send a metafile having meta information for acquiring a predetermined number of audio streams including a plurality of groups of encoded data from a communication network or a broadcast; as well as an information inserting unit configured to insert into the metafile attribute information indicating each attribute of the plurality of groups of encoded data and stream correspondence information indicating in which audio stream the plurality of groups of encoded data are respectively included, Among them, the multiple groups of encoded data include encoded data of a group having attributes compatible with the speaker configuration of the speaker system and the user's selection information, so that the receiving device can decode the encoded data of the group having attributes compatible with the speaker configuration of the speaker system and the selection information based on the attribute information and the stream correspondence information.

2. The transmitting device according to claim 1, wherein: The stream correspondence information is information indicating a correspondence relationship between a group identifier that respectively identifies each of the plurality of groups of encoded data and an identifier that respectively identifies each of the predetermined number of audio streams.

3. A sending method, comprising: a sending step of sending, by a sending unit, a metafile having meta information for acquiring a predetermined number of audio streams including a plurality of groups of encoded data from a communication network or a broadcast; as well as an information inserting step of inserting attribute information indicating each attribute of the plurality of groups of encoded data and stream correspondence information indicating in which audio stream the plurality of groups of encoded data are respectively included into the metafile, Among them, the multiple groups of encoded data include encoded data of a group having attributes compatible with the speaker configuration of the speaker system and the user's selection information, so that the receiving device can decode the encoded data of the group having attributes compatible with the speaker configuration of the speaker system and the selection information based on the attribute information and the stream correspondence information.

4. A receiving method, comprising: a receiving step of receiving, by a receiving unit, a meta file having meta information for acquiring a predetermined number of audio streams including a plurality of groups of encoded data from a communication network or a broadcast, Inserting into the metafile attribute information indicating each attribute of the plurality of groups of coded data, stream correspondence information indicating in which audio stream the plurality of groups of coded data are respectively included, and preset identification information, the preset identification information including a preset identifier associated with a specific group identifier, the preset identifier identifying a preset group set of coded data associated with a specific group of coded data, the specific group identifier identifying the specific group of coded data; as well as A processing step of decoding the encoded data of the group having the attribute compatible with the speaker configuration and selection information of the speaker system based on the attribute information and the stream correspondence information.

5. A receiving device, comprising: a receiving unit configured to receive a meta file having meta information for acquiring a predetermined number of audio streams including a plurality of groups of encoded data from a communication network or a broadcast, Inserting into the metafile attribute information indicating each attribute of the plurality of groups of coded data, stream correspondence information indicating in which audio stream the plurality of groups of coded data are respectively included, and preset identification information, the preset identification information including a preset identifier associated with a specific group identifier, the preset identifier identifying a preset group set of coded data associated with a specific group of coded data, the specific group identifier identifying the specific group of coded data; a processing unit configured to selectively obtain a predetermined group of encoded data from the predetermined number of audio streams based on the attribute information, and reconfigure an audio stream including the predetermined group of encoded data; as well as The stream sending unit is configured to send the reconfigured audio stream to an external device.

6. A receiving method, comprising: a receiving step, in which a receiving unit receives a metafile having meta information, wherein the meta information is used to obtain a predetermined number of audio streams including a plurality of groups of encoded data in a receiving device, Inserting into the metafile attribute information indicating each attribute of the plurality of groups of coded data, stream correspondence information indicating in which audio stream the plurality of groups of coded data are respectively included, and preset identification information, the preset identification information including a preset identifier associated with a specific group identifier, the preset identifier identifying a preset group set of coded data associated with a specific group of coded data, the specific group identifier identifying the specific group of coded data; a processing step of selectively acquiring a predetermined group of encoded data from the predetermined number of audio streams based on the attribute information, and reconfiguring the audio stream including the predetermined group of encoded data; as well as The stream sending step sends the reconfigured audio stream to an external device.

Images