METHOD FOR RECEIVING DATA IN A MULTIMEDIA TRANSMISSION SYSTEM

Description

Technical Field

[0001] The present disclosure relates to a multimedia transmission system, and more particularly, to a method for receiving multimedia dataBackground Art

[0002] The term 'multimedia service' refers to call services such as Video Call, streaming services such as Video on Demand (VOD) service, or services such as Multicast and Broadcast Service. Real-time multimedia services may be divided into call services, interactive services and streaming services depending on the service type, and may also be divided into unicast, multicast and broadcast depending on the number of participating users.

[0003] The conventional broadcast network usually uses Moving Picture Experts Group-2 Transport Stream (MPEG-2 TS), for transmission of multimedia content. MPEG-2 TS is used as a typical transmission technology for transmitting a bit stream, with which multiple broadcast programs (or multiple coded video bit streams) are multiplexed, in an error-ridden transmission environment. MPEG-2 TS is suitable for use in digital TV broadcasting or the like in the multimedia age.

[0004] MPEG-2 TS may have a few limitations in supporting multimedia services. In other words, MPEG-2 TS may have limitations such as the one-way communication, the inefficiency of transmission due to the fixed frame size, and the unnecessary overhead which occurs when data is transmitted using the transport protocol and Internet Protocol (IP) that are specific to audio / video.

[0005] Therefore, MPEG has newly proposed an MPEG Media Transport (MMT) standard as one of the multimedia transmission technologies for supporting multimedia services based on the MPEG technology. For example, the MMT standard may be applied to efficiently transmit hybrid content over the heterogeneous network. The term `hybrid content' as used herein may refer to a collection of content having multimedia elements by video / audio / application. The heterogeneous network may be, for example, a network in which a broadcasting network and a mobile communication network coexist.

[0006] Document US 2011 / 0255558 A1 relates to an interface for providing a media service, including an encapsulation layer for encapsulating coded media data; a delivery layer for transmitting the encapsulated media data to another entity; and a control layer for controlling transmission of the media data.

[0007] Document "Technology under Considerations on MPEG Media Transport" from MPEG Meeting; 28-11-2011 - 2-12-2012; held in Geneva, discloses a method for providing "composition information" in layer E. 1, by combining MMT Assets into MMT Packages.

[0008] Document "Contribution to MMT EE#8 (D.1 Layer Design)" of Kyungmo Park et al, 98. MPEG Meeting; 28-11-2011 - 2-12-2011; held in Geneva, discloses an encapsulation layer design (E.3) to receive MMT Packages and produce MMT payload units that can be transported to the other end using an application transport protocol and / or underlying network transport protocols.

[0009] The MMT standard aims at defining an IP-friendly transmission technology which is the basic technology in a transmission network for multimedia services. To this end, there is a need for a structure of an MMT transport packet for more efficiently delivering logical data objects in a variety of formats, including coded media data, in a changing multimedia service environment.DisclosureTechnical Problem

[0010] An aspect of an embodiment of the present disclosure is to provide a method, according to independent claim 1, for efficiently receiving coded media data in a multimedia transmission system.

[0011] Further embodiments are set out in the dependent claims.Technical Solution

[0012] Embodiments of the present disclosure provide a method for receiving media content in a multimedia system supporting a moving picture experts group, MPEG media transport, MMT, protocol as set out in the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 illustrates a hierarchical structure of an MMT system according to an example not including all the claimed feature. FIG. 2 illustrates a structure of an MMT payload according to an example not including all the claimed feature. FIG. 3 illustrates a logical structure of an MMT package according to an example not including all the claimed feature. FIG. 4 illustrates a flowchart for configuring and transmitting a multimedia data packet including an MMT payload in an MMT system according to an embodiment of the present disclosure. FIG. 5 illustrates an operation of generating a multimedia data packet according to an embodiment of the present disclosure. Best Mode

[0014] Now, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description of the present disclosure, a detailed description of known functions and configurations incorporated herein will be omitted when it may obscure the subject matter of the present disclosure. The terms used herein are defined in consideration of their functions in the present disclosure, and may vary depending on the users, the operator's intention, or the usual practices. Therefore, the definition should be made based on the contents throughout the entire specification.

[0015] A hierarchical structure defined in the MMT standard will be described herein below. Embodiments of the present disclosure provide the structure of a multimedia data packet. To this end, a detailed description will be made of layers for generating a multimedia data packet in the hierarchical structure defined in the MMT standard. The term 'multimedia data packet' as used herein may refer to a delivery frame for an MMT service.

[0016] An MMT Protocol (MMTP) defines an application layer protocol for delivering a delivery frame configured in an MMT Payload Format (PF) over an IP network. An MMT payload is designed to be efficiently delivered by being configured in an MMT payload format. To efficiently deliver MPEG media data over heterogeneous IP networks, MMT defines encapsulation formats, delivery protocols, and signaling message formats.

[0017] FIG. 1 illustrates a hierarchical structure of an MMT system according to an example not including all the claimed feature.

[0018] Referring to FIG. 1, shown are a media coding layer 110, an encapsulation function layer (Layer E) 120, a delivery function layer (Layer D) 130, a transport protocol layer 140, an Internet Protocol (IP) layer 150, and a control function layer (Layer C) 100, which are for configuring a multimedia data packet and transmitting the configured multimedia data packet.

[0019] In accordance with an embodiment of the present disclosure, the media coding layer 110 and the encapsulation function layer 120 may operate as a multimedia data generation unit for generating multimedia data that is based on multimedia content and / or multimedia services. The delivery function layer 130 may operate as a multimedia data configuration unit for configuring a multimedia data packet based on the multimedia data received from the multimedia data generation unit. The delivery function layer 130 corresponding to the multimedia data configuration unit may configure header information by identifying at least one multimedia data provided from the multimedia data generation unit, and configure a multimedia data packet by combining the header information with the least one multimedia data.

[0020] The multimedia data compressed in the media coding layer 110 may be packetized in the form similar to a file format by the encapsulation function layer 120. The encapsulation function layer 120 may generate data segments, which are small units for an MMT service, by receiving the coded media data provided from the media coding layer 110 or the stored media data, and generate access units for an MMT service using the data segments. The encapsulation function layer 120 may generate a packet format for generation / storage and transmission of hybrid content by combining and / or dividing the access units.

[0021] The delivery function layer 130 may convert a data unit(s) output from the encapsulation function layer 120 into an MMT payload format and then, add an MMT transport packet header thereto to configure an MMT transport packet, or may configure a Real-Time Protocol (RTP) packet using an RTP which is the existing transport protocol.

[0022] The packets configured in the delivery function layer 130 may be finally IP-packetized in the IP layer 150 after undergoing the transport protocol layer 140 such as User Datagram Protocol (UDP) or Transport Control Protocol (TCP). The transport protocol layer 140 and the IP layer 150 may operate as a data transmission unit. The control function layer 100, which is optional, may generate control information or signaling information necessary for transmission of data, add the generated information to data, and transmit the data, or may transmit the data through a separate signaling means.

[0023] An MMT payload format generated in the delivery function layer 130 may define a logical structure of a media unit(s) to be delivered, by means of the MMT protocol or the RTP. An MMT payload may be specified by a payload format for delivering the encapsulated data unit or other information by MMT layer protocols or other existing application transport protocols. The MMT payload may provide information about streaming and information about file transfer. In streaming, a data unit may be an MMT Media Fragment Unit (MFU) or an MMT Processing Unit (MPU). In file transfer, a data unit may be an MMT asset and an MMT package.

[0024] FIG. 2 illustrates a structure of an MMT payload according to an example not including all the claimed feature.

[0025] Referring to FIG. 2, an MMT payload 200 is configured to include at least one of at least one MMT MFU 210, at least one MMT MPU 220, at least one MMT asset 230, and at least one MMT package 240.

[0026] The MFU 210 may be a common container format including coded media data, which is independent of any media codec, and can be independently processed by a media decoder. The MFU 210, which represents part of segmented data of the MPU 220, may be the minimum unit that can be decoded independently. As an example, if coding is performed by using one frame as an access unit, the MFU 210 may be one video frame. In another case, the MFU 210 may be one slice included in one frame.

[0027] The MPU 220, which is a container format including one or more MFUs and additional delivery and processing-related information, may include a variety of numbers of MFUs generated from a plurality of different access units. The MPU 220, which means a coded media data unit that can be completely and independently processed by an MMT compliant entity, may have a specific size (e.g., one Group of Picture (GOP) in the case of video) depending on the application environment. As an example, the MPU 220 may have (or consist of) a plurality of picture frames constituting one GOP (e.g., 1-second video), and the PFU 210 may include each picture frame.

[0028] The MMT asset 230, which is a data entity having one or more MPUs, may be the largest data unit, to which the same composition information and transport characteristics are applied. The MMT asset 230 may include only one type of data, including packaged or multiplexed data. For example, at least a part of an Elementary Stream (ES) of audio, at least a part of an ES of video, at least a part of an MPEG-U (User Interface) widget package, at least a part of an MPEG-2 Transport Stream (TS), at least a part of an MPEG-4 (MP4) file, and all or at least a part of an MMT package may be each MMT asset 230.

[0029] The ES, which is defined by a specific media codec, may be logically one or more MMT assets. The MMT asset 230 supporting a layered codec and a multi-view codec may overlap with other MMT assets.

[0030] MMT Composition Information (MMT-CI) means the information defining a spatial and temporal relationship of MMT assets, and MMT Transport Characteristics (MMT-TC) defines Quality of Service (QoS) required for delivery of MMT assets. MMT-TC may be expressed as Asset Delivery Characteristics (ADC), for a specific delivery environment.

[0031] The MMT package 240 may be defined as a collection of coded media data and its related information, which are processed by an MMT compliant entity.

[0032] FIG. 3 illustrates a logical structure of an MMT package according to an example not including all the claimed feature. As illustrated, an MMT package 300 may have (or consist of) one or more MMT assets 320, MMT composition information 310, and one or more ADCs 330 representing MMT transport characteristics. The MMT package 240 may include description information such as an identifier, and locations of the MMT assets 320, and the MMT assets 320 in the MMT package 240 may be multiplexed or concatenated.

[0033] The MMT package 240 may be processed in units of MPUs, and the MMT asset 320 may be a collection of one or more MPUs having the same MMT asset ID. The transport characteristics related to each MMT asset 320 may be expressed by the ADC 330. The ADC 330 may be used by an entity of packetizing an MMT package, to configure parameters of an MMT payload and below-described header information of an MMT packet.

[0034] Among the logical entities that can be included in the MMT package, the MPU may include header fields that are based on the given definition and some of these fields may be essential for processing the MPU. In a case where the outer parts of the boundaries of MFUs in a payload of MPUs are cut off, if the packet is lost, an error may be propagated longer. In media data requiring a low delay, an MFU needs to be immediately transmitted at the time of its generation. In this case, to prevent the transmission delay, the transmission order may be determined to transmit header field information of the MPU after transmitting the generated PFU data.

[0035] Given the above characteristics, an MMT payload needs to be configured to maintain the compatibility between MPU packets and MFU packets.

[0036] FIG. 4 illustrates a flowchart for configuring and transmitting a multimedia data packet including an MMT payload in an MMT system according to an embodiment of the present disclosure. The operation shown in FIG. 4 is performed by the delivery function layer 130 among the layers in FIG. 1.

[0037] Referring to FIG. 4, in operation 410, the delivery function layer 130 receives multimedia data provided from the encapsulation function layer 120. The encapsulation function layer 120 provides multimedia data necessary to configure a multimedia data packet, and information related to each multimedia data.

[0038] In operation 420, the delivery function layer 130 configures payload header information related to an MMT payload of the multimedia data packet based on the header information of the multimedia data. As an example, the payload header information may provide a length of an MMT payload, a type of payload data, and information related to fragmentation and aggregation, and the payload header format may be defined by the system operator or the protocol standard.

[0039] In operation 430, the delivery function layer 130 generates a payload of a multimedia data packet for delivery of multimedia data. In other words, the delivery function layer 130 configures an MMT payload by combining at least a part of the multimedia data received in operation 410, with the payload header information configured in operation 420.

[0040] In operation 440, an MMT packet, which is a multimedia data packet including the MMT payload, may be may be transmitted to a counterpart entity (e.g., a receiver) through a given transport protocol.

[0041] A description will now be made of a detailed procedure for configuring a multimedia data packet by a delivery function layer in charge of transmission of multimedia data for MMT services, which is proposed in an embodiment of the present disclosure, and also made of a payload format. Specifically, a transport data structure that is based on the type of the data generated in the encapsulation function layer 120 may be provided. For configuration of a multimedia data packet, payload header information to be recorded in a header region and multimedia data to be recorded in a payload region may be generated according to a given format.

[0042] FIG. 5 illustrates an operation of generating a multimedia data packet according to an embodiment of the present disclosure.

[0043] Referring to FIG. 5, an MPU file 510 means data that the encapsulation function layer desires to transmit, and an MPU / MFU building block 520 has a data structure for configuring transmission data. The MPU file 510 may include header information 512 such as a File Type (FTYP) box indicating a type of a file, an MMPU box indicating a structure of an MPU, an MOOV box indicating codec setting information, and Movie Fragment (MOOF) box; and an MDAT box 514 having Video Samples (VSs) 516 and MMT Hint Samples (HSs) 518. The MMPU box indicates that a type of the data stored in the MDAT box is MPU, and the MOOV box contains information about a media frame stored in the MDAT box. The MPU / MFU building block 520 may include MPU metadata 522 such as FTYP, MMPU, MOOV and MOOF, and MFUs 524 having related HSs 526 and VSs 528.

[0044] An MMT payload may be configured based on the MPU / MFU building block 520, and carried or transmitted in an MMT packet.

[0045] A detailed procedure for configuring an MMT payload will be as follows. A data configuration unit may analyze data 510 in the form of an MPU file, which is provided from a data generation unit and desired to be transmitted, to determine type information, configuration information of an MPU, setting information of a codec, starting point and size (or length) information of data, from the header information 512 of the MPU data 510. The data configuration unit may analyze the structure of MFUs (corresponding to VSs in FIG. 5) 516 and MFU HSs 518 constituting the MPU data 510, to determine an MFU header containing the size, mutual importance and interconnection (length or the like) information of each MFU, and MFU data which is actual media data, and may also analyze the size of each MFU through the MFU HS 518 indicating the starting offset and length of each MFU.

[0046] Based on the analyzed MPU format, the MPU / MFU building block 520 for configuring a transport packet may be configured. The MFU HS 518 may be an MFU header of the MPU / MFU building block 520.

[0047] The data configuration unit may configure each MFU as a transport unit based on the size of each MFU, which is analyzed from the MFU HS 518. In this case, depending on the size of the MFU, one or multiple payload may be configured. The start, middle or end of each payload may be indicated by start_end_indicator which is optional bits. The MPU metadata 522 may also be treated as one configuration unit of the MPU / MFU building block 520.

[0048] The data type of each payload element may be presented as follows. i. The type of one payload 522 having FTYP, MMPU, MOOV, MOOF and MDAT is called MPU_Partial(), and may be configured to include only the MPU header. ii. If one MFU is configured as one payload, the type of the payload is called MFU_set (). iii. In a case where one MFU is configured as multiple payloads, the type of each payload is MFU_Partial() if the MFU includes only the MFU header, and the type of each payload is MFU_chunk() if the MFU includes an MFU header and part of the MFU data, or includes part of the MFU data. iv. MPU_Partial() may means the type of a payload that includes only the MPU header. v. MPU_Set() may mean the type of a payload that completely includes an MPU.

[0049] Shown below is syntax of an MMT payload according to an embodiment of the present disclosure. The below MMT payload format may be used for UDP transport, exchange of point-to-point chunks, and transport of files.

[0050] As described above, an MMT payload may the MMT payload may include a type field indicating a data type of the payload, and asset_id indicating an identifier of an asset to which the payload (MPU or MFU) belongs. The asset_id may exist to multiplex an MMT bit stream. The asset_id may be omitted, if the MMT payload is delivered through the RTP and a single asset is used.

[0051] Depending on the type field, the following various cases of elements may be included in the MMT payload.

[0052] case 0 may be used if the MMT payload contains configuration data of the data to be transmitted. If asset_id=0, the MMT payload may include package configuration data. Otherwise, asset configuration data may be included.

[0053] An MFU_Partial() element corresponding to case 1 may be used when the MMT payload carries configuration data (header information) of the MFU data. An MFU_chunk() element corresponding to case 2 may be used when the MMT payload contains part of the MFU data together with configuration data (header information) of the MFU data, or carries all or part of the MFU data. An MFU_set() element corresponding to case 3 may be used when the MMT payload contains all (header and data) of the MFU data. An MPU_Partial() element corresponding to case 4 may be used when the MMT payload contains the MPT data. An MPU_Set() element corresponding to case 5 may be used when the MMT payload contains all (header and data) of the MPU data.

[0054] Additionally, the MMT payload may further include other units such as a file unit, signaling information of the control layer, and a protection message.

[0055] MFU_Partial() may be defined to be a set of all of headers that are added to a start of payload bytes and defined in the MFU, when the size of the MFU exceeds the size of the Maximum Transmission Unit (MTU).

[0056] MFU_chunk() may be defined to be a middle or a last part of a payload of the MFU for the headers transmitted in the MFU_Partial() packets.

[0057] MFU_set() may be defined to be a packet that includes a predetermined number of complete MFUs.

[0058] MPU_Partial() may be defined to be a set of all of headers that are added to the additional MFU_set element that is added to the optional MFU_Partial() element, and that are defined in the MPU. The presence / absence of the additional elements may be determined depending on the remaining bytes and the MFU length of the packet.

[0059] A description will now be made of examples of the MPU and MFU elements, together with the mapping between the MMT payload format and the MMT transport packet. A description of the parameter fields that are known in the art or that have nothing to do with embodiments of the present disclosure will be omitted herein, but it should be noted that such parameter fields do not limit the scope of the present disclosure.

[0060] The following represents an example of an MFU element.

[0061] In the above example, start_end_indicator may be optional bits indicating that the payload is a start, middle or end of an MFU. Additionally, start_end_indicator may indicate that the MFU includes Random Access Point (RAP) data, and may include optimal bits indicating a start and / or end position where the data is to be decoded. rap fragment indicator may be optional bits indicting an address of a start point of the data that can be directly decoded, if the payload contains a start point of the randomly accessible data.

[0062] scrambling_key_index may contain information about a series of values capable of decrypting encryption, if the payload is encrypted.

[0063] layer dependency_info may be information indicating the dependency between picture layers. Additionally, it may further include layer identifiers (LayerIDs), a dependency count, signal rolling keys similar to a MPEG-2 TS packet, and the like.

[0064] The following represents an example of an MPU element.

[0065] In the above example, start_end_indicator may be optional bits indicating that the payload is a start, middle or end of an MPU. multiple mfu flag may be set to '1', if the MPU has multiple MFUs. no_mfu may be set to '1', if no MFU is included in the MPU data like in the case of file transfer.

[0066] timing flag may be used to correct the time stamp by indicating whether the current MFU belongs to the same access unit as the previous one, if the previous packet is lost. codec_update_flag may be set to '1', if the payload has codec configuration information, header first may be set to '1', if the payload starts at the header information of the media data. event list flag may be set, if the payload has an event list to be represented for the user. The event list may list events belonging to the content.

[0067] multiple_aus_flag may be set to '1', if the payload contains multiple frames. If a codec such as Advanced Audio Coding (AAC) is used, units for in-band codec configuration may be included in the MPU element.

[0068] codec_update_flag may include codec_config_size indicating the size of codec configuration information and codec_config data indicating the codec configuration information. In an alternative embodiment, if a codec is changed based on the codec configuration information or if the codec configuration information is modified, at the beginning or in the middle of the multimedia streaming service, the presence / absence of the information may be set through the codec_update_flag. downcounter may be used to indicate that a few more payloads containing the codec configuration information exist after the current payload, if the codec configuration information is divided into multiple payloads. As an example, if the codec configuration information is divided into multiple segments and downcounter is set to '3', a receiver may determine that payloads having downcounters of 2, 1 and 0 are further later required to complete the codec configuration information. Upon receiving a payload of downcounter 0, the receiver may determine that the transmission of all payloads related to the codec configuration information is completed.

[0069] If event list flag is set, an event list to be delivered through a transmission unit may exist, and the event list may include num_events indicating the number of events and at least one event data. In an alternative embodiment, if the event data that has already been transmitted is used, the event may be expressed, by including only the event identifier `event ID' instead of the event data. In another alternative embodiment, event data may be transmitted in advance using a predetermined transport means before transmission of the payload. Listed events may be executed after being synchronized with the display of the frame. In a possible embodiment, audio may be optionally changed through the event list during playback of video data.

[0070] If multiple mfu flag is set, num_MFUs indicating how many MFUs the payload has may exist in the MPU element. If multiple mfu flag is not set, num_MFUs is set to '1', and header_first and multiple_au flag are set to '0'. In a possible embodiment, if a multimedia data unit desired to be transmitted has multiple frames, or multiple slices in a single frame, the information for setting the number of multiple entities may be set.

[0071] If timing flag is set, timing information for units in the MPU may be included in the MPU element.

[0072] Additionally, as many MFUs as num_MFUs may be listed in the MPU element. Since the order of MFUs may be different from the order of decoder inputs, decode_order number indicating the order that the MFU is input to the decoder may be optionally included.

[0073] If multiple mfu flag is set to '1', mfu_size indicating the size of each MFU may be further included. In this specification, it is assumed that the MFUs that are input to the data configuration unit have the same size.

[0074] If headers_first is not set to '1', one mmt_mfu_element() may be included.

[0075] If headers_first is set to '1', as many mmt_mfu_element() as num_MFUs may be included in the MPU element.

[0076] The receiver configured in accordance with an embodiment of the present disclosure may include a reception unit configured to receive a multimedia data packet from a counterpart entity, and an interpretation unit configured to extract and interpret at least one MFU or MPU included in the multimedia data packet or at least a part thereof depending on the above-described format of the MMT payload.

[0077] The above-described format of the MMT payload may enable the efficient delivery and aggregation / fragmentation of MPUs and MFUs.

[0078] While the disclosure has been shown and described with reference to certain exemplary implementations thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein, provided that these various changes fall within the scope of the invention as defined by the appended claims.

Claims

1. A method for receiving media content in a multimedia system supporting a moving picture experts group, MPEG media transport, MMT, protocol, the method comprising: receiving, by a delivery function layer (130) of the system, one or more MMT packets generated based on a media processing unit, MPU, (220), the MPU being fragmented into one or more media fragment units, MFUs, (210), each MMT packet including a MMT packet header and a MMT payload (200); and decoding, by the delivery function layer (130, the one or more MMT packets, wherein payload data included in the MMT payload comprises one of MPU metadata of the MPU (220) or media data element derived from the one or more MFUs (210), wherein each of the one or more MMT packets comprises a data type field indicating that the respective payload data comprises the MPU metadata of the MPU (220) in case that the MMT payload comprises the MPU metadata or indicating that the respective payload data comprises the media data element derived from the one or more MFUs (210) in case that the MMT payload comprises the media data element, wherein the MPU metadata of the MPU (220) comprises file type information of a file to be transmitted, wherein the data type field is set to a first value indicating that the respective payload data includes the MPU metadata of the MPU if the respective payload data includes the MPU metadata of the MPU, and wherein the data type field is set to a second value indicating that the respective payload data includes the media data element derived from the one or more MFUs if the payload data includes the media data element derived from the one or more MFUs.

2. The method of claim 1, wherein the MPU metadata of the MPU comprises information regarding configuration of the one or more MFUs.

3. The method of claim 1, wherein the MPU metadata of the MPU comprises a MOOV box having codec configuration information and a MMPU box being placing after the file type information to provide information regarding to the MPU.

4. The method of claim 1, wherein each of the one or more MMT packets comprises a field indicating whether timing information is included in the payload data or not.

5. The method of claim 1, wherein each of the one or more MMT packets comprises information identifying an asset to which data included in the payload data belongs.