Method and apparatus for controlling transmission and reception of content in a communication system

Abstract

The present disclosure relates to a pre-fifth generation (5G) or 5G communication system that will be provided to support higher data rates than fourth generation (4G) communication systems (such as long term evolution (LTE)). A method for receiving media content by a client is provided. The method includes receiving a media presentation description (MPD) including content type information of three-dimensional (3D) media content and configuration type information of the 3D media content from a server, accessing segments of the 3D media content based on the information included in the MPD, and receiving the 3D media content based on the segments. The content type information may indicate an identifier of the 3D media content, and the configuration type information may indicate whether the quality is adjustable for each of the content components of the 3D media content.

Description

Technical Field

[0001] The present disclosure relates to an adaptive streaming technology for three-dimensional (3D) media in a communication system. More specifically, the present disclosure relates to an apparatus and method for efficiently transmitting and receiving three-dimensional (3D) media having properties and structures different from those of 2D media in a communication system. Background Art

[0002] To meet the demand for increased wireless data traffic since the deployment of 4G (fourth generation) communication systems, efforts have been made to develop improved 5G (fifth generation) or pre-5G communication systems. Therefore, 5G or pre-5G communication systems are also called "beyond 4G networks" or "post-LTE systems."

[0003] 5G communication systems are expected to be implemented in higher-frequency (mmWave) bands (e.g., the 60 GHz band) to achieve higher data rates. To reduce radio wave propagation losses and increase transmission distances, beamforming, massive multiple-input multiple-output (MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, and massive antenna technologies are being discussed in 5G communication systems.

[0004] In addition, in 5G communication systems, system network improvements are being developed based on advanced small cells, cloud radio access networks (RAN), ultra-dense networks, device-to-device (D2D) communication, wireless backhaul, mobile networks, collaborative communications, coordinated multi-point (CoMP), and receiver-side interference cancellation.

[0005] In 5G systems, hybrid FSK and QAM modulation (FQAM) and sliding window superposition coding (SWSC) have been developed as advanced coded modulation (ACM), and filter bank multi-carrier (FBMC), non-orthogonal multiple access (NOMA), and sparse code multiple access (SCMA) as advanced access technologies.

[0006] Related art media streaming services provide content representing the service (e.g., a movie) and content components (e.g., video, audio, and subtitles) that constitute the content. Media streaming services can also provide bitstreams encoded with multiple different parameters and a manifest file that provides main attribute information about the bitstream.

[0007] The manifest file provides parameters necessary for receiving a bitstream and a uniform resource locator (URL) for receiving the parameters, and the parameters may be parameters for encoding or parameters describing properties of content components before and after encoding.

[0008] The media streaming service may be provided to the terminal so that the terminal may adaptively select and receive appropriate content or all (or some) of the content according to the performance of the terminal or the real-time environment of the network.

[0009] Examples of such media streaming services include Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol (HTTP) (MPEG-DASH) (ISO / IEC 23009-1), 3rd Generation Partnership Project (3GPP)-DASH (TS26.247), and Real Time Streaming Protocol (RTSP) (RFC 2326).

[0010] The above information is presented as background information only to assist in understanding the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art to the present disclosure. Summary of the Invention

[0011] Technical issues

[0012] Related art media streaming services assume the transmission of two-dimensional (2D) media. Therefore, a new communication scheme between a content provider server and a receiving terminal is required to transmit media such as three-dimensional (3D) media, which has a different structure and properties from 2D media.

[0013] Solution

[0014] Aspects of the present disclosure are to address at least the above-mentioned problems and / or disadvantages and to provide at least the advantages described below. Therefore, one aspect of the present disclosure is to provide an apparatus and method for efficiently transmitting and receiving three-dimensional (3D) media having properties and structures different from those of 2D media in a communication system.

[0015] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0016] According to one aspect of the present disclosure, a method for receiving media content by a client is provided. The method includes receiving a media presentation description (MPD) including content type information and configuration type information of 3D media content from a server, accessing segments of the 3D media content based on the information included in the MPD, and receiving the 3D media content based on the segments.

[0017] According to an embodiment of the present disclosure, the content type information may indicate an identifier of the 3D media content, and the configuration type information may indicate a one-multiplexed content component (OMCC) in which the content consists of one content component or a separately adaptable content component (SACC) in which the quality of each content component is adjustable.

[0018] According to an embodiment of the present disclosure, content type information may be included as an attribute in a subset element or a preselected element as a media content unit included in an MPD.

[0019] According to an embodiment of the present disclosure, configuration type information may be included as an attribute in a subset element or a preselect element as a media content unit included in an MPD.

[0020] According to an embodiment of the present disclosure, when the configuration type information indicates SACC, the MPD may further include role information of each of a plurality of content components, and the role information may be included as an attribute in an AdaptationSet element which is a content component unit.

[0021] According to an embodiment of the present disclosure, the MPD may further include quality priority information of at least one content component. The quality priority information may be included as an attribute in a subset element or a preselect element as a media content unit included in the MPD, or may be included as an attribute in an AdaptationSet element as a content component unit included in the MPD.

[0022] According to an embodiment of the present disclosure, the MPD may further include resolution information of at least one content component. The resolution information may include information required to convert 3D media content into two-dimensional (2D) media and may be included as an attribute in an AdaptationSet element, which is a content component unit included in the MPD.

[0023] According to an embodiment of the present disclosure, the MPD may further include quality assessment information of the final 3D media corresponding to the combination of content components. The quality assessment information may be included as an attribute in a subset element or a preselected element of a media content unit included in the MPD.

[0024] According to another aspect of the present disclosure, a method for providing media content by a server is provided. The method includes sending a media presentation description (MPD) including content type information of 3D media content and configuration type information of the 3D media content to a client, receiving a request for a segment of the 3D media content from the client, and sending the 3D media content to the client.

[0025] According to another aspect of the present disclosure, a client terminal configured to receive media content is provided. The client terminal includes a transceiver and at least one processor, the processor being configured to control and receive an MPD including content type information and configuration type information of 3D media content from a server, access segments of the 3D media content based on the information included in the MPD, and receive the 3D media content based on the segments.

[0026] According to another aspect of the present disclosure, a server configured to provide media content is provided. The server includes a transceiver and at least one processor, the processor being configured to control the sending of an MPD including content type information of 3D media content and configuration type information of the 3D media content to a client, receiving a request for a 3D media content segment from the client, and sending the 3D media content to the client.

[0027] Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various embodiments of the disclosure.

[0028] Beneficial effects

[0029] An aspect of the present disclosure is to provide an apparatus and method for efficiently transmitting and receiving three-dimensional (3D) media having properties and structures different from those of 2D media in a communication system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0031] Figure 1 is a diagram illustrating a communication system according to an embodiment of the present disclosure;

[0032] Figure 2 is a block diagram illustrating a service provider (or content provider) according to an embodiment of the present disclosure;

[0033] Figure 3 is a block diagram illustrating a client terminal according to an embodiment of the present disclosure;

[0034] Figure 4 is a flowchart illustrating a method for playing media content by a client according to an embodiment of the present disclosure; and

[0035] Figure 5 is a flowchart illustrating a method for providing media content by a server according to an embodiment of the present disclosure.

[0036] Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures. DETAILED DESCRIPTION

[0037] The following description, with reference to the accompanying drawings, is provided to facilitate a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist understanding, but these are to be regarded as exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, descriptions of well-known functions and structures may be omitted for clarity and conciseness.

[0038] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustration purposes only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.

[0039] It is to be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.

[0040] The embodiments of the present disclosure may also be applied to communication systems having similar technical backgrounds, with minor changes made without significantly departing from the scope of the present disclosure, and as determined by a person skilled in the art to which the present disclosure pertains. As used herein, the term "communication system" includes a broadcast system, but when a broadcast service is the primary service, the communication system may be clearly referred to as a broadcast system.

[0041] The advantages and features of the present disclosure and the methods for achieving these advantages and features can be understood by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed herein, and various changes may be made thereto. The embodiments disclosed herein are provided solely to inform those of ordinary skill in the art of the scope of the present disclosure. The present disclosure is limited only by the appended claims. Throughout the specification, the same reference numerals represent the same elements.

[0042] The methods described below in conjunction with the embodiments are based on hardware. However, the embodiments of the present disclosure include techniques using both hardware and software, and therefore do not exclude software-based methods.

[0043] As used herein, the terms representing components of signals, files, network entities, service providers or media content provider devices, and media processing terminals are provided as examples for ease of description. The present disclosure is not limited to these terms, and other terms equivalent in technical concepts may also be used.

[0044] According to an embodiment of the present disclosure, a method for identifying three-dimensional (3D) media having a different structure or different role from two-dimensional (2D) media, providing relationship / spatial configuration information for it, assigning a role for it, considering its priority, and providing it with expected quality after reception according to related art is proposed.

[0045] According to an embodiment of the present disclosure, a service provider, a content provider, and a (media reception) terminal may adaptively process 3D media (eg, provide, transmit, or play 3D media).

[0046] According to an embodiment of the present disclosure, 3D media can be configured by combining 2D media. According to an embodiment of the present disclosure, a terminal can receive and play 2D media using a 3D player proposed in the present disclosure instead of a 2D player of the related art, even though the configuration of the 3D media provided by the service provider (or content provider) is 2D media.

[0047] According to an embodiment of the present disclosure, 3D media may be provided in a format different from that of 2D media as all 2D media are multiplexed with one content component, or may be provided as each 2D media is mapped to one content component.

[0048] According to an embodiment of the present disclosure, a service provider, a content provider, and a terminal can exchange at least one of 3D media provision format information and 3D media configuration information among each other. The service provider, the content provider, and the terminal can exchange information for adaptively transmitting / receiving each component of the 3D media according to a network environment, and can exchange information required for post-processing (e.g., scaling) and using the adaptively transmitted / received components.

[0049] According to an embodiment of the present disclosure, a service provider, a content provider, and a terminal can exchange information for inferring the final quality of the 3D media determined for each quality / combination of components of the 3D media when adaptively transmitting / receiving each component of the 3D media according to a network environment. The terminal can receive (or process) a combination for ensuring the minimum quality of key items as determined by a 3D application and a 3D player.

[0050] Figure 1 is a diagram illustrating a communication system according to an embodiment of the present disclosure.

[0051] refer to Figure 1 , the communication system 10 may include a service provider or content provider 100 , a network 110 and a client terminal 120 .

[0052] The service provider or content provider 100 can adaptively provide media content-related services to the client terminal 120 according to the network environment and terminal capabilities.

[0053] According to an embodiment of the present disclosure, the service provider or content provider 100 may be implemented as a server or may be referred to as a server.

[0054] The media content can be 3D augmented reality (AR), mixed reality (MR), extended reality (XR), or virtual reality (VR) media. The media content can include some 2D media in 3D media, or can be composed of a combination of 2D media.

[0055] The 3D or 2D media segments 101 and the manifest information 103 may be provided so as to provide the 2D media segments 101 of the service provider or content provider 10 to provide a media content service. The 3D or 2D media segments may be composed of content and content components constituting the content.

[0056] The network 110 may relay between the service provider or content provider 100 and the client terminal 120 , such that the service provider or content provider 100 may provide media content to the client terminal 120 .

[0057] The client terminal 120 can adaptively receive and play media content provided by the service provider or content provider 100. The client terminal 120 may also be referred to as a client or a terminal.

[0058] Figure 2 is a block diagram illustrating a service provider (or content provider) according to an embodiment of the present disclosure.

[0059] refer to Figure 2 , the service provider or content provider 100 may include a transceiver 102, a memory 104, and a controller 105. The transceiver 102, the memory 104, and the controller 105 may operate according to a communication method of the service provider or content provider 100, which will be described below.

[0060] However, the components of the service provider or content provider 100 are not limited to the above examples. For example, the service provider or content provider 100 may include more or fewer components than the aforementioned components. The transceiver 102, memory 104, and controller 105 may be implemented in the form of a single chip. The controller 105 may include one or more processors.

[0061] The transceiver 102 is generally referred to as a transmitter and a receiver, and can transmit and receive signals to / from a client terminal. These signals may include control information and content-related data. To this end, the transceiver 102 may include a radio frequency (RF) transmitter for up-converting and amplifying transmit signals and an RF receiver for low-noise amplifying receive signals and down-converting the receive signal frequency. However, this is merely an example of the transceiver 102, and the components of the transceiver 102 are not limited to an RF transmitter and an RF receiver.

[0062] The memory 104 may store programs and data required for the operation of the service provider or content provider 100. The memory 104 may store control information or content-related data included in a signal obtained by the service provider or content provider 100. The memory 104 may include a storage medium such as a read-only memory (ROM), a random access memory (RAM), a hard disk, a compact disc (CD)-ROM, and a digital versatile disc (DVD), or a combination of storage media. The memory 104 may be embedded in the controller 105 rather than provided separately.

[0063] According to the embodiments described below, the controller 105 can control a series of processes to operate the service provider or content provider 100. For example, the controller 105 can receive control signals and data signals through the transceiver 102 and process the received control signals and data signals. The controller 105 can transmit the processed control signals and data signals through the transceiver 102. A plurality of controllers 105 may be provided. The controller 105 can control the components of the service provider or content provider 100 by executing a program stored in the memory 104.

[0064] The controller 105 may control sending a media presentation description (MPD) including content type information of 3D media content and configuration type information of 3D media content to the client, control receiving a request for a 3D media content segment from the client, and control sending the 3D media content to the client.

[0065] The content type information may indicate an identifier of 3D media, and the configuration type information may indicate one of a single multiplexed content component (OMCC) in which content consists of one content component and a separately adaptable content component (SACC) in which quality is adjustable for each content component.

[0066] Figure 3 is a block diagram illustrating a client terminal according to an embodiment of the present disclosure.

[0067] refer to Figure 3, the client terminal 120 may include a transceiver 121, a display 122, a memory 123, a processor or controller 124, an external space identifier 125, a 2D media player 126, and a 3D media player 127. The transceiver 121, the display 122, the memory 123, the controller 124, the external space identifier 125, the 2D media player 126, and the 3D media player 127 may operate according to the communication method of the client terminal 120 described below.

[0068] However, the components of the client terminal 120 are not limited thereto. For example, the client terminal 120 may include more or fewer components than those described above.

[0069] According to one embodiment, the client terminal 120 may include a plurality of identical components 121 to 127 inside or outside the client terminal 120 .

[0070] The client terminal 120 may select and receive all (or some) of the appropriate content components according to the network environment or terminal capabilities.

[0071] The transceiver 121 is generally referred to as a transmitter and a receiver, and can send and receive signals to / from a service provider, content provider, or server. These signals may include control information, content request information, and content. To this end, the transceiver 121 may include a radio frequency (RF) transmitter for up-converting and amplifying the transmitted signal, and an RF receiver for low-noise amplification and down-conversion of the received signal. However, this is merely an example of the transceiver 121, and the components of the transceiver 121 are not limited to an RF transmitter and an RF receiver.

[0072] The display 122 may play and display media content received by the client terminal 120 from the service provider or content provider 100 .

[0073] The memory 123 can store programs and data required to operate the client terminal 120. The memory 123 can store control information or content-related data included in the signal obtained by the client terminal 120. The memory 123 can include a storage medium such as ROM, RAM, a hard disk, a CD-ROM, and a DVD, or a combination of storage media. The memory 123 can be embedded in the controller 124 rather than provided separately.

[0074] The controller 124 can control a series of processes to operate the client terminal 120 according to various embodiments described below. For example, the controller 124 can receive control signals and data signals through the transceiver 121 and process the received control signals and data signals. The controller 124 can also transmit the processed control signals and data signals through the transceiver 121. Multiple controllers 124 may be provided. The controller 124 can control the components of the client terminal 120 by executing programs stored in the memory 123.

[0075] The client terminal 120 may display the received media content on a virtual space and match the virtual space with the space identified by the external space identifier 121 , thereby providing the user of the client terminal 120 with an effect as if the media content is displayed on an actual external space.

[0076] The 2D media player 126 may control and allow the 2D media content provided from the service provider or content provider 100 to be played on the client terminal 120 .

[0077] The 3D media player 127 may control to allow 3D media content provided from the service provider or content provider 100 to be played on the client terminal 120 .

[0078] DASH stands for Dynamic Adaptive Streaming over HTTP. DASH can divide the content and representation obtained by encoding the same content components under different conditions (for example, network bandwidth, resolution, or codec) into sufficiently short segments (for example, 1 to 10 seconds), and then provide a multimedia presentation description (used as a manifest file) that specifies the file name, sequence number, start time, and length of each segment.

[0079] These segments consist of an initialization segment with information for decoder configuration to correctly decode the corresponding representation, and a media segment including segmented content components.

[0080] The terminal requests an MPD, receives the MPD from the service provider (or content provider), identifies what representations are provided, then determines, for example, the performance or capabilities of the client terminal and the network speed, selects the appropriate representation, and downloads and plays the segments sequentially.

[0081] If a variable such as network bandwidth is changed, for example if the network speed increases, segments of a representation created with a higher bandwidth may be selected and played, and thus may be played seamlessly, despite changes in image quality depending on the network speed.

[0082] By selection by a user or a client terminal, a change in the available language and media segment combination can be selected from the attribute information of the available adaptation combination set (adaptationSet), and playback can be performed.

[0083] Real Time Streaming Protocol (RTSP) is a protocol standardized by the Internet Engineering Task Force (IETF) for sending and receiving multimedia data such as video and audio data in real time. It is a protocol for remotely controlling a server that provides media data to send and receive streaming data, and a protocol for remotely performing control operations such as "play" and "pause".

[0084] According to the present disclosure, a scheme for configuring 1) a 3D media identifier, 2) a 3D content configuration type, and 3) 3D media configuration information is proposed as a scheme for efficiently transmitting and receiving 3D media having properties and structures different from 2D media in a communication system.

[0085] 3D Media Identifier

[0086] According to the present disclosure, a method for a terminal to identify whether the media provided by a service provider is 3D media or 2D media according to the related art is proposed.

[0087] Related technologies such as DASH do not require separate information indicating whether media is 2D media because all image media are 2D media.

[0088] In related art techniques, the manifest provides the type of content component (eg, whether the content component is video or audio) or compression information (eg, what video encoder has been used for encoding, eg, High Efficiency Video Coding (HEVC)).

[0089] Therefore, a terminal using the related art knows in advance that video and audio content components are required to constitute the content, although there is no separate indication of the content structure of the 2D media, and the terminal can select the adaptation transmission group [adaptationSet] provided for each content component and receive the stored representation file segment through the encoder supported therein. As used herein, the term "representation" refers to the format of the representation content, and there can be high image quality representation and high image quality representation.

[0090] According to the present disclosure, a method for transmitting and receiving new media is provided, which is configured in a composite manner by using content and content components of 2D media of related technologies for different purposes and allows a specific player to provide a new multimedia experience.

[0091] In other words, a method is proposed for identifying that content is new media that cannot be processed (or consumed) by a 2D media player of the related art, allowing an appropriate player (e.g., a 3D media player) to process the media, and supporting the content components of the 3D media player to partially use the 2D media player. The related art is, for example, point cloud compression (VPCC) or mesh media based on ISO / IEC23090-5 video (here, VPCC media may include three or more 2D video content components to perform different roles by decomposing 3D media and metadata related to the three or more 2D video content components. Mesh media may be composed of surface color information and metadata of points, lines, and surfaces by decomposing 3D media).

[0092] According to the present disclosure, a method is proposed for explicitly assigning an identifier to a media type on a per-content or per-content component basis via a manifest to identify whether the provided media is 3D media or 2D media. In a terminal according to the present disclosure, a player capable of playing the media is associated with playback of the media based on the assigned media identifier.

[0093] When a manifest includes multiple adaptation sets (AdaptationSet), in the concept of the present disclosure, each adaptation set can correspond to a content component. In addition, the subset or presentation that is bundled together for playback for a specified adaptation set can correspond to the content.

[0094] @contentType exists as an attribute of AdaptationSet. @contentType is specified as a format identifier representing the content component provided by the AdaptationSet (23009-1 Version 4, Section 5.3.3).

[0095] Therefore, the problem to be solved according to the present disclosure, for example, processing (or consumption) of 3D content composed of 2D content components by a 2D player, may not be solved simply by specifying a type on a per-content component basis.

[0096] According to the present disclosure, since a media type identifier is specified in units of content or content components as described above, it is proposed to add @contentType as an attribute to a content unit of the related art, ie, a subset or a preselected element.

[0097] According to the present disclosure, an appropriate player is selected according to @contentType, and playback of an element including @contentType is performed according to the playback process of the player, thereby preventing an inappropriate player from playing content or content components.

[0098] Therefore, according to an embodiment of the present disclosure, the terminal can first check whether there are subsets or preselections in the received list, and if there are one or more subsets or presentations, search for @contentType and perform a comparison as to whether the @contentType value matches at least one in the list of identifiers supported by the player equipped in the terminal.

[0099] If the @contentType value matches at least one of the list of identifiers supported by the player equipped in the terminal, the terminal can activate the player, transmit the subset or preselection, and receive one of the subset or preselection selected by the player or the user operating the player.

[0100] According to another embodiment of the present disclosure, the terminal may check whether there is a subset or preselection in the received list, and if there is no subset or preselection in the received list, search for @contentType in the AdaptationSet, perform a comparison as to whether the @contentType value in the AdaptationSet matches at least one in the list of identifiers supported by the player equipped in the terminal, and if the @contentType value in the AdaptationSet matches at least one in the list of identifiers, activate the player and transmit the AdaptationSet.

[0101] Table 1 shows an example including a subset of contentType, and Table 2 shows an example including a preselection of contentType.

[0102] [Table 1]

[0103]

[0104] [Table 2]

[0105]

[0106] Although the terminal is capable of playing 3D media, when an identifier describing provided content or content component is not in a format recognizable by the terminal, the terminal may search manifest information for playable content or content component having another identifier.

[0107] Although the terminal is capable of playing 3D media, when the identifier describing the provided content or content component is not in a format recognizable by the terminal, the terminal may play 2D media, receive a program / plug-in / library for supplementing the terminal's player based on the identifier information and play it, or may search for an alternative manifest provided from a server for playable content or content component having another identifier.

[0108] In the search step, the method for providing the alternative list is as follows.

[0109] A content provider can provide a URL list by specifying multiple base URLs based on content delivery networks (CDNs) in different geographic locations, allowing adaptive content receivers to receive content as quickly as possible. However, in the related art, there is no reason to express what different attributes each link has in the base URL list, as it is assumed that the same content as the manifest file received based on the base URL is sent.

[0110] According to an embodiment of the present disclosure, a content provider and a terminal having an adaptive content receiver include one or more 3D media identifiers described in a manifest file received using a base URL as attributes of the base URL. Thus, the adaptive content receiver identifies the identifiers of playable 3D media by communicating with a content player and receives the base URL providing the identifiers.

[0111] Table 3 shows an example of a base URL according to the related art, and Table 4 shows an example of a method for providing an MPD including other types of 3D media.

[0112] [Table 3]

[0113]

[0114] [Table 4]

[0115]

[0116] Configuration type of 3D content

[0117] VPCC can decompose 3D media and compose three or more 2D video content components playing different roles and related metadata, and mesh can decompose 3D media and compose surface color information and metadata from points, lines, and surfaces.

[0118] In new media that provide new multimedia experiences, such as VPCC or mesh, one content is composed of one or more content components, and the plurality of content components are multiplexed and can eventually be recognized as a new format.

[0119] For example, a VPCC may consist of three or more 2D video content components playing different roles and associated metadata (i.e., four or more content components in total), and may include one content component that is a video-based coding (V3C) bitstream in which all components have been multiplexed into one visual volume.

[0120] For example, mesh media may consist of a metadata content component having information of points, lines, and surfaces and a texture image content component corresponding to surface color, and may have only one multiplexed bitstream as content components in which the content components are compressed and combined.

[0121] Obviously, for adaptive transmission when the content consists of only one multiplexed bitstream content component, the image quality adjustment options for several conditions are applied individually to all pre-multiplexed content components, and finally form the target bitrate of the multiplexed bitstream. This is referred to as a single multiplexed content component (OMCC) configuration.

[0122] However, when the configuration information of the content is provided as an identification unit of content components, a target bit rate for each content component can be provided for adaptive transmission. This is referred to herein as SACC configuration.

[0123] The advantage of the OMCC configuration is that a target bit rate can be selected to meet bandwidth requirements based on the network environment, and content can be received adaptively. In other words, when a content component is received with a target bit rate lower than the available bandwidth, all content components belonging to that content can be encoded and multiplexed with interoperable quality. In addition, because all content components are time-synchronized within the received segment, no separate information is required for synchronization (or synchronization).

[0124] The SACC configuration advantageously adjusts the reception quality of each content component, ultimately adjusting the content quality, based on more flexible application usage. For example, when a building is represented as a 3D object and transmitted, the 3D object can be displayed at a sufficient level, while maintaining its geometric detail, despite relatively reduced color information. For example, in the case of clothing with complex patterns in product information for online shopping, the clothing pattern can be displayed with high image quality while the geometric details can be simplified.

[0125] According to an embodiment of the present disclosure, when displaying a content list in a manifest, in addition to identifier information of the type of appropriate player and media, an OMCC or a SACC may be separately displayed according to a content component configuration method.

[0126] In other words, in a method for providing content information supporting OMCC and SACC, a manifest may include content information, the content information may have at least two items, and in a configuration method attribute for the item, one may be indicated as OMCC and the other may be indicated as SACC. In this case, when receiving and playing media, the terminal may select a multiplexed component or select one component from among several components, wherein the selection may be based on quality and received according to application usage.

[0127] Table 5 below shows an example of indicating OMCC or SACC respectively in a subset of the manifest.

[0128] [Table 5]

[0129]

[0130] 3D media configuration information

[0131] According to the present disclosure, a method for identifying a type of each content component for a terminal receiving content representing 3D media and content components of the content according to use of an application in a SACC configuration that can provide a target bit rate for each content component is provided.

[0132] According to this disclosure, a method for specifying each role at the content component level is proposed. In VPCC, there are four main types of content components: geometry, attributes, occupancy, and atlas. Therefore, it is possible to specify that content is VPCC and that the content component is one of the four types.

[0133] Specifically, geometry and attributes can include one content component and two content components, respectively, and one geometry can be represented by one layer and one geometry can be represented by two layers. In this case, the two layers can be included in one content component, or each of the two layers can be included in one of the two content components.

[0134] Therefore, since there may be various combinations of roles that one content component has, it is allowed to define a role scheme and use a combination of one or more of the values ​​defined by the scheme.

[0135] @schemeIdUri is used to identify the role scheme, which is used to identify the role of the content component. An adaptation set or a content component can be assigned to multiple roles within the same scheme.

[0136] Table 6 shows example scenarios, and Table 7 shows example roles.

[0137] [Table 6]

[0138]

[0139] [Table 7]

[0140]

[0141] As described above, since an AdaptationSet has multiple representations, and each representation has a different target bitrate, the AdaptationSet may ultimately provide a group of multiple representations that provides adaptively acceptable choices for components, such as an AdaptationSet.

[0142] According to the present disclosure, in order for a terminal to receive 3D media, when providing different target bit rates, it is necessary to use the representation provided in the AdaptationSet in combination with the characteristics of the 3D media.

[0143] In other words, since some components affect the number of points, some components affect image quality, and other components affect the quality of the entire object, it is necessary to properly predict the influence range of each component, the number of 3D constituent units finally obtained, and the image quality.

[0144] In VPCC technology, when 3D media is converted into 2D media, the best part in the form of fragments is projected onto each surface of a virtual cube surrounding the 3D media according to the 3D constituent units (points, lines or surfaces) of the 3D media in the virtual cube.

[0145] In this case, the color information of the projected 3D building blocks is the attribute, the distance between the 3D building blocks and the cube surface is the geometry, the division between areas of the cube surface with projection information and areas without such information is the occupancy map, and the metadata required to reconstruct the 3D media back into 3D and spatially construct it is the atlas data.

[0146] For example, as the quality of the transferred geometry decreases, the distance information from the cube surface to the 3D component becomes more uncertain, and the reconstructed 3D component may increase due to errors, or if overlapped, they may decrease. In addition, due to the decrease in distance accuracy between the original and reconstructed results, evaluation criteria such as the distance between the original and reconstructed points and the orthogonal distance between the planes containing the original and reconstructed points will be scored as low.

[0147] For example, as the quality of the content component of the transmission attribute decreases, the information about the color assigned to the reconstructed 3D component loses detail. In addition, due to the reduction in color accuracy between the original and reconstructed results, evaluation criteria such as color difference in the YUV color system between the original and reconstructed points will be rated as low.

[0148] For example, as the quality of the content component of the transmitted occupancy map decreases, it becomes unclear what parts of the video image of geometry and attributes are information to be reconstructed and what parts should not be reconstructed. As a result, 3D components that should be reconstructed may not be reconstructed, or vice versa, resulting in the reconstruction of 3D media that contains a lot of noise overall.

[0149] Given these characteristics, in the case of VPCC, it can be recognized that the factors affecting 3D media quality are different for each content component, and the degree of their influence also varies. More generally, for content with multiple types of content components as components, it is recommended to score and list the content components and prioritize their impact. More specifically, it is recommended to specify the performance elements of each content component and describe the resulting score.

[0150] According to embodiments of the present disclosure, the occupancy map content component that constitutes the VPCC content affects whether the reconstructed 3D media includes information read from the geometry and attribute content components, depending on its quality. Therefore, even if high-quality geometry and attributes are received, it is difficult to avoid noise generated by a low-quality occupancy map.

[0151] Furthermore, even after receiving a high-quality occupancy map content component, if low-quality geometry is received, the positions of the points may change overall, and thus the shape of the reconstructed 3D media may change. As described above, since the SACC is provided to allow for different qualities of content components according to the use case of the application, the priority of the content components can be evaluated differently depending on the application.

[0152] Therefore, a method for providing the quality priority (@qualityPriority) of included content components at the content level (Table 8) or providing its own quality priority at the content component level (Table 9) is disclosed.

[0153] Table 8 shows an example of providing a subset of quality priorities at the content level, while Table 9 shows an example of providing a subset of quality priorities at the content component level.

[0154] [Table 8]

[0155]

[0156] [Table 9]

[0157]

[0158] In particular, when the same priority value is assigned to content components, it can be understood that they have the same priority. Since the content component of layer 0 including geometry and the content component of layer 1 including geometry are AdaptationSet1 No. 1 and AdaptationSet1 No. 2 in Table 9, respectively, the same value can be assigned to content components of the same type.

[0159] According to the present disclosure, it is proposed to specify factors that influence the quality of the 3D media to be reconstructed in terms of adaptive selection (ie at the representation level) and to describe the resulting score.

[0160] When describing the following factors representing levels, since not all cases of combinations of various content components are included, only the degree or level of influence that is widely present can be described, i.e., levels 1 to 5 or best / good / normal / poor / worst, or high / medium / low.

[0161] Table 10 shows the quality factor for each content component type, and Table 11 shows an example of applying the quality factor at the presentation level.

[0162] [Table 10]

[0163]

[0164] [Table 11]

[0165]

[0166] According to another embodiment of the present disclosure, it is possible to indicate how much the representation has changed relative to the original representation. The 2D video generated when converting 3D media to 2D media is a video obtained by removing one axis from the XYZ coordinate system of the 3D media through projection and moving it to a 2D plane (e.g., XY or YZ plane) in a one-to-one correspondence.

[0167] Therefore, there is resolution information that is determined to perfectly convert 3D media into 2D media. This disclosure defines this resolution as a nominal resolution, provides the changed resolution for each individual representation, and provides information for allowing a 3D player to activate a scaler to change the received content component, determine the rate at which it will be scaled up or down, and estimate the impact of the error predicted thereby.

[0168] The prior art discloses resolutions per representation. The present disclosure is characterized in that a nominal resolution is provided, thereby determining the degree of impact on quality, and providing the nominal resolution at the level of content components.

[0169] Table 12 shows examples of nominal resolutions at the content component level.

[0170] [Table 12]

[0171]

[0172] Another embodiment of the present disclosure: RTSP

[0173] The Real Time Streaming Protocol (RTSP) is a protocol designed to control a media streaming server, standardized by the IETF (RFC7826), and adopted by 3GPP as a media streaming protocol.

[0174] According to the prior art, when media streaming is performed between a server and a client using RTSP, manifest information of the media streaming (e.g., media component information indicating whether the media is audio or video and encoding information of the media) is determined by pre-negotiation between the server and the client using the Session Description Protocol (SDP) format.

[0175] Described below is a method for explicitly specifying identifiers for media types in units of content and content components in a manifest provided according to the present disclosure when RTSP is used instead of the above-described DASH.

[0176] Table 13 shows an example representation of 2D media manifest information using the SDP format according to the prior art.

[0177] [Table 13]

[0178]

[0179] In the example above, the DESCRIBE line is a command by the server to indicate to the client the properties of the streaming media, and the entire session information (e.g., version and total rate) is given in the subsequent<Generic Session Information> (<General Session Information>). The attributes of each media component are sent in the following<Media Information> (<Media Information>). The line starting with "" indicates the media component (video / audio) and protocol being transmitted, and the subsequent lines convey various properties of the media (for example, encoder and per-media rate).

[0180] Table 14 shows an example of an SDP message for indicating 3D media configuration information according to an embodiment.

[0181] [Table 14]

[0182]

[0183] In the example shown in Table 14, it is assumed that the 3D content consists of three content components (three m-lines), namely occupancy, geometry, and attributes.

[0184] The components (occupancy, geometry, and attributes) are assigned media IDs 101, 102, and 103, respectively (a=mid:10x), and an attribute for grouping them is added to the session information (a=3d-group:v-pcc 101 102 103) to indicate that the three components need to be combined to constitute one 3D content.

[0185] Furthermore, in the above example, multiple payload types (hereinafter referred to as "PTs") are indicated in the mth row to indicate multiple representations (e.g., the same component with different bit rates). In other words, for the geometry component, two representations with PT number 97 and PT number 98 are indicated to indicate that number 97 has been encoded with H.264 and number 98 has been encoded with H.265.

[0186] According to the SDP syntax, the related art 2D content representation expresses the priority of media components as the order of m lines, and this can also be applied to the quality priority of 3D content components. For example, in the above example, it can be said that based on the order of the m lines written, occupancy has the highest priority, while attributes have the lowest priority.

[0187] However, in this case, although the term "priority" in the related art scheme refers to the order in which one m row is selected from among multiple m rows when the same component has multiple m rows (the same component is composed of multiple representations) (i.e., only one m row is ultimately selected), the term "priority" according to the present disclosure refers to the impact of different components (represented by multiple m rows) on the overall quality.

[0188] Provides a Quality of Experience (QOE) table for each component combination

[0189] According to the present disclosure, a method for receiving content representing 3D media and the combination of content components that constitute the content, based on the use of an application, is provided. In this case, the content components affect each other, thereby affecting the quality of the final 3D media played (in which the received content components are combined). Therefore, a function for providing the priority of content components and reflecting it in the terminal's reception policy can be provided.

[0190] However, since the priority merely indicates relatively different degrees of influence on quality, rather than information on 3D media quality, the terminal cannot know the resulting quality according to different qualities of received content components before receiving the content components.

[0191] Therefore, according to the present disclosure, a method is proposed for providing quality assessment information for each quality standard of final 3D media obtained according to a combination of content components at the content level of a manifest, determining, by a 3D player and a 3D application of a terminal, a quality requirement for each content component to be received based on the information, determining a minimum quality combination of key quality assessment items to ensure the quality of the 3D media, and determining to receive it.

[0192] Table 15 shows an example of the quality of experience matrix configured in the subset.

[0193] [Table 15]

[0194]

[0195] According to the present disclosure, component information of content components that determine the final quality of 3D media is provided at the content level (in a subset), as shown in Table 15. In this example, the subset specifies the ID of the AdaptationSet that includes the content component and provides a QoeMatrix list.

[0196] The QoeMatrix list uses schemeUrl to specify what scheme the properties described below follow, and uses schemeIdURL to describe one or more criteria as describable quality criteria.

[0197] The example of the standard is the same as shown in Table 10 above (quality factors for each content component type). The quality achieved by the standard can be expressed as a percentage relative to the highest quality. 100% means it is equal to the highest quality, while 50% means, for example, that if one million points can be reconstructed at the highest quality, only 500,000 points can be reconstructed.

[0198] The composition is a combined list of adaptation identifiers and representation identifiers, and is the minimum requirement for meeting the quality, meaning that if a higher representation quality than the requirement is received, the requirement can be met. In this case, whether a representation is of high quality is preferably determined by referring to properties among the state-of-the-art properties of the representation, which properties have a significant causal relationship, where higher numbers are associated with higher quality, such as bitrate, width, or height.

[0199] In Table 15, the structure specifies that reception of 1:2, 2:2, 3:4, 4:1, 5:1 or higher is required to achieve the 80% point number criterion, which means that for AdaptationSets #1 and #2 (i.e., the two components of the geometry), reception quality #2 or higher is required, for AdaptationSets #3 (i.e., the attribute component), reception quality #4 or higher is required, for AdaptationSets #4 (i.e., the occupancy component), only reception #1 is required, and for AdaptationSets #5 (i.e., the map component), only reception #1 is required.

[0200] By providing the minimum representation combination information that meets the minimum quality requirements as described above, the QoeMatrix list can be expressed as the necessary number of steps for the necessary quality standard, and this is useful when terminals and applications determine the content components to be received according to the network environment, which is more efficient than providing all possible combinations.

[0201] In order for service providers and content providers to manage the quality of terminals more actively, a forcedQoePlay attribute may be provided as a subset or pre-selected additional attribute, and if this attribute is provided and its value is 1, the terminal may be forced to only receive combinations present in the queMatrix.

[0202] Table 16 shows an example of the forcing method.

[0203] [Table 16]

[0204]

[0205] Figure 4 FIG. 4 is a flowchart illustrating a method for playing media content by a client according to an embodiment of the present disclosure.

[0206] Combine Figure 4 The client terminal described refers to Figure 1 and Figure 3 The client terminal shown in FIG. 1 can execute the various embodiments described above.

[0207] Reference Figure 4 In operation S100, the client terminal may receive a media presentation description (MPD) from the server, the media presentation description including content type information of 3D media content, configuration type information of the 3D media content, role information of each of a plurality of content components, quality priority information of at least one content component, resolution information of at least one content component, and at least one of quality assessment information of a final 3D media corresponding to a combination of the content components.

[0208] The server can refer to Figure 1 and 2The service provider or content provider shown in , and the server can perform the above-mentioned various embodiments.

[0209] In operation S110 , the client terminal may access segments of 3D media content based on information included in the MPD.

[0210] In operation S120 , the client terminal may receive 3D media content based on the segments.

[0211] In operation S130 , the client terminal may play 3D media content using a 3D media player implemented therein.

[0212] Figure 5 is a flowchart illustrating a method for providing media content by a server according to an embodiment of the present disclosure.

[0213] The server can refer to Figure 1 and 2 The service provider or content provider shown in , and the server can perform the above-mentioned various embodiments.

[0214] Reference Figure 5 In operation S200, the server may send a media presentation description (MPD) to the client, the media presentation description including content type information of the 3D media content, configuration type information of the 3D media content, role information of each of a plurality of content components, quality priority information of at least one content component, resolution information of at least one content component, and at least one of quality assessment information of a final 3D media corresponding to a combination of the content components.

[0215] In operation S210 , the server may receive a request for a 3D media content segment from a client.

[0216] In operation S220 , the server may transmit the 3D media content to the client.

[0217] The methods according to the embodiments described in the specification or claims of the present disclosure can be implemented by hardware, software, or a combination of hardware and software.

[0218] When implemented in software, a computer-readable storage medium storing one or more programs (software modules) may be provided. The one or more programs stored in the computer-readable storage medium are configured to be executed by one or more processors in an electronic device. The one or more programs include instructions that enable the electronic device to perform the methods according to the embodiments described in the specification or claims of the present disclosure.

[0219] Programs (software modules or software) may be stored in random access memory, nonvolatile memory including flash memory, ROM, electrically erasable programmable read-only memory (EEPROM), magnetic disk storage, optical disk ROM, digital versatile disk (DVD) or other types of optical storage devices, or cassette tapes. Alternatively, programs may be stored in a memory consisting of a combination of all or some programs. Each component memory may include multiple memories.

[0220] The program may be stored in an attachable storage device that can be accessed via a communication network, such as the Internet, an intranet, a local area network (LAN), a wide area network (WAN), or a storage area network (SAN), or a communication network configured by a combination thereof. The storage device may be connected to a device that performs an embodiment of the present disclosure via an external port. An independent storage device on a communication network may be connected to a device that performs an embodiment of the present disclosure.

[0221] In the above-described specific embodiments of the present disclosure, components included in the present disclosure are presented in either singular or plural form, depending on the specific embodiment being proposed. However, the singular or plural form is selected to suit the context suggested for ease of description, and the present disclosure is not limited to singular or plural components. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0222] As apparent from the foregoing description, according to the embodiments of the present disclosure, 3D media having properties and structures different from 2D media may be efficiently transmitted and received in a communication system.

[0223] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. A method for receiving media content by a client, the method comprising: receiving, from a server, a media presentation description (MPD) including content type information of three-dimensional (3D) media content and configuration type information of the 3D media content; accessing segments of the 3D media content based on information included in the MPD; as well as receiving 3D media content based on the fragments, The content type information indicates an identifier of the 3D media content, wherein the configuration type information indicates a single multiplexed content component (OMCC) in which the content consists of one content component or a separately adaptable content component (SACC) in which the quality of each content component is adjustable, wherein when the configuration type information indicates SACC, the MPD further includes role information for each of the plurality of content components, and The role information is included as an attribute in the AdaptationSet element which is a content component unit.

2. The method according to claim 1, wherein the content type information is included as an attribute in a subset element or a preselected element of a media content unit included in the MPD, and The configuration type information is included as an attribute in a subset element or a preselect element of a media content unit included in the MPD.

3. The method according to claim 1, wherein the MPD further includes quality priority information of at least one content component, and The quality priority information is included as an attribute in a Subset element or a Preselect element as a media content unit included in the MPD, or the quality priority information is included as an attribute in an AdaptationSet element as a content component unit included in the MPD.

4. The method according to claim 1, The MPD also includes resolution information of at least one content component. The resolution information includes information necessary to convert 3D media content into two-dimensional (2D) media, and Here, the resolution information is included as an attribute in the AdaptationSet element which is a content component unit included in the MPD.

5. The method according to claim 1, wherein the MPD further includes quality assessment information of the final 3D media corresponding to the combination of content components, and The quality assessment information is included as an attribute in a subset element or a preselected element of a media content unit included in the MPD.

6. A method for providing media content by a server, the method comprising: Sending a media presentation description (MPD) including content type information of three-dimensional (3D) media content and configuration type information of the 3D media content to the client; receiving a request for a segment of 3D media content from a client; and Send 3D media content to the client, The content type information indicates an identifier of the 3D media content, wherein the configuration type information indicates a single multiplexed content component (OMCC) in which the content consists of one content component or a separately adaptable content component (SACC) in which the quality of each content component is adjustable, wherein when the configuration type information indicates SACC, the MPD further includes role information for each of the plurality of content components, and The role information is included as an attribute in the AdaptationSet element which is a content component unit.

7. The method according to claim 6, wherein the content type information is included as an attribute in a subset element or a preselected element of a media content unit included in the MPD, and The configuration type information is included as an attribute in a subset element or a preselect element of a media content unit included in the MPD.

8. The method according to claim 6, Wherein the MPD further includes quality priority information of at least one content component, and The quality priority information is included as an attribute in a Subset element or a Preselect element as a media content unit included in the MPD, or the quality priority is included as an attribute in an AdaptationSet element as a content component unit included in the MPD.

9. The method according to claim 6, The MPD also includes resolution information of at least one content component. The resolution information includes information necessary to convert 3D media content into two-dimensional (2D) media, and Here, the resolution information is included as an attribute in the AdaptationSet element which is a content component unit included in the MPD.

10. The method according to claim 6, wherein the MPD further includes quality assessment information of the final 3D media corresponding to the combination of content components, and The quality assessment information is included as an attribute in a subset element or a preselected element of a media content unit included in the MPD.

11. A client terminal configured to receive media content, the client terminal comprising: transceiver; and at least one processor coupled to the transceiver and configured to control: receiving a media presentation description (MPD) including content type information of three-dimensional (3D) media content and configuration type information of the 3D media content from a server, accessing segments of the 3D media content based on information included in the MPD, and receiving 3D media content based on the fragments, The content type information indicates an identifier of the 3D media content, wherein the configuration type information indicates a single multiplexed content component (OMCC) in which the content consists of one content component or a separately adaptable content component (SACC) in which the quality of each content component is adjustable, wherein when the configuration type information indicates SACC, the MPD further includes role information for each of the plurality of content components, and The role information is included as an attribute in the AdaptationSet element which is a content component unit.

12. The client terminal according to claim 11, wherein the content type information is included as an attribute in a subset element or a preselected element of a media content unit included in the MPD, and The configuration type information is included as an attribute in a subset element or a preselect element of a media content unit included in the MPD.

13. A server configured to provide media content, the server comprising: transceiver; and at least one processor coupled to the transceiver and configured to control: Sending a Media Presentation Description (MPD) including content type information of three-dimensional (3D) media content and configuration type information of the 3D media content to the client, receiving a request for a segment of 3D media content from a client, and Send 3D media content to the client, The content type information indicates an identifier of the 3D media content, wherein the configuration type information indicates a single multiplexed content component (OMCC) in which the content consists of one content component or a separately adaptable content component (SACC) in which the quality of each content component is adjustable, wherein when the configuration type information indicates SACC, the MPD further includes role information for each of the plurality of content components, and The role information is included as an attribute in the AdaptationSet element which is a content component unit.

Images