Associating file format objects and dynamic adaptive streaming media (DASH) objects based on hypertext transfer protocol

By introducing associated ID attributes and camera position information descriptors, the DASH mechanism was optimized, solving the problem of excessively large MPD files in multi-camera VR videos and achieving efficient video streaming and dynamic management of camera positions.

CN118921537APending Publication Date: 2024-11-08HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410518559.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2018-04-06
Filing Date
2019-04-03
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing DASH mechanism cannot efficiently handle camera position changes and time-related metadata representations in multi-camera video streams when processing virtual reality videos, resulting in excessively large MPD files and a lack of descriptors describing static and dynamic camera positions.

Method used

The associated ID attribute and the VWPT descriptor are introduced to reduce redundant associations between representations by using the associated ID attribute, distinguish between static and dynamic aircraft positions by using the VWPT descriptor, and omit metadata representations with time attributes and trajectory references, thereby optimizing the MPD file structure.

Benefits of technology

It effectively reduces MPD file size, improves the efficiency of video streaming, supports dynamic switching of multi-camera VR videos and differentiation of static camera positions, and reduces network resource and storage requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118921537A_ABST
    Figure CN118921537A_ABST
Patent Text Reader

Abstract

A method of associating a file format object with a dynamic adaptive streaming media (DASH) object based on a hypertext transfer protocol includes obtaining media content associated with a first stand and a second stand, where a first position of the first stand is static and a second position of the second stand is dynamic; generating a first VWPT descriptor according to the media content, and describing the first machine position; generating a second VWPT descriptor according to the media content, and describing the second machine position; generating an MPD comprising the first VWPT descriptor and the second VWPT descriptor; and transmitting the MPD.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application. The application number of the original application is 201980024408.3, and the original application date is April 3, 2019. The entire contents of the original application are incorporated into this application by reference.

[0002] Cross-reference to related applications

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 654,034 filed by Futurewei Technologies Inc. on April 6, 2018, entitled “Efficient Association of Dynamic Adaptive HTTP Streaming (DASH) and File Format Objects,” the entire contents of which are incorporated herein by reference. Technical Field

[0004] The disclosed objects generally relate to video encoding, and more particularly to associating file format objects with Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) objects. Background Art

[0005] Virtual Reality (VR), which can also be referred to as omnidirectional media, immersive media and / or 360-degree media, is an interactive recorded and / or computer-generated experience that occurs in a simulated environment and utilizes visual, audio and / or tactile feedback. From a visual perspective, VR presents an image sphere (or a portion of an image sphere) with the user at the center of the sphere. A head mounted display (HMD) or other display unit can present the image sphere. Specifically, through a VR display, a user can view a portion of the sphere through a field of view (FOV). The user can dynamically change the position and / or angle of the viewing angle to experience the environment displayed by the VR video. Each image (also referred to as a frame) in a VR video includes both the area of ​​the sphere within the user's FOV and the area of ​​the sphere outside the user's FOV. Therefore, VR frames include significantly more data than non-VR video images. Content providers are interested in providing streaming-based VR videos. However, compared to traditional videos, VR videos include significantly more data and different attributes. As a result, the traditional video streaming mechanism cannot efficiently stream VR videos. Summary of the invention

[0006] A first aspect relates to a method, comprising: acquiring media content associated with a camera position; generating a camera position information (VWPT) descriptor according to the media content, wherein the VWPT descriptor comprises a value indicating whether the position of the camera position is static; generating a media presentation description (MPD) file comprising the VWPT descriptor; and transmitting the MPD.

[0007] According to the first aspect, in a first possible implementation manner of the method, the value is a value is_static.

[0008] According to the first aspect or any one of the above implementations of the first aspect, in a second implementation of the method, the value is_static is a unique value in the VWPT descriptor, indicating whether the position of the machine position is static or dynamic.

[0009] According to the first aspect or any of the above implementations of the first aspect, in a third implementation of the method, the value being equal to “true” indicates that the position of the camera stand is static.

[0010] According to the first aspect or any of the above implementations of the first aspect, in a fourth implementation of the method, the value being equal to “false” indicates that the position of the aircraft stand is dynamic.

[0011] According to the first aspect or any one of the above implementations of the first aspect, in a fifth implementation of the method, the VWPT descriptor also includes a stand identifier (ID) value indicating a stand ID of the stand.

[0012] According to the first aspect or any of the above implementations of the first aspect, in a sixth implementation of the method, the VWPT descriptor also includes a center x value, indicating the absolute position of the center of the camera position on the x-axis in the global three-dimensional coordinates.

[0013] According to the first aspect or any one of the above implementations of the first aspect, in a seventh implementation of the method, the VWPT descriptor also includes a center y value, indicating the absolute position of the center of the camera position on the y-axis in the global three-dimensional coordinates.

[0014] According to the first aspect or any one of the above implementations of the first aspect, in an eighth implementation of the method, the VWPT descriptor also includes a center z value, indicating the absolute position of the center of the camera position on the z-axis in the global three-dimensional coordinates.

[0015] A second aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is configured to execute the first aspect or any one of the above implementations of the first aspect.

[0016] A third aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, an apparatus performs the first aspect or any of the above implementations of the first aspect.

[0017] A fourth aspect relates to a method, comprising: transmitting a request for an MPD; receiving the MPD in response to the request, wherein the MPD comprises a VWPT descriptor based on media content, the media content is associated with a camera position, the VWPT descriptor comprises a value indicating whether the position of the camera position is static; and processing the MPD.

[0018] According to the fourth aspect, in a first possible implementation of the method, the value is a value is_static.

[0019] According to the fourth aspect or any one of the above implementations of the fourth aspect, in a second implementation of the method, the value is_static is a unique value in the VWPT descriptor, indicating whether the position of the machine position is static or dynamic.

[0020] According to the fourth aspect or any of the above implementations of the fourth aspect, in a third implementation of the method, the value being equal to “true” indicates that the position of the aircraft position is static.

[0021] According to the fourth aspect or any of the above implementations of the fourth aspect, in a fourth implementation of the method, the value being equal to "false" indicates that the position of the aircraft position is dynamic.

[0022] According to the fourth aspect or any of the above implementations of the fourth aspect, in a fifth implementation of the method, the VWPT descriptor also includes a stand ID value, indicating the stand ID of the stand.

[0023] According to the fourth aspect or any of the above implementations of the fourth aspect, in a sixth implementation of the method, the VWPT descriptor also includes a center x value, indicating the absolute position of the center of the camera position on the x-axis in the global three-dimensional coordinates.

[0024] According to the fourth aspect or any of the above implementations of the fourth aspect, in a seventh implementation of the method, the VWPT descriptor also includes a center y value, indicating the absolute position of the center of the camera position on the y-axis in the global three-dimensional coordinates.

[0025] According to the fourth aspect or any of the above implementations of the fourth aspect, in an eighth implementation of the method, the VWPT descriptor also includes a center z value, indicating the absolute position of the center of the camera position on the z-axis in the global three-dimensional coordinates.

[0026] A fifth aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is used to execute the fourth aspect or any one of the above implementations of the fourth aspect.

[0027] A sixth aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, a device performs the fourth aspect or any one of the above implementations of the fourth aspect.

[0028] A seventh aspect relates to a method. The method comprises: acquiring media content; generating a metadata representation with time attributes according to the media content, wherein the metadata representation with time attributes is not associated with the attribute @associationId, the attribute @associationAsId, the attribute @associationVwptId, and the attribute @associationPsId; generating an MPD, wherein the MPD includes the metadata representation with time attributes; and transmitting the MPD.

[0029] According to the seventh aspect, in a first implementation of the method, the metadata representation with time attributes is also not associated with any other such attributes, and any other such attributes are used to associate the metadata representation with time attributes to other DASH objects.

[0030] According to the seventh aspect or any of the above implementations of the seventh aspect, in a second implementation of the method, the metadata track with time attributes is applicable to all media presentations.

[0031] The eighth aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is used to execute the seventh aspect or any one of the above implementations of the seventh aspect.

[0032] A ninth aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, a device performs the seventh aspect or any one of the above implementations of the seventh aspect.

[0033] A tenth aspect relates to a method. The method comprises: transmitting a request for an MPD; receiving the MPD in response to the request, wherein the MPD comprises a metadata representation with a time attribute based on media content, the metadata representation with a time attribute being unassociated with an attribute @associationId, an attribute @associationAsId, an attribute @associationVwptId, and an attribute @associationPsId; and processing the MPD.

[0034] According to the tenth aspect, in a first implementation of the method, the metadata representation with time attributes is also not associated with any other such attributes, and any other such attributes are used to associate the metadata representation with time attributes to other DASH objects.

[0035] According to the tenth aspect or any one of the above implementations of the tenth aspect, in a second implementation of the method, the metadata track with time attributes is applicable to all media presentations.

[0036] The eleventh aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is used to execute the tenth aspect or any one of the above implementations of the tenth aspect.

[0037] A twelfth aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, a device performs the tenth aspect or any one of the above implementations of the tenth aspect.

[0038] A thirteenth aspect relates to a method, comprising: acquiring media content; generating a metadata track with time attributes according to the media content, wherein the metadata track with time attributes is not associated with a 'cdsc' track reference; generating a file format object, wherein the file format object includes the metadata track with time attributes; and transmitting the file format object.

[0039] According to the thirteenth aspect, in a first implementation of the method, the 'cdsc' track reference indicates that the metadata track with time attributes describes each media track respectively.

[0040] According to the thirteenth aspect or any of the above implementations of the thirteenth aspect, in a second implementation of the method, the metadata track with time attributes is applicable to all movies.

[0041] A fourteenth aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is used to execute the twelfth aspect or any one of the above implementations of the twelfth aspect.

[0042] A fifteenth aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, a device performs the thirteenth aspect or any one of the above implementations of the thirteenth aspect.

[0043] A sixteenth aspect relates to a method, comprising: transmitting a request for a file format object; receiving the file format object in response to the request, wherein the file format object comprises a metadata track with time attributes based on media content, the metadata track with time attributes being unassociated with a 'cdsc' track reference; and processing the file format object.

[0044] According to the sixteenth aspect, in a first implementation manner of the method, the 'cdsc' track reference indicates that the metadata track with time attributes describes each media track respectively.

[0045] According to the sixteenth aspect or any of the above implementations of the sixteenth aspect, in a second implementation of the method, the metadata track with time attributes is applicable to all movies.

[0046] A seventeenth aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is used to execute any one of the sixteenth aspect or any one of the above implementations of the sixteenth aspect.

[0047] The eighteenth aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, a device performs the sixteenth aspect or any one of the above implementations of the sixteenth aspect.

[0048] A nineteenth aspect relates to a method. The method includes: acquiring media content associated with a first camera position and a second camera position, wherein a first position of the first camera position is static and a second position of the second camera position is dynamic; generating a first VWPT descriptor according to the media content and describing the first camera position; generating a second VWPT descriptor according to the media content and describing the second camera position; generating an MPD including the first VWPT descriptor and the second VWPT descriptor; and transmitting the MPD.

[0049] According to the nineteenth aspect, in a first implementation of the method, the second camera position is associated with a metadata representation with time attributes, the metadata representation with time attributes includes a metadata track with time attributes, and the metadata track with time attributes includes information describing the second position.

[0050] According to the nineteenth aspect or any one of the above implementations of the nineteenth aspect, in a second implementation of the method, the first VWPT descriptor includes a first value indicating whether the first position is static; the second VWPT descriptor includes a second value indicating whether the second position is static.

[0051] According to the nineteenth aspect or any one of the above implementations of the nineteenth aspect, in a third implementation of the method, the first value is a first value is_static, and the second value is a second value is_static.

[0052] According to the nineteenth aspect or any one of the above implementations of the nineteenth aspect, in a fourth implementation of the method, the first value is_static is a unique value in the first VWPT descriptor, indicating whether the first position of the first camera position is static or dynamic; the second value is_static is a unique value in the second VWPT descriptor, indicating whether the second position of the second camera position is static or dynamic.

[0053] According to the nineteenth aspect or any above implementation of the nineteenth aspect, in a fifth implementation of the method, the first value is equal to "true", indicating that the position of the aircraft position is static, and the second value is equal to "true", indicating that the position of the aircraft position is static.

[0054] According to the nineteenth aspect or any one of the above implementations of the nineteenth aspect, in a sixth implementation of the method, the first value is equal to "false", indicating that the position of the aircraft position is dynamic, and the second value is equal to "false", indicating that the position of the aircraft position is dynamic.

[0055] According to the nineteenth aspect or any one of the above implementations of the nineteenth aspect, in a seventh implementation of the method, the first VWPT descriptor also includes a first camera ID value, indicating the first camera ID of the first camera; the second VWPT descriptor also includes a second camera ID value, indicating the second camera ID of the second camera.

[0056] According to the nineteenth aspect or any above implementation manner of the nineteenth aspect, in an eighth implementation manner of the method, the first VWPT descriptor also includes a first center x value, indicating a first absolute position of a first center of the first camera position on the x-axis and in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center x value, indicating a second absolute position of a second center of the second camera position on the x-axis and in the global three-dimensional coordinates.

[0057] According to the nineteenth aspect or any above implementation manner of the nineteenth aspect, in a ninth implementation manner of the method, the first VWPT descriptor also includes a first center y value, indicating a first absolute position of a first center of the first camera position on the y-axis in the global three-dimensional coordinate system; the second VWPT descriptor also includes a second center y value, indicating a second absolute position of a second center of the second camera position on the y-axis in the global three-dimensional coordinate system.

[0058] According to the nineteenth aspect or any above implementation manner of the nineteenth aspect, in the tenth implementation manner of the method, the first VWPT descriptor also includes a first center z value, indicating a first absolute position of a first center of the first camera position on the z-axis in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center z value, indicating a second absolute position of a second center of the second camera position on the z-axis in the global three-dimensional coordinates.

[0059] The twentieth aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is used to execute the nineteenth aspect or any one of the above implementations of the nineteenth aspect.

[0060] A twenty-first aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, a device performs the nineteenth aspect or any of the above implementations of the nineteenth aspect.

[0061] A twenty-second aspect relates to a method, comprising: transmitting a request for an MPD; receiving the MPD in response to the request, wherein the MPD comprises a first VWPT descriptor and a second VWPT descriptor, wherein the first VWPT descriptor is based on media content and describes a first position, wherein a first position of the first position is static; and the second VWPT descriptor is based on the media content and describes a second position, wherein a second position of the second position is dynamic; and processing the MPD.

[0062] According to the twenty-second aspect, in a first implementation of the method, the second camera position is associated with a metadata representation with time attributes, the metadata representation with time attributes includes a metadata track with time attributes, and the metadata track with time attributes includes information describing the second position.

[0063] According to the 22nd aspect or any one of the above implementations of the 22nd aspect, in a second implementation of the method, the first VWPT descriptor includes a first value, indicating whether the first position is static; the second VWPT descriptor includes a second value, indicating whether the second position is static.

[0064] According to the 22nd aspect or any one of the above implementations of the 22nd aspect, in a third implementation of the method, the first value is a first value is_static, and the second value is a second value is_static.

[0065] According to the twenty-second aspect or any one of the above implementations of the twenty-second aspect, in a fourth implementation of the method, the first value is_static is a unique value in the first VWPT descriptor, indicating whether the first position of the first camera position is static or dynamic; the second value is_static is a unique value in the second VWPT descriptor, indicating whether the second position of the second camera position is static or dynamic.

[0066] According to the 22nd aspect or any one of the above implementations of the 22nd aspect, in a fifth implementation of the method, the first value is equal to "true", indicating that the position of the aircraft position is static, and the second value is equal to "true", indicating that the position of the aircraft position is static.

[0067] According to the 22nd aspect or any one of the above implementations of the 22nd aspect, in a sixth implementation of the method, the first value is equal to "false", indicating that the position of the aircraft position is dynamic, and the second value is equal to "false", indicating that the position of the aircraft position is dynamic.

[0068] According to the 22nd aspect or any one of the above implementations of the 22nd aspect, in a seventh implementation of the method, the first VWPT descriptor also includes a first camera ID value, indicating the first camera ID of the first camera; the second VWPT descriptor also includes a second camera ID value, indicating the second camera ID of the second camera.

[0069] According to the twenty-second aspect or any of the above implementations of the twenty-second aspect, in an eighth implementation of the method, the first VWPT descriptor also includes a first center x value, indicating a first absolute position of a first center of the first camera position on the x-axis and in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center x value, indicating a second absolute position of a second center of the second camera position on the x-axis and in the global three-dimensional coordinates.

[0070] According to the twenty-second aspect or any of the above implementations of the twenty-second aspect, in a ninth implementation of the method, the first VWPT descriptor also includes a first center y value, indicating a first absolute position of a first center of the first camera position on the y-axis in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center y value, indicating a second absolute position of a second center of the second camera position on the y-axis in the global three-dimensional coordinates.

[0071] According to the twenty-second aspect or any of the above implementations of the twenty-second aspect, in the tenth implementation of the method, the first VWPT descriptor also includes a first center z value, indicating a first absolute position of the first center of the first camera position on the z-axis and in the global three-dimensional coordinates, and the second VWPT descriptor also includes a second center z value, indicating a second absolute position of the second center of the second camera position on the z-axis and in the global three-dimensional coordinates.

[0072] A twenty-third aspect relates to a device. The device includes a memory and a processor coupled to the memory. The processor is used to execute any one of the twenty-second aspect or any one of the above implementations of the twenty-second aspect.

[0073] A twenty-fourth aspect relates to a computer program product. The computer program product includes computer executable instructions stored in a non-transitory medium. When a processor executes the computer executable instructions, a device performs the twenty-second aspect or any one of the above implementations of the twenty-second aspect.

[0074] Any of the above embodiments can be combined with any of the other embodiments to generate new embodiments. These and other features will be more clearly understood through the following detailed description in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] For a more complete understanding of the present invention, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0076] Figure 1 A schematic diagram of an exemplary system for VR video encoding.

[0077] Figure 2 A flowchart of an exemplary method for encoding VR video.

[0078] Figure 3 A schematic diagram of an exemplary system for recording multi-camera VR video.

[0079] Figure 4 This is a schematic diagram of an exemplary DASH MPD, which can be used to stream multi-camera VR videos.

[0080] Figure 5 is a schematic diagram of an exemplary video encoding device.

[0081] Figure 6 A schematic diagram of an exemplary DASH content server, which is used to associate DASH objects in an MPD to support streaming of multi-camera VR videos.

[0082] Figure 7 A flowchart of a method for generating and transmitting MPD provided by an embodiment of the present invention.

[0083] Figure 8 A flowchart of a method for receiving and processing MPD provided by an embodiment of the present invention.

[0084] Fig. 9 A flowchart of a method for generating and transmitting MPD provided by another embodiment of the present invention.

[0085] Fig.10 A flowchart of a method for receiving and processing MPD provided by another embodiment of the present invention.

[0086] Fig.11 A flowchart of a method for generating and transmitting a file format object provided by one embodiment of the present invention.

[0087] Fig.12 A flowchart of a method for receiving and processing a file format object provided by one embodiment of the present invention.

[0088] Fig.13 A flowchart of a method for generating and transmitting MPD provided by yet another embodiment of the present invention.

[0089] Fig.14 The present invention is a flowchart of a method for receiving and processing MPD according to another embodiment of the present invention. DETAILED DESCRIPTION

[0090] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or existing. The present invention should in no way be limited to the illustrative implementations, drawings, and techniques described below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims and their full scope of equivalents.

[0091] The following abbreviations are used:

[0092] ASIC: Application-Specific Integrated Circuit

[0093] CPU: Central Processing Unit

[0094] DASH: Dynamic Adaptive Streaming over HTTP

[0095] DSP: Digital Signal Processor

[0096] EO: Electro-optical

[0097] FOV: Field of View

[0098] FPGA: Field Programmable Gate Array

[0099] HMD: Head-mounted display

[0100] HTTP: Hypertext Transfer Protocol

[0101] ID: Identifier

[0102] I / O: Input and / or Output

[0103] IPD: Interpupillary distance

[0104] ISO: International Organization for Standardization

[0105] ISOBMFF: ISO Base Media File Format

[0106] LTE: Long Term Evolution

[0107] MPD: Media Description File

[0108] OE: Optoelectronics

[0109] OMAF: Omnidirectional Media Format

[0110] PC: Personal Computer

[0111] RAM: Random Access Memory

[0112] RF: Radio Frequency

[0113] ROM: Read Only Memory

[0114] RX: Receiver

[0115] SRAM: Static RAM

[0116] TCAM: Ternary Content Addressable Memory

[0117] Tx / Rx: Transceiver unit

[0118] URL: Uniform Resource Locator

[0119] VM: Virtual Machine

[0120] VR: Virtual Reality

[0121] VWPT: Aircraft Information

[0122] XML: Extensible Markup Language

[0123] DASH is a mechanism for streaming video data over a network. DASH provides the client with an MPD file that describes the video. Specifically, the MPD describes various representations of the video and the location of such representations. The above representations can all include the same video content at different resolutions. The client can obtain video clips from these representations and display them to the client. Specifically, the client can monitor the video buffer and / or network communication speed, and switch various representations based on the data in the above MPD, thereby dynamically changing the video resolution based on the current situation.

[0124] DASH allows representations to be associated with other representations. For example, a video representation can be associated with a representation, where these associations include metadata with temporal attributes. This metadata can provide data indicating how the video frames should be displayed. This data can be called hints, and the representations can be called hint tracks. In DASH, each representation can be associated with a representation that contains the relevant hint track. Applying such a scheme to VR video may cause problems.

[0125] First, a VR video may include multiple video streams captured simultaneously from multiple locations called camera positions. During playback of the video, the user is able to switch camera positions. Therefore, each camera position includes multiple representations. In this case, the temporal metadata representation may contain a recommended viewing angle that indicates the suggested camera position and the recommended FOV for that camera position at any given point in time. In this way, the user can choose to give up control of the VR experience (for example, if the user wants the content producer to decide which viewpoint to watch the VR video from). To implement this scheme, each representation of each camera position is associated with a temporal metadata representation. When there are a large number of representations, this approach produces a very large MPD because each representation records an association.

[0126] Second, cameras in DASH can be identified by camera ID and / or location data. In some cases, the location of a camera may change over time. For example, a camera may be mounted on a track and moved to capture different areas of the environment being filmed. In this case, the location information will change. Since the location information of the camera is different at different times, the DASH mechanism may not be able to clearly determine whether certain elements are associated with the same camera.

[0127] Third, DASH lacks an MPD descriptor to indicate static and dynamic positions. In addition, this descriptor does not include a value to indicate whether the associated position is static or dynamic.

[0128] Fourth, metadata representations with temporal attributes are usually applicable to all media presentations. However, DASH lacks a method to associate metadata representations with temporal attributes to all media presentations.

[0129] Fifth, a metadata track with temporal attributes is usually applicable to the entire movie. However, the file format does not have a method to associate a metadata track with temporal attributes to the entire movie.

[0130] Embodiments for associating file format objects and DASH objects are disclosed herein. To address the first problem described above, an association ID attribute is introduced so that DASH objects reference other DASH objects at different levels of specificity (e.g., not limited to strict representation-to-representation associations). For example, DASH organizes various representations into adaptation sets, where an adaptation set contains representations that can be interchanged at runtime (e.g., representations of the same video data containing different resolutions). When there are different camera positions, each camera position may include one or more adaptation sets with one or more corresponding representations. In addition, the MPD may contain preselections, which are combinations of adaptation sets (e.g., audio track adaptation sets and video adaptation sets associated with the same camera position) that form a specific experience and are selected by the content provider to be presented in groups. The association ID disclosed herein may associate representations to adaptation sets, representations to camera positions, representations to preselections, adaptation sets to camera positions, adaptation sets to preselections, camera positions to preselections (and vice versa), and any combination or subcombination thereof. For example, the association ID may include an association ID attribute (@associationId), an association position ID attribute (@associationVwptId), an association adaptation set ID (@associationAsId), an association pre-selection ID (@associationPsId), or a combination thereof. In order to solve the second problem mentioned above, when the position IDs are the same, the position elements are considered to be the same regardless of the position position information. In order to solve the third problem mentioned above, a position descriptor indicates both a static position and a dynamic position. In addition, the position descriptor may include a value indicating whether the associated position is static or dynamic. In order to solve the fourth problem mentioned above, various attributes in the metadata representation with time attributes are omitted. In order to solve the fifth problem mentioned above, track references in the metadata track with time attributes are omitted.

[0131] Figure 11 is a schematic diagram of an exemplary system 100 for VR video encoding. The system 100 includes a multi-directional camera 101, a VR encoding device 104 including an encoder 103, a DASH content server 111, a client 108 including a decoder 107, and a rendering device 109. The multi-directional camera 101 includes an array of camera devices. Each camera device points to a different angle, so that the multi-directional camera 101 can capture multiple directional video streams of the surrounding environment from multiple angles. For example, the multi-directional camera 101 can capture a VR video 121 of the environment as a sphere, wherein the multi-directional camera 101 is located at the center of the sphere. "Sphere" and "spherical video" used herein refer to both a geometric sphere and a sub-part of a geometric sphere, such as a spherical segment, a spherical dome, a spherical segment, etc. For example, the multi-directional camera 101 can capture a 180-degree video to cover half of the environment, so that the production staff can stay behind the multi-directional camera 101. The multi-directional camera 101 is also capable of shooting VR video 121 within 360 degrees (or any subset within the 360-degree range). However, a portion of the floor below the multi-directional camera 101 may be ignored, resulting in the video not being a complete sphere. Therefore, the term "sphere" used here is a general term used for the purpose of clear discussion and should not be considered as limiting from a geometric perspective. It should be noted that the multi-directional camera 101 described above is an exemplary camera capable of capturing VR video 121, and other camera devices (e.g., cameras, fisheye lenses) can also be used to capture VR video.

[0132] The VR video 121 captured by the multi-directional camera 101 is forwarded to the VR encoding device 104. The VR encoding device 104 can be a computing system including dedicated VR encoding software. The VR encoding device 104 may include an encoder 103. In some examples, the encoder 103 may also be included in a computer system independent of the VR encoding device 104. The VR encoding device 104 is used to convert the multiple directional video streams in the VR video 121 into a VR video stream including multiple directional video streams, including the entire area recorded from all relevant angles. This conversion can be called image stitching. For example, the frames in each video stream captured simultaneously can be stitched together to generate a single spherical image. Secondly, a spherical video stream can be generated from the spherical image. For the sake of clarity, it should be noted that the terms "frame" and "picture / image" are used interchangeably in this article unless otherwise specified.

[0133] Next, the spherical video stream can be forwarded to the encoder 103 for compression. The encoder 103 is a device and / or program capable of converting information from one format to another format for the purpose of standardization, acceleration and / or compression. The standardized encoder 103 is used to encode rectangular images and / or square images. Therefore, the encoder 103 is used to map each spherical image in the spherical video stream to multiple rectangular sub-images. These sub-images can then be carried in separate sub-image video streams. Therefore, each sub-image video stream displays a stream of images that changes over time, and each sub-image video stream records a portion of the spherical video stream. Finally, the encoder 103 can encode each sub-image video stream to compress the video stream into a manageable file size. In general, the encoder 103 divides each frame in each sub-image video stream into pixel blocks, compresses the pixel blocks by inter-frame prediction and / or intra-frame prediction to generate coded blocks including prediction blocks and residual blocks, transforms the residual blocks for further compression, and filters these blocks using various filters. The compressed blocks and the corresponding syntax are stored in the codestream, for example, as tracks of ISOBMFF and / or OMAF.

[0134] The encoded tracks in the VR video 121, including the compressed blocks and the associated syntax, constitute part of the media content 123. The media content 123 may include an encoded video file, an encoded audio file, a combined audio and video file, media in multiple languages, subtitled media, metadata, or a combination thereof. The media content 123 may be divided into adaptation sets. For example, from the perspective of a camera, the video may be included in one adaptation set, the audio may be included in another adaptation set, the closed subtitles may be included in another adaptation set, the metadata may be included in another adaptation set, and so on. The adaptation set contains media content 123, and the media content 123 cannot be interchanged with the media content 123 in other adaptation sets. The content in each adaptation set may be stored in a representation, wherein the representations in the same adaptation set are interchangeable. For example, the VR video 121 shot from a single camera position may be downsampled to different resolutions and stored in the corresponding representation. For another example, the audio shot from a single camera position may be downsampled to different qualities, translated into different languages, etc., and stored in the corresponding representation.

[0135] The media content 123 may be forwarded to a DASH content server 111 for distribution to end users via the network 105. The DASH content server 111 may be any device for servicing HTTP requests from clients. The DASH content server 111 may include a dedicated server, a server cluster, a VM in a cloud computing environment, or any other suitable content management entity. The DASH content server 111 may receive the media content 123 from the VR encoding device 104. The DASH content server 111 may generate an MPD 125 describing the media content 123. For example, the MPD 125 may describe preselections, camera positions, adaptation sets, representations, metadata tracks, segments, etc., and the locations where these contents may be obtained via HTTP requests (e.g., HTTP GET).

[0136] The client 108 including the decoder 107 can conduct a media communication session 127 with the DASH content server 111 to obtain the media content 123 through the network 105. The network 105 can include the Internet, a mobile communication network (e.g., a data network based on LTE), or other data communication data systems. The client 108 can be any user-operated device for viewing video content from the media content 123, such as a computer, a television, a tablet device, a smart phone, etc. The media communication session 127 can include issuing media requests, such as HTTP-based requests (e.g., HTTP GET requests). After receiving the initial media request, the DASH content server 111 can forward the MPD 125 to the client 108. The client 108 can then use the information in the MPD 125 to make additional media requests for the media content 123 as part of the media communication session. Specifically, the client 108 can use the data in the MPD 125 to determine which parts of the media content 123 should be obtained based on user preferences, user selections, buffer / network conditions, etc. After selecting the relevant portion of the media content 123, the client 108 uses the data in the MPD 125 to address the media request to the location containing the relevant data in the DASH content server 111. The DASH content server 111 can then respond to the client 108 with the requested portion of the media content 123. In this way, the client 108 receives the requested portion of the media content 123 without having to download the entire media content 123, saving network resources (e.g., time, bandwidth, etc.) on the network 105.

[0137] The decoder 107 is a device located at the user's location (e.g., implemented on the client 108) for performing the reverse process of the encoding process of the encoder 103, decoding the encoded bitstream obtained in the form of a representation from the DASH content server 111. The decoder 107 also merges the resulting sub-image video streams to reconstruct a VR video sequence 129. The VR video sequence 129 contains the portion of the media content 123 reconstructed by the decoder 107 requested by the client 108 according to user selection, preference and / or network conditions. The VR video sequence 129 can then be forwarded to the rendering device 109. The rendering device 109 is a device for displaying the VR video sequence 129 to the user. For example, the rendering device 109 may include an HMD fixed to the user's head and covering the user's eyes. The rendering device 109 may include a screen, a camera, a motion sensor, a speaker, etc. corresponding to each eye, and may communicate with the decoder 108 via a wireless connection and / or a wired connection. In other examples, the rendering device 109 may be a display screen, such as a TV, a computer monitor, a tablet computer, etc. The rendering device 109 can display a portion of the VR video sequence 129 to the user. The portion shown is shot according to the FOV and / or viewing angle of the rendering device 109. For example, the rendering device 109 can use a motion tracking sensor to change the position of the FOV according to the user's head movement. In this way, the user can see different parts of the spherical video stream according to the head movement. In some cases, the rendering device 109 can adjust the FOV corresponding to each eye according to the user's IPD, thereby creating a three-dimensional space effect.

[0138] Figure 2 Flowchart of an exemplary method 200 for encoding VR video. Method 200 is performed by components in system 100, etc. In step 201, a multi-directional camera group such as multi-directional camera 101 is used to capture multiple directional video streams. The multiple directional video streams include views of the environment at different angles. For example, the multiple directional video streams may include videos captured at 360 degrees, 180 degrees, 240 degrees, etc. around the camera on a horizontal plane, and the multi-directional video streams may also include videos captured at 360 degrees, 180 degrees, 240 degrees, etc. around the camera on a vertical plane. The generated video includes information sufficient to cover a spherical area around the camera for a certain period of time.

[0139] In step 203, the multiple directional video streams are synchronized in the time domain. Specifically, each directional video stream includes a series of images shot at corresponding angles. Ensure that the frames captured at the same time domain position in each directional video stream are uniformly processed so that the multiple directional video streams are synchronized. Then, the frames in the directional video streams can be spliced ​​together in the air domain to generate a spherical video stream. Therefore, each frame in the spherical video stream contains data obtained from the frames of all directional video streams, and these frames occur at the same time position.

[0140] In step 205, the spherical video stream is mapped to a rectangular sub-image video stream. This process may also be referred to as projecting the spherical video stream to a rectangular sub-image video stream. Encoders and decoders are typically designed to encode rectangular frames and / or square frames. Therefore, mapping the spherical video stream to a rectangular sub-image video stream generates video streams that can be encoded and decoded by non-VR specific encoders and decoders. It should be noted that steps 203 and 205 involve VR video processing and can therefore be performed by specialized VR hardware, software, or a combination thereof.

[0141] In step 207, the rectangular sub-image video stream is forwarded to an encoder such as encoder 103. Then, the encoder encodes the sub-image video stream into a sub-image code stream of a corresponding media file format. Specifically, the encoder can treat each sub-image video stream as a video signal. The encoder can encode each frame in each sub-image video stream by inter-frame prediction, intra-frame prediction, etc. With regard to the file format, the sub-image video stream may be stored in ISOBMFF. For example, the sub-image video stream is captured at a specified resolution. Then, the sub-image video stream is downsampled to various lower resolutions for encoding. Each resolution can be referred to as a representation. A lower-quality representation results in an unclear image, but reduces the file size. Therefore, compared with a high-quality representation that loses visual quality, a low-quality representation can be transmitted to a user using less network resources (e.g., time, bandwidth). Each representation can be stored in a corresponding track set in a DASH content server (e.g., DASH content server 111). Therefore, tracks can be sent to users, wherein the tracks include sub-image code streams of various resolutions (e.g., visual quality).

[0142] In step 209, the sub-image code stream is sent to the decoder as a track. Specifically, the MPD describing the various representations can be forwarded from the DASH content server to the client. The forwarding can be performed upon receiving a request from the client (e.g., an HTTP GET request). For example, the MPD can describe various adaptation sets containing various representations. The client can then request the relevant representation or part thereof from the desired adaptation set.

[0143] In step 211, a decoder such as decoder 107 receives a requested representation, wherein the requested representation includes a trajectory consisting of a sub-image code stream. The decoder can then decode the sub-image code stream into a sub-image video stream for display. The decoding process involves the inverse process of the encoding process (e.g., through inter-frame prediction and intra-frame prediction). In step 213, the decoder can merge the sub-image video streams into a spherical video stream to be presented to the user as a VR video sequence. The decoder can then forward the VR video sequence to a rendering device (e.g., rendering device 109).

[0144] In step 215, the rendering device renders the perspective of the spherical video stream to show it to the user. As described above, the area outside the FOV in the VR video sequence may not be rendered at each time point.

[0145] Figure 3 Schematic diagram of an exemplary system 300 for recording multi-camera VR video. For example, system 300 can be used to record VR video from multiple cameras. The video of each camera can be divided into one or more adaptive sets, and then stored as representations in the corresponding adaptive sets. Such adaptive sets and representations can be encoded and transmitted to the client for presentation, for example, in combination with system 100 and / or method 200 as described above. Therefore, system 300 can be an embodiment of multi-directional camera 101.

[0146] In one example, the system 300 may be deployed in a stadium 311 where a sporting event is held. The system 300 may use a first fixed camera 301, a second fixed camera 302, and a mobile camera 303. In some examples, cameras 301 to 303 may all be multi-directional cameras for recording VR sequences. Camera 301 is placed at a first camera position, and camera 302 is placed at a second camera position, which is different from the first camera position. During the sporting event in the stadium 311, camera 301 and camera 302 record the same environment from different camera positions, thereby providing the action occurring during the sporting event in the stadium 311 from different angles. Camera 303 may be moved according to instructions from a staff member in the stadium 311. For example, camera 303 may be placed on a track 312, so that it may be moved to an area of ​​particular interest during the sporting event. For example, as the sporting event progresses, camera 303 may be moved near the location of the ball to view the ongoing game from the best angle. Thus, camera 303 is associated with a constantly changing camera position, which is different from the camera positions of camera 301 and camera 302.

[0147] In one example, videos captured by various cameras 301 to 303 can be provided so that the user can dynamically select the video stream of the corresponding camera position on demand. In the DASH scheme, the video and / or audio captured by the cameras 301 to 303 can be saved in the corresponding (one or more) adaptation sets according to the camera position. The video in each adaptation set corresponding to a camera position can be stored in representations of different qualities. Therefore, the user can select a camera position based on the data in the MPD and receive a representation of the video of the camera position based on the video quality suitable for the user's network connection.

[0148] Additionally, a user may want to relinquish control and allow the video producer to select the most relevant camera positions as the sporting event is in progress. This can be accomplished using a temporal metadata track (e.g., a cue track). The temporal metadata tracks are stored as separate representations in separate adaptation sets. For example, the temporal metadata track may indicate that the FOV specified at the camera position of camera 301 contains the most interesting video (e.g., a player holding a ball) for a first time period, the FOV for the camera position of camera 302 for a second time period, the FOV for the camera position of camera 303 for a third time period, and so on. In this way, a client can read the temporal metadata track and obtain the appropriate representation from the camera position indicated by the temporal metadata track to display to the user as directed by the video producer during the sporting event.

[0149] In this scenario, each representation of each camera position (e.g., each adaptation set) is associated with a metadata track with time attributes. When there are many camera positions and many video qualities, the number of associations between metadata tracks with time attributes and corresponding representations becomes more. Such associations are recorded in the MPD file, which greatly increases the file size of the MPD file. Therefore, in addition to the association between representations, the mechanism described in the present invention also indicates the association between various DASH objects. For example, the camera positions of three cameras 301 to 303 can be associated with metadata track representations and / or adaptation sets with time attributes. In this way, the number of associations is reduced from 3 times the number of available video qualities to 3 in the example shown. Various DASH objects and the associations between such objects will be discussed below in conjunction with the accompanying drawings to solve the above-mentioned problems.

[0150] Figure 4 4 is a schematic diagram of an exemplary DASH MPD 400. DASH MPD 400 can be used to stream multi-camera VR videos. For example, when applying method 200 and / or system 100, MPD 400 can be used to describe the video recorded by system 300. Therefore, MPD 400 can be used as MPD 125.

[0151] MPD 400 may include one or more pre-selections 460. Pre-selection 460 is a set of content that (video producers, etc.) select to experience together. For example, pre-selection 460 may include a video, corresponding audio, corresponding subtitles, corresponding haptic feedback, etc. For example, pre-selection 460 may include multiple adaptation sets 420.

[0152] The MPD 400 may also include one or more camera positions 450. A camera position 450 includes multiple adaptation sets 420 associated with (recorded from) the same location / camera. For example, a camera position 450 may include any version of data recorded entirely by camera 301 (at a first camera position 450), entirely by camera 302 (at a second camera position 450), or entirely by camera 303 (at a third camera position 450). A camera position 450 may include one or more adaptation sets 420. For example, a camera position 450 may include an adaptation set 420 of audio recorded at a location, an adaptation set 420 of video recorded at the location, an adaptation set 420 of metadata about the location, and so on. A camera position 450 may or may not be part of a preselection 460, depending on the selection made by the content producer.

[0153] Adaptation set 420 includes one or more representations 430. Specifically, adaptation set 420 includes multiple representations 430, which are of the same type and can be presented interchangeably. For example, since one type of audio data cannot be exchanged with one type of video data, audio data, video data, and metadata are placed in different adaptation sets 420, so as not to affect media presentation. In addition, videos from different camera positions 450 cannot be interchanged because such videos contain different images.

[0154] The media data contained in the representation 430 can be presented to generate a part of a multimedia presentation. In terms of video, the representations 430 in the same adaptation set 420 can contain the same video at different resolutions. Therefore, such representations 430 can be used interchangeably according to the desired video quality. In terms of audio, the representations 430 in the common adaptation set 420 can contain audio with varying quality and audio tracks in different languages. The representations 430 in the adaptation set 420 can also contain metadata such as metadata tracks with time attributes (e.g., prompt tracks). Therefore, the representation 430 containing metadata with time attributes can be used simultaneously with the corresponding video representation 430, audio representation 430, closed caption representation 430, etc. to determine how such media representation 430 should be presented. For example, the metadata representation 430 with time attributes can indicate the preferred camera position 450 and the preferred viewing angle of the camera position 450 according to time. Therefore, the metadata representation 430 with time attributes can indicate which adaptation sets should be acquired and presented when the media presentation is in progress.

[0155] The representation 430 may include a fragment 440. The fragment 440 includes media data within a predetermined time period (e.g., 3 seconds). Thus, the fragment 440 may include a portion of audio data, a portion of video data, etc. that can be accessed over a network using a predetermined URL. The MPD 400 includes data indicating the URL of each fragment 440. Thus, the client can select a desired preselection 460 and / or a camera position 450 to determine the adaptation set 420 that should be presented. The client can then determine the representation 430 to be obtained based on the current network congestion. The client can then request the corresponding fragment 440 in order to display the media presentation to the user.

[0156] 1. Indication of the relationship between representations

[0157] In DASH, a syntax element in a representation 430 may describe an associated representation 430, the syntax element including an association ID attribute (@associationId) and optionally an association type attribute (@associationType). An associated representation 430 refers to a representation 430 that provides information about the relationship between the representation 430 and other representations 430. A fragment 440 in an associated representation 430 may be optional and used to decode and / or present the representation 430 identified by @associationId. It can be considered that such a fragment 440 is supplementary information or descriptive information, and the attribute @associationType indicates the type of association. It should be noted that the attributes @associationId and @associationType are used between representations 430 in different adaptation sets 420 (e.g., in different adaptation sets 420). As described above, a metadata representation 430 with a time attribute may be associated with each other representation 430 in an MPD 400. When there are multiple camera positions 450, there may be a large number of representations 430. In this case, the syntax indicating the association between representation 430 and representation 430 can use a large amount of data to indicate the association between metadata representation 430 with time attributes and representation 430. Specifically, taking representation 430 and representation 430 as an example, the attribute @associationId contains the value of the identifier attribute (@id) of all representations 430.

[0158] The present invention uses various exemplary indication mechanisms to solve this problem and efficiently indicate the association between representations. In a first exemplary embodiment, the attribute @associationId may contain the attribute @id (e.g., AdaptationSet@id) of the adaptation set 420. When the attribute @associationId contains a specific value of AdaptationSet@id, the value of AdaptationSet@id should be unique among the values ​​of the attribute @id of all adaptation sets 420 and all representations 430 in the MPD 400, and should be unique among the ID values ​​of any other type of objects such as the camera position 450 or the pre-selection 460, and the attribute @associationId may contain these ID values. An optional level attribute of the adaptation set 420 may be added to indicate whether the value of the AdaptationSet@id of the adaptation set 420 satisfies uniqueness. The first exemplary embodiment is implemented as follows:

[0159] In this embodiment, the association representation 430, the attribute @associationId and the attribute @associationType are described in detail as follows. An element in the representation 430 describes the association representation 430, includes the attribute @associationId, and optionally includes the attribute @associationType. The association representation 430 refers to a representation 430 that provides information about the relationship between the representation 430 and other representations 430 or adaptation sets 420. The fragments 440 in the association representation 430 may be optional and are used to decode and / or present the representation 430 or adaptation set 420 identified by @associationId. It can be considered that these fragments are supplementary information or descriptive information, and the attribute @associationType indicates the type of association. It should be noted that in this embodiment, the attributes @associationId and @associationType can only be used between representations 430 in different adaptation sets 420. Table 1 defines the attribute @associationId and the attribute @associationType.

[0160] Table 1: Attributes @associationId and @associationType

[0161]

[0162] In this embodiment, the carrying of the metadata track with time attributes of OMAF is specifically described as follows: the metadata track with time attributes of the track sample entry type 'invo' or 'rcvp' can be encapsulated in the DASH representation 430. The attribute @associationId of the metadata representation 430 can include the value of the attribute @id of the representation 430 or the adaptation set 420, and the representation 430 or the adaptation set 420 includes omnidirectional media carried by (one or more) media tracks, and the above media tracks are associated with the metadata track with time attributes. The @associationType of the metadata representation attribute 430 can be set to be equal to 'cdsc'.

[0163] In this embodiment, the optional adaptive set level attributes may be specifically described in Table 2.

[0164] Table 2: Attribute @idUniqueFlag

[0165]

[0166] The ID unique flag attribute (@idUniqueFlag) provides a mechanism for indicating whether the value of AdaptationSet@id is unique within the adaptation set 420.

[0167] In a second exemplary embodiment, the representation level attribute may be represented as an associated adaptation set ID (@associationAsId), which is defined as associating the included representation 430 to one or more adaptation sets 420. In this embodiment, the associated representation 430, the attribute @associationId, the attribute @associationAsId, and the attribute @associationType are specifically described as follows: An element in the representation 430 describes the associated representation 430, contains the attribute @associationId or @associationAsId, and optionally contains the attribute @associationType. The associated representation 430 refers to a representation 430 that provides information about the relationship between the representation 430 and other representations 430 or adaptation sets 420. The fragment 440 in the associated representation 430 may be optional and is used to decode and / or present the representation 430 or adaptation set 420 identified by @associationId or @associationAsId. It can be considered that the fragment 440 is supplementary information or descriptive information, and the attribute @associationType indicates the type of association. It should be noted that the attributes @associationId or @associationAsId and @associationType can only be used between representations 430 in different adaptation sets 420. Table 3 defines the attribute @associationId, the attribute @associationAsId and the attribute @associationType.

[0168] Table 3: Attributes @associationId, @associationAsId, and @associationType

[0169]

[0170]

[0171] In this embodiment, the carrying of the metadata track with time attributes of OMAF can be specifically described as follows. Metadata tracks with time attributes of track sample entry types such as 'invo' or 'rcvp' can be encapsulated in the DASH representation 430. The attribute @associationId or @associationAsId of the metadata representation 430 should respectively include the value of the attribute @id of the representation 430 or the adaptation set 420, indicating that the representation 430 or the adaptation set 420 contains omnidirectional media carried by (one or more) media tracks, and the above media tracks are associated with the metadata track with time attributes. The attribute @associationType of the metadata representation can be set to be equal to 'cdsc'. This embodiment can also use the attribute @idUniqueFlag in the first embodiment.

[0172] Through the first two exemplary embodiments, the properties of the representation 430 are associated with the adaptation set 420, so that all adaptation sets 430 are associated with the representation 430 containing the metadata track with time attributes. In this way, the metadata track with time attributes can refer to the entire video track group instead of referring to each representation 430, which greatly reduces the file size of the MPD 400.

[0173] The third exemplary embodiment is similar to the first exemplary embodiment, but allows the representation 430 to be associated with all the cameras 450. In this embodiment, the attribute @associationId may contain the ID value of the camera 450. When the attribute @associationId contains a specific ID value of the camera 450, the ID value of the camera 450 should be unique among the values ​​of the attribute @id of all cameras 450 and all representations 430 in the MPD 400, and should be unique among the ID values ​​of any other type of objects such as the adaptation set 420 or the pre-selection 460, which the attribute @associationId may contain. The attribute Viewpoint@value may contain a field indicating whether the ID value of the camera 450 satisfies uniqueness. Other values ​​may be exchanged with the camera 450 as described in the first exemplary embodiment, referring to the adaptation set 420.

[0174] The fourth exemplary embodiment is similar to the second exemplary embodiment, but provides a representation level attribute, which can be represented as an association slot ID (@associationVwptId), defined to associate the included representation 430 with one or more slots 450. Other values ​​can be exchanged with the adaptation set 420 and the slot 450 as described in the second exemplary embodiment.

[0175] The fifth exemplary embodiment is similar to the first exemplary embodiment, but allows the representation 430 to be associated with the preselection 460. In this embodiment, the attribute @associationId may contain the attribute @id (e.g., Preselection@id) of the preselection 460. When the attribute @associationId contains a specific ID value of the preselection 460, the ID value of the preselection 460 should be unique among the values ​​of the attribute @id of all preselections 460 and all representations 430 in the MPD 400, and should be unique among the ID values ​​of any other type of object such as the adaptation set 420 or the camera position 450. An optional attribute may be added to the XML schema of the element of the preselection 460 to indicate whether the value of the Preselection@id of the preselection 460 satisfies uniqueness. Other values ​​may be mentioned as described in the first exemplary embodiment, and the adaptation set 420 is exchanged with the preselection 460.

[0176] The sixth exemplary embodiment is substantially similar to the second exemplary embodiment, but provides a representation level attribute, which can be represented as an association ID (@associationPsId) of a preselection 460, defined to associate the included representation 430 with one or more preselections 460. Other values ​​can be exchanged with the adaptation set 420 and the preselection 460 as described in the second exemplary embodiment.

[0177] In the seventh exemplary embodiment, the attributes in exemplary embodiments 1, 2, 3, 4, 5 and / or 6 are implemented as other level attributes of the MPD 400. Specifically, the attribute @associationId, the attribute @associationAsId, the attribute @associationPsId and / or the attribute @associationVwptId as described above may be implemented as (one or more) attributes of the adaptation set 420, the preselection 460 and / or the slot 450. This allows the adaptation set 420, the preselection 460 and / or the slot 450 to be associated with other DASH objects such as other adaptation sets 420, preselections 460 and / or slots 450.

[0178] Through any of Embodiments 1 to 7, it is possible to efficiently indicate the position 450 or some other DASH object corresponding to the associated representation 430, adaptation set 420, preselection 460 or representation 430 (e.g., metadata representation 430 with time attributes) without significantly increasing the size of MPD 400.

[0179] Specific exemplary implementations are described as follows. Metadata representations with time attributes can be uniformly associated with one or more media representations. Metadata representations with time attributes can be uniformly associated with all media representations in a sub-image combination, as described below. The association descriptor is a child element of the DASH representation element in the metadata representation with time attributes. The association descriptor should contain a string in the association element [SubPicCompositionId="aa"] of type AdaptationSet, where "aa" indicates the sub-image combination identifier value. In addition, 'cdtg' can be included as the value of the attribute Association@associationKindList of the association element. Specifically, a metadata track with time attributes containing a 'cdtg' track reference uniformly describes the referenced media tracks and track groups. The 'cdtg' track reference can only exist in a metadata track with time attributes.

[0180] 2. Indicate the seat ID

[0181] Since some cameras 450 may be moved within a video, causing the corresponding position information to change, some systems may have difficulty determining whether two cameras 450 are actually the same. In addition, DASH lacks an MPD descriptor to indicate static cameras and dynamic cameras. Moreover, such a descriptor does not include a value to indicate whether the associated camera is static or dynamic. The following can solve these problems.

[0182] In the first exemplary embodiment, when both elements in the camera position 450 include the attribute @value, the attribute @value contains the same ID value of the camera position 450, then the attribute @value of the two elements in the camera position 450 can be considered to be the same regardless of the position information. Thus, the present embodiment provides a system that can determine whether multiple DASH objects (e.g., adaptation set 420) including an instance of an element of the camera position 450 can be considered to belong to the same camera position 450, thereby avoiding erroneous switching of the camera position 450 when it is not necessary, and vice versa.

[0183] In a second exemplary embodiment, in order to support the situation where there are multiple positions in DASH, the elements in the existing positions 450 should be used, but the position scheme is defined. Specifically, in MPD 400, the position element with the attribute @schemeIdUri equal to "urn:mpeg:mpegI:omaf:2018:vwpt" is called a position information (VWPT) descriptor. The VWPT descriptor is defined to indicate the position 450 in MPD 400. The same descriptor can be used for positions 450 that are static and dynamic or change over time. In one example, there can be at most one VWPT descriptor at the level of the adaptation set 420, and no VWPT descriptor should exist at any other level. The VWPT descriptor indicates the position 450 to which the adaptation set 420 belongs and whether the position of the position 450 is static or dynamic. If the position is static, the VWPT descriptor also provides the position of the position 450. The attribute @value of the VWPT descriptor is a list of comma-separated values ​​as described in Table 4:

[0184] Table 4: Attribute @value of VWPT descriptor

[0185]

[0186] When the position of the camera 450 is dynamic, as indicated by is_static being equal to 'false', the temporal metadata representation 430 shall provide the dynamic position of the camera 450, the temporal metadata representation 430 comprising a temporal metadata track comprising a specific sample entry type, e.g., 'vpps'. The attribute @associationId of the temporal metadata representation 430 shall contain the value of the viewpoint_id of the attribute @value of the VWPT descriptor of the camera 450. In addition, the attribute @associationType of the metadata representation 430 shall be set to 'cdsc'.

[0187] The value is_static may be the only value in a VWPT descriptor, indicating whether the position of the camera is static or dynamic.

[0188] 3. Associate temporal metadata representations with all media presentations

[0189] Metadata representations with time attributes are generally applicable to all media presentations. However, DASH lacks a method to associate metadata representations with time attributes to all media presentations. This problem can be solved as follows. In an exemplary embodiment, metadata representations 430 with time attributes are allowed to not include or omit or not associate attributes @associationId, @associationAsId, @associationVwptId, @associationId, @associationPsId, and any other such attributes used to associate metadata representations 430 with time attributes to other DASH objects. Therefore, it is indicated that metadata representations 430 with time attributes are applicable to all media presentations.

[0190] 4. Associate metadata tracks with time attributes to all videos

[0191] OMAF includes the following specific descriptions for or for associating a metadata track with time attributes to a media track or track group. When a metadata track with time attributes is linked to one or more media tracks with a 'cdsc' track reference, each media track is described separately. The flag bit 0 (the least significant bit is bit 0) of TrackGroupTypeBox is used to indicate the uniqueness of track_group_id. The semantics of this flag are described as follows:

[0192] (flags&1) equal to 1 in a TrackGroupTypeBox for a specific track_group_type indicates that the track_group_id in the TrackGroupTypeBox is not equal to any track_ID value nor to the track_group_id of any TrackGroupTypeBox with a different track_group_type. When (flags&1) is equal to 1 in a TrackGroupTypeBox with specific values ​​of track_group_type and track_group_id, (flags&1) shall be equal to 1 in all TrackGroupTypeBoxes with the same values ​​of track_group_type and track_group_id.

[0193] The semantics of the track_IDs of TrackReferenceBox are changed from

[0194] track_IDs is an array of integers giving the track identifiers of the referenced track. The value 0 should not be present. A value should not be repeated in the array.

[0195] Modified to:

[0196] track_IDs is an array of integers providing the track identifiers of the referenced track or the track_group_id values ​​of the referenced track group. Each value of track_IDs[i] (where i is a valid index into the track_IDs[] array) is an integer such that the included track references the track with track_ID equal to track_IDs[i] or references a track group with both track_group_id equal to track_IDs[i] and TrackGroupTypeBox with (flag&1) equal to 1. When referencing a track_group_id value, the track reference applies to each track in the referenced track group separately, unless otherwise specified in the semantics of the particular track reference type. The value 0 shall not be present. A value shall not be repeated in the array.

[0197] Metadata tracks with temporal attributes are usually applicable to the entire movie. However, the file format does not have a method to associate a metadata track with temporal attributes to the entire movie. The following can solve this problem. In an exemplary embodiment, it is allowed that the metadata track with temporal attributes does not include, omit or associate a 'cdsc' track reference. Therefore, it is indicated that the metadata track with temporal attributes applies to the entire movie.

[0198] Figure 5Schematic diagram of an exemplary video encoding device 500. The video encoding device 500 is suitable for implementing the disclosed examples / embodiments described herein. The video encoding device 500 includes a downlink port 520, an uplink port 550 and / or a transceiver unit (Tx / Rx) 510. The transceiver unit 510 includes a transmitter and / or a receiver for communicating data in the uplink and / or downlink through a network. The video encoding device 500 also includes a processor 530 and a memory 532. The processor 530 includes a logic unit and / or a CPU for processing the data. The memory 532 is used to store the data. The video encoding device 500 may also include an optical-to-electrical (OE) component, an electrical-to-optical (EO) component and / or a wireless communication component coupled to the uplink port 550 and / or the downlink port 520, and perform data communication through an optical communication network or a wireless communication network. The video encoding device 500 may also include an input and / or output (I / O) device 560 for data communication with a user. The I / O device 560 may include an output device, such as a display for displaying video data, a speaker for outputting audio data, etc. The I / O device 560 may also include input devices such as a keyboard, a mouse, a trackball, and corresponding interfaces for interacting with these output devices.

[0199] The processor 530 is implemented by hardware and software. The processor 530 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 530 communicates with the downstream port 520, the transceiver unit 510, the upstream port 550, and the memory 532. The processor 530 includes a video encoding component 514. The video encoding component 514 can implement all or part of the disclosed embodiments described herein. For example, the video encoding component 514 can be used to implement the functions of the video encoding device 104, the encoder 103, the DASH content server 111, the client 108, and / or the decoder 107, depending on the specific circumstances. For example, the video encoding component 514 can execute the relevant part of the method 200. For another example, the video encoding component 514 can receive VR video (e.g., omnidirectional video) from the camera 301, 302, and / or 303, and generate an MPD 400 associated with the file format object to support streaming of multi-camera VR video through a streaming media system. In addition, the video encoding component 514 can encode the MPD using a syntax for associating a representation 430, an adaptation set 420, a position 450, and / or a preselection with another representation 430, another adaptation set 420, another position 450, and / or another preselection, so as to achieve the purpose of associating a DASH object with a metadata track with a time attribute, etc., without encoding the association of each representation 430. This reduces the file size of the MPD, reduces the processing resources used to generate and / or process the MPD, reduces the network resources used to transmit the MPD, and / or reduces the memory resources used to store the MPD. In this way, the video encoding module 514 improves the functionality of the video encoding device 500 and solves problems unique to video encoding technology. In addition, the video encoding module 514 affects the transition of the video encoding device 500 to different states. Alternatively, the video encoding module 514 can be implemented as instructions stored in the memory 532 and executed by the processor 530 (for example, implemented as a computer program product stored in a non-transitory medium).

[0200] The memory 532 includes one or more memory types, such as disk, tape drive, solid-state drive, ROM, RAM, flash memory, TCAM, SRAM, etc. The memory 532 may be used as an overflow data storage device to store programs when such programs are selected for execution and to store instructions and data read during the execution of the programs.

[0201] Figure 6Schematic diagram of an exemplary DASH content server 600. The DASH content server 600 is used to associate DASH objects in an MPD (e.g., MPD 400 and / or MPD 125) to support streaming of multi-camera VR videos from the system 300, etc. Therefore, the DASH content server 600 can be used with the method 200 and / or the video encoding component 514 to support the encoder and / or DASH content server (respectively, the encoder 103 and / or the DASH content server 111, etc. in the system 100) to distribute multi-camera VR media content.

[0202] The DASH content server 600 may include a content receiving module 601 for receiving media content including a VR video sequence, wherein the media content is described as a plurality of adaptation sets. The DASH content server 600 may also include a DASH MPD generating module 603 for generating a DASH MPD including an association ID, wherein the association ID indicates that a portion of the media content is associated with a corresponding first camera position. The DASH content server 600 may also include a request receiving module 605 for receiving a request from a client for media content. The DASH content server 600 may also include a transmitter 607 for: upon receiving the request, transmitting the DASH MPD including the association ID to the client to support transmission of the media content to the client.

[0203] Figure 7 A flowchart of a method 700 for generating and transmitting an MPD provided for an embodiment of the present invention. In step 701, media content associated with a camera position is obtained. In step 703, a VWPT descriptor is generated based on the media content. The VWPT descriptor includes a value indicating whether the position of the camera position is static. For example, the value is_static in Table 4. In step 705, an MPD including the VWPT descriptor is generated. Finally, in step 707, the MPD is transmitted.

[0204] Figure 8 A flowchart of a method 800 for receiving and processing an MPD provided for an embodiment of the present invention. In step 801, a request for an MPD is transmitted. In step 803, the MPD is received in response to the request. The MPD includes a VWPT descriptor based on media content, and the media content is associated with a camera position. The VWPT descriptor includes a value indicating whether the position of the camera position is static. For example, the value is_static in Table 4. Finally, in step 805, the MPD is processed.

[0205] Fig. 9A flowchart of a method 900 for generating and transmitting an MPD provided for another embodiment of the present invention. In step 901, media content is obtained. In step 903, a metadata representation with time attributes is generated based on the media content. The metadata representation with time attributes is not associated with the attribute @associationId, the attribute @associationAsId, the attribute @associationVwptId, and the attribute @associationPsId. In step 905, an MPD is generated, wherein the MPD includes the metadata representation with time attributes. Finally, in step 907, the MPD is transmitted.

[0206] Fig.10 A flowchart of a method 1000 for receiving and processing an MPD provided for another embodiment of the present invention. In step 1001, a request for an MPD is transmitted. In step 1003, the MPD is received in response to the request. The MPD includes a metadata representation with time attributes based on media content. The metadata representation with time attributes is not associated with the attribute @associationId, the attribute @associationAsId, the attribute @associationVwptId, and the attribute @associationPsId. Finally, in step 1005, the MPD is processed.

[0207] Fig.11 A flowchart of a method 1100 for generating and transmitting a file format object provided by an embodiment of the present invention. In step 1101, media content is obtained. In step 1103, a metadata track with time attributes is generated according to the media content. The metadata track with time attributes is not associated with a 'cdsc' track reference. In step 1105, a file format object is generated, wherein the file format object includes the metadata track with time attributes. Finally, in step 1107, the file format object is transmitted.

[0208] Fig.12 A flowchart of a method 1200 for receiving and processing a file format object provided by an embodiment of the present invention. In step 1201, a request for a file format object is transmitted. In step 1203, the file format object is received in response to the request. The file format object includes a metadata track with time attributes based on media content. The metadata track with time attributes is not associated with a 'cdsc' track reference. Finally, in step 1205, the file format object is processed.

[0209] Fig.13A flowchart of a method 1300 for generating and transmitting an MPD provided for an embodiment of the present invention. In step 1301, media content associated with a first camera position and a second camera position is obtained. The first position of the first camera position is static, and the second position of the second camera position is dynamic. In step 1303, a first VWPT descriptor is generated according to the media content, and the first camera position is described. In step 1305, a second VWPT descriptor is generated according to the media content, and the second camera position is described. The first VWPT descriptor and the second VWPT descriptor can be VWPT descriptors described in combination with Table 4. In step 1307, an MPD including the first VWPT descriptor and the second VWPT descriptor is generated. Finally, in step 1309, the MPD is transmitted.

[0210] The second camera position may be associated with a metadata representation with a time attribute, the metadata representation with a time attribute comprising a metadata track with a time attribute. The metadata track with a time attribute comprises information describing the second position. The first VWPT descriptor may include a first value indicating whether the first position is static. The second VWPT descriptor may include a second value indicating whether the second position is static. The first value and the second value may be the value is_static in Table 4.

[0211] Fig.14 Flowchart of method 1400 for receiving and processing MPD according to another embodiment of the present invention. In step 1401, a request for MPD is transmitted. In step 1403, the MPD is received in response to the request. The MPD includes a first VWPT descriptor and a second VWPT descriptor. The first VWPT descriptor is based on media content and describes a first camera position. The first position of the first camera position is static. The second VWPT descriptor is based on the media content and describes a second camera position. The second position of the second camera position is dynamic. The first VWPT descriptor and the second VWPT descriptor may be the VWPT descriptors described in Table 4. Finally, in step 1405, the MPD is processed. For methods 700 to 1400, the transceiver unit 510 may perform the receiving and transmitting steps, and the processor 530 may perform the remaining steps.

[0212] The second camera position may be associated with a metadata representation with a time attribute, the metadata representation with a time attribute comprising a metadata track with a time attribute. The metadata track with a time attribute comprises information describing the second position. The first VWPT descriptor may include a first value indicating whether the first position is static. The second VWPT descriptor may include a second value indicating whether the second position is static. The first value and the second value may be the value is_static in Table 4.

[0213] An apparatus includes a memory element and a processor element coupled to the memory element. The processor element is used to perform the following method: obtain media content associated with a first camera position and a second camera position, wherein a first position of the first camera position is static and a second position of the second camera position is dynamic; generate a first VWPT descriptor according to the media content and describe the first camera position; generate a second VWPT descriptor according to the media content and describe the second camera position; generate an MPD of the first VWPT descriptor and the second VWPT descriptor; and transmit the MPD.

[0214] Although the present invention provides several embodiments, it should be understood that the disclosed systems and methods may be embodied in a variety of other specific forms without departing from the spirit or scope of the present invention. The examples of the present invention should be considered illustrative rather than restrictive, and the present invention is not limited to the details given herein. For example, various elements or components may be combined or integrated in another system, or certain features may be omitted or not implemented.

[0215] In addition, without departing from the scope of the present invention, the techniques, systems, subsystems and methods described and illustrated as discrete or separate in various embodiments may be combined or integrated with other systems, components, techniques or methods. Other items shown or discussed as coupled may also be coupled or communicated indirectly via an interface, device or intermediate component, either electrically, mechanically or otherwise. Other variations, substitutions, and modified examples will be apparent to those skilled in the art and do not depart from the spirit and scope disclosed herein.

Claims

1. A method, characterized in that The method comprises: Get the media content associated with the camera position; Generate a camera position information (VWPT) descriptor according to the media content, wherein the VWPT descriptor includes a camera position identifier (ID) value indicating a camera position ID of the camera position; Generate a media presentation description (MPD) file including the VWPT descriptor; The MPD is transmitted.

2. The method according to claim 1, characterized in that The VWPT descriptor also includes a center x value indicating the absolute position of the center of the camera position on the x-axis in the global three-dimensional coordinates.

3. The method according to any one of claims 1 to 2, characterized in that The VWPT descriptor also includes a center y value indicating an absolute position of the center of the camera position on the y-axis in the global three-dimensional coordinates.

4. The method according to any one of claims 1 to 3, characterized in that The VWPT descriptor also includes a center z value indicating an absolute position of the center of the camera position on the z-axis in the global three-dimensional coordinates.

5. A device, characterized in that: The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 1 to 4.

6. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the device performs any one of claims 1 to 4.

7. A method, characterized in that The method comprises: Transmitting a request for a media presentation description (MPD); In response to the request, receiving the MPD, wherein the MPD includes a video position information (VWPT) descriptor based on media content, the media content is associated with a position, and the VWPT descriptor includes a position identifier (ID) value indicating a position ID of the position; The MPD is processed.

8. The method according to claim 7, characterized in that The VWPT descriptor also includes a center x value indicating the absolute position of the center of the camera position on the x-axis in the global three-dimensional coordinates.

9. The method according to any one of claims 7 to 8, characterized in that The VWPT descriptor also includes a center y value indicating an absolute position of the center of the camera position on the y-axis in the global three-dimensional coordinates.

10. The method according to any one of claims 7 to 9, characterized in that The VWPT descriptor also includes a center z value indicating an absolute position of the center of the camera position on the z-axis in the global three-dimensional coordinates.

11. A device, characterized in that: The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 7 to 10.

12. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the device performs any one of claims 7 to 10.

13. A method, characterized in that The method comprises: Get media content; generating a timed metadata representation based on the media content, wherein the timed metadata representation is not associated with attribute information; generating a media presentation description (MPD) file, wherein the MPD includes the metadata representation with time attributes; The MPD is transmitted.

14. The method according to claim 13, characterized in that The attribute information includes an attribute @associationId, an attribute @associationAsId, an attribute @associationVwptId, and an attribute @associationPsId.

15. The method according to claim 13 or 14, characterized in that The metadata representation with time attributes is also not associated with any other such attributes, any other such attributes are used to associate the metadata representation with time attributes to other Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) objects.

16. The method according to any one of claims 13 to 15, characterized in that: The metadata track with time attributes applies to all media presentations.

17. A device, characterized in that: The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 13 to 16.

18. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the device performs any one of claims 13 to 16.

19. A method, characterized in that The method comprises: Transmitting a request for a media presentation description (MPD); In response to the request, receiving the MPD, wherein the MPD includes a metadata representation with time attributes based on the media content, the metadata representation with time attributes being unassociated with attribute information; The MPD is processed.

20. The method according to claim 19, characterized in that The attribute information includes an attribute @associationId, an attribute @associationAsId, an attribute @associationVwptId, and an attribute @associationPsId.

21. The method according to claim 19 or 20, characterized in that The metadata representation with time attributes is also not associated with any other such attributes, any other such attributes are used to associate the metadata representation with time attributes to other Dynamic Adaptive Streaming over Hypertext Transfer Protocol (DASH) objects.

22. The method according to any one of claims 19 to 21, characterized in that: The metadata track with time attributes applies to all media presentations.

23. A device, characterized in that: The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 19 to 21.

24. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the apparatus performs any one of claims 19 to 22.

25. A method, characterized in that The method comprises: Get media content; generating a metadata track with time attributes according to the media content; generating a file format object, wherein the file format object includes the metadata track with time attributes; Transmits the file format object.

26. The method according to claim 25, characterized in that The metadata track with time attributes is not associated with the 'cdsc' track reference.

27. The method according to claim 25 or 26, characterized in that The 'cdsc' track reference indicates that the metadata track with time attributes describes each media track separately.

28. The method according to any one of claims 25 to 27, characterized in that: The metadata track with time attributes is applicable to all movies.

29. A device, characterized in that: The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 25 to 28.

30. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the device performs any one of claims 25 to 28.

31. A method, characterized in that The method comprises: Transmits a request for a file format object; In response to the request, receiving the file format object, wherein the file format object includes a metadata track with time attributes based on media content; Handles the file format object.

32. The method according to claim 31, characterized in that The metadata track with time attributes is not associated with the 'cdsc' track reference.

33. The method according to claim 31 or 32, characterized in that The 'cdsc' track reference indicates that the metadata track with time attributes describes each media track separately.

34. The method according to any one of claims 31 to 33, characterized in that: The metadata track with time attributes is applicable to all movies.

35. A device, characterized in that: The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 31 to 34.

36. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the device performs any one of claims 31 to 34.

37. A method, characterized in that The method comprises: Acquire media content associated with the first camera and the second camera; Generate a first camera position information (VWPT) descriptor according to the media content, and describe the first camera position; Generate a second VWPT descriptor according to the media content and describe the second camera position; generating a media presentation description (MPD) file including the first VWPT descriptor and the second VWPT descriptor; The MPD is transmitted.

38. The method according to claim 37, characterized in that The first position of the first camera is static, and the second position of the second camera is dynamic.

39. The method according to claim 37 or 28, characterized in that The second camera position is associated with a metadata representation with time attributes, the metadata representation with time attributes comprises a metadata track with time attributes, and the metadata track with time attributes comprises information describing the second position.

40. The method according to any one of claims 37 to 39, characterized in that The first VWPT descriptor further includes a first camera identifier (ID) value indicating a first camera ID of the first camera; the second VWPT descriptor further includes a second camera ID value indicating a second camera ID of the second camera.

41. The method according to any one of claims 37 to 40, characterized in that The first VWPT descriptor also includes a first center x value, indicating a first absolute position of a first center of the first camera position on the x-axis and in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center x value, indicating a second absolute position of a second center of the second camera position on the x-axis and in the global three-dimensional coordinates.

42. The method according to any one of claims 37 to 41, characterized in that The first VWPT descriptor also includes a first center y value, indicating a first absolute position of a first center of the first camera position on the y-axis in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center y value, indicating a second absolute position of a second center of the second camera position on the y-axis in the global three-dimensional coordinates.

43. The method according to any one of claims 37 to 42, characterized in that The first VWPT descriptor also includes a first center z value, indicating a first absolute position of a first center of the first camera position on the z-axis in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center z value, indicating a second absolute position of a second center of the second camera position on the z-axis in the global three-dimensional coordinates.

44. A device, characterized in that The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 37 to 43.

45. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the apparatus performs any one of claims 37 to 43.

46. ​​A method, characterized in that The method comprises: Transmitting a request for a media presentation description (MPD); In response to the request, receiving the MPD, wherein the MPD includes a first camera position information (VWPT) descriptor and a second VWPT descriptor, wherein the first VWPT descriptor is based on the media content and describes the first camera position; and the second VWPT descriptor is based on the media content and describes the second camera position; The MPD is processed.

47. The method according to claim 46, characterized in that The first position of the first camera is static, and the second position of the second camera is dynamic.

48. The method according to claim 46 or 47, characterized in that The second camera position is associated with a metadata representation with time attributes, the metadata representation with time attributes comprises a metadata track with time attributes, and the metadata track with time attributes comprises information describing the second position.

49. The method according to any one of claims 46 to 48, characterized in that The first VWPT descriptor further includes a first camera identifier (ID) value indicating a first camera ID of the first camera; the second VWPT descriptor further includes a second camera ID value indicating a second camera ID of the second camera.

50. The method according to any one of claims 46 to 49, characterized in that The first VWPT descriptor also includes a first center x value, indicating a first absolute position of a first center of the first camera position on the x-axis and in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center x value, indicating a second absolute position of a second center of the second camera position on the x-axis and in the global three-dimensional coordinates.

51. The method according to any one of claims 46 to 50, characterized in that The first VWPT descriptor also includes a first center y value, indicating a first absolute position of a first center of the first camera position on the y-axis in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center y value, indicating a second absolute position of a second center of the second camera position on the y-axis in the global three-dimensional coordinates.

52. The method according to any one of claims 46 to 51, characterized in that The first VWPT descriptor also includes a first center z value, indicating a first absolute position of a first center of the first camera position on the z-axis in the global three-dimensional coordinates; the second VWPT descriptor also includes a second center z value, indicating a second absolute position of a second center of the second camera position on the z-axis in the global three-dimensional coordinates.

53. A device, characterized in that The device comprises: Memory; A processor coupled to the memory, wherein the processor is configured to perform any one of claims 46 to 52.

54. A computer program product, characterized in that The computer program product comprises computer executable instructions stored in a non-transitory medium; when a processor executes the computer executable instructions, the apparatus performs any one of claims 46 to 52.