Video processing method, device and medium

Abstract

An embodiment of the present disclosure provides a solution for video processing. A video processing method is proposed, the method comprising: performing conversion between a bitstream of a video and a media file of the video, where the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

Description

[Technical field]

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 298,461, filed January 11, 2022, the entire contents of which are expressly incorporated herein by reference.

[0002] FIELD OF THE DISCLOSURE Embodiments of the present disclosure relate generally to video processing techniques, and more specifically to signaling external stream tracks within a media file. [Background technology]

[0003] Media streaming applications are typically based on Internet Protocol (IP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP) transport methods, and typically rely on file formats such as ISO Base Media File Format (ISOBMFF). One such streaming system is Dynamic Adaptive Streaming over HTTP (DASH). In DASH, multiple representations may exist for video and / or audio data of multimedia content, and different representations may correspond to different coding characteristics (e.g., different profiles or levels of a video coding standard, different bit rates, different spatial resolutions, etc.). In addition, Enhanced Dependent Random Access Point (EDRAP) image-based video coding and streaming has been proposed. Therefore, it is worth studying the signaling of external stream tracks. Summary of the Invention

[0004] An embodiment of the present disclosure provides a solution for video processing.

[0005] In a first aspect, a video processing method is proposed, comprising: performing a conversion between a bitstream of a video and a media file of the video, where the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples being encapsulated based on a restricted scheme, the sample entry including an indication associated with the restricted scheme, a value of the indication indicating that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

[0006] According to the method according to the first aspect of the present disclosure, a new restriction scheme is adopted to indicate that a sample in an external stream track is allowed to contain multiple coded images. Compared with the conventional solutions, the proposed method has the advantage that it can support EDRAP-based technologies more efficiently.

[0007] In a second aspect, an apparatus for processing video data is proposed, the apparatus comprising a processor and a non-transitory memory having instructions recorded thereon that, when executed by the processor, cause the processor to perform a method according to the first aspect of the present disclosure.

[0008] In a third aspect, a non-transitory computer readable storage medium is proposed, said non-transitory computer readable storage medium storing instructions for causing a processor to perform the method according to the first aspect of the present disclosure.

[0009] In a fourth aspect, another non-transitory computer-readable recording medium is proposed, the non-transitory computer-readable recording medium storing a video bitstream generated by a method executed by a video processing device, the method comprising: performing a conversion between the bitstream and a media file of the video, the media file including an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples being encapsulated based on a restricted scheme, the sample entry including an indication associated with the restricted scheme, a value of the indication indicating that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

[0010] In a fifth aspect, a method for storing a bitstream of a video is proposed, comprising the steps of performing a conversion between the bitstream and a media file of the video, and storing the bitstream on a non-transitory computer-readable recording medium, where the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

[0011] In a sixth aspect, another non-transitory computer-readable recording medium is proposed, the non-transitory computer-readable recording medium storing a media file of a video generated by a method executed by a video processing device, the method comprising: performing a conversion between a bitstream of the video and the media file, the media file including an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples being encapsulated based on a restricted scheme, the sample entry including an indication associated with the restricted scheme, a value of the indication indicating that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

[0012] In a seventh aspect, a method for storing a media file of a video is proposed, the method comprising the steps of performing a conversion between a bitstream of the video and the media file, and storing the media file on a non-transitory computer-readable recording medium, where the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

[0013] This description is provided to introduce a selection of concepts in a simplified form that are more fully described in the detailed description. This description is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. [Brief description of the drawings]

[0014] The above and other objects, features, and advantages of the embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: In the exemplary embodiments of the present disclosure, like reference numerals generally refer to like components.

[0015] [Figure 1] 1 shows a block diagram of an example video coding system according to some embodiments of the present disclosure.

[0016] [Diagram 2] 1 shows a block diagram of an example video encoder according to some embodiments of the present disclosure.

[0017] [Diagram 3] 2 shows a block diagram of an exemplary video decoder according to some embodiments of the present disclosure.

[0018] [Figure 4] FIG. 1 is a diagram illustrating the concept of a random access point (RAP).

[0019] [Diagram 5] FIG. 2 is another diagram illustrating the concept of RAP.

[0020] [Figure 6] A diagram illustrating the Dependent Random Access Point (DRAP) concept.

[0021] [Figure 7] FIG. 2 is another diagram illustrating the concept of DRAP.

[0022] [Figure 8] FIG. 1 illustrates the concept of Enhanced Dependent Random Access Point (EDRAP).

[0023] [Figure 9] FIG. 2 is another diagram illustrating the concept of EDRAP.

[0024] [Figure 10]A diagram showing EDRAP-based video streaming.

[0025] [Figure 11] FIG. 2 is another diagram for illustrating EDRAP-based video streaming.

[0026] [Figure 12] 1 illustrates a flowchart of a video processing method according to some embodiments of the present disclosure.

[0027] [Figure 13] FIG. 1 illustrates a block diagram of a computing device capable of implementing various embodiments of the present disclosure.

[0028] Throughout the drawings, the same or similar reference numbers typically refer to the same or similar elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Next, the principles of the present disclosure will be described with reference to some embodiments. It should be understood that these embodiments are provided for illustrative purposes only, to help those skilled in the art understand and embody the present disclosure, and do not imply any limitations on the scope of the present disclosure. The disclosure described herein can be embodied in various ways other than the methods described below.

[0030] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0031] References in this disclosure to "one embodiment," "one embodiment," "exemplary embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an exemplary embodiment, it is noted that it is within the knowledge of one of ordinary skill in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0032] Terms such as "first" and "second" may be used herein to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0033] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. It will be further understood that the terms "comprise," "comprising," "having," "having," "containing," and / or "including," as used herein, specify the presence of stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0034] Example Environment 1 is a block diagram illustrating an example video coding system 100 that may utilize techniques of this disclosure. As shown, video coding system 100 may include a source device 110 and a destination device 120. Source device 110 may also be referred to as a video encoding device, and destination device 120 may also be referred to as a video decoding device. In operation, source device 110 may be configured to generate encoded video data, and destination device 120 may be configured to decode the encoded video data generated by source device 110. Source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0035] Video source 112 may include sources such as video capture devices. Examples of video capture devices include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system that generates video data, and / or combinations thereof.

[0036] The video data may include one or more images. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a series of bits forming a coded representation of the video data. The bitstream may include coded images and associated data. A coded image is a coded representation of an image. The associated data may include sequence parameter sets, image parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The encoded video data may be transmitted directly to the destination device 120 through the network 130A via the I / O interface 116. The encoded video data may be stored in a storage medium / server 130B for access by the destination device 120.

[0037] Destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130B. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120 configured to interface with an external display device.

[0038] Video encoder 114 and video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or future standards.

[0039] FIG. 2 is a block diagram illustrating an example of a video encoder 200, which may be an example of the video encoder 114 in the system 100 shown in FIG. 1, according to some embodiments of the disclosure.

[0040] Video encoder 200 may be configured to embody any or all of the techniques of this disclosure. In the example of FIG. 2, video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0041] In some embodiments, the video encoder 200 may include a division unit 201, a prediction unit 202, which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy encoding unit 214.

[0042] In other examples, video encoder 200 may include more, fewer, or different functional components. In one example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference image is the image in which the current video block is located.

[0043] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be integrated but are depicted separately in the example of FIG. 2 for illustrative purposes.

[0044] The division unit 201 may divide an image into one or more video blocks. The video encoder 200 and the video decoder 300 may support a variety of video block sizes.

[0045] The mode selection unit 203 may, for example, select one of intra or inter coding modes based on the error result, provide the resulting intra-coded or inter-coded block to the residual generation unit 207 to generate residual block data, and provide the encoded block to the reconstruction unit 212 to reconstruct and use as a reference image. In some examples, the mode selection unit 203 may select a combined intra and inter prediction (CIIP) mode, in which prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 may select a resolution of the motion vector of the block (e.g., sub-pixel or integer pixel accuracy).

[0046] To perform inter prediction on the current video block, motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 may determine a prediction video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.

[0047] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on a current video block depending on whether the current video block is in an I slice, a P slice, or a B slice, for example. As used herein, an "I slice" may refer to a portion of an image composed of macroblocks, all of which are based on macroblocks within the same image. Additionally, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of an image composed of macroblocks that are independent of macroblocks within the same image.

[0048] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may look for a reference image in list 0 or list 1 for a reference video block of the current video block. Motion estimation unit 204 may then generate a reference index indicating a reference image in list 0 or list 1 that contains the reference video block, and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information of the current video block. Motion compensation unit 205 may generate a predictive video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0049] Alternatively, in other examples, motion estimation unit 204 may perform bidirectional prediction on the current video block. Motion estimation unit 204 may look up a reference image in list 0 for a reference video block of the current video block, and may look up a reference image in list 1 for another reference video block of the current video block. Motion estimation unit 204 may then generate a reference index indicating the reference images in lists 0 and 1 that contain the reference video blocks, and a motion vector indicating a spatial displacement between the reference video block and the current video block. Motion estimation unit 204 may output the reference index and the motion vector of the current video block as motion information of the current video block. Motion compensation unit 205 may generate a predictive video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0050] In some examples, motion estimation unit 204 may output a full set of motion information for the decoding process of the decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal the motion information of the current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0051] In one example, motion estimation unit 204 may indicate, in a syntax structure associated with the current video block, a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0052] In another example, motion estimation unit 204 may identify another video block and a motion vector differential (MVD) in a syntax structure associated with the current video block. The motion vector differential indicates a difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may determine the motion vector of the current video block using the motion vector and the motion vector differential of the indicated video block.

[0053] As discussed above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0054] Intra prediction unit 206 may perform intra prediction on the current video block. When intra prediction unit 206 performs intra prediction on the current video block, intra prediction unit 206 may generate predictive data for the current video block based on decoded samples of other video blocks in the same image. The predictive data for the current video block may include a predicted video block and various syntax elements.

[0055] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., as indicated by a minus sign) the prediction video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.

[0056] In other examples, such as in skip mode, residual data for the current video block may not be present, and residual generation unit 207 may not perform the subtraction operation.

[0057] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0058] After transform processing unit 208 generates the transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0059] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in buffer 213.

[0060] After reconstruction unit 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video blocking artifacts in the video blocks.

[0061] Entropy encoding unit 214 may receive data from other functional components of video encoder 200. Once entropy encoding unit 214 receives the data, it may perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream including the entropy encoded data.

[0062] FIG. 3 is a block diagram illustrating an example of a video decoder 300, which may be an example of the video decoder 124 in the system 100 shown in FIG. 1, according to some embodiments of the disclosure.

[0063] The video decoder 300 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 3, the video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of the video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0064] 3, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. The video decoder 300 may, in some examples, perform a decoding path that is generally inverse to the encoding path described with respect to the video encoder 200.

[0065] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 may decode the entropy coded video data, from which the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information, for example, by performing AMVP and merge mode. AMVP is used and includes derivation of some most likely candidates based on data from neighboring PBs and reference pictures. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and, in the case of prediction regions in B slices, identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from spatially or temporally neighboring blocks.

[0066] The motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter used with sub-pixel precision may be included in the syntax element.

[0067] Motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using an interpolation filter used by video encoder 200 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filter used by video encoder 200 according to received syntax information and generate a predictive block using the interpolation filter.

[0068] The motion compensation unit 302 may use at least a portion of the syntactic information to determine the size of blocks used to encode frames and / or slices of the encoded video sequence, partition information describing how each macroblock of the image of the encoded video sequence is divided, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-encoded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently from other slices of the same image with respect to entropy coding, signal prediction, and residual signal reconstruction. A slice may be either an entire image or a region of an image.

[0069] The intra prediction unit 303 may form a prediction block from spatially neighboring blocks, e.g., using an intra prediction mode received in the bitstream. The inverse quantization unit 304 inverse quantizes, or dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0070] The reconstruction unit 306 may, for example, obtain a decoded block by adding the residual block and a corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. Optionally, a deblocking filter may be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and also generates the decoded video for presentation on a display device.

[0071] Some exemplary embodiments of the present disclosure will be described in detail below. Section headings are used herein for ease of understanding, but it should be understood that they do not limit the embodiments disclosed in a section to only that section. Furthermore, although certain embodiments are described with reference to versatile video coding or other specific video codecs, the disclosed techniques are applicable to other video coding techniques. Furthermore, although some embodiments describe video coding steps in detail, it will be understood that a corresponding decoding step that undoes the coding is performed by a decoder. Furthermore, the term video processing encompasses video coding or compression, video decoding or decompression, and video transcoding, which represents video pixels from one compressed format to another compressed format or at a different compressed bit rate. 1. Overview This disclosure relates to media file formats, specifically to signaling an External Stream Track (EST) with appropriate sample entry types, allowing one sample in the EST to contain multiple coded pictures. The EST provides the external pictures required for random access from an Extended Dependent Random Access Point (EDRAP) picture in a Main Stream Track (MST). These ideas may be applied individually or in various combinations to media files according to any media file format, such as the ISO Base Media File Format (ISOBMFF) and file formats derived from ISOBMFF. 2. Background 2.1 Video Coding Standards Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure in which temporal prediction and transform coding are utilized. In 2015, the Joint Video Exploration Team (JVET) was established by VCEG and MPEG to explore future video coding technologies beyond HEVC. Since then, many new methods have been adopted by JVET and incorporated into reference software called the Joint Exploration Model (JEM). JVET was renamed to Joint Video Experts Team (JVET) when the Versatile Video Coding (VVC) project was officially launched. VVC is a new coding standard that targets a 50% bitrate reduction compared to HEVC and was finalized at the 19th meeting of JVET, which ended on July 1, 2020. The Versatile Video Coding (VVC) standard (ITU-T H.266 | ISO / IEC 23090-3) and the associated Versatile Supplementary Enhancement Information (VSEI) standard (ITU-T H.274 | ISO / IEC 23002-7) are designed to be used in the widest range of applications, including traditional uses such as television broadcasting, videoconferencing, and playback from storage media, as well as new and advanced uses such as adaptive bitrate streaming, video region extraction, composition and merging of content from multiple coded video bitstreams, multiview video, scalable layered coding, and viewport-adaptive 360° immersive media. 2.2 File Format Standards Media streaming applications are typically based on IP, TCP, and HTTP transport methods and typically rely on file formats such as ISO Base Media File Format (ISOBMFF). One such streaming system is Dynamic Adaptive Streaming over HTTP (DASH). When using video formats with ISOBMFF and DASH, file format specifications specific to the video format, such as AVC file format and HEVC file format, may be required for encapsulation of video content in ISOBMFF tracks and DASH representations and segments. Important information about the video bitstream, such as profile, tier, level, and many other information, may need to be exposed as file format level metadata and / or DASH Media Presentation Description (MPD) for content selection purposes, e.g., for selection of appropriate media segments both for initialization at the start of a streaming session and for stream adaptation during a streaming session. Similarly, when using image formats with ISOBMFF, file format specifications specific to the image format, such as AVC image file format and HEVC image file format, may be required. 2.3 Random Access and its Support in HEVC and VVC Random access refers to starting the access and decoding of a bitstream from a picture that is not the first picture of the bitstream in decoding order. To support tuning and channel switching in broadcast / multicast and multiparty videoconferencing, local playback and seeking in streaming, and stream adaptation in streaming, the bitstream needs to contain frequent random access points, which are typically intra-coded pictures, but could also be inter-coded pictures (e.g., in the case of gradual decoding updates). HEVC includes signaling of Intra Random Access Point (IRAP) pictures in the NAL unit header through the NAL unit type. Three types of IRAP pictures are supported: Instantaneous Decoder Refresh (IDR), Clean Random Access (CRA), and Broken Link Access (BLA) pictures. IDR pictures constrain the inter-picture prediction structure to not reference pictures prior to the current Group of Pictures (GOP) and are traditionally called Closed GOP Random Access Points. CRA pictures are less restrictive by allowing certain pictures to reference pictures prior to the current GOP that are all discarded in the case of random access. CRA pictures are traditionally called Open GOP Random Access Points. BLA pictures typically result from splicing of two bitstreams or parts thereof at a CRA picture, such as during a stream switch. To enable better system usage of IRAP pictures, a total of six different NAL units are defined to signal characteristics of IRAP pictures, which can be used to better match the Stream Access Point types defined in the ISO Base Media File Format (ISOBMFF), which are utilized for random access support in Dynamic Adaptive Streaming over HTTP (DASH). VVC supports three types of IRAP pictures, two types of IDR pictures (one with or one without an associated RADL picture), and one type of CRA picture, which are essentially the same as HEVC. The BLA picture type in HEVC is not included in VVC mainly for two reasons: i) the basic functionality of a BLA picture can be realized by a CRA picture and an end-of-sequence NAL unit, and the presence of an end-of-sequence NAL unit indicates that the following picture starts a new CVS in a single-layer bitstream; and ii) during the development of VVC, it is desired to specify fewer NAL unit types than HEVC, as indicated by using 5 bits instead of 6 bits for the NAL unit type field in the NAL unit header. Another important difference between VVC and HEVC random access support is that VVC supports GDR in a more prescriptive manner. In GDR, bitstream decoding can start with an inter-coded picture, and initially the entire picture area can be decoded correctly, and after a few pictures the entire picture area will be correct. AVC and HEVC also support GDR, and use the recovery point SEI message to signal GDR random access points and recovery points. In VVC, a new NAL unit type is specified to indicate GDR pictures, and recovery points are signaled in the picture header syntax structure. CVS and bitstreams are allowed to start with a GDR picture. This means that the entire bitstream is allowed to contain only inter-coded pictures, not a single intra-coded picture. The main advantage of specifying GDR support in this way is that it provides a behavior that is compatible with GDR. GDR allows an encoder to smooth the bitrate of the bitstream by distributing intra-coded slices or blocks across multiple pictures, as opposed to intra-coding the entire picture, which can significantly reduce end-to-end delay, which is becoming more important than ever as ultra-low latency applications such as wireless display, online gaming, and drone-based applications become more prevalent. Another GDR-related feature in VVC is virtual boundary signaling. The boundary between the refreshed (i.e. correctly decoded) and non-refreshed regions in the image between the GDR image and its recovery point can be signaled as a virtual boundary, and when notified, no in-loop filtering is applied across the boundary, so that no decoding mismatch of some samples at or near the boundary occurs. This is useful if the application decides to display the correctly decoded region during the GDR process. IRAP images and GDR images are collectively referred to as Random Access Point (RAP) images. 2.4 Enhanced Dependent Random Access Point (EDRAP) based video coding, storage and streaming 2.4.1 Concepts and Standards Support The concept of EDRAP-based video coding, storage and streaming is explained with reference to the following Figures 4 to 11. Figures 4 and 5 are diagrams showing the existing concept of random access points (RAP). The application (e.g. adaptive streaming) decides the frequency of random access points (RAP), for example, RAP period 1 second or 2 seconds. Conventionally, RAP is provided by coding IRAP pictures, as shown in Figure 4. Note that inter-prediction references for non-key pictures between RAP pictures are not shown, and the output order is from left to right. In case of random access from CRA6, the decoder receives and correctly decodes the pictures, as shown in Figure 5. Figures 6 and 7 show the concept of Dependent Random Access Point (DRAP). The DRAP approach provides improved coding efficiency by allowing DRAP pictures (and subsequent pictures) to reference previous IRAP pictures for inter prediction, as shown in Figure 6. Note that inter prediction for non-key pictures between RAP pictures is not shown, and the output order is left to right. When randomly accessing from DRAP6, the decoder receives and correctly decodes the pictures as shown in Figure 7. Figures 8 and 9 show the concept of Enhanced Dependent Random Access Point (EDRAP). The EDRAP approach offers a bit more flexibility by allowing EDRAP pictures (and subsequent pictures) to reference some of the previous RAP pictures (IRAP or EDRAP), as shown in Figure 8. Note that inter-prediction for non-key pictures between RAP pictures is not shown, and the output order is from left to right. When randomly accessing from EDRAP6, the decoder receives and correctly decodes the pictures as shown in Figure 9. 10 and 11 are diagrams showing EDRAP-based video streaming. When random access is performed from a segment starting from EDRAP6 or when switching to the segment, the decoder receives and decodes the segment as shown in FIG. EDRAP-based video coding is supported by the EDRAP Indication SEI message included in the developing amendments to the VSEI standard, the storage portion is supported by the EDRAP Sample Group and associated External Stream Track References included in the developing amendments to the ISOBMFF standard, and the streaming portion is supported by the Main Stream Representation (MSR) and External Stream Representation (ESR) descriptors included in the developing amendments to the DASH standard. 2.4.2 EDRAP Indication SEI Message An amendment to the VSEI standard is under development. The latest draft specification of this amendment includes the specification of the EDRAP indication SEI message. The syntax and semantics of the EDRAP indication SEI message are as follows: [Table 1] The image associated with an Extended DRAP (EDRAP) Indication SEI message is called an EDRAP image. The presence of the EDRAP indication SEI message indicates that the picture ordering and picture referencing constraints specified in this subclause apply. These constraints allow a decoder to properly decode an EDRAP picture in the same layer and the pictures that follow it in both decoding order and output order without having to decode any other pictures in the same layer, except for the list of referenceablePictures, which consists of a list of IRAP or EDRAP pictures in the same CLVS and in decoding order identified by the edrap_ref_rap_id[ i ] syntax element. The constraints implied by the presence of an EDRAP indication SEI message, all of which shall apply, are as follows: - The EDRAP image is a trailing image. - EDRAP images have a temporal sublayer identifier equal to 0. - EDRAP images do not include images in the same layer in the active entry of the reference image list, except for referenceablePictures. - Any image that is in the same layer and that follows the EDRAP image in both decoding order and output order shall not include in its active entries of its reference picture list any images that are in the same layer and that precede the EDRAP image in decoding order or output order, except for referenceablePictures. - any image in the list referenceablePictures does not contain in its active entry of its reference picture list an image that is not in the same layer and in its previous position in the list referenceablePictures. NOTE - As a result, the first image in referenceablePictures will not contain any images from the same layer in the active entry of its reference image list, even if it is an EDRAP image rather than an IRAP image. edrap_rap_id_minus1 plus 1 specifies the RAP image identifier (denoted as RapPicId) of the EDRAP image. Each IRAP or EDRAP image has an associated RapPicId value. The RapPicId value for an IRAP image is inferred to be equal to 0. The RapPicId values ​​for any two EDRAP images associated with the same IRAP image shall be different. Specifies that when edrap_leading_pictures_decodable_flag is equal to 1, both of the following constraints apply: - Any image that is in the same layer and that follows an EDRAP image in decoding order shall follow in output order the image that is in the same layer and that precedes the EDRAP image in decoding order. - Any image that is in the same layer and that follows the EDRAP image in decoding order and precedes the EDRAP image in output order shall not include in its active entries of its reference picture list any images that are in the same layer and precede the EDRAP image in decoding order, except for referenceablePictures. If edrap_leading_pictures_decodable_flag is equal to 0, no such constraint is imposed. edrap_reserved_zero_12bits SHALL be equal to 0 in bitstreams conforming to this version of this specification. Other values ​​of edrap_reserved_zero_12bits are reserved for future use by ITU-T | ISO / IEC. Decoders SHALL ignore values ​​of edrap_reserved_zero_12bits. edrap_num_ref_rap_pics_minus1 plus 1 indicates the number of IRAP or EDRAP pictures that are in the same CLVS as the EDRAP picture and that can be included in the active entries of the EDRAP picture's reference picture list. edrap_ref_rap_id[ i ] indicates the RapPicId of the ith RAP image that may be included in the active entry of the reference image list of the EDRAP image. The ith RAP image shall be either an IRAP image associated with the current EDRAP image or an EDRAP image associated with the same IRAP image as the current EDRAP image. 2.4.3 EDRAP Sample Groups and Associated External Stream Track References An amendment to the ISOBMFF standard is under development. The latest draft specification of this amendment includes the specification of an EDRAP sample group and associated external stream track references. The specifications of these two ISOBMFF features are as follows: 3.1 Definition External Elementary Streams An elementary stream including access units comprising an external image External Images A picture in an external elementary stream in EST that is required for inter-prediction reference in decoding of an elementary stream in MST when randomly accessing from a specific EDRAP picture in MST. External Stream Tracks (EST) Tracks containing external elementary streams Main Stream Track (MST) A track that contains the video elementary stream. 3.2 Abbreviations EDRAP Extended Dependent Random Access Point EST External Stream Track MST Mainstream Truck 8.3.3.4 Related external stream track references A track reference of type "aest" (meaning "associated external stream track") may be included in the MST that references an EST. If the MST has a track reference of type "aest", the following applies: - The MST shall have at least one sample containing an EDRAP image. - For each sample, sampleA, in the MST that contains an EDRAP picture, there shall be exactly one sample, sampleB, in the associated EST that has the same decoding time as sampleA, and starting from sampleB, consecutive samples in the associated EST shall exclusively contain all external pictures required when randomly accessing from the EDRAP picture contained in sampleA. All samples in the EST shall be identified as sync samples. The EST track header flags shall have both track_in_movie and track_in_preview set to 0. 10.11 Extended DRAP (EDRAP) Sample Group 10.11.1 Definition This sample group is similar to the DRAP sample group specified in sub-clause 10.8, but allows for more flexible cross-RAP referencing. An EDRAP sample is a sample from which all samples in decoding order and output order can be correctly decoded if the closest initial sample preceding the EDRAP sample and one or more other identified EDRAP samples that are earlier in decoding order than the EDRAP sample are available for reference. An initial sample is a sync sample or a SAP sample of SAP type 1, 2 or 3 marked as such by a SAP sample group. For example, if the 32nd sample in a file is an initial sample consisting of an I-picture, then the 48th sample may consist of a P-picture and be marked as a member of the EDRAP sample group, thereby indicating that random access can be performed on the 48th sample by first decoding the 32nd sample (ignoring samples 33-47) and then continuing decoding from the 48th sample. NOTE: EDRAP samples may only be used in combination with SAP samples of types 1, 2, and 3. This is to enable the creation of a decodable sample sequence by concatenating the preceding SAP sample and zero or more other identified EDRAP samples that are earlier in decoding order than the EDRAP sample with the EDRAP sample and the samples that follow the EDRAP sample in decoding and output order. A sample can become a member of the EDRAP sample group only if the following conditions are met: - An EDRAP sample refers only to the closest preceding initial sample and to one or more other identified EDRAP samples that are earlier in the decoding order than the EDRAP sample. - An EDRAP sample and all samples following the EDRAP sample in decoding and output order may be correctly decoded if decoding begins at the EDRAP sample after decoding the closest preceding SAP sample of type 1, 2 or 3 that is marked as a Sync sample or a SAP sample group, and after decoding zero or more other identified EDRAP samples that precede the EDRAP sample in decoding order. 10.11.2 Syntax

number

number

number

number

number

[0072] Further details of the embodiments of the present disclosure related to the signaling of external stream tracks in a media file are described below. The embodiments of the present disclosure should be considered as examples for explaining the general concept, and should not be interpreted in a narrow sense. Moreover, these embodiments can be applied individually or in any combination.

[0073] As used herein, the term "sample" may refer to all data associated with a single time. The term "sample entry" may refer to a structure that defines and describes the format of several samples in a track. The term "track" may refer to a timing sequence of related samples. The term "box" may refer to an object-oriented building block defined by a unique type identifier and length.

[0074] FIG. 12 shows a flowchart of a method 1200 of video processing according to some embodiments of the present disclosure. The method 1200 may be embodied in a client or a server. The term "client" as used herein may refer to computer hardware or software that accesses services provided by a server as part of a client-server model of a computer network. As an example, a client may be a smartphone or a tablet. The term "server" as used herein may refer to a device capable of computing, where the client accesses services through a network. The server may be a physical computing device or a virtual computing device.

[0075] As shown in Figure 12, at 1202, a conversion is performed between a bitstream of a video and a media file of the video. A media file is a collection of data that establishes a bounded or unbounded presentation of media content in the context of a file format (e.g., an International Organization for Standardization (ISO) based media file format). In some embodiments, the conversion may include generating a media file and storing the bitstream in the media file. Additionally or alternatively, the conversion may include parsing the media file to reconstruct the bitstream.

[0076] In some embodiments, the media file includes an external stream track. For example, in the context of EDRAP-based video processing, the external stream track may be referenced by a corresponding main stream track. The external stream track includes a set of samples. A sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme. By way of example and not limitation, a sample entry transformation using the restricted scheme may be applied to hide the sample entry type. The sample entry type may be the original sample entry type of the sample entry. It should be understood that the sample entry type may be any other sample entry type, such as the current sample entry type of the sample entry. The scope of the present disclosure is not limited in this respect.

[0077] Moreover, the sample entry includes an indication associated with the restricted scheme, the value of the indication indicating that a sample in the set of samples is permitted to include multiple coded images of the external stream track. By way of example and not limitation, the sample entry may include a first data structure (e.g., a restricted scheme information box, also referred to as RestrictedSchemeInfoBox) including information about a restricted scheme. The first data structure may include a second data structure (e.g., a scheme type box, also referred to as SchemeTypeBox) identifying the restricted scheme.

[0078] Further, the indication may include a first field (e.g., a scheme_type field) in the second data structure. In one example, the value of the first field may be equal to a predefined code, such as "aest". If a restricted scheme with a scheme type field equal to the predefined value is used for the track, the samples associated with the sample entry may include multiple samples of the original track. It should be appreciated that the predefined code may be any other suitable code, such as "spkt". It should also be appreciated that the above discussion and / or examples are provided for illustrative purposes only. The scope of the present disclosure is not limited in this respect.

[0079] Considering the above, a new restricted scheme is adopted to indicate that a sample in an external stream track is allowed to contain multiple coded images. Compared to traditional solutions, the proposed method has the advantage of being able to support EDRAP-based techniques more efficiently.

[0080] In some embodiments, the Uniform Resource Identifier (URI) of the restricted scheme may not be present in the media file. For example, the sample entry may include a first data structure (e.g., a restricted scheme information box) that includes information about the restricted scheme, and the first data structure may include a second data structure (e.g., a scheme type box) that identifies the restricted scheme. The value of the least significant bit (i.e., Bit 0) of the flags field in the second data structure may be equal to 0.

[0081] In some embodiments, a sample entry type may be encapsulated with a predefined sample entry code, such as "resv." Additionally, the sample entry may include a first data structure (e.g., a Restricted Scheme Information Box) that includes information about a restricted scheme. The first data structure may include a third data structure (e.g., an original format box, also referred to as OriginalFormatBox) that stores the sample entry type.

[0082] In some additional or alternative embodiments, the media file may include a fourth data structure (e.g., an External Stream Information Box, also referred to as EasternStreamInfoBox) that includes information associated with the external stream track. By way of example and not limitation, the fourth data structure may include at least one of: (1) a field indicating a track identity (ID) of a main stream track associated with the external stream track; (2) a field indicating a maximum allowable number of coded pictures in a sample in the external stream track; (3) a field indicating whether a first coded picture in each sample in the external stream track may be an Intra Random Access Point (IRAP) picture, the first coded picture being at a first position in the respective sample; (4) a field indicating whether each sample in the external stream track includes all parameter sets required for decoding the coded pictures in the respective sample; or (5) a field indicating whether, for the external stream track, a first picture of a corresponding EDRAP picture in the main stream track associated with the external stream track is decodable when a decoding process starts from an EDRAP picture.

[0083] In some embodiments, the first picture of a corresponding EDRAP picture in a main stream track associated with an external stream track is decodable when the decoding process starts with the EDRAP picture if the following two constraints are met: (1) any picture in the same layer and following the EDRAP picture in decoding order shall follow in output order the picture in the same layer and preceding the EDRAP picture in decoding order, and (2) any picture in the same layer and following the EDRAP picture in decoding order and preceding the EDRAP picture in output order shall not include in its active entry of reference picture list any picture in the same layer and preceding the EDRAP picture in decoding order, except for referenceablePictures.

[0084] According to an embodiment of the present disclosure, a non-transitory computer-readable recording medium is proposed. A video bitstream is stored in the non-transitory computer-readable recording medium. The bitstream may be generated by a method executed by a video processing device. According to the method, conversion between the bitstream and a video media file is performed. The media file includes an external stream track including a set of samples. A sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme. The sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

[0085] According to an embodiment of the present disclosure, a method for storing a video bitstream is proposed. In the method, a conversion between a video bitstream and a media file is performed. The media file includes an external stream track including a set of samples. A sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme. The sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is allowed to include multiple coded images of the external stream track. Furthermore, the bitstream is stored in a non-transitory computer-readable recording medium.

[0086] According to an embodiment of the present disclosure, another non-transitory computer-readable recording medium is proposed. A video media file is stored in the non-transitory computer-readable recording medium. The bitstream may be generated by a method executed by a video processing device. According to the method, conversion between the video bitstream and the media file is performed. The media file includes an external stream track including a set of samples. A sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme. The sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is allowed to include multiple coded images of the external stream track.

[0087] According to an embodiment of the present disclosure, a method for storing a video media file is proposed. In the method, conversion between a video bitstream and a media file is performed. The media file includes an external stream track including a set of samples. A sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme. The sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is allowed to include multiple coded images of the external stream track. Furthermore, the media file is stored on a non-transitory computer-readable recording medium.

[0088] Implementations of the present disclosure may be described in light of the following clauses, the features of which may be combined in any reasonable manner.

[0089] Clause 1. A video processing method, comprising: performing conversion between a bitstream of a video and a media file of the video, wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples being encapsulated based on a restricted scheme, the sample entry including an indication associated with the restricted scheme, and a value of the indication indicating that samples within the set of samples are permitted to include multiple coded images of the external stream track.

[0090] Clause 2. The method of clause 1, wherein the sample entry type is an original sample entry type.

[0091] Clause 3. The method of any one of clauses 1 or 2, wherein the sample entry includes a first data structure containing information about the restricted scheme, the first data structure including a second data structure identifying the restricted scheme, and the instruction includes a first field within the second data structure.

[0092] Clause 4. The method of clause 3, wherein the first data structure is a restricted scheme information box, the second data structure is a scheme type box, and the first field is a scheme type field.

[0093] Clause 5. The method of any one of clauses 3 or 4, wherein the value of the first field is equal to a predetermined code.

[0094] Clause 6. The method of any one of clauses 1 to 5, wherein a Uniform Resource Identifier (URI) of the restricted scheme is not present in the media file.

[0095] Clause 7. The method of any one of clauses 1 to 6, wherein the sample entry includes a first data structure including information about the restricted scheme, the first data structure including a second data structure identifying the restricted scheme, and a value of a least significant bit of a flags field in the second data structure is equal to 0.

[0096] Clause 8. The method of clause 7, wherein the first data structure is a restricted scheme information box and the second data structure is a scheme type box.

[0097] Clause 9. The method of any one of clauses 1 to 8, wherein the sample entry type is encapsulated in a predefined sample entry code.

[0098] Clause 10. A method according to any one of clauses 1 to 9, wherein the sample entry comprises a first data structure containing information about the restricted scheme, the first data structure comprising a third data structure storing the sample entry type.

[0099] Clause 11. The method of clause 10, wherein the first data structure is a restricted scheme information box and the third data structure is an original format box.

[0100] Clause 12. The method of any one of clauses 1 to 11, wherein the media file includes a fourth data structure including information associated with the external stream track.

[0101] Clause 13. The method of clause 12, wherein the fourth data structure includes at least one of a field indicating a track identity (ID) of a mainstream stream track associated with the external stream track, a field indicating a maximum allowable number of coded pictures in a sample in the external stream track, a field indicating whether a first coded picture in each sample in the external stream track is an Intra Random Access Point (IRAP) picture, where the first coded picture is at a first position in the respective sample, a field indicating whether each sample in the external stream track includes all parameter sets required for decoding the coded pictures in the respective sample, or a field indicating, for each sample in the external stream track, whether a first picture of a corresponding Extended Dependent Random Access Point (EDRAP) picture in the mainstream stream track associated with the external stream track is decodable when the decoding process starts from the EDRAP picture.

[0102] Clause 14. The method of any one of clauses 1 to 13, wherein the fourth data structure is an External Stream Information Box.

[0103] Clause 15. The method of any one of clauses 1 to 14, wherein the media file is in an International Organization for Standardization (ISO) based media file format.

[0104] Clause 16. The method of any one of clauses 1 to 15, wherein the converting comprises generating the media file and storing the bitstream in the media file.

[0105] Clause 17. The method of any one of clauses 1 to 15, wherein the conversion comprises parsing the media file to reconstruct the bitstream.

[0106] Clause 18. Apparatus for processing video data comprising a processor and a non-transitory memory having instructions recorded thereon, the instructions, when executed by the processor, causing the processor to perform a method according to any one of clauses 1 to 17.

[0107] Clause 19. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any one of clauses 1 to 17.

[0108] Clause 20. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, the method comprising the step of performing a conversion between the bitstream and a media file of the video, wherein the media file comprises an external stream track comprising a set of samples, a sample entry type of a sample entry for the set of samples being encapsulated based on a restricted scheme, the sample entry comprising an indication associated with the restricted scheme, and a value of the indication indicating that samples within the set of samples are permitted to include multiple coded images of the external stream track.

[0109] Clause 21. A method for storing a bitstream of a video, comprising the steps of performing a conversion between the bitstream and a media file of the video, and storing the bitstream on a non-transitory computer-readable recording medium, wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry including an indication associated with the restricted scheme, and a value of the indication indicating that samples in the set of samples are permitted to include multiple coded images of the external stream track.

[0110] Clause 22. A non-transient computer-readable recording medium storing a media file of a video generated by a method performed by a video processing device, the method comprising the step of performing a conversion between a bitstream of the video and the media file, wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is permitted to include multiple coded images of the external stream track.

[0111] Clause 23. A method for storing a media file of a video, comprising: performing a conversion between a bitstream of the video and the media file; and storing the media file on a non-transitory computer-readable recording medium, wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry including an indication associated with the restricted scheme, and a value of the indication indicating that samples in the set of samples are permitted to include multiple coded images of the external stream track.

[0112] Exemplary Devices 13 illustrates a block diagram of a computing device 1300 capable of implementing various embodiments of the present disclosure. The computing device 1300 may be embodied as or included in the source device 110 (or the video encoder 114 or 200) or the destination device 120 (or the video decoder 124 or 300).

[0113] It will be appreciated that the computing device 1300 shown in FIG. 13 is for illustrative purposes only and is not intended to suggest any limitation on the functionality and scope of the embodiments of the present disclosure.

[0114] 13, the computing device 1300 includes a general-purpose computing device 1300. The computing device 1300 may include at least one or more processors or processing units 1310, a memory 1320, a storage unit 1330, one or more communication units 1340, one or more input devices 1350, and one or more output devices 1360.

[0115] In some embodiments, the computing device 1300 may be embodied as any user terminal or server terminal having computing capabilities. The server terminal may be a server provided by a service provider, a large-scale computing device, etc. The user terminal may be any type of mobile, fixed, or portable terminal, including, for example, a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof (including accessories and peripherals of these devices, or any combination thereof). It is contemplated that the computing device 1300 may support any type of interface to a user, such as "wearable" circuitry.

[0116] The processing unit 1310 may be a physical or virtual processor and may implement various processes based on programs stored in the memory 1320. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of the computing device 1300. The processing unit 1310 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0117] Computing device 1300 typically includes a variety of computer storage media. Such media may be any media accessible by computing device 1300, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 1320 may be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory), or any combination thereof. Storage unit 1330 may be any removable or non-removable medium, including a machine-readable medium such as a memory, a flash memory drive, a magnetic disk, or another medium that can be used to store information and / or data and that can be accessed by computing device 1300.

[0118] The computing device 1300 may further include additional removable / non-removable, volatile / non-volatile memory media. Although not shown in FIG. 13, a magnetic disk drive that reads from and writes to a removable non-volatile magnetic disk and an optical disk drive that reads from and writes to a removable non-volatile optical disk may be provided. In such a case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0119] The communication unit 1340 communicates with further computing devices via a communication medium. Moreover, the functionality of the components in the computing device 1300 may be embodied by a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the computing device 1300 can operate in a networked environment using logical connections with one or more other servers, networked personal computers (PCs), or further general network nodes.

[0120] The input device(s) 1350 may be one or more of a variety of input devices such as a mouse, a keyboard, a tracking ball, a voice input device, etc. The output device(s) 1360 may be one or more of a variety of output devices such as a display, a loudspeaker, a printer, etc. The communication unit 1340 may enable the computing device 1300 to further communicate with one or more external devices (not shown), such as a storage device and a display device, one or more devices allowing a user to interact with the computing device 1300, or any device (such as a network card, a modem, etc.) allowing the computing device 1300 to communicate with one or more other computing devices, as needed. Such communication may be performed via an input / output (I / O) interface (not shown).

[0121] In some embodiments, instead of being integrated into a single device, some or all of the components of computing device 1300 may be located in a cloud computing architecture. In a cloud computing architecture, the components may be provided remotely and work together to perform the functions described in this disclosure. In some embodiments, cloud computing provides computing, software, data access, and storage services without the end user needing to be aware of the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services over a wide area network (such as the Internet) using appropriate protocols. For example, a cloud computing provider provides applications over a wide area network that can be accessed through a web browser or other computing components. The software or components of the cloud computing architecture and corresponding data may be stored on a server located in a remote location. Computing resources in a cloud computing environment may be consolidated or distributed at remote data center locations. A cloud computing infrastructure may act as a single access point for users but provide services through a shared data center. Thus, a cloud computing architecture may be used to provide the components and functions described herein from a service provider located in a remote location. Alternatively, they may be provided from a traditional server or installed directly or otherwise on a client device.

[0122] The computing device 1300 may be used to implement video encoding / decoding in embodiments of the present disclosure. The memory 1320 may include one or more video processing modules 1325 having one or more program instructions. These modules are accessible and executable by the processing unit 1310 to perform the functions of various embodiments described herein.

[0123] In an example embodiment performing video encoding, the input device 1350 may receive video data to be encoded as input 1370. The video data may be processed, for example, by the video processing module 1325 to generate an encoded bitstream. The encoded bitstream may be provided as output 1380 via the output device 1360.

[0124] In an example embodiment performing video decoding, the input device 1350 may receive an encoded bitstream as an input 1370. The encoded bitstream may be processed, for example, by the video processing module 1325 to generate decoded video data. The decoded video data may be provided as output 1380 via the output device 1360.

[0125] Although the present disclosure has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application as defined by the appended claims. Such variations are intended to be within the scope of the present application. Accordingly, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. 1. A video processing method comprising the steps of: performing conversion between a bitstream of a video and a media file of the video; wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that samples within the set of samples are permitted to include multiple coded images of the external stream track.

2. The method of claim 1 , wherein the sample entry type is an original sample entry type.

3. 3. The method of claim 1, wherein the sample entry includes a first data structure containing information about the restricted scheme, the first data structure including a second data structure identifying the restricted scheme, and the indication includes a first field in the second data structure.

4. 4. The method of claim 3, wherein the first data structure is a restricted scheme information box, the second data structure is a scheme type box, and the first field is a scheme type field.

5. 5. The method according to claim 3, wherein the value of the first field is equal to a predetermined code.

6. The method of claim 1 , wherein a Uniform Resource Identifier (URI) of the restricted scheme is not present in the media file.

7. 7. The method of claim 1, wherein the sample entry includes a first data structure containing information about the restricted scheme, the first data structure including a second data structure identifying the restricted scheme, and wherein a value of a least significant bit of a flags field in the second data structure is equal to 0.

8. 8. The method of claim 7, wherein the first data structure is a restricted scheme information box and the second data structure is a scheme type box.

9. The method of claim 1 , wherein the sample entry type is encapsulated in a predefined sample entry code.

10. 10. The method of claim 1, wherein the sample entry includes a first data structure that includes information about the restricted scheme, the first data structure including a third data structure that stores the sample entry type.

11. The method of claim 10 , wherein the first data structure is a restricted scheme information box and the third data structure is an original format box.

12. The method of claim 1 , wherein the media file includes a fourth data structure that includes information associated with the external stream track.

13. The fourth data structure is A field indicating a track identity (ID) of a main stream track associated with the external stream track; a field indicating the maximum allowed number of coded pictures in a sample in said external stream track; a field indicating whether the first coded picture in each sample in the external stream track is an Intra Random Access Point (IRAP) picture, the first coded picture being in the first position within the respective sample; a field indicating whether each sample in the external stream track contains all parameter sets necessary for decoding the coded image in the respective sample, or a field indicating, for each sample in the external stream track, whether a first image of a corresponding Extended Dependent Random Access Point (EDRAP) image in a main stream track associated with the external stream track is decodable when a decoding process starts from the EDRAP image; The method of claim 12 , comprising at least one of:

14. The method of claim 1 , wherein the fourth data structure is an External Stream Information Box.

15. The method of claim 1 , wherein the media file is in an International Organization for Standardization (ISO) based media file format.

16. The method of claim 1 , wherein the converting comprises generating the media file and storing the bitstream in the media file.

17. The method of claim 1 , wherein the conversion comprises parsing the media file to reconstruct the bitstream.

18. 18. An apparatus for processing video data comprising a processor and a non-transitory memory having instructions recorded thereon, the instructions, when executed by the processor, causing the processor to perform a method according to any one of claims 1 to 17.

19. A non-transitory computer readable storage medium storing instructions that cause a processor to perform the method of any one of claims 1 to 17.

20. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by a video processing device, comprising: The method includes performing a conversion between the bitstream and a media file of the video; wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that samples within the set of samples are permitted to include multiple coded images of the external stream track.

21. 1. A method for storing a video bitstream, comprising: performing conversion between the bitstream and a media file of the video; storing the bitstream on a non-transitory computer-readable recording medium; wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that samples within the set of samples are permitted to include multiple coded images of the external stream track.

22. 1. A non-transitory computer-readable recording medium for storing a media file of a video generated by a method performed by a video processing device, comprising: The method includes performing a conversion between the video bitstream and the media file; wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that a sample in the set of samples is permitted to include multiple coded images of the external stream track.

23. 1. A method for storing a video media file, comprising: performing a conversion between the video bitstream and the media file; storing the media file on a non-transitory computer-readable recording medium; wherein the media file includes an external stream track including a set of samples, a sample entry type of a sample entry for the set of samples is encapsulated based on a restricted scheme, the sample entry includes an indication associated with the restricted scheme, and a value of the indication indicates that samples within the set of samples are permitted to include multiple coded images of the external stream track.

Images