Pictures and layer contained in vvc image item

Abstract

To provide a method for processing a coded representation of an image according to a file format.SOLUTION: A method includes a step of performing conversion between a visual media file and a bitstream. A media file format specifies that an image item of a particular type value in the visual media file includes a single access unit of the bitstream. The single access unit is either an Intra Random Access Picture (IRAP) access unit or a Gradual Decoding Refresh (GDR) access unit according to a video coding format. All pictures in the GDR access unit are identified as recovery points in the bitstream.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application is a divisional application of Japanese Patent Application No. 2023 - 135957, which is a divisional application of Japanese Patent Application No. 2021 - 142854, claiming priority and the benefit thereof to U.S. Provisional Patent Application No. 63 / 073829, filed on September 2, 2020. For all purposes under the law, the entire disclosure of the above - mentioned application is incorporated by reference as part of the disclosure of this application.

[0002] [Technical Field] This patent document is related to the encoding and decoding of images and videos.

Background Art

[0003] Digital videos account for the largest bandwidth usage on the Internet and other digital communication networks. As the number of user devices capable of receiving and displaying videos increases, the bandwidth demand for digital video usage is expected to continue growing.

Summary of the Invention

[0004] This patent document discloses techniques that can be used by a video encoder and a decoder to process a coded representation of a video or an image according to a file format.

[0005] In one exemplary aspect, a method of processing image data includes performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that an image item of a particular type value in the visual media file includes a single access unit of the bitstream. The single access unit is either an Intra Random Access Picture (IRAP) access unit according to the video coding format or a Gradual Decoding Refresh (GDR) access unit according to the video coding format. All pictures within the GDR access unit are identified as recovery points within the bitstream.

[0006] In another exemplary aspect, a method of processing image data includes performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies excluding layers that do not belong to a target output layer set for an image item of a particular type value in the visual media file.

[0007] In another exemplary aspect, a method of processing image data includes performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units each consisting of one or more pictures according to a video coding format. The bitstream is coded according to a video coding format. The media file format specifies that an image item of a particular type value in the visual media file includes at least a portion of an access unit in which a picture has one or more sub-pictures.

[0008] In another exemplary aspect, a method of processing image data includes performing a conversion between a visual media file and a bitstream. The visual media file has image items each having a sequence of one or more pictures according to a media file format. The bitstream includes access units each consisting of one or more pictures each belonging to a layer according to a video coding format. The media file format specifies that an image item having a picture originating from the bitstream is allowed to be associated with different instances of a characteristic descriptor indicating a high-level characteristic of the bitstream.

[0009] In another exemplary aspect, a method of processing image data includes performing a conversion between a visual media file and a bitstream. The visual media file has image items each including a sequence of one or more pictures according to a media file format, and the bitstream includes access units each consisting of one or more pictures each belonging to a layer according to a video coding format. The media file format specifies that in response to a record of an operating point included in a characteristic descriptor of an operating point indicating a high-level characteristic of the bitstream, at least one of a value of a first syntax element in the record or a value of a second syntax element in the record is constrained to be a predetermined value.

[0010] In another exemplary aspect, a video processing method is disclosed. The method includes performing a conversion between a visual media including a sequence of one or more images and a bitstream representation according to a file format, where the file format is configured to include one or more syntax elements indicating transition characteristics between one or more of the images during display of the one or more images.

[0011] In another exemplary aspect, another video processing method is disclosed. The method includes performing a conversion between a visual media including a sequence of one or more images and a bitstream representation according to a file format, where the file format specifies that the file format is restricted according to rules when the visual media is represented by a file having a specific file brand.

[0012] In another exemplary aspect, another video processing method is disclosed. The method includes performing a conversion between a visual media including a sequence of one or more images and a bitstream representation according to a file format, where the file format is configured to indicate an image type of one or more images according to rules.

[0013] In yet another exemplary aspect, a video encoder device is disclosed. The video encoder has a processor configured to implement the above method.

[0014] In yet another exemplary aspect, a video decoder device is disclosed. The video decoder has a processor configured to implement the above method.

[0015] In yet another exemplary aspect, a computer-readable medium storing code is disclosed. The code embodies one of the methods described herein in a form of code executable by a processor.

[0016] In yet another example aspect, a computer-readable medium storing a bitstream is disclosed. The bitstream is generated using the methods described herein.

[0017] These and other features are described throughout this specification.

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Modes for Carrying Out the Invention

[0019] Section headings are used in this specification for ease of understanding and are not intended to limit the applicability of the technologies and embodiments disclosed in each section to that section only. Further, H.266 terminology is used in some descriptions only for ease of understanding and not to limit the scope of the disclosed technologies. Accordingly, the technologies described herein are applicable to other video codec protocols and designs as well. In this specification, editorial changes are indicated in the text by showing deleted text with a strikethrough and added text in highlighted (including bold italic) form with respect to the current draft of the VVC standard.

[0020] 1. Overview This specification relates to image file formats. Specifically, it relates to the signaling and storage of images and image transitions in media files based on the ISO base media file format. The ideas may be applied individually or in various combinations to images coded by any codec, such as the Versatile Video Coding (VVC) standard, and to any image file format, such as the VVC image file format under development.

[0021] 2. Abbreviations AU Access Unit AUD Access Unit Delimiter AVC Advanced Video Coding BP Buffering Period CLVS Coded Layer Video Sequence CLVSS Coded Layer Video Sequence Start CPB Coded Picture Buffer CRA Clean Random Access CTU Coding Tree Unit CVS Coded Video Sequence DCI Decoding Capability Information DPB Decoded Picture Buffer DUI Decoding Unit Information EOB End Of Bitstream EOS End Of Sequence GDR Gradual Decoding Refresh HEVC High Efficiency Video Coding HRD Hypothetical Reference Decoder IDR Instantaneous Decoding Refresh ILP Inter-Layer Prediction ILRP Inter-Layer Reference Picture IRAP Intra Random Access Picture JEM Joint Exploration Model LTRP Long-Term Reference Picture MCTS Motion-Constrained Tile Sets NAL Network Abstraction Layer OLS Output Layer Set PH Picture Header POC Picture Order Count PPS Picture Parameter Set PT Picture Timing PTL Profile, Tier and Level PU Picture Unit RAP Random Access Point RBSP Raw Byte Sequence Payload SEI Supplemental Enhancement Information SLI Subpicture Level Information SPS Sequence Parameter Set STRP Short-Term Reference Picture SVC Scalable Video Coding VCL Video Coding Layer VPS Video Parameter Set VTM VVC Test Model VUI Video Usability Information VVC Versatile Video Coding

[0022] 3. Initial discussion 3.1 Video coding standard Video coding standards have evolved mainly through the development of well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, and ISO / IEC produced MPEG-1 and MPEG-4 Visual. The two organizations jointly produced H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure that utilizes temporal prediction and transform coding. To explore future video coding technologies beyond HEVC, the JVET (Joint Video Exploration Team) was jointly established by VCEG and MPEG in 2015. Since then, many new methods have been introduced by the JVET and placed in a reference software named JEM (Joint Exploration Model). The JVET was later renamed to JVET (Joint Video Experts Team) when the VVC (Versatile Video Coding) project officially started. VVC is a new coding standard aiming for a 50% bitrate reduction compared to HEVC, which was summarized by the JVET at its 19th meeting that ended on July 1, 2020.

[0023] The VVC (Versatile Video Coding) standard specification (ITU-T H.266 | ISO / IEC 23090-3) and the related VSEI (Versatile Supplemental Enhancement Information) standard specification (ITU-T H.274 | ISO / IEC 23002-7) are designed for use in the widest possible range of applications, including both conventional uses such as television broadcasting, video conferencing, or playback from storage media, and newer and more advanced use cases such as adaptive bitrate streaming, video region extraction, synthesis and merging of content from multiple coded video bitstreams, multi-view video, scalable layer coding, and viewport-adaptive 360° immersive media.

[0024] 3.2 File Format Standard Specification Media streaming applications typically rely on IP, TCP, and HTTP transport methods and usually depend on file formats such as the ISO Base Media File Format (ISOBMFF). One such streaming system is Dynamic Adaptive Streaming over HTTP (DASH). To use video formats according to ISOBMFF and DASH, file format specifications specific to video formats such as the AVC file format and the HEVC file format are required for the encapsulation of video content in ISOBMFF tracks and in DASH representations and segments. Important information regarding the video bitstream, such as profile, tier, and level, among many others, needs to be published as file format level metadata and / or DASH Media Presentation Description (MPD) for the purpose of content selection, e.g., for the selection of appropriate media segments for both the initialization at the start of a streaming session and the stream adaptation during the streaming session.

[0025] Similarly, to use image formats according to ISOBMFF, file format specifications specific to image formats such as the AVC image file format and the HEVC image file format may be required.

[0026] 3.3 VVC Video File Format The VVC video file format is a file format for the storage of VVC video content based on ISOBMFF and is currently under development by MPEG.

[0027] 3.4 VVC Image File Format and Image Transition The VVC image file format is a file format for storing image content coded using VVC, based on ISOBMFF, and is currently under development by MPEG.

[0028] In some cases, the design for slideshow signaling includes push - behind of image transition effects such as wipes, zooms, fades, splits, dissolves, etc. The transition effects are signaled in a transition effect characteristic structure, which is associated with the first of two consecutive items involved in the transition, includes the transition type, and, if applicable, notifies other transition information such as the transition direction and transition shape.

[0029] 4. Examples of technical problems solved by the disclosed technical solutions The latest designs of the VVC image file format and the signaling of image transition effects have the following problems.

[0030] 1) In an application based on a slideshow or other type of image with a transition effect from one image to another, the time for the transition often does not need to be exact, but for a good user experience, it should not be too long or too short. And the best transition period depends on the content and the transition type. Therefore, from the perspective of the user experience, it is useful to notify the recommended transition period, and the recommended value is determined by the content creator.

[0031] 2) In the latest VVC image file format draft specification, depending on a specific VVC image item type and file brand, an access unit including multiple pictures of multiple layers can be included in the VVC bitstream of the image item. Here, some pictures may be inter-coded, that is, including B or P slices predicted using inter-layer prediction specified in VVC. In other words, there is no interoperability point through either the image item type or the file brand, and the image item can only include one picture that is intra-coded (that is, only includes an intra-coded I slice). In the VVC standard itself, such interoperability points are provided by the definitions of two still image profiles, the Main10 still image profile and the Main10 4:4:4 still image profile.

[0032] 3) An item of type 'vvc1' is specified as follows: An item of type 'vvc1' consists of NAL units of a VVC bitstream segmented by length as specified below, and the bitstream strictly contains one access unit. Remark 2 An item of type 'vvc1' may consist of an IRAP access unit defined in ISO / IEC 23090-3, can include more than one coded picture, and includes at most one coded picture with any specific value of nuh_layer_id. However, no access unit can be an access unit in such an image item. Therefore, the first part of the above Remark 2 should be moved to the basic definition (that is, the first part quoted above), and the missing part of the GDR access unit should be added.

[0033] 4) The following sentence exists: The 'vvc1' image item should contain the layers included in the layer set identified by the associated TargetOlsProperty, and may also include other layers. When allowing other layers than those included in the identified OLS, which entity within the application system is supposed to set the correct value of the target OLS index in the associated TargetOlsProperty? In any case, this value needs to be set accurately, for example, by the file composer. Since discarding unnecessary pictures in unnecessary layers is an easy operation for the file composer, it makes sense not to allow any unnecessary pictures in unnecessary layers at all.

[0034] 5) The following constraints exist: Image items originating from the same bitstream should be associated with the same VvcOperatingPointsInformationProperty. However, the VVC bitstream may contain multiple CVSs that may have different operating points.

[0035] 6) In the following text, the values of other syntax elements of VvcOperatingPointRecord, such as ptl_max_temporal_id[i] (the temporal ID of the top - level sub - layer representation where the level information exists in the i - th profile_tier_level() syntax structure) and op_max_temporal_id, should also be constrained: When included in VvcOperatingPointsInformationProperty, the values of the syntax elements of VvcOperatingPointsRecord are constrained as follows: frame_rate_info_flag must be equal to 0. As a result, avgFrameRate and constantFrameRate do not exist and their semantics are not specified. The bit_rate_info_flag shall be equal to 0. As a result, maxBitRate and avgBitRate do not exist and their semantics are not specified.

[0036] 7) The following text exists: When a VVC sub-picture item is suitable for being decoded by a VVC decoder and consumed without other VVC sub-picture items, the VVC sub-picture item shall be stored as an item of type 'vvc1'. Otherwise, the VVC sub-picture item shall be stored as an item of type 'vvs1' and shall be formatted as a series of NAL units preceded by a length field, as defined in L.2.2.1.2. There are the following problems with this: a) This condition is not clear enough to be used as a compliance requirement condition (for example, when considering how to verify whether the requirement is satisfied), so it needs to be clarified. b) Here, the bitstream of an image item of type 'vvc1' can include exactly a subset of a VVC access unit, so the use of an image item of type 'vvc1' does not strictly conform to the above definition that the bitstream contains exactly one VVC access unit. c) It is not clear whether it is allowed to have one VVC image item of type 'vvc1' that includes a picture containing multiple 'extractable' sub-pictures.

[0037] 8) The following statement does not include an OPI NAL unit: VPS, DCI, SPS, PPS, AUD, PH, EOS, and EOB NAL units shall not be present in both the item and the samples of a 'vvs1' item. However, the operation point information (OPI) NAL unit shall be treated similarly here.

[0038] 9) Only one transition effect (e.g., zoom, rotation) is allowed for a given image or for a region within a given image. However, in actual applications, multiple effects may be applied to one image or to one region within a given image.

[0039] 5. Exemplary Embodiments and Solutions To solve the above problems and others, the methods summarized below are disclosed. The items should be regarded as examples for explaining general concepts and should not be interpreted in a narrow sense. Furthermore, these items may be applied individually or may be combined in some way.

[0040] 1) To solve Problem 1, a recommended transition period may be notified for the transition from one image to another. a. In one example, alternatively, a mandatory transition period is notified for the transition from one image to another. b. In one example, the value notified, i.e., the value of the recommended or mandatory transition period, is determined by the content creator. c. In one example, one transition period is notified for each transition characteristic. d. In one example, one transition period is notified for each type of transition. e. In one example, one transition period is notified for a list of transition characteristics. f. In one example, one transition period is notified for a list of types of transitions. g. In one example, one transition period is notified for all transitions.

[0041] 2) To solve Problem 2, one or more file brands are defined such that the VVC bitstream included in an image item conforming to such a brand contains only one access unit that contains only one picture (or a portion thereof) that is intra-coded. a. Alternatively, one or more file brands are defined such that the VVC bitstream included in an image item conforming to such a brand shall contain exactly one access unit that contains exactly one picture (or a portion thereof) that is intra / IBC / palette-coded. i. Alternatively, one or more file brands are defined such that the VVC bitstream included in an image item conforming to such a brand shall contain exactly one access unit that contains exactly one I picture (or a portion thereof). b. In one example, the values of such file brands are specified as 'vvic', 'vvi1', 'vvi2'. c. In one example, additionally, the VVC bitstream included in such an image item shall conform to the Main10 still picture profile, Main10 4:4:4 still picture profile, Main10 profile, Main10 4:4:4 profile, multi-layer Main10 profile, or multi-layer Main10 4:4:4 profile. i. Alternatively, additionally, the VVC bitstream included in such an image item shall conform to the Main10 still picture profile, Main10 4:4:4 still picture profile, Main10 profile, or Main10 4:4:4 profile. ii. Alternatively, additionally, the VVC bitstream included in such an image item shall conform to the Main10 still picture profile or Main10 4:4:4 still picture profile. d. In one example, it may be specified that an image item conforming to such a brand shall not have any of the following characteristics: Target Output Layer Set Property (TargetOlsProperty), VVC Operating Points Information Property (VvcOperatingPointsInformationProperty).

[0042] 3) To solve problem 2, one or more image item types are defined such that the VVC bitstream included in an image item of such type includes only one access unit that contains only pictures that are intra-coded. a. Alternatively, one or more image item types are defined such that the VVC bitstream included in an image item of such type includes only one access unit that contains only pictures that are intra / palette / IBC-coded. i. Alternatively, one or more image item types are defined such that the VVC bitstream included in an image item of such type includes only one access unit that contains only I pictures. b. In one example, the type value of such an image item type is specified as 'vvc1' or 'vvc2'. c. In one example, additionally, the bitstream in such an image item is required to conform to the Main10 still picture profile, Main10 4:4:4 still picture profile, Main10 profile, Main10 4:4:4 profile, multi-layer Main10 profile, or multi-layer Main10 4:4:4 profile. i. Alternatively, additionally, the bitstream in such an image item is required to conform to the Main10 still picture profile, Main10 4:4:4 still picture profile, Main10 profile, or Main10 4:4:4 profile. ii. Alternatively, additionally, the bitstream in such an image item is required to conform to the Main10 still picture profile or Main10 4:4:4 still picture profile. d. In one example, such a type of image item may be specified as not having any of the following characteristics: Target Output Layer Set Property (TargetOlsProperty), VVC Operating Points Information Property (VvcOperatingPointsInformationProperty).

[0043] 4) To solve Problem 3, for example, a VVC image item of type 'vvc1' is defined to consist of exactly one access unit which is an IRAP access unit defined in ISO / IEC 23090-3 or a GDR access unit having ph_recovery_poc_cnt equal to 0 such that all pictures are defined in ISO / IEC 23090-3, from an NAL unit of a VVC bitstream.

[0044] 5) To solve Problem 4, for example, a VVC image item of type 'vvc1' is not allowed to include pictures within a layer that does not belong to the target output layer set.

[0045] 6) To solve Problem 5, image items originating from the same bitstream are allowed to be associated with different instances of VvcOperatingPointsInformationProperty.

[0046] 7) To solve Problem 6, the syntax elements ptl_max_temporal_id[i] and op_max_temporal_id of VvcOperatingPointsRecord are constrained to be specific values when VvcOperatingPointsRecord is included in VvcOperatingPointsInformationProperty.

[0047] 8) To solve Problem 7, either of the following is allowed, for example, for a VVC image item of 'vvc1': a. It includes the entire VVC access unit where each picture may contain a plurality of "extractable" sub - pictures. b. It includes a subset of VVC access units where for each layer present in the bitstream, there is one or more "extractable" sub - pictures that collectively form a rectangular region. Here, an "extractable" sub - picture refers to a sub - picture for which the corresponding flag sps_subpic_treated_as_pic_flag[i] specified in VVC is equal to 1.

[0048] 9) To solve Problem 8, the OPI NAL unit may be specified not to be present in both the item and the sample of the 'vvs1' item.

[0049] 10) To solve Problem 9, it is proposed that multiple transition effects from one image (or its region) to another image (or its region) are enabled in a slideshow. a. In one example, the indication of multiple transition effects may be signaled, for example, by having a plurality of transition effect characteristic structures associated with the first of two consecutive image items. b. In one example, the indication of the number of transition effects applied to two consecutive image items may be signaled in the file. c. Alternatively, furthermore, the method of applying multiple transition effects may be signaled in the file, or predefined, or derived on - the - fly. i. In one example, the order of applying multiple transition effects may be signaled in the file. ii. In one example, the order of applying multiple transition effects may be derived according to the order of the indication of multiple effects in the bitstream.

[0050] 11) To solve problem 9, it is proposed to enable multiple transition effects from one image to another in a slide show. Here, each of the multiple transition effects is applied to a specific region in two image items involved in that transition. a. In one example, the specific regions in two image items involved in a transition to which the transition effect is applied are signaled by transition effect characteristics.

[0051] 12) To solve problem 9, it is proposed to enable multiple alternative transition effects to be signaled for a pair of consecutive image items, and selecting one of the multiple transition effects to be applied depends on the file player. a. In one example, the priority (or preference order) of the multiple transition effects is signaled in the file, or predefined, or derived according to the order of signaling of the transition characteristics.

[0052] 6. Embodiments The following are some example embodiments for some of the inventive aspects briefly described above in Section 5, which can be applied to the VVC image file format and the standard specifications for slide show support. The most relevant parts that are added or changed are underlined in bold italic, and some of the parts that are deleted are indicated using [[ ]].

[0053] 6.1 First Embodiment This embodiment relates to at least items 1, 1.b and 1.c.

Table 1

[0054] 6.2 Second Embodiment This embodiment relates to at least items 4 and 5. [Table 2]

[0055] 6.3 Third Embodiment This embodiment relates to at least items 6 and 7. [Table 3]

[0056] 6.4 Fourth Embodiment This embodiment relates to at least item 9. [Table 4]

[0057] FIG. 1 is a block diagram showing an exemplary video processing system 1900 in which various techniques disclosed herein may be implemented. For various implementations, some or all of the components of system 1900 may be included. System 1900 may include an input unit 1902 that receives video content. The video content may be received in a raw or uncompressed format, for example, 8- or 10-bit multi-component pixel values, or may be in a compressed or encoded format. Input unit 1902 may correspond to a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet®, Passive Optical Network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0058] System 1900 may include a coding component 1904 that can implement various coding or encoding methods described herein. The coding component 1904 may reduce the average bit rate of the video from the input unit 1902 to the output unit of the coding component 1904 so as to generate a coded representation of the video. Accordingly, coding techniques are sometimes referred to as video compression or video transcoding techniques. The output of the coding component 1904 may be stored or transmitted via a connected communication as represented by the component 1906. The stored or communicated bitstream (or coded) representation of the video received at the input unit 1902 may be used by a component 1908 that generates a displayable video that is sent to the pixel values or the display interface 1910. The process of generating a video that a user can view from the bitstream is sometimes referred to as video decompression. Further, while certain video processing operations are referred to as "coding" operations or tools, it will be understood that such coding tools or operations are used in an encoder and the corresponding decoding tools or operations that reverse the results of the coding will be performed by a decoder.

[0059] Examples of a peripheral bus interface or a display interface may include a Universal Serial Bus (USB) or a High-Definition Multimedia Interface (HDMI (registered trademark)) or a Displayport (registered trademark), etc. Examples of a storage interface include SATA (Serial Advanced Technology Attachment), PCI, an IDE interface, etc. The techniques described herein may be embodied in various electronic devices such as a cellular phone, a laptop, a smartphone, or other devices capable of performing digital data processing and / or video display.

[0060] FIG. 2 is a block diagram of a video processing apparatus 3600. The apparatus 3600 may be used to implement one or more of the methods described herein. The apparatus 3600 may be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 3600 may include one or more processors 3602, one or more memories 3604, and video processing hardware 3606. The processor 3602 may be configured to implement one or more of the methods described herein. The memory(ies) 3604 may be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 3606 may be used to implement some of the techniques described herein in a hardware circuit. In some embodiments, the video processing hardware 3606 may be at least partially included in the processor 3602, e.g., a graphics coprocessor.

[0061] FIG. 4 is a block diagram depicting an exemplary video coding system 100 that may utilize the techniques of the present disclosure.

[0062] As shown in FIG. 4, the video coding system 100 may include a source device 110 and a destination device 120. The source device 110 may generate encoded video data and may be referred to as a video encoding device. The destination device 120 may be able to decode the encoded video data generated by the source device 110 and may be referred to as a video decoding device.

[0063] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0064] Video source 112 may include a source such as a video capture device, an interface to receive video data from a video content provider, and / or a computer graphics system that generates video data, or a combination of such sources. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a coded representation of the video data. The bitstream may include coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data may include a sequence parameter set, a picture parameter set, and other syntax structures. I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be transmitted directly through network 130a to destination device 120 via I / O interface 116. The encoded video data may also be stored in storage medium / server 130b for access by a transmitter.

[0065] Destination device 120 may include I / O interface 126, video decoder 124, and display device 122.

[0066] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain the encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to the user. Display device 122 may be integrated with destination device 120 or may be configured to interface with an external display device and be outside of destination device 120.

[0067] Video encoder 114 and video decoder 124 may operate according to video compression standards such as the HEVC (High Efficiency Video Coding) standard, the VVC (Versatile Video Coding) standard, and other current and / or future standard specifications.

[0068] FIG. 5 is a block representing an example of video encoder 200, which may be video encoder 114 of system 100 shown in FIG. 4.

[0069] Video encoder 200 may be configured to perform any or all of the techniques of the present disclosure. In the example of FIG. 5, video encoder 200 includes a plurality of functional components. The techniques described in the present disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in the present disclosure.

[0070] The functional components of video encoder 200 may include partition unit 201, prediction unit 202 that may include mode selection unit 203, motion estimation unit 204, motion compensation unit 205, and intra prediction unit 206, residual generation unit 207, transform unit 208, quantization unit 209, inverse quantization unit 210, inverse transform unit 211, reconstruction unit 212, buffer 213, and entropy encoding unit 214.

[0071] In other examples, video encoder 200 may include more, fewer, or different functional components. In an example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode where at least one reference picture is the picture in which the current video block is located.

[0072] Furthermore, some components such as the motion estimation unit 204 and the motion compensation unit 205 may be highly integrated, but are shown separately in the example of FIG. 5 for the sake of explanation.

[0073] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0074] The mode selection unit 203 may select, for example, one of the intra or inter coding modes based on an error result, and supply the resulting intra or inter coded block to the residual generation unit 207 that generates residual block data, and to the reconstruction unit 212 that reconstructs the coded block for use as a reference picture. In some examples, the mode selection unit 203 may select an intra and inter prediction combination (CIIP) mode where the prediction is based on an inter prediction signal and an intra prediction signal. The mode selection unit 203 may also select the resolution (e.g., sub-pixel or integer pixel accuracy) for the motion vector of the block in the case of inter prediction.

[0075] To perform inter prediction for the current video block, the motion estimation unit 204 may generate motion information of the current video block by comparing one or more reference frames from the buffer 213 with the current video block. The motion compensation unit 205 may determine a predicted video block of the current video block based on the motion information and the decoded samples of the picture from the buffer 213 other than the picture associated with the current video block.

[0076] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations for the current video block depending on, for example, whether the current video block is an I slice, a P slice, or a B slice.

[0077] In some examples, the motion estimation unit 204 may perform uni - directional prediction for the current video block. The motion estimation unit 204 may search for reference pictures in list 0 or list 1 for the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index indicating the reference picture in list 0 or list 1 that includes the reference video block, and a motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.

[0078] In other examples, the motion estimation unit 204 may perform bi - directional prediction for the current video block. The motion estimation unit 204 may search for a reference picture in list 0 for the reference video block for the current video block, and may also search for a reference picture in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a reference index indicating the reference pictures in lists 0 and 1 that include the reference video blocks, and a motion vector indicating the spatial displacement between the reference video blocks and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0079] In some examples, the motion estimation unit 204 may output a full set of motion information for the decoder's decoding process.

[0080] In some examples, the motion estimation unit 204 may not output a full set of motion information for the current video. Instead, the motion estimation unit 204 may signal the motion information of the current video block by referring to the motion information of other video blocks. For example, the motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of adjacent video blocks.

[0081] In one example, the motion estimation unit 204 may indicate a value to the video decoder 300 in a syntax structure related to the current video block, indicating that the current video block has the same motion information as other video blocks.

[0082] In other examples, the motion estimation unit 204 may identify other video blocks and a Motion Vector Difference (MVD) in a syntax structure related to the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the identified video block. The video decoder 300 may use the motion vector of the identified video block and the motion vector difference to determine the motion vector of the current video block.

[0083] As described above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0084] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks within the same picture. The prediction data for the current video block may include the predicted video block and various syntaxes.

[0085] The residual generation unit 207 may generate residual data for the current video block by subtracting the predicted video block of the current video block from the current video block (e.g., indicated by a minus sign). The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples within the current video block.

[0086] In other examples, for example, in skip mode, the residual data for the current video block may not exist, and the residual generation unit 207 may not perform a subtraction operation.

[0087] The transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0088] After the transform processing unit 208 generates the transform coefficient video block associated with the current video block, the quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0089] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block respectively to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more predicted video blocks generated by the prediction unit 202 and generate a reconstructed video block related to the current block for storage in the buffer 213.

[0090] After the reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0091] The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. When the entropy encoding unit 214 receives data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy encoded data and generate a bitstream including the entropy encoded data.

[0092] FIG. 6 is a block diagram illustrating an example of a video decoder 300, which may be the video decoder 124 of the system 100 represented in FIG. 4.

[0093] The video decoder 300 may be configured to execute any or all of the techniques of the present disclosure. In the example of FIG. 6, the video decoder 300 includes a plurality of functional components. The techniques described in the present disclosure may be shared among various components of the video decoder 300. In some examples, a processor may be configured to execute any or all of the techniques described in the present disclosure.

[0094] In the example of FIG. 6, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. The video decoder 300 may perform a decoding path generally inverse to the encoding path described with respect to the video encoder 200 (FIG. 5) in some examples.

[0095] The entropy decoding unit 301 may extract the encoded bitstream. The encoded bitstream may include entropy-coded video data (e.g., an encoded block of video data). The entropy decoding unit 301 may decode the entropy-coded video data, and from the entropy-decoded video data, the motion compensation unit 302 may determine motion information including a motion vector, motion vector precision, a reference picture list index, and other motion information. The motion compensation unit 302 may determine such information, for example, by performing AMVP and merge mode.

[0096] The motion compensation unit 302 may optionally perform interpolation based on an interpolation filter to generate a motion-compensated block. An identifier for the interpolation filter used at sub-pixel precision may be included in the syntax element.

[0097] The motion compensation unit 302 may use the interpolation filter used by the video encoder 200 during encoding of the video block to calculate interpolation values for sub-integer pixels of the reference block. The motion compensation unit 302 may determine the interpolation filter used by the video encoder 200 according to the received syntax information and use the interpolation filter to generate a prediction block.

[0098] The motion compensation unit 302 may use some of the syntax information to determine the size of the blocks used to encode the frames and / or slices of the encoded video sequence, the partition information that describes how each macroblock of the pictures of the encoded video sequence is partitioned, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.

[0099] The intra prediction unit 303 may use, for example, the intra prediction mode received in the bitstream to form a prediction block from spatially adjacent blocks. The inverse quantization unit 304 inverse quantizes, i.e., dequantizes, the quantized video block coefficients supplied in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0100] The reconstruction unit 306 may add the corresponding prediction block and residual block generated by the motion compensation unit 302 or the intra prediction unit 303 to form the decoded block. If desired, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in the buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and further generates the decoded video for presentation on a display device.

[0101] An enumeration of the solutions desired by some embodiments is given next.

[0102] The following solutions show exemplary embodiments of the techniques discussed in the previous section (e.g., items 1, 10, and 11).

[0103] 1. A visual media processing method (e.g., method 700 represented in FIG. 3), comprising A step (702) of performing a conversion between visual media including a sequence of one or more images and a bitstream representation according to a file format, wherein the file format is configured to include one or more syntax elements indicating transition characteristics between one or more of the images during display of the one or more images, method.

[0104] 2. The method of Solution 1, wherein the transition characteristic is a transition time, and the file format includes other syntax elements indicating the type of the transition time, and the type has an obligatory transition time or a recommended transition time, method.

[0105] 3. The method of Solution 1, wherein the transition characteristic has one or more transition effects between one or more of the images, method.

[0106] 4. The method of Solution 2, wherein the file format includes one or more syntax elements describing one or more transition effects applicable to transitions between consecutive images or portions of consecutive images, method.

[0107] 5. The method of Solution 3, wherein the file format includes a syntax structure specifying a plurality of transition effects and corresponding portions of the images to which the plurality of transition effects are applicable during a transition from one image to the next image, method.

[0108] The following solutions show exemplary embodiments of the techniques discussed in the previous section (e.g., item 2).

[0109] 6. A method for visual media processing, wherein Performing a conversion between visual media including a sequence of one or more images and a bitstream representation according to a file format, wherein when the visual media is represented by a file having a specific file brand, the file format is restricted according to rules, Method.

[0110] 7. The method of solution 6, wherein the rules define that only one access unit of a portion of an image coded using a specific coding tool is included, Method.

[0111] 8. The method of solutions 6 to 7, wherein the specific coding tool has an intra coding tool, Method.

[0112] 9. The method of solutions 6 to 7, wherein the specific coding tool has an intra block copy coding tool, Method.

[0113] 10. The method of solution 6, wherein the specific coding tool has a palette coding tool, Method.

[0114] 11. The method of solution 6, wherein the rules define that the file format is not permitted to store one or more images coded according to coding characteristics, Method.

[0115] 12. The method of solution 11, wherein the coding characteristics have target output layer set characteristics, Method.

[0116] The following solutions show exemplary embodiments of the techniques discussed in the previous section (e.g., items 3, 4, 5, and 8).

[0117] 13. A visual media processing method, comprising: performing a conversion between visual media including a sequence of one or more images and a bitstream representation according to a file format, wherein the file format is configured to indicate an image type of one or more images according to rules, the method.

[0118] 14. The method according to solution 13, wherein the rules further specify that for one image type, the file format allows inclusion of only one access unit including an intra-coded image, the method.

[0119] 15. The method according to solution 13, wherein the rules specify that a particular image type is allowed to include only a network abstraction layer unit including exactly one access unit that is an intra-random access picture unit, the method.

[0120] 16. The method according to solution 13, wherein the rules specify that for a particular image type, the file format is not allowed to include pictures within a layer from different target output layer sets, the method.

[0121] 17. The method according to solution 13, wherein the rules specify that for a particular image type, the file format is allowed to include an entire access unit including one or more pictures including a plurality of extractable sub-pictures, the method.

[0122] 18. Any one of the methods 1 to 17, wherein the conversion comprises encoding one or more images to generate a bitstream representation according to a file format. Method.

[0123] 19. The method of solution 18, wherein the bitstream representation according to the file format is stored on a computer-readable medium or transmitted via a communication connection. Method.

[0124] 20. Any one of the methods 1 to 17, wherein the conversion comprises decoding and reconstructing one or more images from a bitstream representation. Method.

[0125] 21. The method of solution 20, further comprising prompting to display one or more images after decoding and reconstruction. Method.

[0126] 22. A video decoding apparatus having a processor configured to implement the method according to any one of solutions 1 to 21.

[0127] 23. A video encoding apparatus having a processor configured to implement the method according to any one of solutions 1 to 21.

[0128] 24. A computer program product storing computer code, wherein when the code is executed by a processor, it causes the processor to implement the method according to any one of solutions 1 to 21. Computer program product.

[0129] 25. A computer-readable medium storing a bitstream representation according to a file format generated by any one of solutions 1 to 21.

[0130] 26. The method, apparatus, or system described in this specification.

[0131] FIG. 8 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 800 includes, at operation 810, performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that an image item of a specific type value in the visual media file includes a single access unit of the bitstream. The single access unit is either an Intra Random Access Picture (IRAP) access unit according to the video coding format or a Gradual Decoding Refresh (GDR) access unit according to the video coding format. All pictures within the GDR access unit are identified as recovery points within the bitstream.

[0132] FIG. 9 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 900 includes, at operation 910, performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies excluding layers that do not belong to a target output layer set for an image item of a specific type value in the visual media file.

[0133] FIG. 10 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 1000 includes, in operation 1010, performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to a video coding format. The media file format specifies that an image item of a particular type value in the visual media file includes at least a portion of an access unit in which a picture has one or more sub-pictures.

[0134] The following are examples of the techniques discussed in relation to FIGS. 8-10.

[0135] 1. An example of a method for processing video data, including performing a conversion between a visual media file and a bitstream, wherein the visual media file has a sequence of one or more pictures according to a media file format, the bitstream has one or more access units according to a video coding format, the bitstream is coded according to a video coding format, and the media file format specifies that an image item of a particular type value in the visual media file includes a single access unit of the bitstream, the single access unit being either an intra-random access picture (IRAP) access unit according to the video coding format or a graceful decode refresh (GDR) access unit according to the video coding format, and all pictures within the GDR access unit are identified as recovery points within the bitstream. Method.

[0136] 2. The method of Example 1, The video coding format corresponds to the VVC (Versatile Video Coding) standard according to ISO / IEC 23090-3, Method.

[0137] 3. The method according to Example 1 or 2, wherein a specific type value is specified as 'vvc1', Method.

[0138] 4. The method according to any one of Examples 1 to 3, wherein each of all pictures in the GDR access unit includes a picture header field having a value zero indicating that the corresponding picture is a recovery point, Method.

[0139] 5. The method according to Example 4, wherein the picture header field corresponds to ph_recovery_poc_cnt_field, Method.

[0140] 6. A method for processing video data, including the step of performing conversion between a visual media file and a bitstream, wherein the visual media file has a sequence of one or more pictures according to a media file format, the bitstream has one or more access units according to a video coding format, the bitstream is coded according to a video coding format, and the media file format specifies excluding a layer to which an image item of a specific type value in the visual media file does not belong from a target output layer set. Method.

[0141] 7. The method according to Example 6, wherein the video coding format corresponds to the VVC (Versatile Video Coding) standard according to ISO / IEC 23090-3, Method

[0142] 8. The method according to Example 6 or 7, wherein a specific type value is specified as 'vvc1', Method

[0143] 9. The method according to any one of Examples 6 to 8, wherein the image item includes layers within an output layer set identified by characteristics indicating a target output layer set and does not include other layers, Method

[0144] 10. A method for processing video data, comprising the step of performing conversion between a visual media file and a bitstream, wherein the visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format, wherein the bitstream is coded according to a video coding format, wherein the media file format specifies that an image item of a specific type value in the visual media file includes at least a portion of an access unit in which a picture has one or more sub-pictures, Method

[0145] 11. The method according to Example 10, wherein the video coding format conforms to the VVC (Versatile Video Coding) standard according to ISO / IEC 23090-3, and a specific type value is specified as 'vvc1', Method

[0146] 12. The method according to Example 10 or 11, wherein the image item includes the entire access unit, Method

[0147] 13. Any one of the methods of Examples 10 to 12, The image item includes a portion of the access unit, For each layer present in the bitstream, one or more sub-pictures form a rectangular region Method.

[0148] 14. A video processing apparatus having a processor, The processor is configured to execute any one of the methods of Examples 1 to 13, Video processing apparatus.

[0149] 15. A non-transitory computer-readable recording medium storing a bitstream of a video generated by any one of the methods of Examples 1 to 13 executed by a video processing apparatus.

[0150] FIG. 11 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 1100 includes, in operation 1110, performing a conversion between a visual media file and a bitstream. The visual media file has image items each having a sequence of one or more pictures according to a media file format. The bitstream includes access units each consisting of one or more pictures respectively belonging to layers according to a video coding format. The media file format specifies that an image item having a picture originating from the bitstream is allowed to be associated with different instances of a characteristic descriptor indicating high-level characteristics of the bitstream.

[0151] FIG. 12 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 1200 includes, at operation 1210, performing a conversion between a visual media file and a bitstream. The visual media file has image items each including a sequence of one or more pictures according to a media file format. The bitstream has access units each consisting of one or more pictures respectively belonging to layers according to a video coding format. The media file format is specified such that at least one of the value of a first syntax element in the record or the value of a second syntax element in the record is constrained to be a predetermined value in response to a record of an operating point included in a characteristic descriptor of an operating point indicating high-level characteristics of the bitstream.

[0152] The following are examples of the technology discussed in relation to FIGS. 11-12.

[0153] 1. An example solution as a method for processing image data, including performing a conversion between a visual media file and a bitstream, wherein the visual media file has image items each having a sequence of one or more pictures according to a media file format, and the bitstream includes access units each consisting of one or more pictures respectively belonging to layers according to a video coding format, and the media file format is specified such that an image item having a picture originating from the bitstream is allowed to be associated with different instances of a characteristic descriptor indicating high-level characteristics of the bitstream. Method.

[0154] 2. The method of example solution 1, wherein the video coding format conforms to the VVC (Versatile Video Coding) standard according to ISO / IEC 23090-3. Method.

[0155] 3. A method according to Example Solution 1 or 2, wherein the characteristic descriptor is represented as VvcOperatingPointsInformationProperty, method.

[0156] 4. An example solution for a method of processing image data, comprising the step of performing a conversion between a visual media file and a bitstream, wherein the visual media file has image items each containing a sequence of one or more pictures according to a media file format, and the bitstream contains access units each consisting of one or more pictures respectively belonging to layers according to a video coding format, wherein the media file format is specified such that in response to a record of an operating point included in a characteristic descriptor of the operating point indicating a high-level characteristic of the bitstream, at least one of the value of the first syntax element in the record or the value of the second syntax element in the record is constrained to be a predetermined value, method.

[0157] 5. A method according to Example Solution 4, wherein the video coding format conforms to the VVC (Versatile Video Coding) standard according to ISO / IEC 23090-3, method.

[0158] 6. A method according to Example Solution 4 or 5, wherein the first syntax element specifies a maximum temporal identification related to the i-th profile tier level syntax structure, and i ranges from 0 to (the number of profile tiers - 1), method.

[0159] 7. A method according to Example Solution 6, wherein the first syntax element is represented as ptl_max_temporal_id[i], Method

[0160] 8. A method according to any one of exemplary solutions 4 to 7, wherein the second syntax element specifies a maximum time identification related to the recording of the operating point, Method

[0161] 9. A method according to the method of exemplary solution 8, wherein the second syntax element is represented as max_temporal_id, Method

[0162] 10. A method according to any one of exemplary solutions 4 to 9, wherein the recording includes a third syntax element that specifies whether frame rate information exists, and the value of the third syntax element is constrained to be a predetermined value, Method

[0163] 11. A method according to any one of exemplary solutions 4 to 10, wherein the recording includes a fourth syntax element that specifies whether bit rate information exists, and the value of the fourth syntax element is constrained to be a predetermined value, Method

[0164] 12. A method according to any one of exemplary solutions 4 to 10, wherein the predetermined value is equal to 0, Method

[0165] 13. A video processing apparatus having a processor, wherein the processor is configured to execute a method according to any one of Examples 1 to 12, Video processing apparatus

[0166] 14. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method according to any one of Examples 1 to 12 executed by a video processing apparatus.

[0167] With the solution described in this specification, the encoder may conform to the formatting rules by generating the coded representation according to the formatting rules. With the solution described in this specification, the decoder may use the formatting rules to parse the syntax elements in the coded representation knowing the presence or absence of the syntax elements according to the formatting rules in order to generate the decoded video.

[0168] As used herein, the term "video processing" may refer to video encoding, video decoding, video compression or video decompression. For example, a video compression algorithm may be applied during the conversion from the pixel representation of a video to the corresponding bitstream representation or vice versa. The bitstream representation of a current video block may correspond to bits that are either at the same position in the bitstream or spread over different locations, as defined by the syntax, for example. For example, a macroblock may be encoded using the bits in the header and other fields in the bitstream with respect to the transformed and coded error residue values. Further, during the conversion, the decoder may parse the bitstream based on a determination, knowing the possibility of the presence or absence of some fields, as described in the above solution. Similarly, the encoder may determine whether a particular syntax field should be included and, accordingly, generate the bitstream representation by including or excluding the syntax field from the bitstream representation.

[0169] The disclosures and other solutions, examples, embodiments, modules, and functional operations described in this specification can be implemented in digital electronic circuits, or in computer software, firmware, or hardware that includes the structures disclosed herein and their structural equivalents, or in combinations of one or more of them. The disclosed embodiments and other embodiments can be implemented as one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage carrier, a memory device, a composition that provides a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" includes, by way of example, all apparatus, devices, and machines for processing data, including programmable processors, computers, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, for example, an electrical, optical, or electromagnetic signal generated by a machine and generated to encode information for transmission to an appropriate receiver device.

[0170] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a single file dedicated to the program in question, or in multiple cooperating files (e.g., files that store one or more modules, subprograms, or portions of code), or in portions of files that hold other programs or data (e.g., one or more scripts stored in a markup language document). A computer program can be deployed to be executed on one computer or on a single location, or distributed across multiple computers located in one or more locations and interconnected by a communication network.

[0171] The processes and logic flows described herein can be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data to generate output. The processes and logic flows can also be executed by dedicated logic circuits, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and the apparatus can be implemented as such.

[0172] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. In general, a processor will receive instructions and data from a read-only memory or a random access memory or both. Essential elements of a computer are a processor for executing the instructions and one or more memory devices for storing the instructions and data. In general, a computer will also include one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks, or be operatively coupled for receiving data from or transferring data to one or more such mass storage devices or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and all forms of nonvolatile memory, media, and memory devices including CDROM and DVD-ROM disks. The processor and the memory may be enhanced or incorporated in a dedicated logic circuit.

[0173] This specification includes many details, but they are to be construed not as limitations on the scope of any subject or of anything that may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. The particular features described herein in connection with separate embodiments may be implemented in combination with a single embodiment. Conversely, the various features described in connection with a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Further, features may be described and even initially claimed as operating in a particular combination, but in some cases, one or more features from the claimed combination may be excisable from that combination, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination.

[0174] Similarly, operations are represented in the drawings in a particular order, but this should not be understood as requiring that such operations be performed in that particular order or in a sequential order to achieve the desired result, or that all of the operations shown be performed. Further, the separation of the various system components in the embodiments described herein should not be understood as requiring such separation in all embodiments.

[0175] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

[Claim 1] 1. A method for processing image data, comprising the steps of: performing a conversion between a visual media file and a bitstream; the visual media file comprises a sequence of one or more pictures according to a media file format, and the bitstream comprises one or more access units according to a video coding format; the bitstream is coded according to the video coding format; the media file format specifies that an image item of a first type value in the visual media file comprises a single access unit of the bitstream, the single access unit being either an Intra Random Access Picture (IRAP) access unit according to a video coding format or a Gradual Decoding Refresh (GDR) access unit according to a video coding format, and all pictures within the GDR access unit are identified as recovery points within the bitstream; The media file format further specifies that no Operation Point Information (OPI) Network Abstraction Layer (NAL) units are present in a sub-picture item of a second type value or in a sample of the sub-picture item of the second type value. method.

Images