Pictures and layers included in VVC image items

Abstract

To provide systems, methods, and apparatus for processing image data.SOLUTION: A method includes a step of performing a conversion between a visual media file and a bitstream. The visual media file comprises a sequence of one or more pictures according to a media file format, and the bitstream comprises one or more access units according to a video coding format. The media file format specifies that an image item of a specific type value in the visual media file includes a single access unit of the bitstream. The single access unit is either an Intra Random Access Picture (IRAP) access unit or a Gradual Decoding Refresh (GDR) access unit according to the video coding format. All pictures in the GDR access unit are identified as a recovery point in the bitstream.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application is a divisional application of Japanese Patent Application No. 2021 - 142854, which claims priority and the benefit thereof from U.S. Provisional Patent Application No. 63 / 073829, filed on September 2, 2020. For all purposes under the law, the entire disclosure of the above application is incorporated by reference as part of the disclosure of this application.

[0002] [Technical Field] This patent document relates to the encoding and decoding of images and videos.

Background Art

[0003] Digital videos account for the largest bandwidth usage on the Internet and other digital communication networks. As the number of user devices capable of receiving and displaying videos increases, the bandwidth demand for digital video usage is expected to continue growing.

Summary of the Invention

[0004] This patent document discloses techniques that can be used by video encoders and decoders to process the coded representations of videos or images according to a file format.

[0005] In one exemplary embodiment, a method for processing image data includes the step of performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that an image item of a particular type value in the visual media file contains a single access unit in the bitstream. The single access unit is either an Intra Random Access Picture (IRAP) access unit according to the video coding format or a Gradual Decoding Refresh (GDR) access unit according to the video coding format. All pictures within the GDR access unit are identified as recovery points in the bitstream.

[0006] In another exemplary embodiment, a method for processing image data includes the step of performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that image items of a particular type value in the visual media file exclude layers that do not belong to the target output layer set.

[0007] In another exemplary embodiment, a method for processing image data includes the step of performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that an image item of a particular type value in the visual media file includes at least a portion of an access unit in which the picture has one or more subpictures.

[0008] In another exemplary embodiment, a method for processing image data includes the step of performing a conversion between a visual media file and a bitstream. The visual media file has image items, each having a sequence of one or more pictures according to a media file format. The bitstream includes access units, each consisting of one or more pictures, each belonging to a layer according to a video coding format. The media file format specifies that image items having pictures originating from a bitstream are allowed to be associated with different instances of a characteristic descriptor that exhibits high-level characteristics of the bitstream.

[0009] In another exemplary embodiment, a method for processing image data includes the step of performing a conversion between a visual media file and a bitstream. The visual media file has image items, each containing a sequence of one or more pictures according to a media file format, and the bitstream includes access units, each consisting of one or more pictures, each belonging to a layer according to a video coding format. The media file format specifies that, in response to a recording of an operating point contained in an operating point characteristic descriptor that indicates a high-level characteristic of the bitstream, at least one of the values ​​of a first syntax element or a second syntax element in the recording is constrained to a predetermined value.

[0010] In another exemplary embodiment, a video processing method is disclosed. The method includes the step of performing a conversion between a visual medium containing a sequence of one or more images and a bitstream representation according to a file format, wherein the file format is configured to include one or more syntax elements indicating transition characteristics between one or more images during the display of one or more images.

[0011] In other exemplary embodiments, other video processing methods are disclosed. The method includes the step of performing a conversion between a visual media containing a sequence of one or more images and a bitstream representation according to a file format, wherein the file format is restricted according to rules when the visual media is represented in a file having a particular file brand.

[0012] In other exemplary embodiments, other video processing methods are disclosed. The method includes the step of performing a conversion between a visual medium containing a sequence of one or more images and a bitstream representation, according to a file format, wherein the file format is configured to indicate the image types of one or more images according to a rule.

[0013] In yet another exemplary embodiment, a video encoder device is disclosed. The video encoder has a processor configured to implement the method described above.

[0014] In yet another exemplary embodiment, a video decoder device is disclosed. The video decoder has a processor configured to implement the method described above.

[0015] In yet another exemplary embodiment, a computer-readable medium storing code is disclosed. The code embodies one of the methods described herein in the form of code executable by a processor.

[0016] In yet another example aspect, a computer-readable medium storing a bitstream is disclosed. The bitstream is generated using the methods described herein.

[0017] These and other features are described throughout this specification.

Brief Description of the Drawings

[0018] [Figure 1] It is a block diagram of an exemplary video processing system. [Figure 2] It is a block diagram of a video processing apparatus. [Figure 3] It is a flowchart of an example of a method of video processing. [Figure 4] It is a block diagram representing a video coding system according to some embodiments of the present disclosure. [Figure 5] It is a block diagram representing an encoder according to some embodiments of the present disclosure. [Figure 6] It is a block diagram representing a decoder according to some embodiments of the present disclosure. [Figure 7] An example of an encoder block diagram is shown. [Figure 8] It is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. [Figure 9] It is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. [Figure 10] It is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. [Figure 11] It is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. [Figure 12] It is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology.

Mode for Carrying Out the Invention

[0019] Section headings are used in this document for ease of understanding and are not intended to limit the applicability of the technologies and embodiments disclosed in each section to only that section. Further, H.266 terminology is used in some descriptions only for ease of understanding and not for limiting the scope of the disclosed technologies. Accordingly, the technologies described herein are applicable to other video codec protocols and designs. In this document, editorial changes are indicated in the text by showing deleted text with a strikethrough and added text in emphasized form (including bold italic).

[0020] 1. Overview This document is related to image file formats. Specifically, it is related to the signaling and storage of images and image transitions in media files based on the ISO base media file format. The idea may be applied individually or in various combinations to images coded by any codec, such as the Versatile Video Coding (VVC) standard, and to any image file format, such as the VVC image file format under development.

[0021] 2. Abbreviations AU Access Unit AUD Access Unit Delimiter AVC Advanced Video Coding BP Buffering Period CLVS Coded Layer Video Sequence CLVSS Coded Layer Video Sequence Start CPB Coded Picture Buffer CRA Clean Random Access CTU Coding Tree Unit CVS Coded Video Sequence DCI Decoding Capability Information DPB Decoded Picture Buffer DUI Decoding Unit Information EOB End Of Bitstream EOS End Of Sequence GDR Gradual Decoding Refresh HEVC High Efficiency Video Coding HRD Hypothetical Reference Decoder IDR Instantaneous Decoding Refresh ILP Inter-Layer Prediction ILRP Inter-Layer Reference Picture IRAP Intra Random Access Picture JEM Joint Exploration Model LTRP Long-Term Reference Picture MCTS Motion-Constrained Tile Sets NAL Network Abstraction Layer OLS Output Layer Set PH Picture Header POC Picture Order Count PPS Picture Parameter Set PT Picture Timing PTL Profile, Tier and Level PU Picture Unit RAP Random Access Point RBSP Raw Byte Sequence Payload SEI Supplemental Enhancement Information SLI Subpicture Level Information SPS Sequence Parameter Set STRP Short-Term Reference Picture SVC Scalable Video Coding VCL VideoCoding Layer VPS Video Parameter Set VTM VVC Test Model VUI Video Usability Information VVC Versatile Video Coding

[0022] 3. Initial discussion 3.1 Video Coding Standards Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T created H.261 and H.263, ISO / IEC created MPEG-1 and MPEG-4 Visual, and the two organizations jointly created the H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure, utilizing time prediction and transformation coding. To explore future video coding technologies beyond HEVC, JVET (Joint Video Exploration Team) was jointly established in 2015 by VCEG and MPEG. Since then, many new methods have been introduced by JVET and placed in reference software called JEM (Joint Exploration Model). JVET was later renamed JVET (Joint Video Experts Team) when the VVC (Versatile Video Coding) project officially began. VVC is a new coding standard that aims to reduce the bitrate by 50% compared to HEVC, which was finalized by JVET at its 19th meeting, which concluded on July 1, 2020.

[0023] The VVC (Versatile Video Coding) standard (ITU-T H.266|ISO / IEC 23090-3) and the related VSEI (Versatile Supplemental Enhancement Information) standard (ITU-T H.274|ISO / IEC 23002-7) are designed for use in the widest possible range of applications, including both traditional uses such as television broadcasting, video conferencing, or playback from storage media, and newer and more advanced use cases such as adaptive bitrate streaming, video region extraction, content synthesis and merging from multiple coded video bitstreams, multiview video, scalable layered coding, and viewport-adaptive 360° immersive media.

[0024] 3.2 File Format Standards Media streaming applications are typically based on IP, TCP, and HTTP transport methods and usually rely on file formats such as the ISO Base Media File Format (ISOBMFF). One such streaming system is Dynamic Adaptive Streaming over HTTP (DASH). To use video formats with ISOBMFF and DASH, file format standards specific to video formats, such as the AVC file format and HEVC file format, have become necessary for encapsulating video content in ISOBMFF tracks and in DASH representations and segments. Important information about the video bitstream, such as profiles, tiers, and levels, and many others, needs to be exposed as file format-level metadata and / or DASH Media Presentation Description (MPD) for content selection purposes, such as selecting appropriate media segments for both initialization at the start of a streaming session and stream adaptation during a streaming session.

[0025] Similarly, using image formats based on ISOBMFF may require specific file format standards for those image formats, such as the AVC and HEVC image file formats.

[0026] 3.3 VVC Video File Format The VVC video file format is a file format for storing VVC video content based on ISOBMFF, and is currently under development by MPEG.

[0027] 3.4 VVC Image File Format and Image Transitions The VVC image file format is a file format for storing image content coded using VVC, based on ISOBMFF, and is currently under development by MPEG.

[0028] In some cases, the design for slideshow signaling includes the enhancement of image transition effects such as wipe, zoom, fade, split, and dissolve. Transition effects are signaled by a transition effect characteristic structure, which relates to the first of two consecutive items involved in the transition, includes the transition type, and, where applicable, also indicates other transition information such as the transition direction and transition shape.

[0029] 4. Examples of technical problems solved by the disclosed technical solutions The latest design for the VVC image file format and signaling for image transition effects has the following problems:

[0030] 1) In slideshows or other image-based applications with transition effects from one image to another, the timing of the transitions often does not need to be precise, but for a good user experience, it should not be too long or too short. The best transition duration depends on the content and the type of transition. Therefore, from a user experience perspective, it is useful to inform users of recommended transition durations, and these recommended values ​​are determined by the content creator.

[0031] 2) In the latest VVC image file format draft specification, certain VVC image item types and file brands may allow an access unit containing multiple pictures in multiple layers within the VVC bitstream of an image item, where some pictures may be intercoded, i.e., containing B or P slices predicted using the interlayer prediction specified in VVC. In other words, there is no interoperability point via either the image item type or file brand, and an image item can only contain one intracoded picture (i.e., only an intracoded I slice). In the VVC standard itself, such an interoperability point is given by the definition of two still image profiles: the Main10 still image profile and the Main10 4:4:4 still image profile.

[0032] 3) Items of type 'vvc1' are specified as follows: An item of type 'vvc1' consists of NAL units of a VVC bitstream whose length is delimited as specified below, and the bitstream contains exactly one access unit. Note 2: An item of type 'vvc1' may consist of IRAP access units as defined in ISO / IEC 23090-3, and may contain more than one coded picture, and at most one coded picture having a nuh_layer_id of any particular value. However, no access unit can be an access unit for such image items. Therefore, the first part of Note 2 above should be moved to the basic definition (i.e., the first part quoted above), and the missing part of the GDR access unit should be added.

[0033] 4) The following sentence exists: The 'vvc1' image item should contain layers included in the layer set identified by the associated TargetOlsProperty, and may also contain other layers. If layers other than those included in the identified OLS should be allowed, which entity within the application system is responsible for setting the correct value of the target OLS index in the relevant TargetOlsProperty? In any case, this value needs to be set precisely, for example, by the file composer, and since it is easy for the file composer to discard unnecessary pictures in unnecessary layers, it makes sense to not allow unnecessary pictures in unnecessary layers at all.

[0034] 5) The following constraints exist: Image items originating from the same bitstream should be associated with the same VvcOperatingPointsInformationProperty. However, a VVC bitstream may contain multiple CVSs that may have different operating points.

[0035] 6) In the following text, the values ​​of other syntax elements of VvcOperatingPointRecord, such as ptl_max_temporal_id[i] (the time ID of the top-level sublayer representation in the i-th profile_tier_level() syntax structure where level information exists) and op_max_temporal_id, should also be constrained: When included in VvcOperatingPointsInformationProperty, the values ​​of the syntax elements in VvcOperatingPointsRecord are constrained as follows: The `frame_rate_info_flag` must be equal to 0. As a result, `avgFrameRate` and `constantFrameRate` do not exist, and their semantics are not specified. bit_rate_info_flag must be equal to 0. As a result, maxBitRate and avgBitRate do not exist, and their semantics are not specified.

[0036] 7) The following text exists: If a VVC subpicture item is suitable to be decoded by a VVC decoder and consumed without other VVC subpicture items, that VVC subpicture item should be stored as an item of type 'vvc1'. Otherwise, the VVC subpicture item should be stored as an item of type 'vvs1' and formatted as a series of NAL units led by a length field, as defined in L.2.2.1.2. This presents the following problems: a) This condition is not clear enough to be used as a condition for conformity (for example, when considering how to verify whether the requirement is satisfied) and therefore needs to be clarified. b) Here, the use of an image item of type 'vvc1' does not strictly conform to the above definition that the bitstream contains exactly one VVC access unit, since the bitstream of an image item of type 'vvc1' can contain exactly a subset of VVC access units. c) It is unclear whether it is permissible to have a single VVC image item of type 'vvc1' that contains a picture containing multiple "extractable" subpictures.

[0037] 8) The following statement does not contain an OPI NAL unit: VPS, DCI, SPS, PPS, AUD, PH, EOS, and EOB NAL units are items and vvs1 It should not be present in both the item sample and the actual item. However, the Operating Point Information (OPI)NAL unit should be treated similarly here.

[0038] 9) Only one transition effect (e.g., zoom, rotate) is permitted for a given image or region within a given image. However, in practical applications, multiple effects may be applied to a single image or region within a given image.

[0039] 5. Exemplary Embodiments and Solutions To address the above issues and others, a method summarized below is disclosed. The items should be considered examples illustrating general concepts and should not be interpreted in a narrow sense. Furthermore, these items may be applied individually or combined in some way.

[0040] 1) To resolve Problem 1, a recommended transition period may be provided for transitions from one image to another. a. As an alternative, a mandatory transition period may be notified for the transition from one image to another. b. For example, the notified value, i.e., the recommended or mandatory transition period value, is determined by the content creator. c. For example, one transition period is notified for each transition characteristic. d. For example, one transition period is notified for each type of transition. e. For example, one transition period is notified about a list of transition characteristics. f. For example, one transition period is notified about a list of transition types. g. For example, one transition period is notified for all transitions.

[0041] 2) To resolve Problem 2, one or more file brands are defined such that the VVC bitstream contained in an image item conforming to such a brand is required to contain only one access unit containing only one picture (or a portion thereof) that is intracoded. a. Alternatively, one or more file brands are defined such that the VVC bitstream contained within an image item conforming to such a brand is required to contain only one access unit containing only one picture (or a portion thereof) that is intra / IBC / palette coded. i. Alternatively, one or more file brands are defined such that the VVC bitstream contained within an image item conforming to such a brand is required to contain only one access unit containing only one I picture (or a portion thereof). b. For example, the values ​​for such file brands are specified as 'vvic', 'vvi1', and 'vvi2'. c. For example, additionally, the VVC bitstream contained in such image items is required to conform to the Main10 still image profile, Main10 4:4:4 still image profile, Main10 profile, Main10 4:4:4 profile, multilayer Main10 profile, or multilayer Main10 4:4:4 profile. i. Alternatively, additionally, the VVC bitstream contained in such image items is required to conform to the Main10 still image profile, Main10 4:4:4 still image profile, Main10 profile, or Main10 4:4:4 profile. ii. Alternatively, and additionally, the VVC bitstream contained in such image items is required to conform to the Main10 still image profile or the Main10 4:4:4 still image profile. d. For example, it may be specified that image items conforming to such a brand should not have any of the following properties: TargetOlsProperty, VvcOperatingPointsInformationProperty.

[0042] 3) To resolve Problem 2, one or more image item types are defined such that the VVC bitstream contained within such image item types contains only one access unit that contains only intracoded pictures. a. Alternatively, one or more image item types are defined such that the VVC bitstream contained within such image item types contains only one access unit that contains only intra / palette / IBC coded pictures. i. Alternatively, one or more image item types are defined such that the VVC bitstream contained within such image item types contains only one access unit that contains only picture. b. For example, the type value for such an image item type is specified as 'vvc1' or 'vvc2'. c. For example, the bitstream in such an image item is required to conform to the Main10 still image profile, Main10 4:4:4 still image profile, Main10 profile, Main10 4:4:4 profile, multilayer Main10 profile, or multilayer Main10 4:4:4 profile. i. Alternatively, and additionally, the bitstream in such image items is required to conform to the Main10 still image profile, the Main10 4:4:4 still image profile, the Main10 profile, or the Main10 4:4:4 profile. ii. Alternatively, and additionally, the bitstream in such image items is required to conform to the Main10 still image profile or the Main10 4:4:4 still image profile. d. For example, it may be specified that such an image item should not have any of the following properties: TargetOlsProperty, VVCOperatingPointsInformationProperty.

[0043] 4) To solve problem 3, for example, a VVC picture item of type 'vvc1' is defined to consist of a NAL unit of a VVC bitstream that contains exactly one access unit, which is either an IRAP access unit as defined in ISO / IEC 23090-3 or a GDR access unit where all pictures have ph_recovery_poc_cnt equal to 0 as defined in ISO / IEC 23090-3.

[0044] 5) To resolve problem 4, for example, a VVC image item of type 'vvc1' is not allowed to contain pictures in layers that do not belong to the target output layer set.

[0045] 6) To solve problem 5, image items originating from the same bitstream are allowed to be associated with different instances of VvcOperatingPointsInformationProperty.

[0046] 7) To solve problem 6, the syntax elements ptl_max_temporal_id[i] and op_max_temporal_id of VvcOperatingPointsRecord are constrained to be specific values ​​if VvcOperatingPointsRecord is included in VvcOperatingPointsInformationProperty.

[0047] 8) To solve Problem 7, one of the following is allowed for a VVC image item, for example, 'vvc1': a. Includes an entire VVC access unit in which each picture may contain multiple "extractable" subpictures. b. A subset of VVC access units, where each layer present in the bitstream contains one or more "extractable" subpictures that collectively form a rectangular region. Here, an "extractable" subpicture refers to a subpicture where the corresponding flag sps_subpic_treated_as_pic_flag[i] specified in the VVC is equal to 1.

[0048] 9) To resolve problem 8, it may be specified that the OPI NAL unit should not be present in either the item or the sample of the 'vvs1' item.

[0049] 10) To solve problem 9, it is proposed that multiple transition effects be made possible in the slideshow, from one image (or region thereof) to another image (or region thereof). a. For example, instructions for multiple transition effects may be signaled by having multiple transition effect characteristic structures associated with the first of two consecutive image items. b. For example, the number of transition effects to apply to two consecutive image items may be signaled in the file. c. Alternatively, the method for applying multiple transition effects may be signaled in a file, predefined, or derived on the fly. i. For example, the order in which multiple transition effects are applied may be signaled in a file. ii. For example, the order in which multiple transition effects are applied may be derived according to the order in which the instructions for the multiple effects in the bitstream are given.

[0050] 11) To solve problem 9, it is proposed to enable multiple transition effects from one image to another in the slideshow. Here, each of the multiple transition effects is applied to a specific area in the two image items involved in the transition. a. For example, specific regions in two image items involved in a transition to which a transition effect is applied are signaled by the transition effect characteristics.

[0051] 12) To solve problem 9, it is proposed that multiple selective transition effects be signaled for a pair of consecutive image items, and that it is up to the file player to select one of the multiple transition effects to apply. a. For example, the priority (or preference order) of multiple transition effects may be signaled in a file, predefined, or derived according to the signaling order of the transition characteristics.

[0052] 6. Embodiments The following are some example embodiments of some of the embodiments of the invention briefly described earlier in Section 5, which may be applicable to the standard specification for VVC image file format and slideshow support. The most relevant additions or modifications are underlined in bold italics, and some deletions are indicated with [[]].

[0053] 6.1 First Embodiment This embodiment relates to at least items 1, 1.b, and 1.c. [Table 1] TIFF0007910978000002.tif222161TIFF0007910978000003.tif254154TIFF0007910978000004.tif242161TIFF0007910978000005.tif174161

[0054] 6.2 Second Embodiment This embodiment relates to at least items 4 and 5. [Table 2]

[0055] 6.3 Third Embodiment This embodiment relates to at least items 6 and 7. [Table 3]

[0056] 6.4 Fourth Embodiment This embodiment relates to at least item 9. [Table 4]

[0057] Figure 1 is a block diagram showing an exemplary video processing system 1900 in which various technologies disclosed herein may be implemented. Various implementations may include some or all of the components of system 1900. System 1900 may include an input unit 1902 that receives video content. The video content may be received in raw or uncompressed format, for example, as 8 or 10-bit multi-component pixel values, or in a compressed or encoded format. The input unit 1902 may correspond to a network interface, peripheral bus interface, or storage interface. Examples of network interfaces include wired interfaces such as Ethernet® and Passive Optical Network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0058] System 1900 may include a coding component 1904 that can implement various coding or encoding methods described herein. The coding component 1904 may reduce the average bitrate of video from the input 1902 to the output of the coding component 1904 in order to produce a coded representation of the video. Thus, the coding technique is sometimes referred to as video compression or video transcoding technique. The output of the coding component 1904 may be stored or transmitted via connected communication, as represented by component 1906. The stored or transmitted bitstream (or coded) representation of the video received at the input 1902 may be used by component 1908 to generate pixel values ​​or a displayable video to be sent to the display interface 1910. The process of generating a video from the bitstream that can be viewed by the user is sometimes referred to as video decompression. Furthermore, it will be understood that while certain video processing operations are referred to as “coding” operations or tools, such coding tools or operations are used in an encoder, and the corresponding decoding tools or operations that replace the coding results are performed by a decoder.

[0059] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI®), or DisplayPort®. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, and IDE interfaces. The technologies described herein may be embodied in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0060] Figure 2 is a block diagram of a video processing device 3600. The device 3600 may be used to implement one or more of the methods described herein. The device 3600 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. The device 3600 may include one or more processors 3602, one or more memories 3604, and video processing hardware 3606. The processor 3602 may be configured to implement one or more of the methods described herein. The memories (multiple memories) 3604 may be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 3606 may be used in hardware circuitry to implement some of the techniques described herein. In some embodiments, the video processing hardware 3606 may be at least partially included in the processor 3602, for example, a graphics coprocessor.

[0061] Figure 4 is a block diagram representing an exemplary video coding system 100 that may utilize the technology of this disclosure.

[0062] As shown in Figure 4, the video coding system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data and may be called a video coding device. The destination device 120 can decode the encoded video data generated by the source device 110 and may be called a video decoding device.

[0063] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0064] The video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a coded representation of the video data. The bitstream may include coded pictures and associated data. A coded picture is a coded representation of a picture. Associated data may include a sequence parameter set, a picture parameter set, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or transmitter. The encoded video data may be transmitted directly through the network 130a to the destination device 120 via the I / O interface 116. The encoded video data may also be stored in a storage medium / server 130b for access by the transmitter.

[0065] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122.

[0066] The I / O interface 126 may include a receiver and / or modem. The I / O interface 126 may acquire encoded video data from the source device 110 or storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or it may be located outside the destination device 120 and configured to interface with an external display device.

[0067] The video encoder 114 and video decoder 124 may operate in accordance with video compression standards such as the HEVC (High Efficiency Video Coding) standard, the VVC (Versatile Video Coding) standard, and other current and / or further standards.

[0068] Figure 5 is a block representing an example of a video encoder 200, which may be the video encoder 114 of the system 100 shown in Figure 4.

[0069] The video encoder 200 may be configured to perform any or all of the technologies described herein. In the example in Figure 5, the video encoder 200 includes several functional components. The technologies described herein may be shared among the various components of the video encoder 200. In some examples, a processor may be configured to perform any or all of the technologies described herein.

[0070] The functional components of the video encoder 200 may include a partition unit 201, a prediction unit 202 which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra-prediction unit 206, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0071] In other examples, the video encoder 200 may include more, fewer, or different functional components. In the example, the prediction unit 202 may include an intrablock copy (IBC) unit. The IBC unit may perform prediction in IBC mode, where at least one reference picture is the picture on which the current video block is located.

[0072] Furthermore, several components, such as the motion estimation unit 204 and the motion compensation unit 205, may be highly integrated, but for illustrative purposes, they are shown separately in the example in Figure 5.

[0073] The partition unit 201 may partition the picture into one or more video blocks. The video encoder 200 and video decoder 300 may support various video block sizes.

[0074] The mode selection unit 203 may, for example, select one of the intra or intercoding modes based on the error result and supply the resulting intra or intercoded blocks to the residual generation unit 207, which generates residual block data, and to the reconstruction unit 212, which reconstructs the encoded blocks for use as a reference picture. In some examples, the mode selection unit 203 may select a Combination of Intra and Inter Prediction (CIIP) mode, where the prediction is based on inter-prediction signals and intra-prediction signals. The mode selection unit 203 may also select the resolution (e.g., sub-pixel or integer pixel precision) for the block motion vectors in the case of inter-prediction.

[0075] To perform interpretation on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. The motion compensation unit 205 may determine the predicted video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.

[0076] The motion estimation unit 204 and the motion compensation unit 205 may perform different actions for the current video block depending, for example, whether the current video block is an I-slice, a P-slice, or a B-slice.

[0077] In some examples, the motion estimation unit 204 may perform a one-way prediction for the current video block, and the motion estimation unit 204 may look up a reference picture in List 0 or List 1 for a reference video block for the current video block. The motion estimation unit 204 may then generate a reference index indicating the reference picture in List 0 or List 1 containing the reference video block, and a motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the predicted direction indicator, and the motion vector as motion information for the current video block. The motion compensation unit 205 may generate a predicted video block for the current block based on the reference video block indicated by the motion information for the current video block.

[0078] In another example, the motion estimation unit 204 may perform bidirectional prediction for the current video block, and may look up a reference picture in List 0 for a reference video block for the current video block, and may also look up a reference picture in List 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a reference index indicating the reference pictures in List 0 and List 1 containing the reference video blocks, and a motion vector indicating the spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and motion vector of the current video block as motion information for the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information for the current video block.

[0079] In some cases, the motion estimation unit 204 may output a full set of motion information for the decoder's decoding process.

[0080] In some cases, the motion estimation unit 204 does not need to output the full set of motion information for the current video. Rather, the motion estimation unit 204 may signal the motion information for the current video block by referring to the motion information of other video blocks. For example, the motion estimation unit 204 may determine that the motion information for the current video block is sufficiently similar to the motion information of an adjacent video block.

[0081] For example, the motion estimation unit 204 may indicate a value in a syntax structure related to the current video block that indicates to the video decoder 300 that the current video block has the same motion information as other video blocks.

[0082] In other examples, the motion estimation unit 204 may identify other video blocks and motion vector differences (MVDs) in a syntax structure related to the current video block. The motion vector difference represents the difference between the motion vector of the current video block and the motion vector of the identified video block. The video decoder 300 may use the motion vectors and motion vector differences of the identified video blocks to determine the motion vector of the current video block.

[0083] As described above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0084] The intra-prediction unit 206 may perform intra-prediction on the current video block. When the intra-prediction unit 206 performs intra-prediction on the current video block, it may generate prediction data for the current video block based on decoded samples from other video blocks in the same picture. The prediction data for the current video block may include predictions for the video block and various syntaxes.

[0085] The residual generation unit 207 may generate residual data for the current video block by subtracting the predicted video block for the current video block from the current video block (e.g., indicated by a minus sign). The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples within the current video block.

[0086] In other examples, for instance, in skip mode, residual data for the current video block may not exist, and the residual generation unit 207 may not perform a subtraction operation.

[0087] The conversion processing unit 208 may generate one or more conversion coefficient video blocks of the current video block by applying one or more conversions to the residual video blocks associated with the current video block.

[0088] After the conversion processing unit 208 generates a conversion coefficient video block associated with the current video block, the quantization unit 209 may quantize the conversion coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0089] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transformed coefficient video block to reconstruct the residual video block from the transformed coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to the corresponding sample from one or more predicted video blocks generated by the prediction unit 202, and generate a reconstructed video block associated with the current block for storage in the buffer 213.

[0090] After the reconstruction unit 212 has reconstructed the video block, loop filtering may be performed to reduce video blocking artifacts in the video block.

[0091] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives data, it may perform one or more entropy coding operations to generate entropy coded data and generate a bitstream containing the entropy coded data.

[0092] Figure 6 is a block diagram showing an example of a video decoder 300, which may be the video decoder 124 of system 100 shown in Figure 4.

[0093] The video decoder 300 may be configured to perform any or all of the technologies described herein. In the example in Figure 6, the video decoder 300 includes several functional components. The technologies described herein may be shared among the various components of the video decoder 300. In some examples, the processor may be configured to perform any or all of the technologies described herein.

[0094] In the example in Figure 6, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 may perform a decoding path that is generally the reverse of the encoding path described for the video encoder 200 (Figure 5).

[0095] The entropy decoding unit 301 may extract the encoded bitstream. The encoded bitstream may contain entropy-coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 may decode the entropy-coded video data, and from the entropy-decoded video data, the motion compensation unit 302 may determine motion information, including motion vectors, motion vector precision, reference picture list index, and other motion information. The motion compensation unit 302 may determine such information, for example, by performing AMVP and merge modes.

[0096] The motion compensation unit 302 may optionally perform interpolation based on an interpolation filter to generate motion-compensated blocks. Identifiers for the interpolation filters used with subpixel precision may be included in the syntax element.

[0097] The motion compensation unit 302 may use the interpolation filter used by the video encoder 200 during video block encoding to calculate interpolation values ​​for sub-integer pixels of the reference block. The motion compensation unit 302 may determine the interpolation filter used by the video encoder 200 according to the received syntax information and use that interpolation filter to generate a predicted block.

[0098] The motion compensation unit 302 may use some syntax information to determine the size of the blocks used to encode the frames and / or slices of the encoded video sequence, partition information describing how each macroblock of the picture in the encoded video sequence is partitioned, one or more reference frames (and a list of reference frames) for each encoded block, and other information for decoding the encoded video sequence.

[0099] The intra-prediction unit 303 may use, for example, an intra-prediction mode received in a bitstream to form prediction blocks from spatially adjacent blocks. The inverse quantization unit 304 inverse quantizes, i.e., dequantizes, the quantized video block coefficients supplied in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies the inverse transform.

[0100] The reconstruction unit 306 may add the corresponding predicted blocks and residual blocks generated by the motion compensation unit 302 or the intra-prediction unit 303 to form a decoded block. If desired, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-prediction and further generates the decoded video for presentation on a display device.

[0101] A list of desired solutions by several embodiments is given below.

[0102] The following solutions illustrate exemplary embodiments of the techniques discussed in previous sections (e.g., items 1, 10, and 11).

[0103] 1. A visual media processing method (for example, method 700 shown in Figure 3), The process includes a step (702) of performing a conversion between a visual media containing a sequence of one or more images and a bitstream representation according to the file format, The file format is configured to include one or more syntax elements that indicate transition characteristics between one or more images during the display of one or more images. method.

[0104] 2. The method of Solution 1, Transition characteristics are transition times, and the file format includes other syntactic elements indicating the type of transition time, which can be either a mandatory transition time or a recommended transition time. method.

[0105] 3. The method of Solution 1, Transition characteristics have one or more transition effects between one or more images. method.

[0106] 4. The method of Solution 2, The file format includes one or more syntax elements that describe one or more transition effects applicable to transitions between consecutive images or parts of consecutive images. method.

[0107] 5. The method of Solution 3, The file format includes a syntax structure that specifies multiple transition effects and the corresponding parts of an image to which those transition effects can be applied during a transition from one image to the next. method.

[0108] The following solutions illustrate exemplary embodiments of the techniques discussed in the previous section (e.g., item 2).

[0109] 6. A method for processing visual media, The process includes a step of performing a conversion between a visual media containing a sequence of one or more images and a bitstream representation according to the file format, When visual media is represented in files belonging to a specific file brand, the file format is restricted according to the rules. method.

[0110] 7. The method of solution 6, The rules stipulate that the access unit must include only one access unit for the portion of the image coded using a specific coding tool. method.

[0111] 8. The solutions are methods 6-7, Certain coding tools have intracoding tools. method.

[0112] 9. The solutions are methods 6-7, Certain coding tools have an intrablock copy coding tool. method.

[0113] 10. The method of Solution 6, Certain coding tools have a palette coding tool. method.

[0114] 11. The method of Solution 6, The rules stipulate that a file format is not permitted to store one or more images coded according to its coding characteristics. method.

[0115] 12. The method of solution 11, The coding characteristics have target output layer set characteristics. method.

[0116] The following solutions illustrate exemplary embodiments of the techniques discussed in previous sections (e.g., items 3, 4, 5, and 8).

[0117] 13. A method for processing visual media, The process includes a step of performing a conversion between a visual media containing a sequence of one or more images and a bitstream representation according to the file format, The file format is configured to indicate the image type of one or more images according to a rule. method.

[0118] 14. Solution 13 method, The rules further specify that, for each image type, the file format allows the inclusion of only one access unit containing the intracoded image. method.

[0119] 15. Solution 13 methods, The rule specifies that a particular image type is allowed to contain only a network abstraction layer unit that includes exactly one access unit, which is an intra-random access picture unit. method.

[0120] 16. Solution 13 methods, The rules specify that for certain image types, the file format does not allow pictures within layers that are from different target output layer sets. method.

[0121] 17. Solution 13 method, The rule specifies that a file format may contain an entire access unit that includes one or more pictures containing multiple extractable subpictures for a given image type. method.

[0122] 18. One of the solutions 1 to 17, The conversion involves encoding one or more images to generate a bitstream representation according to the file format. method.

[0123] 19. Solution 18 method, A bitstream representation conforming to a file format is stored on a computer-readable medium or transmitted via a communication connection. method.

[0124] 20. One of the solutions 1 to 17, The transformation involves decoding and reconstructing one or more images from a bitstream representation. method.

[0125] 21. Solution 20 method, The process further includes a step of prompting the user to display one or more images after decoding and reconstruction. method.

[0126] 22. A video decoding device having a processor configured to implement one or more of the methods described in Solutions 1 to 21.

[0127] 23. A video encoding device having a processor configured to implement one or more of the methods described in Solutions 1 to 21.

[0128] 24. A computer program product that stores computer code, The code, when executed by the processor, causes the processor to implement one of the methods described in Solution 1 through 21. Computer program products.

[0129] 25. A computer-readable medium recording a bitstream representation that conforms to a file format generated by any of Solutions 1 through 21.

[0130] 26. Any method, apparatus, or system described herein.

[0131] Figure 8 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 800 includes performing a conversion between a visual media file and a bitstream in operation 810. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that an image item of a particular type value in the visual media file contains a single access unit in the bitstream. The single access unit is either an Intra Random Access Picture (IRAP) access unit according to the video coding format or a Gradual Decoding Refresh (GDR) access unit according to the video coding format. All pictures within the GDR access unit are identified as recovery points in the bitstream.

[0132] Figure 9 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 900 includes performing a conversion between a visual media file and a bitstream in operation 910. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that image items of a particular type value in the visual media file exclude layers that do not belong to the target output layer set.

[0133] Figure 10 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 1000 includes performing a conversion between a visual media file and a bitstream in operation 1010. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format. The media file format specifies that an image item of a particular type value in the visual media file includes at least a portion of an access unit in which the picture has one or more subpictures.

[0134] The following are examples of technologies discussed in relation to Figures 8-10.

[0135] 1. An example of a method for processing video data, This includes the step of performing a conversion between a visual media file and a bitstream, A visual media file has a sequence of one or more pictures according to the media file format, and a bitstream has one or more access units according to the video coding format. The bitstream is coded according to the video coding format. A media file format specifies that an image item of a particular type value in a visual media file contains a single access unit of a bitstream, where the single access unit is either an intra-random access picture (IRAP) access unit following a video coding format or a progressive decoded refresh (GDR) access unit following a video coding format, and all pictures within a GDR access unit are identified as recovery points in the bitstream. method.

[0136] 2. The method of Example 1, The video coding format conforms to the VVC (Versatile Video Coding) standard in accordance with ISO / IEC 23090-3. method.

[0137] 3. The method of Example 1 or 2, The specific type value is specified as 'vvc1'. method.

[0138] 4. Any method from Examples 1 to 3, Each picture in the GDR access unit includes a picture header field with a value of zero, indicating that the corresponding picture is a recovery point. method.

[0139] 5. The method of Example 4, The picture header field corresponds to ph_recovery_poc_cnt_field. method.

[0140] 6. A method for processing video data, This includes the step of performing a conversion between a visual media file and a bitstream, A visual media file has a sequence of one or more pictures according to the media file format, and a bitstream has one or more access units according to the video coding format. The bitstream is coded according to the video coding format. The media file format specifies that image items of a particular type in a visual media file should exclude layers that do not belong to the target output layer set. method.

[0141] 7. The method of Example 6, The video coding format conforms to the VVC (Versatile Video Coding) standard in accordance with ISO / IEC 23090-3. method.

[0142] 8. The method of Example 6 or 7, The specific type value is specified as 'vvc1'. method.

[0143] 9. Any method from Examples 6 to 8, An image item contains layers within an output layer set, identified by a characteristic indicating the target output layer set, but does not contain other layers. method.

[0144] 10. A method for processing video data, This includes the step of performing a conversion between a visual media file and a bitstream, A visual media file has a sequence of one or more pictures according to the media file format, and a bitstream has one or more access units according to the video coding format. The bitstream is coded according to the video coding format. A media file format specifies that an image item of a particular type value in a visual media file contains at least a portion of an access unit in which the picture has one or more subpictures. method.

[0145] 11. The method of Example 10, The video coding format conforms to the VVC (Versatile Video Coding) standard according to ISO / IEC 23090-3, and the specific type value is specified as 'vvc1'. method.

[0146] 12. The method of Example 10 or 11, The image item includes the entire access unit. method.

[0147] 13. Any method from Examples 10 to 12, The image item includes the portion of the access unit, For each layer present in the bitstream, one or more subpictures form a rectangular region. method.

[0148] 14. A video processing device having a processor, The processor is configured to perform one of the methods shown in Examples 1 through 13. Video processing device.

[0149] 15. A non-temporary computer-readable recording medium storing a video bitstream generated by any of the methods in Examples 1 to 13 performed by a video processing device.

[0150] Figure 11 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 1100 includes performing a conversion between a visual media file and a bitstream in operation 1110. The visual media file has image items, each having a sequence of one or more pictures according to a media file format. The bitstream includes access units, each consisting of one or more pictures, each belonging to a layer according to a video coding format. The media file format specifies that image items having pictures originating from a bitstream are allowed to be associated with different instances of a characteristic descriptor that exhibits high-level characteristics of the bitstream.

[0151] Figure 12 is a flowchart representation of a method for processing image data according to one or more embodiments of the present technology. Method 1200 includes performing a conversion between a visual media file and a bitstream in operation 1210. The visual media file has image items, each containing a sequence of one or more pictures according to a media file format. The bitstream has access units, each consisting of one or more pictures, each belonging to a layer according to a video coding format. The media file format specifies that, in response to an operating point record contained in an operating point characteristic descriptor that indicates high-level characteristics of the bitstream, at least one of the values ​​of a first syntax element or a second syntax element in the record is constrained to a predetermined value.

[0152] The following are examples of technologies discussed in relation to Figures 11-12.

[0153] 1. An example solution for processing image data, This includes the step of performing a conversion between a visual media file and a bitstream, A visual media file has image items, each having a sequence of one or more pictures according to the media file format, and a bitstream includes access units, each consisting of one or more pictures belonging to layers according to the video coding format. The media file format specifies that image items having pictures originating from a bitstream may be associated with different instances of a characteristic descriptor that exhibits high-level characteristics of the bitstream. method.

[0154] 2. An example of solution 1, The video coding format conforms to the VVC (Versatile Video Coding) standard in accordance with ISO / IEC 23090-3. method.

[0155] 3. An example of solution 1 or 2, The characteristic descriptor is represented as VvcOperatingPointsInformationProperty. method.

[0156] 4. An example solution for processing image data, This includes the step of performing a conversion between a visual media file and a bitstream, A visual media file has an image item, each containing a sequence of one or more pictures according to the media file format, and a bitstream has access units, each consisting of one or more pictures belonging to layers according to the video coding format. The media file format specifies that, in response to a recording of an operating point included in a characteristic descriptor of the operating point that exhibits high-level characteristics of the bitstream, at least one of the values ​​of a first syntax element or a second syntax element in that recording is constrained to a predetermined value. method.

[0157] 5. An example of solution 4, The video coding format conforms to the VVC (Versatile Video Coding) standard in accordance with ISO / IEC 23090-3. method.

[0158] 6. An example of solution 4 or 5, The first syntax element specifies the maximum time identification associated with the i-th profile tier level syntax structure, where i ranges from 0 to (number of profile tier levels - 1). method.

[0159] 7. An example of solution 6, The first syntax element is represented as ptl_max_temporal_id[i], method.

[0160] 8. Any of the example solutions 4 through 7, The second syntax element specifies the maximum time identifier associated with recording the operating point. method.

[0161] 9. An example of solution 8, The second syntax element is represented as max_temporal_id. method.

[0162] 10. Any of the example solutions 4 through 9, The recording includes a third syntax element that specifies whether frame rate information exists, and the value of the third syntax element is constrained to a predetermined value. method.

[0163] 11. Any of the example solutions 4 through 10, The record includes a fourth syntax element that specifies whether bitrate information exists, and the value of the fourth syntax element is constrained to a predetermined value. method.

[0164] 12. Any of the example solutions 4 through 10, The given value is equal to 0. method.

[0165] 13. A video processing device having a processor, The processor is configured to perform one of the methods shown in Examples 1 through 12. Video processing device.

[0166] 14. A non-temporary computer-readable recording medium storing a video bitstream generated by any of the methods in Examples 1 to 12 performed by a video processing device.

[0167] In the solutions described herein, the encoder may conform to the format rules by generating the coded representation in accordance with the format rules. In the solutions described herein, the decoder may use the format rules to parse the syntax elements in the coded representation, knowing whether or not they are present, in order to generate the decoded video.

[0168] In this specification, the term “video processing” may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of video to a corresponding bitstream representation, or vice versa. The bitstream representation of the current video block may correspond to bits that are either in the same position or spread out in different locations within the bitstream, as defined, for example, by syntax. For example, a macroblock may be encoded with respect to the converted and coded error residue, and further with respect to bits in the header and other fields within the bitstream. Furthermore, during the conversion, the decoder may parse the bitstream knowing, based on a decision, the possible presence or absence of certain fields, as described in the solutions above. Similarly, the encoder may determine whether a particular syntax field should be included and, accordingly, generate the bitstream representation by including or excluding the syntax field from the bitstream representation.

[0169] The disclosures and other solutions, examples, embodiments, modules, and functional operations described herein can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. The disclosed embodiments and other embodiments can be implemented as one or more modules of computer program instructions encoded on a computer-readable medium for execution by or control of the operation of one or more computer program products, for example, a data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage carrier, a memory device, a composition that provides a machine-readable propagating signal, or one or more combinations thereof. The term “data processing device” encompasses all devices, machines, and equipment that process data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code that creates an execution environment for the computer program in question, such as processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. The propagating signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0170] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. Computer programs do not necessarily correspond to files in a file system. A program can be stored in a single file dedicated to the program in question, or in multiple coordinated files (e.g., a file storing one or more modules, subprograms, or parts of code), or in parts of a file that hold other programs or data (e.g., one or more scripts stored in markup language documents). Computer programs can be deployed to run on one computer, or on multiple computers located in one place or distributed across multiple locations and interconnected by a communication network.

[0171] The processes and logic flows described herein are executable by one or more programmable processors that execute one or more computer programs to perform a function by acting on input data to produce an output. The processes and logic flows are also executable by dedicated logic circuits, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs), and the device can be implemented as such.

[0172] Processors suitable for executing computer programs include, for example, one or more processors of any kind, both general-purpose and dedicated microprocessors, as well as any digital computer. Generally, a processor will read instructions and data from read-only memory or random-access memory, or both. Essential elements of a computer are a processor that executes instructions and one or more memory devices that store instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks, or will be operablely coupled for receiving data from or transferring data to or from such one or more mass storage devices, or both. However, a computer is not required to have such devices. Computer-readable media suitable for storing computer program instructions and data include, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and all forms of non-volatile memory, media, and memory devices, including CD-ROM and DVD-ROM disks. Processors and memory may be enhanced by or incorporated into dedicated logic circuits.

[0173] This specification contains numerous details, which should be interpreted not as limitations on the scope of any subject or potentially claimed subject matter, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described herein in relation to separate embodiments can also be implemented in combination with a single embodiment. Conversely, various features described in relation to a single embodiment can also be implemented separately in multiple embodiments or in some appropriate subcombination. Furthermore, features may be described first as operating in a particular combination, and may even be initially claimed as such, but in some cases one or more features from a claimed combination can be removed from that combination, and the claimed combination may be directed towards subcombinations or variations of subcombinations.

[0174] Similarly, although the operations are shown in a specific order in the drawings, this should not be understood as requiring that such operations be performed in that specific order or sequentially in which they are shown, or that all the operations shown be performed, in order to achieve the desired result. Furthermore, the isolation of various system components in the embodiments described herein should not be understood as requiring such isolation in all embodiments.

[0175] Only a few implementations and examples are described, and other implementations, enhancements, and modifications may be made based on those described and illustrated in this patent document.

Claims

1. A method for processing image data, The process includes a step of performing a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format, The media file format specifies that an image item of a first type value in the visual media file includes a single access unit of the bitstream, the single access unit being either an intra-random access picture (IRAP) access unit according to the video coding format or a progressive decoded refresh (GDR) access unit according to the video coding format, and all pictures within the GDR access unit are identified as recovery points in the bitstream. The media file format further specifies that the operating point information (OPI) network abstraction layer (NAL) unit is not present in the subpicture item of the second type value and in the sample of the subpicture item of the second type value. method.

2. The aforementioned video coding format conforms to the VVC (Versatile Video Coding) standard in accordance with ISO / IEC 23090-3. The aforementioned first type value is specified as 'vvc1'. The method according to claim 1.

3. Each of the pictures in the GDR access unit includes a picture header field having a value of zero indicating that the corresponding picture is the recovery point. The method according to claim 1 or 2.

4. The aforementioned picture header field corresponds to ph_recovery_poc_cnt_field, The method according to claim 3.

5. The media file format specifies that the image items of the first type value in the visual media file exclude layers that do not belong to the target output layer set. The method according to any one of claims 1 to 4.

6. The aforementioned image item includes a layer in the output layer set identified by the characteristic indicating the target output layer set, but does not include any other layers. The method according to claim 5.

7. The media file format specifies that the image item of the first type value in the visual media file includes at least a portion of an access unit in which the picture has one or more subpictures. The method according to any one of claims 1 to 4.

8. The aforementioned image item includes the entire access unit, The method according to claim 7.

9. The aforementioned image item includes a portion of the access unit, and for each layer present in the bitstream, the one or more subpictures form a rectangular region. The method according to claim 7.

10. The aforementioned video coding format conforms to the VVC (Versatile Video Coding) standard in accordance with ISO / IEC 23090-3. The aforementioned second type value is specified as 'vvs1'. The method according to claim 1.

11. The conversion includes encoding the bitstream into the visual media file. The method according to any one of claims 1 to 10.

12. The conversion includes decoding the bitstream from the visual media file. The method according to any one of claims 1 to 10.

13. A device for processing visual media file data, Processor and Non-temporary memory containing instructions and It has, When the aforementioned instruction is executed by the processor, it causes the processor to perform a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format, The media file format specifies that an image item of a first type value in the visual media file includes a single access unit of the bitstream, the single access unit being either an intra-random access picture (IRAP) access unit according to the video coding format or a progressive decoded refresh (GDR) access unit according to the video coding format, and all pictures within the GDR access unit are identified as recovery points in the bitstream. The media file format further specifies that the operating point information (OPI) network abstraction layer (NAL) unit is not present in the subpicture item of the second type value and in the sample of the subpicture item of the second type value. Device.

14. A non-temporary computer-readable storage medium that stores instructions, The instruction causes the processor to perform a conversion between a visual media file and a bitstream. The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format, The media file format specifies that an image item of a first type value in the visual media file includes a single access unit of the bitstream, the single access unit being either an intra-random access picture (IRAP) access unit according to the video coding format or a progressive decoded refresh (GDR) access unit according to the video coding format, and all pictures within the GDR access unit are identified as recovery points in the bitstream. The media file format further specifies that the operating point information (OPI) network abstraction layer (NAL) unit is not present in the subpicture item of the second type value and in the sample of the subpicture item of the second type value. A non-temporary computer-readable storage medium.

15. A method for storing a visual media file generated by a method performed by a video processing device, The steps include generating the aforementioned visual media file from a video bitstream, The process includes the step of storing the aforementioned visual media file on a non-temporary computer-readable recording medium, The visual media file has a sequence of one or more pictures according to a media file format, and the bitstream has one or more access units according to a video coding format. The bitstream is coded according to the video coding format, The media file format specifies that an image item of a first type value in the visual media file includes a single access unit of the bitstream, the single access unit being either an intra-random access picture (IRAP) access unit according to the video coding format or a progressive decoded refresh (GDR) access unit according to the video coding format, and all pictures within the GDR access unit are identified as recovery points in the bitstream. The media file format further specifies that the operating point information (OPI) network abstraction layer (NAL) unit is not present in the subpicture item of the second type value and in the sample of the subpicture item of the second type value. method.

Images