A method for low overhead image file format

Abstract

An apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: signal, into a bitstream, information with a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

Description

A Method For Low Overhead Image File FormatTECHNICAL FIELD

[0001] The examples and non-limiting embodiments relate generally to a method for low overhead image file format.BACKGROUND

[0002] It is known to process and render video in a multimedia system.SUMMARY OF THE INVENTION

[0003] According to some embodiments, there is provided the subject matter of the independent claims. Some further embodiments are defined in the dependent claims.BRIEF DESCRIPTION OF DRAWINGS

[0004] FIG. 1 shows an encoder according to an embodiment.

[0005] FIG. 2 shows a decoder according to an embodiment.

[0006] FIG. 3 is a block diagram illustrating a system in accordance with an example.

[0007] FIG. 4 is an example apparatus configured to implement the examples described herein.

[0008] FIG. 5 shows a representation of an example of non-volatile memory media used to store instructions that implement the examples described herein.

[0009] FIG. 6 is an example method based on the examples described herein.

[0010] FIG. 7 is an example method based on the examples described herein.DETAILED DESCRIPTION

[0011] ISO / IEC 23008-12 - High Efficiency Image File Format or HEIF is designed to enable the interchange of images and image sequences, as well as their associated metadata. Images can be stored as items using the support for untimed data storage in ISOBMFF, utilizing the MetaBox. A file may contain any number of image items. ISO corresponds to International Standards Organization, and I EC corresponds to International Electrotechnical Commission.

[0012] ISO base media file format

[0013] Available media file format standards include International Standards Organization (ISO) base media file format (ISO / IEC 14496-12, which may be abbreviated ISOBMFF), Moving Picture Experts Group (MPEG)-4 file format (ISO / IEC 14496-14, also known as the MP4 format), file format for NAL (Network Abstraction Layer) unit structured video (ISO / IEC 14496-15) and High Efficiency Video Coding standard (HEVC or H.265 / HEVC).

[0014] Some concepts, structures, and specifications of ISOBMFF are described below as an example of a container file format, based on which some embodiments may be implemented. The aspects of thedisclosure are not limited to ISOBMFF, but rather the description is given for one possible basis on top of which at least some embodiments may be partly or fully realized.

[0015] A basic building block in the ISO base media file format is called a box. Each box has a header and a payload. The box header indicates the type of the box and the size of the box in terms of bytes. Box type is typically identified by an unsigned 32-bit integer, interpreted as a four character code (4CC). A box may enclose other boxes, and the ISO file format specifies which box types are allowed within a box of a certain type. Furthermore, the presence of some boxes may be mandatory in each file, while the presence of other boxes may be optional. Additionally, for some box types, it may be allowable to have more than one box present in a file. Thus, the ISO base media file format may be considered to specify a hierarchical structure of boxes.

[0016] ISOBMFF defines the FileTypeBox as below.Box Type: 'ftyp'Container: File, or OriginalFileTypeBoxMandatory: YesQuantity: Exactly one (but see below)

[0017] Files with no FileTypeBox should be read as if they contained a FileTypeBox with Major_brand='mp4T, minor_version=0, and the single compatible brand 'mp41

[0018] A media-file structure may be compatible with more than one detailed specification, and it is therefore not always possible to speak of a single ‘type’ or ‘brand’ for the file. This means that the utility of the file name extension and multipurpose internet mail extension (MIME) type are somewhat reduced.

[0019] This box shall be placed as early as possible in the file (e.g. after any obligatory signature, but before any significant variable-size boxes such as a MovieBox, MediaDataBox, or FreeSpaceBox). It identifies which specification is the ‘best use’ of the file, and a minor version of that specification; and also a set of other specifications to which the file complies. Readers implementing this format should attempt to read files that are marked as compatible with any of the specifications that the reader implements. Any incompatible change in a specification should therefore register a new ‘brand’ identifier to identify files conformant to the new specification.

[0020] The minor version is informative only. It does not appear for compatible-brands, and is not used to determine the conformance of a file to a standard. It may allow more precise identification of the major specification, for inspection, debugging, or improved decoding.

[0021] Files would normally be externally identified (e.g. with a file extension or mime type) that identifies the ‘best use’ (major brand), or the brand that may provide the greatest compatibility.Syntaxaligned(8) class GeneralTypeBox(code) extends Box(code) {unsigned int(32) major_brand;unsigned int(32) minor_version;unsigned int(32) compati ble_brands[]; / / to end of the boxaligned(8) class FileTypeBox extends GeneralTypeBox ('ftyp'){}

[0022] Semantics

[0023] This box identifies the specifications to which this file complies.

[0024] Each brand is a four character code, registered with ISO, that identifies a precise specification.major_brand - is a brand identifierminor_version - is an informative integer for the minor version of the major brand compatible_brands - is a list, to the end of the box, of brands

[0025] In files conforming to the ISO base media file format, the media data may be provided in one or more instances of MediaDataBox (‘mdat‘) and the MovieBox (‘moov’) may be used to enclose the metadata for timed media. In some cases, for a file to be operable, both of the ‘mdat’ and ‘moov’ boxes may be required to be present. The ‘moov’ box may include one or more tracks, and each track may reside in one corresponding TrackBox (‘trak’). Each track is associated with a handler, identified by a four-character code, specifying the track type. Video, audio, and image sequence tracks can be collectively called media tracks, and they contain an elementary media stream. Other track types comprise hint tracks and timed metadata tracks.

[0026] Tracks comprise samples, such as audio or video frames. For video tracks, a media sample may correspond to a coded picture or an access unit.

[0027] A media track refers to samples (which may also be referred to as media samples) formatted according to a media compression format (and its encapsulation to the ISO base media file format). A hint track refers to hint samples, containing cookbook instructions for constructing packets for transmission over an indicated communication protocol. A timed metadata track may refer to samples describing referred media and / or hint samples.

[0028] The 'trak' box includes in its hierarchy of boxes the SampleDescriptionBox, which gives detailed information about the coding type used, and any initialization information needed for that coding. The SampleDescriptionBox contains an entry-count and as many sample entries as the entry-count indicates. The format of sample entries is track-type specific but derived from generic classes (e.g. VisualSampleEntry,Aud ioSamp leE ntry). Which type of sample entry form is used for derivation of the track-type specific sample entry format is determined by the media handler of the track.

[0029] The track reference mechanism can be used to associate tracks with each other. The TrackReferenceBox includes box(es), each of which provides a reference from the containing track to a set of other tracks. These references are labeled through the box type (e.g., the four-character code of the box) of the contained box(es).

[0030] The ISO Base Media File Format contains three mechanisms for timed metadata that can be associated with particular samples: sample groups, timed metadata tracks, and sample auxiliary information. A derived specification may provide similar functionality with one or more of these three mechanisms.

[0031] A sample grouping in the ISO base media file format and its derivatives, such as the advanced video coding (AVC) file format and the scalable video coding (SVC) file format, may be defined as an assignment of each sample in a track to be a member of one sample group, based on a grouping criterion. A sample group in a sample grouping is not limited to being contiguous samples and may contain non-adjacent samples. As there may be more than one sample grouping for the samples in a track, each sample grouping may have a type field to indicate the type of grouping. Sample groupings may be represented by two linked data structures: (1) a SampleToGroupBox (sbgp box) represents the assignment of samples to sample groups; and (2) a SampleGroupDescriptionBox (sgpd box) contains a sample group entry for each sample group describing the properties of the group. There may be multiple instances of the SampleToGroupBox and SampleGroupDescriptionBox based on different grouping criteria. These may be distinguished by a type field used to indicate the type of grouping. SampleToGroupBox may comprise a grouping_type_parameter field that can be used e.g. to indicate a sub-type of the grouping.

[0032] In ISOMBFF, an edit list provides a mapping between the presentation timeline and the media timeline. Among other things, an edit list provides for the linear offset of the presentation of samples in a track, provides for the indication of empty times and provides for a particular sample to be dwelled on for a certain period of time. The presentation timeline may be accordingly modified to provide for looping, such as for the looping videos of the various regions of the scene. One example of the box that includes the edit list, the EditListBox, is provided below:aligned(8) class EditListBox extends FullBoxfelst’, version, flags) {unsigned int(32) entry_count;for (i=1; i <= entry_count; i++) {if (version==1) {unsigned int(64) segment_duration;int(64) media_time;} else { / / version==0unsigned int(32) segment_duration;int(32) media_time;}int(16) media_rate_integer;int(16) media_rate_fraction = 0;}}

[0033] In ISOBMFF, an EditListBox may be contained in EditBox, which is contained in TrackBox ('trak').

[0034] In this example of the edit list box, flags specifies the repetition of the edit list. By way of example, setting a specific bit within the box flags (the least significant bit, i.e., flags & 1 in ANSI-C notation, where & indicates a bit-wise AND operation) equal to 0 specifies that the edit list is not repeated, while setting the specific bit (i.e., flags & 1 in ANSI-C notation) equal to 1 specifies that the edit list is repeated. The values of box flags greater than 1 may be defined to be reserved for future extensions. As such, when the edit list box indicates the playback of zero or one samples, (flags & 1) shall be equal to zero. When the edit list is repeated, the media at time 0 resulting from the edit list follows immediately the media having the largest time resulting from the edit list such that the edit list is repeated seamlessly. ANSI-C is the C programming language standard.

[0035] In ISOBMFF, a Track group enables grouping of tracks based on certain characteristics or the tracks within a group have a particular relationship. Track grouping, however, does not allow any image items in the group.

[0036] The syntax of TrackGroupBox in ISOBMFF is as followsaligned(8) class TrackGroupBox extends Box('trgr') {}aligned(8) class TrackGroupTypeBox(unsigned int(32) track_group_type) extends FullBox(track_group_type, version = 0, flags = 0) { unsigned int(32) track_group_id; / / the remaining data may be specified for a particular track_group_type}

[0037] track_group_type indicates the grouping_type and shall be set to one of the following values, or a value registered, or a value from a derived specification or registration.

[0038] 'msrc' indicates that this track belongs to a multi-source presentation. The tracks that have the same value of track_group_id within a TrackGroupTypeBox of track_group_type 'msrc' are mapped as being originated from the same source. For example, a recording of a video telephony call may have both audioand video for both participants, and the value of track_group_id associated with the audio track and the video track of one participant differs from value of track_group_id associated with the tracks of the other participant.

[0039] The pair of track_group Jd and track_g rou p_ty pe identifies a track group within the file. The tracks that contain a particular TrackGroupTypeBox having the same value of track_group_id and track_group_type belong to the same track group.

[0040] The Entity grouping is similar to track grouping but enables grouping of both tracks and image items in tha same group.

[0041] The syntax of EntityToGroupBox in ISOBMFF is as follows.aligned(8) class EntityToGroupBox(grouping_type, version, flags)extends FullBox(grouping_type, version, flags) {unsigned int(32) groupjd;unsigned int(32) num_entities_in_group;for(i=0; i<num_entities_in_group; i++)unsigned int(32) entity Jd;}

[0042] groupjd is a non-negative integer assigned to the particular grouping that shall not be equal to any groupjd value of any other EntityToGroupBox, any itemJD value of the hierarchy level (file, movie, or track) that contains the GroupsListBox, or any track D value (when the GroupsListBox is contained in the file level).

[0043] num_entitiesjn_group specifies the number of entityjd values mapped to this entity group.

[0044] entityjd is resolved to an item, when an item with item D equal to entityjd is present in the hierarchy level (file, movie or track) that contains the GroupsListBox, or to a track, when a track with trackJD equal to entityjd is present and the GroupsListBox is contained in the file level.

[0045] Files conforming to the ISOBMFF may contain any non-timed objects, referred to as items, meta items, or metadata items, in a meta box (four-character code: ‘meta’). While the name of the meta box refers to metadata, items can generally contain metadata or media data. The meta box may reside at the top level of the file, within a movie box (four-character code: ‘moov’), and within a track box (four-character code: rak’), but at most one meta box may occur at each of the file level, movie level, or track level. The meta box may be required to contain a ‘hdlr’ box indicating the structure or format of the ‘meta’ box contents. The meta box may list and characterize any number of items that can be referred and each one of them can be associated with a file name and are uniquely identified with the file by item identifier (itemjd) which is an integer value. The metadata items may be for example stored in the 'idat' box of the meta box or in an 'mdat' box or reside in a separate file. If the metadata is located external to the file then its location may be declaredby the DatalnformationBox (four-character code: ‘dinf ). In the specific case that the metadata is formatted using extensible markup language (XML) syntax and is required to be stored directly in the MetaBox, the metadata may be encapsulated into either the XMLBox (four-character code: ‘xml ‘) or the BinaryXMLBox (four-character code: ‘bxml’). An item may be stored as a contiguous byte range, or it may be stored in several extents, each being a contiguous byte range. In other words, items may be stored fragmented into extents, e.g. to enable interleaving. An extent is a contiguous subset of the bytes of the resource. The resource can be formed by concatenating the extents.

[0046] A common base structure is used to contain general untimed metadata. This structure is called the MetaBox as it was originally designed to carry metadata, i.e. data that is annotating other data. However, it is now used for a variety of purposes including the carriage of data that is not annotating other data, especially when present at ‘file level’.

[0047] The MetaBox is required to contain a HandlerBox indicating the structure or format of the MetaBox contents.

[0048] All other contained boxes are specific to the format specified by the HandlerBox.

[0049] The other boxes defined here may be defined as optional or mandatory for a given format. If they are used, then they shall take the form specified here. These optional boxes include a DatalnformationBox, which documents other files in which metadata values (e.g. pictures) are placed, and an ItemLocationBox, which documents where in those files each item is located (e.g. in the common case of multiple pictures stored in the same file).

[0050] At most one MetaBox may occur at each of the file level, segment, movie level, or track level.

[0051] If an ItemProtectionBox occurs, then some or all of the metadata, including possibly the primary resource, may have been protected and be un-readable unless the protection system is taken into account.

[0052] The MetaBox is unusual in that it is a container box yet extends FullBox, not Box.

[0053] Metadata items are identified by item_ID. Within a given MetaBox, a given item_l D shall uniquely refer to a single item. When an item is updated in movie fragments, the item_l D refers to the latest received version. Derived specifications may further restrict the criteria for uniqueness: unique among the item J Ds in both file and movie-level boxes, or unique within that set extended with the trackJD of the tracks in a movie box. The item_l D value of 0 should not be used, and shall not be used when the set is extended to include trackJDs.

[0054] There are three scopes for item J Ds: file and segments; MovieBox and MovieFragmentBox; and TrackBox and TrackFragmentBox. In other words, there shall be only one item with a given item_l D within a given scope (e.g. in the TrackBox and all TrackFragmentBox with the same trackJD).aligned(8) class MetaBox (handlerjype)extends FullBox('meta', version = 0, 0) {HandlerBox(handler_type) theHandler;PrimaryltemBox primary_resource; / / optional DatalnformationBox fileJocations; / / optionalItemLocationBox itemjocations: / / optionalItemProtectionBox protections; / / optionalItemlnfoBox itemjnfos; / / optionalIPMPControlBox IPMP_control; / / optional ItemReferenceBox item_refs; / / optionalItemDataBox item_data; / / optionalBox other_boxesD; / / optional}

[0055] The structure or format of the metadata is declared by the handler. In the case that the primary data is identified by a primary item, and that primary item has an item information entry with an item_type, the handler type may be the same as the item_type. IPMP is an abbreviation for internet protocol (IP) maintenance protocol.

[0056] The ItemPropertiesBox enables the association of any item with an ordered set of item properties. Item properties may be regarded as small data records. The ItemPropertiesBox consists of two parts: ItemPropertyContainerBox that contains an implicitly indexed list of item properties, and one or more ItemPropertyAssociationBox(es) that associate items with item properties.

[0057] Applications are informed about the different operating points provided by a given WC elementary stream and their constitution by using the Operating Points Information sample group ('vopi'). Each operating point is related to an output layer set, a max Temporalld value, and a profile, tier, and level signalling. All this information is captured by the 'vopi' sample group. Apart from this information, this sample group also provides the dependency information between layers.

[0058] When more than one WC track is present for a WC elementary stream and no Operating Point entity group is present for the WC elementary stream, both of the following apply:- Among the WC tracks for the WC elementary stream there shall be one and only one track that carries a 'vopi' sample group.- All the other WC tracks of the WC elementary stream shall have a track reference of type 'vref to the track that carries the 'vopi' sample group.

[0059] For any specific sample in a given track, the temporally collocated sample in another track is defined as the one with the same decoding time as that of this specific sample. For each sample SN in atrack TN that has a 'vref track reference to the track Tk that carries the 'vopi' sample group, the following applies:- If there is a temporally collocated sample Sk in the track Tk, then the sample SN is associated with the same 'vopi' sample group entry as the sample Sk.- Otherwise, the sample SN is associated with the same 'vopi' sample group entry as the last of the samples in the track Tk that precede the sample SN in decoding time.

[0060] When several VPSs are referenced by a VVC elementary stream, it may be needed to include several entries in the sample group description box with groupi ng_type equal to 'vopi'. When one and only one VPS is present, it is recommended to use the default sample group description index and include the operating points information sample group in the sample table box, rather than including it in each track fragment.

[0061] The grouping_type_parameter is not defined for the SampleToGroupBox with grouping type 'vopi'.

[0062] Syntaxclass VvcOperatingPointsRecord {unsigned int(8) num_profile_tier_level_minus1;for (i=0; i<=num_profile_tier_level_minus1 ; i++) {bit(5) reserved = 0;unsigned int(3) ptl_max_temporal_id[i];VvcPTLRecord(ptl_max_temporal_id[i]+1 ) ptl [i] ;}unsigned int(1) alljndependent_layers_flag;if (all_independent_layers_flag){unsigned int(1) each_layer_is_an_ols_flag;bit(1) reserved = 0;} elseunsigned int(2) olsjnodejdc;bit(2) reserved = 0;unsigned int(1) incomplete_operating_points_flag;unsigned int(1) num_ref_sublayer_for_ols_present_flag;unsigned int(9) num_olss;for (i=0; i<num_olss; i++) {unsigned int(8) ptl Jdx[i];unsigned int(9) ols_idx[i];unsigned int(6) layer_count[i];bit(1) reserved = 0;for 0=0; j<layer_count[i]; j++) {unsigned int(6) layer_id[i][j];unsigned int(1) is_output_layer[i][j];bit(1) reserved = 0;if(!is_outputlayer[i][j] && num_ref_sublayer_for_ols_present_flag) unsigned int(8) num_ref_sublayers Jn_layer_in_ols[i][j]; }}bit(4) reserved = 0;unsigned int(12) num_operating_points;for (i=0; i<num_operating_points; i++) {unsigned int(9) ols_loop_entry_idx;unsigned int(3) max_temporal_id;unsigned int(1) frame_rate_info_flag;unsigned int(1) bit_rate_info_flag;if (incomplete_operating_points_flag)unsigned int(2) op_availability_idc;elsebit(2) reserved = 0;bit(3) reserved = 0;unsigned int(2) chroma_format_idc;unsigned int(3) bit_depth_minus8;u nsig ned J nt(16) max_picture_width;u nsig ned J nt(16) max_picture_height;if (frame_rate_info_flag) {unsigned int(16) avg_frame_rate;bit(6) reserved = 0;unsigned int(2) constant_frame_rate;}if (bit_rate_info_flag) {unsigned int(32) max_bit_rate;unsigned int(32) avg_bit_rate;}}if (Sail Jndependent_layers_flag) {unsigned int(8) max_layer_count;for (i=0; i<max_layer_count; i++) {unsigned int(8) layerjd;unsigned int(8) num_direct_ref_layers;for 0=0; j<num_direct_ref_layers; j++)unsigned int(8) direct_ref_layer_id;}}}class VvcOperatingPointsinformation extends VisualSampleGroupEntry ('vopi') { VvcOperatingPointsRecord oinf;}

[0063] Semantics

[0064] num_profile_tier_level_minus1 plus 1 gives the number of the subsequent profiles, tier, and level combinations as well as the associated fields. ptl_max_temporal_id[i] gives the maximum TemporallD of NAL units of the associated bitstream for the specified i-th profile, tier, and level structure. The semantics of ptl_max_temporal Jd[i] and max_temporal_id of an operating point, given below, are different even though they can carry the same numerical value. ptl[i] specifies the i-th profile, tier, and level structure.

[0065] all Jndependent_layers_flag, each_layer_is_an_ols_flag, olsjnodejdc and max_tid Jl_ref_pics_plus1 are defined in ISO / IEC 23090-3. ide is an abbreviation for indicator.

[0066] incomplete_operating_points_flag equal to 1 specifies that some operating points declared in the structure may be incomplete (i.e., some output layer sets or temporal sublayers may be missing in some operating points). incomplete_operating_points_flag equal to 0 specifies that all operating points declared in the structure are complete (i.e., all output layer sets and temporal sublayers associated with the operating points are present in the file).

[0067] num_ref_sublayer_for_ols_present_flag equal to 1 specifies that the syntax element num_ref_sublayers_in Jayer Jn_ols[i] [j] may be present.

[0068] num_ref_sublayer_for_ols_present_flag equal to 0 specifies that the syntax element num_ref_sublayersjn_layer_in_ols[i][j] is not present.

[0069] num_olss specifies the number of output layer sets signalled in this syntax structure. The value of num_olss shall be less than or equal to the value of TotalNumOlss as specified in ISO / IEC 230903.

[0070] ptl Jdx[i] specifies the zero-based index of the listed profile, tier, and level structure for the i-th output layer set signalled in this syntax structure. olsjdx[i] is the output layer set index of the i-th output layer set signalled in this syntax structure. Iayer_count[i] specifies the number of layers in the i-th output layer set signalled in this syntax structure. Iayer_id[i][j] specifies the nuhjayerjd value for the j-th layer in the i-th output layer set signalled in this syntax structure, idx is an abbreviation for index.

[0071] is_output_layer[i][j] equal to 1 specifies that the j-th layer is an output layer in the i-th output layer set signalled in this syntax structure. is_output_layer[i][j] equal to 0 specifies that the j-th layer in not an output layer in the i-th output layer set signalled in this syntax structure.

[0072] num_ref_sublayers_in Jayer J n_ols[i] [j] equal to 0 specifies that the pictures of the layer with layer Id equal to layer_id[i][j] that are neither IRAP pictures nor gradual decoder refresh (GDR) pictures with ph_recovery_poc_cnt equal to 0 are not used as inter-layer reference pictures for decoding of pictures of other layers in the output layer set. num_ref_sublayers_in Jayer J n_ols[i] [j] greater than 0 specifies that, for decoding pictures of the other layers of the output layer set, no picture from the layer with layer Id equal to layer_id[i][j] with Temporalld greater than num_ref_sublayersjn_layer_in_ols[i][j] - 1 is used as inter-layer reference picture and no APS with nuhjayerjd equal to layerjd[i][j] and Temporalld greater than num_ref_sublayersjnjayerjn_ols[i][j] - 1 is referenced. When not present, the value of num_ref_sublayersjnjayerjn_ols[i][j] is inferred to be equal to 7. num_operating_points gives the number of operating points for which the information follows. The list of operating points shall include all operating points that are present in sample entry of tracks representing the bitstream.

[0073] olsjoop_entryjdx is the index to the list of output layer sets signalled in this syntax structure for the operating point. The variable olsldx is set equal to olsjdx[ olsjoop_entryjdx ]for the present loop entry.

[0074] max_temporal Jd indicates the maximum Temporalld of NAL units of this operating point.

[0075] frame_ratejnfo_flag equal to 0 indicates that no frame rate information is present for the operating point. The value 1 indicates that frame rate information is present for the operating point.

[0076] bit_rate J nfo_flag equal to 0 indicates that no bitrate information is present for the operating point. The value 1 indicates that bitrate information is present for the operating point.

[0077] op_availabilityjdc equal to 0 indicates that the operating point is complete. op_availabilityjdc equal to 1 indicates that at least one layer is missing in the operating point. op_availabilityjdc equal to 2 indicates that at least one temporal sublayer is missing in the operating point. op_availability Jdc equal to 3indicates that at least one layer is missing in the operating point and additionally at least one temporal sublayer is missing in the operating point.

[0078] The process specified in subclause 11.5.4 is specified for operating points when incomplete_operating_points_flag is equal to 0 or op_availability_idc is equal to 0 (operating point is complete). When op_availability Jdc is equal to 1 , 2 or 3, a file reader needs to inspect the file to determine if the operating point can be processed and might need to alter the bitstream reconstruction process compared to what is specified in subclause 11.5.4.

[0079] chroma_format_idc indicates the chroma format that applies to this operating point. The following constraints apply for chroma_format_idc: If this operating point contains only one layer, the value of sps_chroma_format_idc, as defined in ISO / IEC 23090-3, shall be the same in all SPSs referenced by the VCL NAL units in the VVC bitstream of this operating point, and the value of chroma_format_idc shall be equal to that value of sps_chroma_format_idc. Otherwise (this operating point contains more than one layer), the value of chroma_format_idc shall be equal to the value of vps_ols_dpb_chroma_format[ MultiLayerOlsl dx[ olsldx ] ], as defined in ISO / IEC 23090-3.

[0080] bit_depth_minus8 indicates the bit depth that applies to this operating point. The following constraints apply for bit_depth_minus8: If this operating point contains only one layer, the value of sps_bitdepth_minus8, as defined in ISO / IEC 23090-3, shall be the same in all SPSs referenced by the VCL NAL units in the WC bitstream of this operating point, and the value of bit_depth_minus8 shall be equal to that value of sps_bitdepth_minus8. Otherwisefthis operating point contains more than one layer), the value of bit_depth_minus8 shall be equal to the value of vps_ols_dpb_bitdepth_minus8[ Multi LayerOlsldx[ olsldx ] ], as defined in ISO / IEC 23090-3.

[0081] max_picture_width indicates the maximum picture width, in units of luma, L, samples, that applies to this operating point. The following constraints apply for max_picture_width: If this operating point contains only one layer, the value of sps_pic_width_max_in_luma_samples, as defined in ISO / IEC 23090-3, shall be the same in all SPSs referenced by the VCL NAL units in the VVC bitstream of this operating point, and the value of max_picture_width shall be equal to that value of sps_pic_width_max_in_luma_samples. Otherwise (this operating point contains more than one layer), the value of max_picture_width shall be equal to the value of vps_ols_dpb_pic_width[ MultiLayerOlsldx[ olsldx ] ], as defined in ISO / IEC 23090-3.

[0082] max_picture_height indicates the maximum picture height, in units of luma samples, that applies to this operating point. The following constraints apply for max_picture_height: If this operating point contains only one layer, the value of sps_pic_height_max_in_luma_samples, as defined in ISO / IEC 23090-3, shall be the same in all SPSs referenced by the VCL NAL units in the WC bitstream of this operating point, and the value of max_picture_height shall be equal to that value of sps_pic_height_max_in_luma_samples.Otherwise (this operating point contains more than one layer), the value of max_picture_height shall be equal to the value of vps_ols_dpb_pic_height[ MultiLayerOlsl dx[ olsldx ] ], as defined in ISO / IEC 23090-3.

[0083] avg_frame_rate gives the average frame rate in units of frames / (256 seconds) for the operating point. Value 0 indicates an unspecified average frame rate. When the bitstream of the operating point contains multiple layers, this gives the average access unit rate.

[0084] constant_frame_rate equal to 1 indicates that the bitstream of the operating point is of constant frame rate. Value 2 indicates that the representation of each sublayer in the bitstream of the operating point is of constant frame rate. Value 0 indicates that the bitstream of the operating point may or may not be of constant frame rate. constant_frame_rate equal to 3 is reserved. When the bitstream of the operating point contains multiple layers, this gives the indication of whether the bitstream of the operating point has constant access unit rate.

[0085] max_bit_rate gives the maximum bit rate in bits / second of the bitstream of the operating point, over any window of one second. avg_bit_rate gives the average bit rate in bits / second of the bitstream of the operating point. max_layer_count specifies the count of all unique layers in all of the operating points described in this syntax structure, layerjd specifies nuhjayerjd of a layer for which all the direct reference layers are given in the following loop of direct_refjayerjd. num_direct_ref_layers specifies the number of direct reference layers for the layer with nuhjayerjd equal to layerjd. When alljndependentjayers_flag is equal to 1, num_direct_refjayers is inferred to be equal to 0 for each layer. direct_refjayerjd indicates nuhjayerjd of the direct reference layer.

[0086] max id Jl_ref_pics_plus1 equal to 0 specifies that the pictures of the layer with nuhjayerjd equal to direct_refjayerjd that are neither I RAP pictures nor GDR pictures with ph_recovery_poc_cnt equal to 0 are not used as inter-layer reference pictures for decoding of pictures of the layer with nuhjayerjd equal to layerjd. A value greater than 0 specifies that, for decoding pictures of the layer with nuhjayerjd equal to layerjd, no picture from the layer with nuhjayerjd equal to direct_refjayerjd with Temporalld greater than maxjid Jl_ref_pics_plus1 - 1 is used as an inter-layer reference picture and no APS with nuhjayerjd equal to direct_refjayerjd and Temporalld greater than max_tidjl_ref_pics_plus1 - 1 is referenced.

[0087] High Efficiency Image File Format (HEIF)

[0088] High Efficiency Image File Format (HEIF) is a standard developed by the Moving Picture Experts Group (MPEG) for storage of images and image sequences. Among other things, the standard facilitates file encapsulation of data coded according to the High Efficiency Video Coding (HEVC) standard. HEIF includes features building on top of the used ISO Base Media File Format (ISOBMFF).

[0089] The ISOBMFF structures and features are used to a large extent in the design of HEIF. The basic design for HEIF comprises still images that are stored as items and image sequences that are stored as tracks. An item in HEIF is defined as the data that does not require timed processing, as opposed to sample data, and is described by the boxes contained in a MetaBox.

[0090] In the context of HEIF, the following boxes may be contained within the root-level 'meta' box and may be used as described in the following. In HEIF, the handler value of the Handler box of the 'meta' box is 'pict'. The resource (whether within the same file, or in an external file identified by a uniform resource identifier) containing the coded media data is resolved through the Data Information ('dinf') box, whereas the Item Location ('Hoc') box stores the position and sizes of every item within the referenced file. The Item Reference ('iref) box documents relationships between items using typed referencing. If there is an item among a collection of items that is in some way to be considered the most important compared to others then this item is signaled by the Primary Item ('pitm') box. Apart from the boxes mentioned here, the 'meta' box is also flexible to include other boxes that may be necessary to describe items.

[0091] Any number of image items can be included in the same file. Given a collection of images stored by using the 'meta' box approach, it sometimes is essential to qualify certain relationships between images. Examples of such relationships include indicating a cover image for a collection, providing thumbnail images for some or all of the images in the collection, and associating some or all of the images in a collection with an auxiliary image such as an alpha plane. A cover image among the collection of images is indicated using the 'pitm' box. A thumbnail image or an auxiliary image is linked to the primary image item using an item reference of type 'thmb' or 'auxl', respectively.

[0092] Layer selectionDefinitionBox type:Property type: Descriptive item propertyContainer: ItemPropertyContainerBoxMandatory (per item): NoQuantity (per item): Zero or one

[0093] If the decoding of a multi-layer image item results into more than one reconstructed image, the 'Isel' item property shall be associated with the image item. Otherwise, the 'Isel' item property shall not be associated with an image item.

[0094] essential shall be equal to 1 for an 'Isel' item property.

[0095] The 'Isel' item property, if any, shall precede, in the item property association order, all transformative item properties.

[0096] This property is used to select which of the reconstructed images is described by subsequent descriptive item properties in the item property association order and manipulated by transformative item properties, if any, to generate an output image of the image item.Syntaxaligned(8) class LayerSelectorPropertyextends ItemProperty('lsel') {unsigned int(16) layerjd;}

[0097] layerjd specifies the layer identifier of the image among the reconstructed images that is described by subsequent descriptive item properties in the item property association order and manipulated by transformative item properties, if any, to generate an output image of the image item. The semantics of layerjd are specific to the coding format and are therefore defined for each coding format for which the decoding of a multi-layer image item can result into more than one reconstructed images.

[0098] Target output layer set

[0099] DefinitionBox type: 'tols'Property type: Descriptive item propertyContainer: ItemPropertyContainerBoxMandatory (per item): No, unless otherwise specified by the coding formatQuantity (per item): Zero or one for a coded image item

[0100] TargetOlsProperty provides the output layer set index to be used as input for the decoding process of the associated coded image item.

[0101] essential shall be equal to 1 for a TargetOlsProperty.aligned(8) class TargetOlsPropertyextends ltemFullProperty('tols', version = 0, flags = 0){unsigned int(16) target_olsjdx;}

[0102] target_olsjdx is the output layer set index to be provided to the decoding process. The semantics of target_olsjdx are specific to the coding format of the image item and are therefore defined for each coding format that requires its use.

[0103] A WC image item contains independently coded WC image.

[0104] An item of type 'vvd' consists of the NAL units of a WC bitstream that are length-delimited as specified below, and the VVC bitstream contains an entire WC access unit, wherein each WC picture may contain multiple "extractable" WC subpictures, where:

[0105] NAL units with nuh JayerJd greater than 0 may be present in items of type 'vvd Readers shall process NAL units with nuhjayerjd greater than 0 in an item of type 'vvd' in a similar manner as NAL units with nuhjayerjd equal to 0.

[0106] In an image item of type 'vvd', WC pictures in non-independent layers may use inter-layer prediction and hence may contain inter coded slices. A WC picture that does not use inter-layer prediction in a WC image item may be an IDR or CRA picture or a GDR picture with ph_recovery_poc_cnt equal to 0 as defined in ISO / IEC 23090-3.

[0107] An item of type 'vvd' contains at most one WC-coded picture for any specific value of nuhjayerjd.

[0108] All image items of type 'vvd' with multiple layers shall have an associated item property VvcOperatingPointsInformationProperty. The VvcOperatingPointsInformationProperty provides a summary of the high-level characteristics of the WC bitstream contained in the WC image item with multiple layers, similar to the 'vopi' sample grouping of ISO / IEC 14496-15.

[0109] All image items of type 'vvd' shall have zero or one associated item property TargetOlsProperty. The TargetOlsProperty contains the target_olsjdx, which provides the output layer set index to be used as input for the decoding process of WC coded image item. The target_olsjdx is used as the value of the TargetOlsIdx variable and is specified in the same coding format as in ISO / IEC 23090-3. The number of TargetOlsProperty shall not be zero unless there is only one WC picture in the WC-coded image item. The 'vvd' image item shall include the layers that are included in the output layer set identified by the associated TargetOlsProperty and shall not include other layers.

[0110] The target_olsjdx in an associated TargetOlsProperty, when present, shall be same as the olsjdx value (when ptl_present_flag is equal to 1) of the VvcDecoderConfigurationRecord.

[0111] When the value that would be derived for the TargetOlsIdx variable by the WC decoding process would be different from the target_olsjdx in an associated TargetOlsProperty, when present, or otherwise from the olsjdx when ptl_present_flag is equal to 1 in the VvcDecoderConfigurationRecord in the WC configuration item property, the reader shall provide the target_olsjdx in the associated TargetOlsProperty or otherwise the olsjdx value of the VvcDecoderConfigurationRecord to the decoder through external means by including it in an operating point information, OPI, NAL unit that is inserted into the WC bitstream reconstructed from the image item.

[0112] An image item of type 'vvd ' may have an associated item property LayerSelectorProperty. The LayerSelectorProperty shall contain layerjd that is among the nuhjayerjd values of the output layers of the output layer set identified by TargetOlsProperty associated with the same VVC image item.

[0113] Syntaxaligned(8) class VVCItemData{for (i=0; i<item_size; ) / / item_size from summing the extents / / in the ItemLocationBox{unsigned int((DecoderConfigurationRecord.LengthSizeMinusOne+1)*8) NALUnitLength;bit(NALUnitl_ength * 8) NALUnit;i ■+= (DecoderConfigurationRecord.LengthSizeMinusOne+1) +NALUnitLength;}}

[0114] In the syntax above, the following applies: The value of item_size is equal to the sum of the extentjength values of each extent of the item, as specified in the ItemLocationBox, and DecoderConfigurationRecord indicates the record in the associated configuration initialization property.

[0115] NALUnitLength indicates the size of a NAL unit measured in bytes. The length field includes the size of both the two-byte NAL header and the NAL unit payload but does not include the length field itself.

[0116] NALUnit contains a single NAL unit. The syntax of a NAL unit shall conform to the syntax specified in ISO / IEC 23090-3 and includes both the two-byte NAL unit header and the variable-length NAL unit payload.

[0117] WC operating points information property

[0118] DefinitionBox type: 'vopi'Property type: Descriptive item propertyContainer: ItemPropertyContainerBoxMandatory (per item): No, for an image item of type 'vvd 'Quantity (per item): Zero or one for an image item of type 'vv '

[0119] The VvcOperatingPointsInformationProperty is similar to the VvcOperatingPointsInformation specified in ISO / IEC 14496-15 but applies to VVC image items.

[0120] The VvcOperatingPointsInformationProperty informs about the different operating points provided by a WC bitstream and their constitution. Each operating point is related to an output layer set and a combination of a profile, tier, and level. TargetOlsProperty associated with a VVC image item provides the output layer set index that can be used to select which operating-point-specific information of the VvcOperatingPointsInformationProperty applies to the WC image item. The VvcOperatingPointsInformationProperty also provides the dependency information between layers.

[0121] Syntaxaligned(8) class VvcOperatingPointsInformationPropertyextends ItemFullPropertyfvopi', version = 0, flags = 0){VvcOperatingPointsRecord opjnfo; / / specified in ISO / IEC 14496-15}

[0122] The semantics of VvcOperatingPointsRecord are specified in ISO / IEC 14496-15. When included in VvcOperatingPointsInformationProperty, the values of the syntax elements of VvcOperatingPointsRecord are constrained as follows:

[0123] frame_rate_i nfo_flag shall be equal to 0. Consequently, avg_frame_rate and constant_frame_rate are not present and their semantics are not specified.

[0124] bit_rate_info_flag shall be equal to 0. Consequently, max_bit_rate and avg_bit_rate are not present and their semantics are not specified.

[0125] ptl_max_temporal_id[i] for each value of i in the range of 0 to num_profile_tier_level_minus1, inclusive, shall be equal to 0 when conformance to the 'pred' brand is not indicated.

[0126] max_temporal Jd shall be equal to 0 when conformance to the 'pred' brand is not indicated.

[0127] Background on Video coding

[0128] Video codec consists of an encoder and a decoder. The encoder transforms the input video into a compressed representation suited for storage / transmission. The decoder can uncompress the compressed video representation back into a viewable form. A video encoder and / or a video decoder may also be separate from each other, i.e. need not form a codec. The encoder may discard some information in the original video sequence in order to represent the video in a more compact form (that is, at lower bitrate).

[0129] Hybrid video encoders, for example many encoder implementations of ITU-T H.263 and H.264, may encode the video information in two phases. Firstly pixel values in a certain picture area (or “block”) are predicted for example by motion compensation means (finding and indicating an area in one of the previously coded video frames that corresponds closely to the block being coded) or by spatial means (using the pixel values around the block to be coded in a specified manner). Secondly the prediction error, i.e. the differencebetween the predicted block of pixels and the original block of pixels, is coded. This may be done by transforming the difference in pixel values using a specified transform (e.g. Discrete Cosine Transform (DCT) or a variant of it), quantizing the coefficients and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, encoder can control the balance between the accuracy of the pixel representation (picture quality) and size of the resulting coded video representation (file size or transmission bitrate).

[0130] In temporal prediction, the sources of prediction are previously decoded pictures (a.k.a. reference pictures). In intra block copy (IBC; a.k.a. intra-block-copy prediction), prediction is applied similarly to temporal prediction but the reference picture is the current picture and only previously decoded samples can be referred in the prediction process. Inter-layer or inter-view prediction may be applied similarly to temporal prediction, but the reference picture is a decoded picture from another scalable layer or from another view, respectively. In some cases, inter prediction may refer to temporal prediction only, while in other cases inter prediction may refer collectively to temporal prediction and any of intra block copy, inter-layer prediction, and inter-view prediction provided that they are performed with the same or similar process than temporal prediction. Inter prediction or temporal prediction may sometimes be referred to as motion compensation or motion-compensated prediction.

[0131] Inter prediction, which may also be referred to as temporal prediction, motion compensation, or motion-compensated prediction, reduces temporal redundancy. In inter prediction the sources of prediction are previously decoded pictures. Intra prediction utilizes the fact that adjacent pixels within the same picture are likely to be correlated. Intra prediction can be performed in spatial or transform domain, i.e., either sample values or transform coefficients can be predicted. Intra prediction is typically exploited in intra coding, where no inter prediction is applied.

[0132] One outcome of the coding procedure is a set of coding parameters, such as motion vectors and quantized transform coefficients. Many parameters can be entropy-coded more efficiently if they are predicted first from spatially or temporally neighboring parameters. For example, a motion vector may be predicted from spatially adjacent motion vectors and only the difference relative to the motion vector predictor may be coded. Prediction of coding parameters and intra prediction may be collectively referred to as inpicture prediction.

[0133] The H.264 / AVC standard was developed by the Joint Video Team (JVT) of the Video Coding Experts Group (VCEG) of the Telecommunications Standardization Sector of International Telecommunication Union (ITU-T) and the Moving Picture Experts Group (MPEG) of International Organisation for Standardization (ISO) / International Electrotechnical Commission (IEC). The H.264 / AVC standard is published by both parent standardization organizations, and it is referred to as ITU-TRecommendation H.264 and ISO / IEC International Standard 14496-10, also known as MPEG-4 Part 10 Advanced Video Coding (AVC). There have been multiple versions of the H.264 / AVC standard, integrating new extensions or features to the specification. These extensions include Scalable Video Coding (SVC) and Multiview Video Coding (MVC).

[0134] The High Efficiency Video Coding standard (which may be abbreviated HEVC or H.265 / HEVC) was developed by the Joint Collaborative Team - Video Coding (JCT-VC) of VCEG and MPEG. The standard is published by both parent standardization organizations, and it is referred to as ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also known as MPEG-H (MPEG high efficiency) Part 2 High Efficiency Video Coding (HEVC). Extensions to H.265 / HEVC include scalable, multiview, three-dimensional, and fidelity range extensions, which may be referred to as SHVC, MV-HEVC, 3D-HEVC, and range extension (REXT), respectively. The references in this description to H.265 / HEVC, SHVC, MV-HEVC, 3D-HEVC and REXT that have been made for the purpose of understanding definitions, structures or concepts of these standard specifications are to be understood to be references to the latest versions of these standards that were available before the date of this application, unless otherwise indicated.

[0135] SHVC, MV-HEVC, and 3D-HEVC use a common basis specification, specified in Annex F of the version 2 of the HEVC standard. This common basis comprises for example high-level syntax and semantics e.g. specifying some of the characteristics of the layers of the bitstream, such as inter-layer dependencies, as well as decoding processes, such as reference picture list construction including inter-layer reference pictures and picture order count derivation for multi-layer bitstream. Annex F may also be used in potential subsequent multi-layer extensions of HEVC. It is to be understood that even though a video encoder, a video decoder, encoding methods, decoding methods, bitstream structures, and / or embodiments may be described in the following with reference to specific extensions, such as SHVC and / or MV-HEVC, they are generally applicable to any multi-layer extensions of HEVC, and even more generally to any multi-layer video coding scheme. The Versatile Video Coding standard (which may be abbreviated WC, H.266, or H.266 / WC) was developed by the Joint Video Experts Team (JVET), which is a collaboration between the ISO / IEC MPEG and ITU-T VCEG. Extensions to VVC are presently under development. A specification of the AV1 bitstream format and decoding process were developed by the Alliance of Open Media (AOM). The AV1 specification was published in 2018. AOM is reportedly working on the AV2 specification.

[0136] Some definitions, bitstream and coding structures, and concepts of H.264 / AVC and HEVC are described in this section as an example of a video encoder, decoder, encoding method, decoding method, and a bitstream structure, wherein the embodiments may be implemented. Some of the definitions, bitstream and coding structures, and concepts of H.264 / AVC are the same as in HEVC - hence, they are described below jointly. The aspects described herein are not limited to H.264 / AVC or HEVC, but rather the descriptionis given for one possible basis on top of which the examples described herein may be partly or fully realized. Many aspects described below in the context of H.264 / AVC or HEVC may apply to VVC, and the aspects described herein may hence be applied to WC.

[0137] Similarly to many earlier video coding standards, the bitstream syntax and semantics as well as the decoding process for error-free bitstreams are specified in H.264 / AVC and HEVC. The encoding process is not specified, but encoders must shall generate conforming bitstreams. Bitstream and decoder conformance can be verified with the Hypothetical Reference Decoder (HRD). The standards contain coding tools that help in coping with transmission errors and losses, but the use of the tools in encoding is optional and no decoding process has been specified for erroneous bitstreams.

[0138] The elementary unit for the input to an H.264 / AVC or HEVC encoder and the output of an H.264 / AVC or HEVC decoder, respectively, is a picture. A picture given as an input to an encoder may also be referred to as a source picture, and a picture decoded by a decoded may be referred to as a decoded picture. The source and decoded pictures are each comprised of one or more sample arrays, such as one of the following sets of sample arrays: Luma (Y) only (monochrome); Luma and two chroma (YCbCr or YCgCo); Green, Blue and Red (GBR, also known as RGB), were Cg is the green chrominance component and Co is the orange chrominance component; Arrays representing other unspecified monochrome or tristimulus color samplings (for example, YZX, also known as XYZ). XYZ and YZX are color models where X represents the red and green parts of a color, Y represents lightness or luminance or luma component of a color, and Z represents the blue and yellow parts of a color.

[0139] In the following, these arrays may be referred to as luma (or L or Y) and chroma, where the two chroma arrays may be referred to as Cb for the blue chrominance component and Cr for the red chrominance component; regardless of the actual color representation method in use. The actual color representation method in use can be indicated e.g. in a coded bitstream e.g. using the Video Usability Information (VUI) syntax of H.264 / AVC and / or HEVC. A component may be defined as an array or single sample from one of the three sample arrays (luma and two chroma) or the array or a single sample of the array that compose a picture in monochrome format.

[0140] In H.264 / AVC and HEVC, a picture may either be a frame or a field. A frame comprises a matrix of luma samples and possibly the corresponding chroma samples. A field is a set of alternate sample rows of a frame and may be used as encoder input, when the source signal is interlaced. Chroma sample arrays may be absent (and hence monochrome sampling may be in use) or chroma sample arrays may be subsampled when compared to luma sample arrays. Chroma formats may be summarized as follows:

[0141] In monochrome sampling there is only one sample array, which may be nominally considered the luma array. In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of theluma array. In 4:2:2 sampling, each of the two chroma arrays has the same height and half the width of the luma array. In 4:4:4 sampling when no separate color planes are in use, each of the two chroma arrays has the same height and width as the luma array.

[0142] In H.264 / AVC and HEVC, it is possible to code sample arrays as separate color planes into the bitstream and respectively decode separately coded color planes from the bitstream. When separate color planes are in use, each one of them is separately processed (by the encoder and / or the decoder) as a picture with monochrome sampling. A partitioning may be defined as a division of a set into subsets such that each element of the set is in exactly one of the subsets.

[0143] When describing the operation of HEVC encoding and / or decoding, the following terms may be used. A coding block may be defined as an NxN block of samples for some value of N such that the division of a coding tree block into coding blocks is a partitioning. A coding tree block (CTB) may be defined as an NxN block of samples for some value of N such that the division of a component into coding tree blocks is a partitioning. A coding tree unit (CTU) may be defined as a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples of a picture that has three sample arrays, or a coding tree block of samples of a monochrome picture or a picture that is coded using three separate color planes and syntax structures used to code the samples. A coding unit (CU) may be defined as a coding block of luma samples, two corresponding coding blocks of chroma samples of a picture that has three sample arrays, or a coding block of samples of a monochrome picture or a picture that is coded using three separate color planes and syntax structures used to code the samples. A CU with the maximum allowed size may be named as LCU (largest coding unit) or coding tree unit (CTU) and the video picture is divided into non-overlapping LCUs.

[0144] A CU consists of a prediction unit (PU) or one or more prediction units defining the prediction process for the samples within the CU and a transform unit (TU) or one or more transform units defining the prediction error coding process for the samples in the said CU. Typically, a CU consists of a square block of samples with a size selectable from a predefined set of possible CU sizes. Each PU and TU can be further split into smaller PUs and TUs in order to increase granularity of the prediction and prediction error coding processes, respectively. Each PU has prediction information associated with it defining what kind of a prediction is to be applied for the pixels within that PU (e.g. motion vector information for inter predicted PUs and intra prediction directionality information for intra predicted PUs).

[0145] Each TU can be associated with information describing the prediction error decoding process for the samples within the said TU (including e.g. DCT coefficient information). It may be signaled at CU level whether prediction error coding is applied or not for each CU. In the case there is no prediction error residual associated with the CU, it can be considered there are no TUs for the said CU. The division of the imageinto CDs, and division of CUs into PUs and TUs may be signaled in the bitstream allowing the decoder to reproduce the intended structure of these units.

[0146] In HEVC, a picture can be partitioned in tiles, which are rectangular and contain an integer number of LCDs. In HEVC, the partitioning to tiles forms a regular grid, where heights and widths of tiles differ from each other by one LCU at the maximum. In HEVC, a slice is defined to be an integer number of coding tree units contained in one independent slice segment and all subsequent dependent slice segments (if any) that precede the next independent slice segment (if any) within the same access unit. In HEVC, a slice segment is defined to be an integer number of coding tree units ordered consecutively in the tile scan and contained in a single network abstraction layer, NAL, unit (NALU). The division of each picture into slice segments is a partitioning. In HEVC, an independent slice segment is defined to be a slice segment for which the values of the syntax elements of the slice segment header are not inferred from the values for a preceding slice segment, and a dependent slice segment is defined to be a slice segment for which the values of some syntax elements of the slice segment header are inferred from the values for the preceding independent slice segment in decoding order. In HEVC, a slice header is defined to be the slice segment header of the independent slice segment that is a current slice segment or is the independent slice segment that precedes a current dependent slice segment, and a slice segment header is defined to be a part of a coded slice segment containing the data elements pertaining to the first or all coding tree units represented in the slice segment. The CUs are scanned in the raster scan order of LCUs within tiles or within a picture, if tiles are not in use. Within an LCU, the CUs have a specific scan order.

[0147] An intra-coded slice (also called I slice) is such that only contains intra-coded blocks. The syntax of an I slice may exclude syntax elements that are related to inter prediction. An inter-coded slice is such where blocks can be intra- or inter-coded. Inter-coded slices may further be categorized into P and B slices, where P slices are such that blocks may be intra-coded or inter-coded but only using uni-prediction, and blocks in B slices may be intra-coded or inter-coded with uni- or bi-prediction.

[0148] The decoder reconstructs the output video by applying prediction means similar to the encoder to form a predicted representation of the pixel blocks (using the motion or spatial information created by the encoder and stored in the compressed representation) and prediction error decoding (inverse operation of the prediction error coding recovering the quantized prediction error signal in spatial pixel domain). After applying prediction and prediction error decoding means the decoder sums up the prediction and prediction error signals (pixel values) to form the output video frame. The decoder (and encoder) can also apply additional filtering means to improve the quality of the output video before passing it for display and / or storing it as prediction reference for the forthcoming frames in the video sequence.

[0149] The filtering may for example include one more of the following: deblocking, sample adaptive offset (SAO), and / or adaptive loop filtering (ALF). H.264 / AVC includes a deblocking, whereas HEVC includes both deblocking and SAO.

[0150] In video codecs the motion information may be indicated with motion vectors associated with each motion compensated image block, such as a prediction unit. Each of these motion vectors represents the displacement of the image block in the picture to be coded (in the encoder side) or decoded (in the decoder side) and the prediction source block in one of the previously coded or decoded pictures. In order to represent motion vectors efficiently those are typically coded differentially with respect to block specific predicted motion vectors. In video codecs, the predicted motion vectors may be created in a predefined way, for example calculating the median of the encoded or decoded motion vectors of the adjacent blocks. Another way to create motion vector predictions is to generate a list of candidate predictions from adjacent blocks and / or co-located blocks in temporal reference pictures and signalling the chosen candidate as the motion vector predictor. In addition to predicting the motion vector values, it can be predicted which reference picture(s) are used for motion-compensated prediction and this prediction information may be represented for example by a reference index of previously coded / decoded picture. The reference index is typically predicted from adjacent blocks and / or co-located blocks in temporal reference picture. Moreover, high efficiency video codecs may employ an additional motion information coding / decoding mechanism, often called merging / merge mode, where all the motion field information, which includes motion vector and corresponding reference picture index for each available reference picture list, is predicted and used without any modification / correction. Similarly, predicting the motion field information is carried out using the motion field information of adjacent blocks and / or co-located blocks in temporal reference pictures and the used motion field information is signaled among a list of motion field candidate list filled with motion field information of available adjacent / co-located blocks.

[0151] In video codecs the prediction residual after motion compensation may be first transformed with a transform kernel (like DCT) and then coded. The reason for this is that often there still exists some correlation among the residual and transform can in many cases help reduce this correlation and provide more efficient coding.

[0152] Video encoders may utilize Lagrangian cost functions to find optimal coding modes, e.g., the desired coding mode for a block and associated motion vectors. This kind of cost function uses a weighting factor A to tie together the (exact or estimated) image distortion due to lossy coding methods and the (exact or estimated) amount of information that is required to represent the pixel values in an image area:C = D +AR

[0153] In the above equation, C is the Lagrangian cost to be minimized, D is the image distortion (e.g. Mean Squared Error) with the mode and motion vectors considered, and R the number of bits needed to represent the required data to reconstruct the image block in the decoder (including the amount of data to represent the candidate motion vectors).

[0154] Video coding standards and specifications may allow encoders to divide a coded picture to coded slices or alike. In-picture prediction is typically disabled across slice boundaries. Thus, slices can be regarded as a way to split a coded picture to independently decodable pieces. In H.264 / AVC and HEVC, in-picture prediction may be disabled across slice boundaries. Thus, slices can be regarded as a way to split a coded picture into independently decodable pieces, and slices are therefore often regarded as elementary units for transmission. In many cases, encoders may indicate in the bitstream which types of in-picture prediction are turned off across slice boundaries, and the decoder operation takes this information into account for example when concluding which prediction sources are available. For example, samples from a neighboring CU may be regarded as unavailable for intra prediction, if the neighboring CU resides in a different slice.

[0155] An elementary unit for the output of an H.264 / AVC or HEVC encoder and the input of an H.264 / AVC or HEVC decoder, respectively, is a Network Abstraction Layer (NAL) unit. For transport over packet-oriented networks or storage into structured files, NAL units may be encapsulated into packets or similar structures. A bytestream format has been specified in H.264 / AVC and HEVC for transmission or storage environments that do not provide framing structures. The bytestream format separates NAL units from each other by attaching a start code in front of each NAL unit. To avoid false detection of NAL unit boundaries, encoders run a byte-oriented start code emulation prevention algorithm, which adds an emulation prevention byte to the NAL unit payload if a start code would have occurred otherwise. In order to enable straightforward gateway operation between packet- and stream-oriented systems, start code emulation prevention may always be performed regardless of whether the bytestream format is in use or not. A NAL unit (NALU) may be defined as a syntax structure containing an indication of the type of data to follow and bytes containing that data in the form of an RBSP interspersed as necessary with emulation prevention bytes. A raw byte sequence payload (RBSP) may be defined as a syntax structure containing an integer number of bytes that is encapsulated in a NAL unit. An RBSP is either empty or has the form of a string of data bits containing syntax elements followed by an RBSP stop bit and followed by zero or more subsequent bits equal to 0.

[0156] NAL units consist of a header and payload. In H.264 / AVC and HEVC, the NAL unit header indicates the type of the NAL unit.

[0157] In HEVC, a two-byte NAL unit header is used for all specified NAL unit types. The NAL unit header contains one reserved bit, a six-bit NAL unit type indication, a three-bit nuh_temporal_id_plus1 (nuhcorresponds to NAL unit header) indication for temporal level (may be required to be greater than or equal to 1) and a six-bit nuhjayerjd syntax element. The temporal Jd_plus1 syntax element may be regarded as a temporal identifier for the NAL unit, and a zero-based Temporalld variable may be derived as follows: Temporalld = temporaljd_plus1 - 1. The abbreviation TID may be used to interchangeably with the Temporalld variable. Temporalld equal to 0 corresponds to the lowest temporal level. The value of temporal Jd_plus1 is required to be non-zero in order to avoid start code emulation involving the two NAL unit header bytes. The bitstream created by excluding all VCL NAL units having a Temporalld greater than or equal to a selected value and including all other VCL NAL units remains conforming. Consequently, a picture having Temporalld equal to tid_value does not use any picture having a Temporalld greater than tid_value as inter prediction reference. A sub-layer or a temporal sub-layer may be defined to be a temporal scalable layer (or a temporal layer, TL) of a temporal scalable bitstream, consisting of VCL NAL units with a particular value of the Temporalld variable and the associated non-VCL NAL units, nuhjayerjd can be understood as a scalability layer identifier.

[0158] NAL units can be categorized into Video Coding Layer (VCL) NAL units and non-VCL NAL units. VCL NAL units may be coded slice NAL units. In HEVC, VCL NAL units contain syntax elements representing one or more CU.

[0159] In HEVC, abbreviations for picture types may be defined as follows: trailing (TRAIL) picture, Temporal Sub-layer Access (TSA), Step-wise Temporal Sub-layer Access (STSA), Random Access Decodable Leading (RADL) picture, Random Access Skipped Leading (RASL) picture, Broken Link Access (BLA) picture, Instantaneous Decoding Refresh (I DR) picture, Clean Random Access (CRA) picture.

[0160] A Random Access Point (RAP) picture, which may also be referred to as an intra random access point (IRAP) picture in an independent layer contains only intra-coded slices. An IRAP picture belonging to a predicted layer may contain P, B, and I slices, cannot use inter prediction from other pictures in the same predicted layer, and may use inter-layer prediction from its direct reference layers. In the present version of HEVC, an IRAP picture may be a BLA picture, a CRA picture or an IDR picture. The first picture in a bitstream containing a base layer is an IRAP picture at the base layer. Provided the necessary parameter sets are available when they need to be activated, an IRAP picture at an independent layer and all subsequent non-RASL pictures at the independent layer in decoding order can be correctly decoded without performing the decoding process of any pictures that precede the IRAP picture in decoding order. The IRAP picture belonging to a predicted layer and all subsequent non-RASL pictures in decoding order within the same predicted layer can be correctly decoded without performing the decoding process of any pictures of the same predicted layer that precede the IRAP picture in decoding order, when the necessary parameter sets are available when they need to be activated and when the decoding of each direct reference layer of thepredicted layer has been initialized . There may be pictures in a bitstream that contain only intra-coded slices that are not I RAP pictures.

[0161] A non-VCL NAL unit may be for example one of the following types: a sequence parameter set, a picture parameter set, a supplemental enhancement information (SEI) NAL unit, an access unit delimiter, an end of sequence NAL unit, an end of bitstream NAL unit, or a filler data NAL unit. Parameter sets may be needed for the reconstruction of decoded pictures, whereas many of the other non-VCL NAL units are not necessary for the reconstruction of decoded sample values.

[0162] Parameters that remain unchanged through a coded video sequence may be included in a sequence parameter set. In addition to the parameters that may be needed by the decoding process, the sequence parameter set may optionally contain video usability information (VUI), which includes parameters that may be important for buffering, picture output timing, rendering, and resource reservation. In HEVC a sequence parameter set RBSP includes parameters that can be referred to by one or more picture parameter set RBSPs or one or more SEI NAL units containing a buffering period SEI message. A picture parameter set contains such parameters that are likely to be unchanged in several coded pictures. A picture parameter set RBSP may include parameters that can be referred to by the coded slice NAL units of one or more coded pictures. In HEVC, a video parameter set (VPS) may be defined as a syntax structure containing syntax elements that apply to zero or more entire coded video sequences as determined by the content of a syntax element found in the SPS referred to by a syntax element found in the PPS referred to by a syntax element found in each slice segment header. A video parameter set RBSP may include parameters that can be referred to by one or more sequence parameter set RBSPs.

[0163] The relationship and hierarchy between video parameter set (VPS), sequence parameter set (SPS), and picture parameter set (PPS) may be described as follows. VPS resides one level above SPS in the parameter set hierarchy and in the context of scalability and / or three-dimensional, 3D, video. VPS may include parameters that are common for all slices across all (scalability or view) layers in the entire coded video sequence. SPS includes the parameters that are common for all slices in a particular (scalability or view) layer in the entire coded video sequence, and may be shared by multiple (scalability or view) layers. PPS includes the parameters that are common for all slices in a particular layer representation (the representation of one scalability or view layer in one access unit) and are likely to be shared by all slices in multiple layer representations.

[0164] VPS may provide information about the dependency relationships of the layers in a bitstream, as well as many other information that are applicable to all slices across all (scalability or view) layers in the entire coded video sequence. VPS may be considered to comprise two parts, the base VPS and a VPS extension, where the VPS extension may be optionally present.

[0165] Out-of-band transmission, signaling or storage can additionally or alternatively be used for other purposes than tolerance against transmission errors, such as ease of access or session negotiation. For example, a sample entry of a track in a file conforming to the ISO Base Media File Format may comprise parameter sets, while the coded data in the bitstream is stored elsewhere in the file or in another file. The phrase along the bitstream (e.g. indicating along the bitstream) may be used in claims and described embodiments to refer to out-of-band transmission, signaling, or storage in a manner that the out-of-band data is associated with the bitstream. The phrase decoding along the bitstream or alike may refer to decoding the referred out-of-band data (which may be obtained from out-of-band transmission, signaling, or storage) that is associated with the bitstream.

[0166] A SEI NAL unit may contain one or more SEI messages, which are not required for the decoding of output pictures but may assist in related processes, such as picture output timing, rendering, error detection, error concealment, and resource reservation. Several SEI messages are specified in H.264 / AVC and HEVC, and the user data SEI messages enable organizations and companies to specify SEI messages for their own use. H.264 / AVC and HEVC contain the syntax and semantics for the specified SEI messages but no process for handling the messages in the recipient is defined. Consequently, encoders are required to follow the H.264 / AVC standard or the HEVC standard when they create SEI messages, and decoders conforming to the H.264 / AVC standard or the HEVC standard, respectively, are not required to process SEI messages for output order conformance. One of the reasons to include the syntax and semantics of SEI messages in H.264 / AVC and HEVC is to allow different system specifications to interpret the supplemental information identically and hence interoperate. It is intended that system specifications can require the use of particular SEI messages both in the encoding end and in the decoding end, and additionally the process for handling particular SEI messages in the recipient can be specified.

[0167] In HEVC, there are two types of SEI NAL units, namely the suffix SEI NAL unit and the prefix SEI NAL unit, having a different nal_unit_type value from each other. The SEI message(s) contained in a suffix SEI NAL unit are associated with the VCL NAL unit preceding, in decoding order, the suffix SEI NAL unit. The SEI message(s) contained in a prefix SEI NAL unit are associated with the VCL NAL unit following, in decoding order, the prefix SEI NAL unit. A coded picture is a coded representation of a picture.

[0168] In HEVC, a coded picture may be defined as a coded representation of a picture containing all coding tree units of the picture. In HEVC, an access unit (AU) may be defined as a set of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and contain at most one picture with any specific value of nuhjayerjd. In addition to containing the VCL NAL units of the coded picture, an access unit may also contain non-VCL NAL units. Said specifiedclassification rule may for example associate pictures with the same output time or picture output count value into the same access unit.

[0169] A bitstream may be defined as a sequence of bits, in the form of a NAL unit stream or a byte stream, that forms the representation of coded pictures and associated data forming one or more coded video sequences. A first bitstream may be followed by a second bitstream in the same logical channel, such as in the same file or in the same connection of a communication protocol. An elementary stream (in the context of video coding) may be defined as a sequence of one or more bitstreams. The end of the first bitstream may be indicated by a specific NAL unit, which may be referred to as the end of bitstream (EOB) NAL unit and which is the last NAL unit of the bitstream. In HEVC and its current draft extensions, the EOB NAL unit is required to have nuhjayerjd equal to 0.

[0170] A coded video sequence may be defined as such a sequence of coded pictures in decoding order that is independently decodable and is followed by another coded video sequence or the end of the bitstream or an end of sequence NAL unit. In HEVC, a coded video sequence may additionally or alternatively (to the specification above) be specified to end, when a specific NAL unit, which may be referred to as an end of sequence (EOS) NAL unit, appears in the bitstream and has nuhjayerjd equal to 0.

[0171] A group of pictures (GOP) and its characteristics may be defined as follows. A GOP can be decoded regardless of whether any previous pictures were decoded. An open GOP is such a group of pictures in which pictures preceding the initial intra picture in output order might not be correctly decodable when the decoding starts from the initial intra picture of the open GOP. In other words, pictures of an open GOP may refer (in inter prediction) to pictures belonging to a previous GOP. An HEVC decoder can recognize an intra picture starting an open GOP, because a specific NAL unit type, ORA NAL unit type, may be used for its coded slices. A closed GOP is such a group of pictures in which all pictures can be correctly decoded when the decoding starts from the initial intra picture of the closed GOP. In other words, no picture in a closed GOP refers to any pictures in previous GOPs. In H.264 / AVC and HEVC, a closed GOP may start from an IDR picture. In HEVC a closed GOP may also start from a broken link access with RADL (BLA_W_RADL) or a broken link access with no leading (BLA_N_LP) picture. An open GOP coding structure is potentially more efficient in the compression compared to a closed GOP coding structure, due to a larger flexibility in selection of reference pictures.

[0172] A Structure of Pictures (SOP) may be defined as one or more coded pictures consecutive in decoding order, in which the first coded picture in decoding order is a reference picture at the lowest temporal sub-layer and no coded picture except potentially the first coded picture in decoding order is a RAP picture. All pictures in the previous SOP precede in decoding order all pictures in the current SOP and all pictures in the next SOP succeed in decoding order all pictures in the current SOP. A SOP may represent a hierarchicaland repetitive inter prediction structure. The term group of pictures (GOP) may sometimes be used interchangeably with the term SOP and having the same semantics as the semantics of SOP.

[0173] A Decoded Picture Buffer (DPB) may be used in the encoder and / or in the decoder. There are two reasons to buffer decoded pictures, for references in inter prediction and for reordering decoded pictures into output order. As H.264 / AVC and HEVC provide a great deal of flexibility for both reference picture marking and output reordering, separate buffers for reference picture buffering and output picture buffering may waste memory resources. Hence, the DPB may include a unified decoded picture buffering process for reference pictures and output reordering. A decoded picture may be removed from the DPB when it is no longer used as a reference and is not needed for output.

[0174] In many coding modes of H.264 / AVC and HEVC, the reference picture for inter prediction is indicated with an index to a reference picture list. The index may be coded with variable length coding, which usually causes a smaller index to have a shorter value for the corresponding syntax element. In H.264 / AVC and HEVC, two reference picture lists (reference picture list 0 and reference picture list 1) are generated for each bi-predictive (B) slice, and one reference picture list (reference picture list 0) is formed for each intercoded (P) slice.

[0175] A reference picture list, such as the reference picture list 0 and the reference picture list 1 , may be constructed in two steps: First, an initial reference picture list is generated. The initial reference picture list may be generated for example on the basis of frame_num, picture order count (POC), temporal Jd, or information on the prediction hierarchy such as a GOP structure, or any combination thereof. Second, the initial reference picture list may be reordered by reference picture list reordering (RPLR) syntax, also known as reference picture list modification syntax structure, which may be contained in slice headers. The initial reference picture lists may be modified through the reference picture list modification syntax structure, where pictures in the initial reference picture lists may be identified through an entry index to the list.

[0176] Many coding standards, including H.264 / AVC and HEVC, may have decoding process to derive a reference picture index to a reference picture list, which may be used to indicate which one of the multiple reference pictures is used for inter prediction for a particular block. A reference picture index may be coded by an encoder into the bitstream is some inter coding modes or it may be derived (by an encoder and a decoder) for example using neighboring blocks in some other inter coding modes.

[0177] Several candidate motion vectors may be derived for a single prediction unit. For example, motion vector prediction HEVC includes two motion vector prediction schemes, namely the advanced motion vector prediction (AMVP) and the merge mode. In the AMVP or the merge mode, a list of motion vector candidates is derived for a PU. There are two kinds of candidates: spatial candidates and temporal candidates, where temporal candidates may also be referred to as TMVP candidates.

[0178] A candidate list derivation may be performed for example as follows, while it should be understood that other possibilities may exist for candidate list derivation. If the occupancy of the candidate list is not at maximum, the spatial candidates are included in the candidate list first if they are available and not already exist in the candidate list. After that, if occupancy of the candidate list is not yet at maximum, a temporal candidate is included in the candidate list. If the number of candidates still does not reach the maximum allowed number, the combined bi-predictive candidates (for B slices) and a zero motion vector are added in. After the candidate list has been constructed, the encoder decides the final motion information from candidates for example based on a rate-distortion optimization (RDO) decision and encodes the index of the selected candidate into the bitstream. Likewise, the decoder decodes the index of the selected candidate from the bitstream, constructs the candidate list, and uses the decoded index to select a motion vector predictor from the candidate list.

[0179] A motion vector anchor position may be defined as a position (e.g., horizontal and vertical coordinates) within a picture area relative to which the motion vector is applied. A horizontal offset and a vertical offset for the anchor position may be given in the slice header, slice parameter set, tile header, tile parameter set, or the like.

[0180] An example encoding method taking advantage of a motion vector anchor position comprises: encoding an input picture into a coded constituent picture; reconstructing, as a part of said encoding, a decoded constituent picture corresponding to the coded constituent picture; encoding a spatial region into a coded tile, the encoding comprising: determining a horizontal offset and a vertical offset indicative of a regionwise anchor position of the spatial region within the decoded constituent picture; encoding the horizontal offset and the vertical offset; determining that a prediction unit at position of a first horizontal coordinate and a first vertical coordinate of the coded tile is predicted relative to the region-wise anchor position, wherein the first horizontal coordinate and the first vertical coordinate are horizontal and vertical coordinates, respectively, within the spatial region; indicating that the prediction unit is predicted relative to a predictionunit anchor position that is relative to the region-wise anchor position; deriving a prediction-unit anchor position equal to sum of the first horizontal coordinate and the horizontal offset, and the first vertical coordinate and the vertical offset, respectively; determining a motion vector for the prediction unit; and applying the motion vector relative to the prediction-unit anchor position to obtain a prediction block.

[0181] An example decoding method wherein a motion vector anchor position is used comprises: decoding a coded tile into a decoded tile, the decoding comprising: decoding a horizontal offset and a vertical offset; decoding an indication that a prediction unit at position of a first horizontal coordinate and a first vertical coordinate of the coded tile is predicted relative to a prediction-unit anchor position that is relative to the horizontal and vertical offset; deriving a prediction-unit anchor position equal to sum of the first horizontalcoordinate and the horizontal offset, and the first vertical coordinate and the vertical offset, respectively; determining a motion vector for the prediction unit; and applying the motion vector relative to the predictionunit anchor position to obtain a prediction block.

[0182] Scalable video coding may refer to coding structure where one bitstream can contain multiple representations of the content, for example, at different bitrates, resolutions or frame rates. In these cases the receiver can extract the desired representation depending on its characteristics (e.g. resolution that matches best the display device). Alternatively, a server or a network element can extract the portions of the bitstream to be transmitted to the receiver depending on e.g. the network characteristics or processing capabilities of the receiver. A meaningful decoded representation can be produced by decoding only certain parts of a scalable bit stream. A scalable bitstream may consist of a “base layer” providing the lowest quality video available and one or more enhancement layers that enhance the video quality when received and decoded together with the lower layers. In order to improve coding efficiency for the enhancement layers, the coded representation of that layer typically depends on the lower layers. E.g., the motion and mode information of the enhancement layer can be predicted from lower layers. Similarly the pixel data of the lower layers can be used to create prediction for the enhancement layer.

[0183] In some scalable video coding schemes, a video signal can be encoded into a base layer and one or more enhancement layers. An enhancement layer may enhance, for example, the temporal resolution (i.e., the frame rate), the spatial resolution, or simply the quality of the video content represented by another layer or part thereof. Each layer together with all its dependent layers is one representation of the video signal, for example, at a certain spatial resolution, temporal resolution and quality level. In this description, a scalable layer together with all of its dependent layers is referred to as a “scalable layer representation”. The portion of a scalable bitstream corresponding to a scalable layer representation can be extracted and decoded to produce a representation of the original signal at certain fidelity.

[0184] Scalability modes or scalability dimensions may include but are not limited to the following:

[0185] - Quality scalability: Base layer pictures are coded at a lower quality than enhancement layer pictures, which may be achieved for example using a greater quantization parameter value (i.e., a greater quantization step size for transform coefficient quantization) in the base layer than in the enhancement layer. Quality scalability may be further categorized into fine-grain or fine-granularity scalability (FGS), mediumgrain or medium-granularity scalability (MGS), and / or coarse-grain or coarse-granularity scalability (CGS), as described below.

[0186] - Spatial scalability: Base layer pictures are coded at a lower resolution (i.e. have fewer samples) than enhancement layer pictures. Spatial scalability and quality scalability, particularly its coarse-grain scalability type, may sometimes be considered the same type of scalability.

[0187] - View scalability, which may also be referred to as multiview coding. The base layer represents a first view, whereas an enhancement layer represents a second view. A view may be defined as a sequence of pictures representing one camera or viewpoint. It may be considered that in stereoscopic or two-view video, one video sequence or view is presented for the left eye while a parallel view is presented for the right eye.

[0188] - Depth scalability, which may also be referred to as depth-enhanced coding. A layer or some layers of a bitstream may represent texture view(s), while other layer or layers may represent depth view(s).

[0189] It should be understood that many of the scalability types may be combined and applied together.

[0190] The term “layer” may be used in context of any type of scalability, including view scalability and depth enhancements. An enhancement layer may refer to any type of an enhancement, such as sig nal-to-noise ratio (SNR), spatial, multiview, and / or depth enhancement. A base layer may refer to any type of a base video sequence, such as a base view, a base layer for SNR / spatial scalability, or a texture base view for depth-enhanced video coding.

[0191] A sender, a gateway, a client, or another entity may select the transmitted layers and / or sub-layers of a scalable video bitstream. Terms layer extraction, extraction of layers, or layer down-switching may refer to transmitting fewer layers than what is available in the bitstream received by the sender, the gateway, the client, or another entity. Layer up-switching may refer to transmitting additional layer(s) compared to those transmitted prior to the layer up-switching by the sender, the gateway, the client, or another entity, i.e. restarting the transmission of one or more layers whose transmission was ceased earlier in layer downswitching. Similarly to layer down-switching and / or up-switching, the sender, the gateway, the client, or another entity may perform down- and / or up-switching of temporal sub-layers. The sender, the gateway, the client, or another entity may also perform both layer and sub-layer down-switching and / or up-switching. Layer and sub-layer down-switching and / or up-switching may be carried out in the same access unit or alike (i.e. virtually simultaneously) or may be carried out in different access units or alike (i.e. virtually at distinct times).

[0192] A scalable video encoder for quality scalability (also known as Signal-to-Noise or SNR) and / or spatial scalability may be implemented as follows. For a base layer, a non-scalable video encoder and decoder may be used. The reconstructed / decoded pictures of the base layer are included in the reference picture buffer and / or reference picture lists for an enhancement layer. In case of spatial scalability, the reconstructed / decoded base-layer picture may be upsampled prior to its insertion into the reference picture lists for an enhancement-layer picture. The base layer decoded pictures may be inserted into a reference picture list(s) for coding / decoding of an enhancement layer picture similarly to the decoded reference pictures of the enhancement layer. Consequently, the encoder may choose a base-layer reference picture as an inter prediction reference and indicate its use with a reference picture index in the coded bitstream. The decoderdecodes from the bitstream, for example from a reference picture index, that a base-layer picture is used as an inter prediction reference for the enhancement layer. When a decoded base-layer picture is used as the prediction reference for an enhancement layer, it is referred to as an inter-layer reference picture.

[0193] While the previous paragraph described a scalable video codec with two scalability layers with an enhancement layer and a base layer, it needs to be understood that the description can be generalized to any two layers in a scalability hierarchy with more than two layers. In this case, a second enhancement layer may depend on a first enhancement layer in encoding and / or decoding processes, and the first enhancement layer may therefore be regarded as the base layer for the encoding and / or decoding of the second enhancement layer. Furthermore, it needs to be understood that there may be inter-layer reference pictures from more than one layer in a reference picture buffer or reference picture lists of an enhancement layer, and each of these inter-layer reference pictures may be considered to reside in a base layer or a reference layer for the enhancement layer being encoded and / or decoded. Furthermore, it needs to be understood that other types of inter-layer processing than reference-layer picture upsampling may take place instead or additionally. For example, the bit-depth of the samples of the reference-layer picture may be converted to the bit-depth of the enhancement layer and / or the sample values may undergo a mapping from the color space of the reference layer to the color space of the enhancement layer.

[0194] A scalable video coding and / or decoding scheme may use multi-loop coding and / or decoding, which may be characterized as follows. In the encoding / decoding, a base layer picture may be reconstructed / decoded to be used as a motion-compensation reference picture for subsequent pictures, in coding / decoding order, within the same layer or as a reference for inter-layer (or inter-view or intercomponent) prediction. The reconstructed / decoded base layer picture may be stored in the DPB. An enhancement layer picture may likewise be reconstructed / decoded to be used as a motion-compensation reference picture for subsequent pictures, in coding / decoding order, within the same layer or as reference for inter-layer (or inter-view or inter-component) prediction for higher enhancement layers, if any. In addition to reconstructed / decoded sample values, syntax element values of the base / reference layer or variables derived from the syntax element values of the base / reference layer may be used in the inter-layer / inter-component / inter-view prediction.

[0195] Inter-layer prediction may be defined as prediction in a manner that is dependent on data elements (e.g., sample values or motion vectors) of reference pictures from a different layer than the layer of the current picture (being encoded or decoded). Many types of inter-layer prediction exist and may be applied in a scalable video encoder / decoder. The available types of inter-layer prediction may for example depend on the coding profile according to which the bitstream or a particular layer within the bitstream is being encoded or, when decoding, the coding profile that the bitstream or a particular layer within the bitstream is indicatedto conform to. Alternatively or additionally, the available types of inter-layer prediction may depend on the types of scalability or the type of an scalable codec or video coding standard amendment (e.g. SHVC, MV-HEVC, or 3D-HEVC) being used.

[0196] A direct reference layer may be defined as a layer that may be used for inter-layer prediction of another layer for which the layer is the direct reference layer. A direct predicted layer may be defined as a layer for which another layer is a direct reference layer. An indirect reference layer may be defined as a layer that is not a direct reference layer of a second layer but is a direct reference layer of a third layer that is a direct reference layer or indirect reference layer of a direct reference layer of the second layer for which the layer is the indirect reference layer. An indirect predicted layer may be defined as a layer for which another layer is an indirect reference layer. An independent layer may be defined as a layer that does not have direct reference layers. In other words, an independent layer is not predicted using inter-layer prediction. A nonbase layer may be defined as any other layer than the base layer, and the base layer may be defined as the lowest layer in the bitstream. An independent non-base layer may be defined as a layer that is both an independent layer and a non-base layer.

[0197] Similarly to MVC, in MV-HEVC, inter-view reference pictures can be included in the reference picture list(s) of the current picture being coded or decoded. SHVC uses multi-loop decoding operation (unlike the SVC extension of H.264 / AVC). SHVC may be considered to use a reference index based approach, i.e. an inter-layer reference picture can be included in a one or more reference picture lists of the current picture being coded or decoded (as described above).

[0198] For the enhancement layer coding, the concepts and coding tools of HEVC base layer may be used in SHVC, MV-HEVC, and / or alike. However, the additional inter-layer prediction tools, which employ already coded data (including reconstructed picture samples and motion parameters a.k.a motion information) in reference layer for efficiently coding an enhancement layer, may be integrated to SHVC, MV-HEVC, and / or alike codec.

[0199] Video coding specifications may contain a set of constraints for associating data units (e.g. NAL units in H.264 / AVC or HEVC) into access units. These constraints may be used to conclude access unit boundaries from a sequence of NAL units. For example, the following is specified in the HEVC standard:

[0200] - An access unit consists of one coded picture with nuhjayerjd equal to 0, zero or more VCL NAL units with nuhjayerjd greater than 0 and zero or more non-VCL NAL units.

[0201] - Let firstBIPicNalUnit be the first VCL NAL unit of a coded picture with nuhjayerjd equal to 0. The first of any of the following NAL units preceding firstBIPicNalUnit and succeeding the last VCL NAL unit preceding firstBIPicNalUnit, if any, specifies the start of a new access unit: access unit delimiter NAL unit with nuhjayerjd equal to 0 (when present), VPS NAL unit with nuhjayerjd equal to 0 (when present),SPS NAL unit with nuhjayerjd equal to 0 (when present), PPS NAL unit with nuhjayerjd equal to 0 (when present), Prefix SEI NAL unit with nuhjayerjd equal to 0 (when present), NAL units with nal_unit_type in the range of RSV_NVCL41..RSV_NVCL44 with nuhjayerjd equal to 0 (when present), NAL units with nal_unit_type in the range of UNSPEC48..UNSPEC55 with nuhjayerjd equal to 0 (when present). RSV is an abbreviation for reserved. UNSPEC is an abbreviation for unspecified.

[0202] The first NAL unit preceding firstBIPicNalUnit and succeeding the last VCL NAL unit preceding firstBIPicNalUnit, if any, can only be one of the above-listed NAL units. When there is none of the above NAL units preceding firstBIPicNalUnit and succeeding the last VCL NAL preceding firstBIPicNalUnit, if any, firstBIPicNalUnit starts a new access unit. Access unit boundary detection may be based on but may not be limited to one or more of the following:

[0203] - Detecting that a VCL NAL unit of a base-layer picture is the first VCL NAL unit of an access unit, e.g. on the basis that: the VCL NAL unit includes a block address or alike that is the first block of the picture in decoding order; and / or the picture order count, picture number, or similar decoding or output order or timing indicator differs from that of the previous VCL NAL unit(s).

[0204] - Having detected the first VCL NAL unit of an access unit, concluding based on pre-defined rules e.g. based on nal_unit_type which non-VCL NAL units that precede the first VCL NAL unit of an access unit and succeed the last VCL NAL unit of the previous access unit in decoding order belong to the access unit.

[0205] The Versatile Video Coding (WC) includes new coding tools compared to HEVC or H.264 / AVC. These coding tools are related to, for example, intra prediction; inter-picture prediction; transform, quantization and coefficients coding; entropy coding; in-loop filter; screen content coding; 360-degree video coding; high-level syntax and parallel processing. Some of these tools are briefly described in the following:

[0206] - Intra prediction: 67 intra mode with wide angles mode extension, Block size and mode dependent 4 tap interpolation filter, Position dependent intra prediction combination (PDPC), Cross component linear model intra prediction (CCLM), Multi-reference line intra prediction, Intra sub-partitions, Weighted intra prediction with matrix multiplication.

[0207] - Inter-picture prediction: Block motion copy with spatial, temporal, history-based, and pairwise average merging candidates, Affine motion inter prediction, sub-block based temporal motion vector prediction, Adaptive motion vector resolution, 8x8 block-based motion compression for temporal motion prediction, High precision (1 / 16 pel) motion vector storage and motion compensation with 8-tap interpolation filter for luma component and 4-tap interpolation filter for chroma component, T riangular partitions, Combined intra and inter prediction, Merge with motion vector difference (MVD) (MMVD), Symmetrical MVD coding, Bidirectional optical flow, Decoder side motion vector refinement, Bi-prediction with CU-level weight.

[0208] -Transform, quantization and coefficients coding: Multiple primary transform selection with DCT2, DST7 and DCT8, where DST is an abbreviation for discrete sine transform, Secondary transform for low frequency zone, Sub-block transform for inter predicted residual, Dependent quantization with max quantization parameter, QP, increased from 51 to 63, Transform coefficient coding with sign data hiding, Transform skip residual coding.

[0209] - Entropy Coding: Arithmetic coding engine with adaptive double windows probability update

[0210] - In loop filter: In-loop reshaping, Deblocking filter with strong longer filter, Sample adaptive offset, Adaptive Loop Filter

[0211] - Screen content coding: Current picture referencing with reference region restriction

[0212] - 360-degree video coding: Horizontal wrap-around motion compensation

[0213] - High-level syntax and parallel processing: Reference picture management with direct reference picture list signalling, Tile groups with rectangular shape tile groups.

[0214] In WC, each picture may be partitioned into coding tree units (CTUs) similar to HEVC. A CTU may be split into smaller CUs using quaternary tree structure. Each CU may be partitioned using quad-tree and nested multi-type tree including ternary and binary split. There are specific rules to infer partitioning in picture boundaries. The redundant split patterns are disallowed in nested multi-type partitioning.

[0215] In some video coding schemes, such as HEVC and WC, a picture is divided into one or more tile rows and one or more tile columns. The partitioning of a picture to tiles forms a tile grid that may be characterized by a list of tile column widths and a list of tile row heights. A tile may be required to contain an integer number of elementary coding blocks, such as CTUs in HEVC and WC. Consequently, tile column widths and tile row heights may be expressed in the units of elementary coding blocks, such as CTUs in HEVC and WC.

[0216] A tile may be defined as a sequence of elementary coding blocks, such as CTUs in HEVC and WC, that covers one "cell" in the tile grid, i.e., a rectangular region of a picture. Elementary coding blocks, such as CTUs, may be ordered in the bitstream in raster scan order within a tile.

[0217] Some video coding schemes may allow further subdivision of a tile into one or more bricks, each of which consisting of a number of CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a tile is not referred to as a tile.

[0218] In some video coding schemes, such as H.264 / AVC, HEVC and WC, a coded picture may be partitioned into one or more slices. A slice may be decodable independently of other slices of a picture and hence a slice may be considered as a preferred unit for transmission. In some video coding schemes, such as H.264 / AVC, HEVC, and WC, a video coding layer (VCL) NAL unit contains exactly one slice.

[0219] A slice may comprise an integer number of elementary coding blocks, such as CTUs in HEVC or WC.

[0220] In some video coding schemes, such as WC, a slice contains an integer number of tiles of a picture or an integer number of CTU rows of a tile.

[0221] In some video coding schemes, two modes of slices may be supported, namely the raster-scan slice mode and the rectangular slice mode. In the raster-scan slice mode, a slice contains a sequence of tiles in a tile raster scan of a picture. In the rectangular slice mode, a slice contains an integer number of tiles of a picture or an integer number of CTU rows of a tile that collectively form a rectangular region of the picture.

[0222] A non-VCL NAL unit may be for example one of the following types: a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a supplemental enhancement information (SEI) NAL unit, a picture header (PH) NAL unit, an end of sequence NAL unit, an end of bitstream NAL unit, or a filler data NAL unit. Some non-VCL NAL units, such as parameter sets and picture headers, may be needed for the reconstruction of decoded pictures, whereas many of the other non-VCL NAL units might not be necessary for the reconstruction of decoded sample values.

[0223] Some coding formats specify parameter sets that may carry parameter values needed for the decoding or reconstruction of decoded pictures. Some examples of different types of parameter sets are briefly described in this paragraph. A video parameter set (VPS) may include parameters that are common across multiple layers in a coded video sequence or describe relations between layers. Parameters that remain unchanged through a coded video sequence (in a single-layer bitstream) or in a coded layer video sequence may be included in a sequence parameter set (SPS). In addition to the parameters that may be needed by the decoding process, the sequence parameter set may optionally contain video usability information (VUI), which includes parameters that may be important for buffering, picture output timing, rendering, and resource reservation. A picture parameter set (PPS) contains such parameters that are likely to be unchanged in several coded pictures. A picture parameter set may include parameters that can be referred to by the coded image segments of one or more coded pictures. A header parameter set (HPS) may contain such parameters that may change on picture basis. In WC, an Adaptation Parameter Set (APS) may comprise parameters for decoding processes of different types, such as adaptive loop filtering or luma mapping with chroma scaling.

[0224] A parameter set may be activated when it is referenced e.g., through its identifier. For example, a header of an image segment, such as a slice header, may contain an identifier of the PPS that is activated for decoding the coded picture containing the image segment. A PPS may contain an identifier of the SPSthat is activated, when the PPS is activated. An activation of a parameter set of a particular type may cause the deactivation of the previously active parameter set of the same type.

[0225] Instead of or in addition to parameter sets at different hierarchy levels (e.g., sequence and picture), video coding formats may include header syntax structures, such as a sequence header or a picture header. A sequence header may precede any other data of the coded video sequence in the bitstream order. A picture header may precede any coded video data for the picture in the bitstream order.

[0226] Video coding specifications may enable the use of supplemental enhancement information (SEI) messages or alike. Some video coding specifications include SEI NAL units, and some video coding specifications contain both prefix SEI NAL units and suffix SEI NAL units. A prefix SEI NAL unit can start a picture unit or alike; and a suffix SEI NAL unit can end a picture unit or alike. Hereafter, an SEI NAL unit may equivalently refer to a prefix SEI NAL unit or a suffix SEI NAL unit. An SEI NAL unit includes one or more SEI messages, which are not required for the decoding of output pictures but may assist in related processes, such as picture output timing, post-processing of decoded pictures, rendering, error detection, error concealment, and resource reservation.

[0227] Several SEI messages are specified in H.264 / AVC, H.265 / HEVC, H.266 / WC, and H.274 / VSEI standards (VSEI corresponds to versatile supplemental enhancement information), and the user data SEI messages enable organizations and companies to specify SEI messages for specific use. The standards may contain the syntax and semantics for the specified SEI messages but a process for handling the messages in the recipient might not be defined. Consequently, encoders may be required to follow the standard specifying a SEI message when they create SEI message(s), and decoders might not be required to process SEI messages for output order conformance. One of the reasons to include the syntax and semantics of SEI messages in standards is to allow different system specifications to interpret the supplemental information identically and hence interoperate. It is intended that system specifications can require the use of particular SEI messages both in the encoding end and in the decoding end, and additionally the process for handling particular SEI messages in the recipient can be specified.

[0228] A coded picture is a coded representation of a picture.

[0229] A bitstream may be defined as a sequence of bits, which may in some coding formats or standards be in the form of a NAL unit stream or a byte stream, that forms the representation of coded pictures and associated data forming one or more coded video sequences. A first bitstream may be followed by a second bitstream in the same logical channel, such as in the same file or in the same connection of a communication protocol. An elementary stream (in the context of video coding) may be defined as a sequence of one or more bitstreams. In some coding formats or standards, the end of the first bitstream may be indicated by aspecific NAL unit, which may be referred to as the end of bitstream (EOB) NAL unit and which is the last NAL unit of the bitstream.

[0230] A coded video sequence (CVS) may be defined as such a sequence of coded pictures in decoding order that is independently decodable and is followed by another coded video sequence or the end of the bitstream.

[0231] The subpicture feature of VVC allows for partitioning of the VVC bitstream in a flexible manner as multiple rectangles representing subpictures, where each subpicture comprises one or more slices. In other words, a subpicture may be defined as a rectangular region of one or more slices within a picture, wherein the one or more slices are complete. Consequently, a subpicture consists of one or more slices that collectively cover a rectangular region of a picture. The slices of a subpicture may be required to be rectangular slices.

[0232] In VVC, the feature of subpictures enables efficient extraction of subpicture(s) from one or more bitstream and merging the extracted subpictures to form another bitstream without excessive penalty in compression efficiency and without modifications of VCL NAL units (i.e., slices).

[0233] The use of subpictures in a coded video sequence (CVS), however, requires appropriate configuration of the encoder and other parameters such as SPS / PPS and so on. In WC, a layout of partitioning of a picture to subpictures may be indicated in and / or decoded from an SPS. A subpicture layout may be defined as a partitioning of a picture to subpictures. In WC, the SPS syntax indicates the partitioning of a picture to subpictures by providing for each subpicture syntax elements indicative of: the x and y coordinates of the top-left corner of the subpicture, the width of the subpicture, and the height of the subpicture, in CTU units. One or more of the following properties may be indicated (e.g. by an encoder) or decoded (e.g. by a decoder) or inferred (e.g. by an encoder and / or a decoder) for the subpictures collectively or per each subpicture individually: i) whether or not a subpicture is treated like a picture in the decoding process (or equivalently, whether or not subpicture boundaries are treated like picture boundaries in the decoding process); in some cases, this property excludes in-loop filtering operations, which may be separately indicated / decoded / inferred; ii) whether or not in-loop filtering operations are performed across the subpicture boundaries. When a subpicture is treated like a picture in the decoding process, any references to sample locations outside the subpicture boundaries are saturated to be within the subpicture boundaries. This may be regarded being equivalent to padding samples outside subpicture boundaries with the boundary sample values for decoding the subpicture. Consequently, motion vectors may be allowed to cause references outside subpicture boundaries in a subpicture that is extractable.

[0234] An independent subpicture (a.k.a. an extractable subpicture) may be defined as a subpicture i) with subpicture boundaries that are treated as picture boundaries and ii) without loop filtering across thesubpicture boundaries. A dependent subpicture may be defined as a subpicture that is not an independent subpicture. In video coding, an isolated region may be defined as a picture region that is allowed to depend only on the corresponding isolated region in reference pictures and does not depend on any other picture regions in the current picture or in the reference pictures. The corresponding isolated region in reference pictures may be for example the picture region that collocates with the isolated region in a current picture. A coded isolated region may be decoded without the presence of any picture regions of the same coded picture. A VVC subpicture with boundaries treated like picture boundaries may be regarded as an isolated region.

[0235] A motion-constrained tile set (MCTS) is a set of tiles such that the inter prediction process is constrained in encoding such that no sample value outside the MCTS, and no sample value at a fractional sample position that is derived using one or more sample values outside the motion-constrained tile set, is used for inter prediction of any sample within the motion-constrained tile set. Additionally, the encoding of an MCTS is constrained in a manner that no parameter prediction takes inputs from blocks outside the MCTS. For example, the encoding of an MCTS is constrained in a manner that motion vector candidates are not derived from blocks outside the MCTS. In HEVC, this may be enforced by turning off temporal motion vector prediction of HEVC, or by disallowing the encoder to use the temporal motion vector prediction (TMVP) candidate or any motion vector prediction candidate following the TMVP candidate in a motion vector candidate list for prediction units located directly left of the right tile boundary of the MCTS except the last one at the bottom right of the MCTS.

[0236] In general, an MCTS may be defined to be a tile set that is independent of any sample values and coded data, such as motion vectors, that are outside the MCTS. An MCTS sequence may be defined as a sequence of respective MCTSs in one or more coded video sequences or alike. In some cases, an MCTS may be required to form a rectangular area. It should be understood that depending on the context, an MCTS may refer to the tile set within a picture or to the respective tile set in a sequence of pictures. The respective tile set may be, but in general need not be, collocated in the sequence of pictures. A motion-constrained tile set may be regarded as an independently coded tile set, since it may be decoded without the other tile sets. An MCTS is an example of an isolated region.

[0237] The WC (Versatile Video Coding) has a functionality of subpictures which may be regarded to improve the motion constrained tiles. WC support for real-time conversational and low latency use cases are important to fully exploit the functionality and end user benefit with modern networks (e.g., ultra reliable and low latency communication, ULLRC, fifth generation, 5G, networks, over the top, OTT, delivery, etc.). WC encoding and decoding is computationally complex. With increasing computational complexity, the enduser devices consuming the content are heterogeneous for example devices supporting single decodinginstances to devices supporting multiple decoding instances and more sophisticated devices having multiple decoders. Consequently, the system carrying the payload should be able to support a variety of scenarios for scalable deployments. There has been rapid growth in the resolution (e.g., 8K) of the video consumed via CE (Consumer Electronics) devices (e.g., televisions (TVs), mobile devices) which can benefit with the ability to execute multiple parallel decoders. One example use case can be parallel decoding for low latency unicast or multicast delivery of 8K WC encoded content.

[0238] The AV1 codec supports input video signals in the 4:0:0 (monochrome), 4:2:0, 4:2:2, and 4:4:4 formats. The allowed pixel representations are 8, 10, and 12 bit. The AV1 codec operates on pixel blocks. Each pixel block is processed in a predictive-transform coding scheme, where the prediction comes from either intraframe reference pixels, interframe motion compensation, or some combinations of the two. The residuals undergo a 2-D unitary transform to further remove the spatial correlations, and the transform coefficients are quantized. Both the prediction syntax elements and the quantized transform coefficient indexes are then entropy coded using arithmetic coding. There are three optional in-loop postprocessing filter stages to enhance the quality of the reconstructed frame for reference by subsequent coded frames. A normative film grain synthesis unit is also available to improve the perceptual quality of the displayed frames.

[0239] The AV1 bitstream is packetized into open bitstream units (OBUs). An ordered sequence of OBUs is fed into the AV1 decoding process, where each OBU comprises a variable length string of bytes. An OBU contains a header and a payload. The header identifies the OBU type and specifies the payload size. The OBU types may include the following.

[0240] 1) Sequence Header contains information that applies to the entire sequence, e.g., sequence profile and whether to enable certain coding tools.

[0241] 2) Temporal Delimiter indicates the frame presentation time stamp. All displayable frames following a temporal delimiter OBU uses this time stamp, until the next temporal delimiter OBU arrives. A temporal delimiter and its subsequent OBUs of the same time stamp are referred to as a temporal unit. In the context of scalable coding, the compression data associated with all representations of a frame at various spatial and fidelity resolutions are in the same temporal unit.

[0242] 3) Frame Header sets up the coding information for a given frame, including signaling inter or intraframe type, indicating the reference frames and signaling probability model update method.

[0243] 4) Tile Group contains the tile data associated with a frame. Each tile can be independently decoded. The collective reconstructions form the reconstructed frame after potential loop filtering.

[0244] 5) Frame contains the frame header and tile data. The frame OBU is largely equivalent to a frame header OBU and a tile group OBU but allows less overhead cost.

[0245] 6) Metadata carries information, such as high dynamic range, scalability, and timecode.

[0246] 7) Tile List contains tile data similar to a tile group OBU. However, each tile here has an additional header that indicates its reference frame index and position in the current frame. This allows the decoder to process a subset of tiles and display the corresponding part of the frame, without the need to fully decode all the tiles in the frame.

[0247] MPEG-5 Part 2 Low Complexity Enhancement Video Coding (LCEVC) is published as ISO / IEC 23094-2. LCEVC works by encoding a lower resolution (and potentially also lower bit depth) version of a source video using any existing codec (the “base codec”) and then coding the differences between the lower resolution video and the full resolution source, up to mathematically lossless coding if needed, using a different compression method (the “enhancement”).

[0248] This enhancement is achieved by a combination of processing an input video at a lower resolution with an existing single-layer codec and using a simple and small set of highly specialized tools to correct impairments, upscale and add details to the processed video.

[0249] Encoder

[0250] The encoding process to create an LCEVC conformant bitstream and can be depicted in three major steps.

[0251] Base codec

[0252] Firstly, the input sequence is fed into two consecutive non-normative downscalers and is processed according to the chosen scaling modes. Any combination of the three available options (2-dimensional scaling, 1 -dimensional scaling in the horizontal direction only or no scaling) can be used. The output then invokes the base codec which produces a base bitstream according to its own specification. This encoded base can be included as part of the LCEVC bitstream.

[0253] Enhancement sub-layer 1

[0254] The reconstructed base picture may be upscaled to undo the downscaling process and is then subtracted from the first-order downscaled input sequence in order to generate the sub-layer 1 (L-1) residuals. These residuals form the starting point of the encoding process of the first enhancement sub-layer. A number of coding tools, which are described further in the following section, process the input and generate entropy encoded quantized transform coefficients.

[0255] Enhancement sub-layer 2

[0256] As a last step of the encoding process, the enhancement data for sub-layer 2 (L-2) needs to be generated. In order to create the residuals, the coefficients from sub-layer 1 are processed by an in-loop LCEVC decoder to achieve the corresponding reconstructed picture. Depending on the chosen scaling mode, the reconstructed picture is processed by an upscaler. Finally, the residuals are calculated by a subtraction of the input sequence and the upscaled reconstruction. Similar to sub-layer 1, the samples areprocessed by a few coding tools. In addition, a temporal prediction can be applied on the transform coefficients in order to achieve a better removal of redundant information. The entropy encoded quantized transform coefficients of sub-layer 2, as well as a temporal layer specifying the use of the temporal prediction on a block basis, are included in the LCEVC bitstream.

[0257] Decoder

[0258] For the creation of the output sequence, the decoder analyses the LCEVC conformant bitstream. The process can again be divided into three parts:

[0259] Base codec

[0260] In order to generate the Decoded Base Picture (Layer 0) the base decoder is fed with the extracted base bitstream. According to the chosen scaling mode in the configuration, this reconstructed picture might be upscaled and is afterwards called Preliminary Intermediate Picture.

[0261] Enhancement sub-layer 1

[0262] Following the base layer, the enhancement part needs to be decoded. Firstly, the coefficients belonging to enhancement sub-layer 1 are decoded using the inverse tools of the encoding process. Additionally, an L-1 filter might be applied in order to smooth the boundaries of a transform block. The output is then referred to as Enhancement Sub-Layer 1 and is added to the preliminary intermediate picture which results in the Combined Intermediate Picture. Again, depending on the scaling mode, an upscaler might be applied and the resulting Preliminary Output Picture has then the same dimensions as the overall output picture.

[0263] Enhancement sub-layer 2

[0264] As a final step, the second enhancement sub-layer is decoded. According to the temporal layer, a temporal prediction might be applied to the dequantized transform coefficients. This Enhancement SubLayer 2 is then added to the Preliminary Output Picture to form the Combined Output Picture as a final output of the decoding process.

[0265] Bitstream structure

[0266] The LCEVC bitstream contains a base layer, which may be at a lower resolution, and an enhancement layer consisting of up to two sub-layers. This subsection briefly explains the structure of this bitstream and how the information can be extracted. While the base layer can be created using any video encoder and is not specified further in the LCEVC specification, the enhancement layer must is expected to follow the structure as specified. Similar to other MPEG codecs, the syntax elements are encapsulated in network abstraction layer (NAL) units which also help synchronize the enhancement layer information with the base layer decoded information. Depending on the position of the frame within a group of pictures (GOP), additional data specifying the global configuration and controlling the decoder may be present. The data ofone enhancement picture is encoded into several chunks. These data chunks are hierarchically organized. For each processed plane (nPlanes), up to two enhancement sub-layers (nLevels) are extracted. Each of them again unfolds into numerous coefficient groups of entropy encoded transform coefficients. The amount depends on the chosen type of transform (nLayers). Additionally, if the temporal prediction is used, for each processed plane an additional chunk with temporal data for Enhancement sub-layer 2 is present.

[0267] MPEG-5 LCEVC has some similarities with a scalable codec (i.e. spatial scalability thanks to the upsampler) but it is also substantially different for the following reasons:

[0268] Generally, in scalable codec, the base layer is encoded with the same standard of the enhancement layer. As specified in the description LCEVC is codec agnostic. The base layer used in LCEVC can be any codec. This particular feature allows LCEVC to be used with any standard, such as H.264 / AVC or H.266 / WC, and also with any other video codec (e.g., AV1, VP8, VP9 etc.).

[0269] The MPEG-5 LCEVC structure is using simple tools specifically designed for the sparse nature of the residual data which allow to keep the complexity low and limit the overhead associated with the enhancement layers, a common problem of scalable codecs. This makes possible to have a software version of the MPEG-5 LCEVC that can run on existing hardware and on top of existing base codec with no need to develop a specific hardware for it. As a consequence, the base codec can work more efficiently and faster given the ability of LCEVC to work with a base codec running at a quarter of the resolution.

[0270] Differently to most of scalable codecs, MPEG-5 LCEVC provides two levels of enhancement that can be applied at different stages or resolutions. Each level has its own independent quantization module and sublayer of the bitstream that can easily decoupled from the other. This also allows bitrate allocation flexibility to cope with different type of content. It may be noted that MPEG-5 LCEVC offers up to two cascade scaling processes in order to further improve the efficiency of the base layer. Each scaler can be user defined, along the following degrees of freedom: kernel size, type of upscaling (i.e. which sub-layer, L-1 or L-2) and kernel values. MPEG-5 LCEVC offers 4 normative upsamplers and one 4 taps user defined kernel. Scalable codecs are generally offering only one fixed scaling engine and it is not programmable.

[0271] MPEG-5 LCEVC can handle different bit depths up to 14 bits per pixel in the main profile. The standard allows the base layer to work on a different bit depth compared the input signal one. This operation can effectively enhance a base layer working at a lower bit depth to a higher one contributing to maintain the fidelity of the input signal. An example of this application is delivering high dynamic range (HDR) with technologies that cannot deliver more than 8-bit per color component, like AVC High Profile.

[0272] A uniform resource identifier (URI) may be defined as a string of characters used to identify a name of a resource. Such identification enables interaction with representations of the resource over a network, using specific protocols. A URI is defined through a scheme specifying a concrete syntax andassociated protocol for the URI. The uniform resource locator (URL) and the uniform resource name (URN) are forms of URI. A URL may be defined as a URI that identifies a web resource and specifies the means of acting upon or obtaining the representation of the resource, specifying both its primary access mechanism and network location. A URN may be defined as a URI that identifies a resource by name in a particular namespace. A URN may be used for identifying a resource without implying its location or how to access it.

[0273] In many video communication or transmission systems, transport mechanisms, and multimedia container file formats, there are mechanisms to transmit or store a scalability layer separately from another scalability layer of the same bitstream, e.g. to transmit or store the base layer separately from the enhancement layer(s). It may be considered that layers are stored in or transmitted through separate logical channels. For example in ISOBMFF, the base layer can be stored as a track and each enhancement layer can be stored in another track, which may be linked to the base-layer track using so-called track references.

[0274] ISO / IEC 23008-12:2024 Commitee Draft Amendment (CDAM) 2 :2024(E)

[0275] 'mif3'-compliant WC image and image collection brand

[0276] L.4.4.1 Requirements on files

[0277] Files shall include ’mif3’ as the major_brand and the brand 'vvi3' as the minor_version in the FileTypeBox and conform to the specifications in 0.2.1 of this description ('mif3' structural brand).

[0278] The 'vvi3' brand defines the coded image item type to be 'vvcT as defined in L.2.2.1.2 and the alpha_item_codec_config (if present), gainmap_item_codec_config (if present) and main_item_codec_config to have the format defined by CompactVvcDecoderConfigurationRecord specified in L.4.4.3 of this description.

[0279] Files that include ’mif3’ as the major_brand and the brand 'vvi3' as the minor_version in the FileTypeBox shall comply with the specifications in L.2 and shall additionally be constrained as follows: explicit_codec_types_flag in the MinimizedlmageBox shall be equal to 0, floatjlag in the MinimizedlmageBox shall be 0. If high_bit_depth_flag in the MinimizedlmageBox is not 0, bit_depth_minus9 in the MinimizedlmageBox shall be between 0 and 7 inclusive. If gainmap_item_data_size in the MinimizedlmageBox is not 0, gainmap_float_flag in the MinimizedlmageBox shall be 0. If gainmap_item_data_size in the MinimizedlmageBox is not 0 and high_bit_depth_flag in the MinimizedlmageBox is not 0, gainmap_bit_depth_minus9 in the MinimizedlmageBox shall be between 0 and 7 inclusive. If alphajlag in the MinimizedlmageBox is equal to 1, alpha_item_data_size in the MinimizedlmageBox shall not be equal to 0. The data of each codec configuration property shall use the syntax defined in subclause L.4.4.3.2 of this description. Each coded image item shall contain a single IDR picture per layer. The data of each coded image item shall consist of only one NAL unit excluding the length and NAL unit header fields, which are inferred by the reader as specified in subclause L.4.4.2 of this description. The equivalent file as specified in L.4.4.1.2 shall conform to the 'vvic' brand.

[0280] L.4.4.2 Requirements on readers

[0281] The requirements on readers specified in 0.2.1 of this description ('mif3' structural brand) shall be supported.

[0282] As a response to ’mif3’ as the major_brand and the brand 'vvi3' as the minor_version in the FileTypeBox, the readers shall treat the file as if an equivalent file were created containing FileTypeBox with 'vvic' as the equivalent major_brand and the equivalent MetaBox derived with the following additional operations in addition to those specified in subclause 0.4: infer infejype to be equal to 'vvcT, infer codec_config_type to be equal to 'vvcC, expand the data of each codec configuration property from the compact syntax to the full structure as defined in section L.4.4.3.4 of this description, when multijayerjlag is equal to 0, expand the data of the respective image item so that the NAL unit length is inferred from the item length, and so that the NAL unit header fields nal_unit_type, nuh_temporal_id_plus1 and nuhjayerjd are set to 8 (IDR slice with no leading pictures, or IDRJMJP), 1 and 0, respectively.

[0283] L.4.4.3 Compact VVC decoder configuration

[0284] L.4.4.3.1 Definition

[0285] This subclause specifies the compact decoder configuration information for ISO / IEC 23090-3 video content. The compact decoder configuration provides essential parameters that are relevant for still images.

[0286] L.4.4.3.2 Syntaxaligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multijayerjlag;unsigned int(2) lengthSizeMinusOne;if (multijayer lag) {unsigned int(1) ptLpresentJIag;if (ptLpresentJIag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_presentjlag;}if (multi Jayer_flag == 0 1| nal_units_present_flag) {unsigned int(1) additional_nal_unitjlag;if (additional_nal_unit Jlag) {unsigned int(3) num_aps_nal_unit;unsigned int(3) num_sei_nal_unit;}if (multi_layer_flag)unsigned int(1) array_completeness;trailing_bits();if (multi_layer_flag) {unsigned int(8) vps_nal_unit_length;bit(8*vps_nal_unit_length) vps_nal_unit;}unsigned int(8) sps_nal_unit_length;bit(8*sps_nal_unit_length) sps_nal_unit;unsigned int(8) pps_nal_unit_length;bit(8*pps_nal_unit_length) pps_nal_unit;if (additional_nal_unit_flag) {for (i=0; i< num_aps_nalus; i++) {unsigned int(8) aps_nal_unit_length;bit(8*aps_nal_unit_length) aps_nal_unit;}for (i=0; i< num_sei_nalus; i++) {unsigned int(8) sei_nal_unit_length;bit(8*sei_nal_unit_length) sei_nal_unit;}}}}

[0287] L.4.4.3.3 Semantics

[0288] multijayerjlag: When equal to 0, the data of the coded image item shall consist of only one NAL unit that excludes the length and NAL unit header fields and has nal_unit_type, nuh_temporal_id_plus1 and nuhjayerjd equal to 8 (IDR_N_LP), 1 and 0, respectively. When equal to 1, the data of the coded image item may include any number of NAL units, which include the length and NAL unit header fields.

[0289] nal_units_present_flag: indicates that NAL units are present in the decoder configuration. additional_nal_unit_flag: equal to 1 indicates the presence of additional NAL units in the decoder configuration record. additional_nal_unit_flag equal to 0 indicates the absence of additional NAL units in the decoderconfiguration record. num_aps_nal_unit: indicates the number of APS NAL units included in the configuration record for the referenced CVS. num_sei_nal_unit: indicates the number of SEI NAL units included in the configuration record for the referenced CVS. trailing_bits: padding bits to ensure payloads are 8-bit aligned. Shall all be 0. vps_nal_unitjength: indicates the length in bytes of the NAL unit. When equal to 0, the VPS NAL unit is not present. vps_nal_unit: contains the VPS NAL unit as specified in ISO / IEC 23090-3. sps_nal_unit_length: indicates the length in bytes of the NAL unit. When equal to 0, the SPS NAL unit is not present. sps_nal_unit: contains the SPS NAL unit as specified in ISO / IEC 23090-3. pps_nal_unitjength: indicates the length in bytes of the NAL unit. When equal to 0, the PPS NAL unit is not present. pps_nal_unit: contains the PPS NAL unit as specified in ISO / IEC 23090-3. aps_nal_unit_length: indicates the length in bytes of the APS NAL unit. aps_nal_unit: contains the APS NAL unit as specified in ISO / IEC 23090-3. sei_nal_unit_length: indicates the length in bytes of the SEI NAL unit. sei_nal_unit: contains the SEI NAL unit as specified in ISO / IEC 23090-3. The semantics of the other parameters are the same as for VvcDecoderConfigurationRecord as defined in ISO / IEC 14496-15.

[0290] L.4.4.3.4 Equivalence with the VVC decoder configuration

[0291] CompactVvcDecoderConfigurationRecord shall be considered equivalent to VvcDecoderConfigurationRecord as defined in ISO / IEC 14496-15 with the following fields:

[0292] - if ptl_present_flag is present and set to 1 : num_sublayers is set to 1 , constant_frame_rate is set to 1, if the codec configuration property is associated with the main image item: chromajormatjdc is set to the value of the chroma_subsampling field from the MinimizedlmageBox, if the codec configuration property is associated with the alpha auxiliary image item: chromajormatjdc is set to 0, if the codec configuration property is associated with the main image item or with the alpha auxiliary image item: bit_depth_minus8 is set to 0 if the value of the high_bit_depth_flag field from the MinimizedlmageBox is 0, or to the value of the bit_depth_minus9 field from the MinimizedlmageBox plus 1 , max_picture_width is set to the value plus 1 of the width_minus1 field from the MinimizedlmageBox, max_picture_height is set to the value plus 1 of the height_minus1 field from the MinimizedlmageBox, if the codec configuration property is associated with the gain map image item: bit_depth_minus8 is set to 0 if the value of the gainmap_high_bit_depth_flag field from the MinimizedlmageBox is 0, or to the value of the gainmap J)it_depth_minus9 field from the MinimizedlmageBox plus 1, max_picture_width is set to the value plus 1 of the gainmap_width_minus1 field from the MinimizedlmageBox, max_picture_height is set to the value plus 1 of the gainmap Jieight_minus1 field from the MinimizedlmageBox, avg Jrame_rate is set to 0,

[0293] - if nal_units_present_flag is set to 1: num_of_arrays is set to the number of entries below: if vps_nal_unitjength is present and not 0, there is a VPS NAL unit array with: NAL_unitJype set to 14 (VPS_NUT as defined in ISO / IEC 23090-3), num_nalus set to 1, naljjnitJength set to vps_nal_unitjength,nal_unit set to vps_nal_unit; if sps_nal_unit_length is not 0, there is a SPS NAL unit array with: NAL_u nitjy pe set to 15 (SPS_NUT as defined in ISO / IEC 23090-3), num_nalus set to 1, naljjnitJength set to sps_nal_unit_length, nal_unit set to sps_nal_unit; if pps_nal_unit_length is not 0, there is a PPS NAL unit array with: NAL_unit_type set to 16 (PPS_NUT as defined in ISO / IEC 23090-3), num_nalus set to 1, naljjnitJength set to pps_nal_unit_length, nal_unit set to pps_nal_unit; if additional_nal_unit_flag is set to 1 and num_aps_nal_unit is not 0, there is a prefix APS NAL unit array with: NAL_unit_ty pe set to 17 (PREFIX_APS_NUT as defined in ISO / IEC 23008-2), num_nalus set to num_aps_nal_unit, and for each prefix APS NAL unit: nal_unit_length is set to aps_nal_unit_length, naljjnit is set to aps_nal_unit; if additional_nal_unit_flag is set to 1 and num_sei_nal_unit is not 0, there is a prefix SEI NAL unit array with: NAL_unit_type set to 23 (PREFIX_SEI_NUT as defined in ISO / IEC 23008-2), num_nalus set to num_sei_nal_unit, and for each prefix SEI NAL unit: nal_uni tjength is set to sei_nal_uni tjength, naljjnit is set to seijial jinit.

[0294] - if array_completeness is not present in CompactVvcDecoderConfigurationRecord, it is set equal to 1 in VvcDecoderConfigurationRecord.

[0295] The other parameters are carried over as is, and repeated if needed.

[0296] Low-overhead image file format

[0297] O.1 General

[0298] The low-overhead image file format provides a more compact representation of the image file format for specific use-cases. This format is designed for small and simple files where the use of the MetaBox would result in significant overhead relative to the size of the image and / or metadata payloads. In this format, the top-level MetaBox is replaced by a MinimizedlmageBox, which logically maintains the presence of the MetaBox by representing its contents.

[0299] When a parser encounters a MinimizedlmageBox, it shall expand it to a MetaBox as described in Clause O.4, at which point the specifications of Clause 6 apply.

[0300] When a brand specified in Clause O.2 is the major brand or among the compatible brands of a file, either explicitly or implicitly, the requirements specified in this annex shall be applied.

[0301] O.2 Brands

[0302] 0.2.1 'mif3' structural brand

[0303] 0.2.1.1 Requirements on files

[0304] Files containing the brand ’mif3’ as the major brand or in the compatible brands array of the FileTypeBox shall conform to the constraints defined in this subclause.

[0305] When the ’mif3’ brand is the major brand or present among the compatible brands of the FileTypeBox, the file may be identified by the MIME type defined in Annex P. When the 'mif3' brand is the major brand, the defined file extension and MIME type defined in Annex P should be used.

[0306] The boxes listed in Table 1 are required in a file under the ’mif3’ brand. The Version column in the following table lists the versions of the boxes allowed by this brand. Other versions of the boxes shall not be present. Other file-level boxes shall not be present.Table 1 — Required boxes in a file under the 'mif3' brand

[0307] When the 'mif3' brand is present as the major brand of the FileTypeBox, the minor version of the FileTypeBox shall be 0 or a brand.

[0308] 0.2.1.2 Requirements on readers

[0309] When the 'mif3' brand is present as the major_brand of the FileTypeBox, the minor_version of the FileTypeBox shall be 0 or a brand that is either structurally compatible with the 'mif3' brand, such as a codec brand complying with the 'mif3' structural brand, or a brand to which the file conforms after the equivalent MetaBox has been transformed from MinimizedlmageBox as specified in Clause O.4.Table 2 — Boxes to be supported under the 'mif3' brandin addition to those required for readers of 'mif1 ' brand

[0310] A file containing a MinimizedlmageBox shall be treated as if:

[0311] - it contained the 'miff brand in the compatible_brands array in the FileTypeBox,

[0312] - it contained the equivalent MetaBox as described in Clause O.4,

[0313] - the major_brand of the FileTypeBox was the equivalent brand specified by the brand provided in the minor_version, if the major_brand is 'mif3' and the minor_version is not 0,

[0314] - it contained the 'tmap' brand in the compatible_brands array in the FileTypeBox if gainmap_flag as defined in clause 0.3.3 is set to 1 in the MinimizedlmageBox.

[0315] O.3 Minimized Image Box

[0316] 0.3.1 DefinitionBox type: 'mini'Container: FileMandatory: NoQuantity: Zero or one

[0317] The minimized image box format provides a more compact representation of the MetaBox for a subset of use cases. It is meant to be used for small and simple files where the full MetaBox would result in considerable overhead compared to the image data payload.

[0318] When MinimizedlmageBox is present in a file, the ’mif3’ brand or a derived brand that implies the ’mif3’ brand shall be the major brand or present among the compatible brands in the FileTypeBox.

[0319] 0.3.2 Syntaxaligned(8) class MinimizedlmageBox extends Box('mini') {bit(2) version = 0; / / flagsbit(1 ) exp I icit_codec_types_fl ag ;bit(1) float_flag;bit(1) full_range_flag;bit(1) alpha_flag;bit(1) explicit_cicp_flag;bit(1) hdr_flag;bit(1) icc_flag;bit(1) exif_flag;bit(1) xmp_flag;bit(2) chroma_subsampling;bit(3) orientation_minus1 ; / / Spatial extentsbit(1) large_dimensions_flag;unsigned int(large_dimensions_flag ? 15 : 7) widthjninusl;unsigned int(large_dimensions_flag ? 15 : 7) height_minus1 ; / / Pixel informationif (chroma_subsampling == 1 || chroma_subsampling == 2)bit(1 ) chroma_is_horizontally_centered;if (chroma_subsampling == 1)bit(1 ) chroma_is_vertically_centered;if (float_flag)bit(2) bit_depth Jog2_minus4;else {bit(1) high_bit_depth_flag;if (high_bit_depth_flag)bit(3) bit_depth_minus9;}if (alpha_flag)bit(1) alpha_is_premultiplied; / / Colour propertiesif (explicit_cicp_flag) {bit(8) colour_primaries;bit(8) transfer_characteristics;if (chroma_subsampling != 0)bit(8) matrix_coefficients;elsematrix_coefficients = 2;}else {colour_primaries = icc_flag ? 2 : 1 ;transfer_ch aracteristics = icc_flag ? 2 : 13;matrix_coefficients = chroma_subsampling == 0 ? 2 : 6;}if (explicit_codec_types_flag) {bit(32) infe_type;bit(32) codec_config_type;} else { / / codec information is defined by the brand in the minor_version field of FileTypeBox } / / High Dynamic Range propertiesif (hdr_flag) {bit(1) gainmap_flag;if (gainmap_flag) {unsigned int(large_dimensions_flag ? 15 : 7) gainmap_width_minus1; unsigned int(large_dimensions_flag ? 15 : 7) gainmap Jieight_minus1; bit(8) gainmap jriatrix_coefficients;bit(1) gainmap_full_range_flag;bit(2) gainmap_chroma_subsampling;if (gainmap_chroma_subsampling == 1 || gainmap_chroma_subsampling == 2) bit(1) gainmap_chroma_is_horizontally_centered;if (gainmap_chroma_subsampling == 1)bit(1 ) gainmap_chroma_is_vertically_centered;bit(1) gainmap_float_flag;if (gainmap_float_flag)bit(2) gainmap J)itjtepthJog2_minus4;else {bit(1 ) g ai nmap_h ig h_bit_depth_fl ag ;if (gainmap_high_bit_depth_flag)bit(3) gainmap_bit_depth_minus9;}bit(1) tmap_icc_flag;bit(1) tmap_explicit_cicp_flag;if (tmap_explicit_cicp_flag) {bit(8) tmap_colour_primaries;bit(8) tmap_transfer_characteristics;bit(8) tmap_matrix_coefficients;bit(1) tmap_full_range_flag;}else {tmap_colour_primaries = 1 ; tmap_transfer_characteristics = 13; tmap_matrix_coefficients = 6; tmap_full_range_flag = 1;}}bit(1) cllLflag;bit(1) mdcv_flag;bit(1) cclv_flag;bit(1) amve_flag;bit(1) reve_flag;bit(1) ndwt_flag;if (clli_flag)ContentLightLevel clli;if (mdcv_flag) MasteringDisplayColourVolume mdcv; if (ccl v_flag)ContentColourVolume cclv;if (amve_flag) AmbientViewingEnvironment amve; if (reve_flag) ReferenceViewingEnvironment reve; if (ndwt_flag)NominalDiffuseWhite ndwt;if (gainmap_flag) {bit(1) tmap_clli_flag;bit(1) tmap_mdcv_flag;bit(1) tmap_cclv_flag;bit(1) tmap_amve_flag;bit(1) tmap_reve_flag;bit(1) tmap_ndwt_flag;if (tmap_clli_flag)ContentLightLevel tmap_clli;if (tmap_mdcv_flag)MasteringDisplayColourVolume tmapjndcv;if (tmap_cclv_flag)ContentColourVolume tmap_cclv;if (tmap_amve_flag)AmbientViewingEnvironment tmap_amve;if (tmap_reve_flag)ReferenceViewingEnvironmenttmap_reve;if (tmap_ndwt_flag)NominalDiffuseWhite tmap_ndwt;}} / / Chunk sizesif (icc_flag || exif_flag || xmp_flag || (hdr_flag && gainmap_flag))bit(1) large_metadata_flag;bit(1) large_codec_config_flag;bit(1) large_item_data_flag;if (icc_flag)unsigned int(large_metadata_flag ? 20 : 10) icc_data_size_minus1; if (hdr_flag && gainmap_flag && tmap_icc_flag)unsigned int(large_metadata_flag ? 20 : 10) tmap_icc_data_size_minus1; if (hdr_flag && gainmap_flag)unsigned int(large_metadata_flag ? 20 : 10) gainmap_metadata_size; if (hdr_flag && gainmap_flag)unsigned int(large_item_data_flag ? 28 : 15) gainmap Jtem_data_size;if (hdr_flag && gainmap_flag && gainmap Jtem_data_size > 0) unsigned int(large_codec_config_flag ? 12 : 3) gainmap Jtem_codec_config_size; unsigned int(large_codec_config_flag ? 12 : 3) main_item_codec_config_size; unsigned int(large_item_data_flag ? 28 : 15) main_item_data_size_minus1;if (alpha_flag)unsigned int(large_item_data_flag ? 28 : 15) alpha_item_data_size;if (alpha_flag && alpha_item_data_size > 0)unsigned int(large_codec_config_flag ? 12 : 3) alpha_item_codec_config_size; if (exif_flag)unsigned int(large_metadata_flag ? 20 : 10) exif_data_size_minus1;if (xmp_flag)unsigned int(large_metadata_flag ? 20 : 10) xmp_data_size_minus1; trailing_bits(); / / bit padding till byte alignment / / Chunksif (alpha_flag && alpha_item_data_size > 0 && alpha_item_codec_config_size > 0) unsigned int(8) alpha_item_codec_config[alpha_item_codec_config_size];if (hdr_flag && gainmap_flag && gainmap Jtem_codec_config_size > 0) unsigned int(8) gainmap Jtem_codec_config[gainmap_item_codec_config_size]; if (main_item_codec_config_size > 0)unsigned int(8) main_item_codec_config[main_item_codec_config_size];if (icc_flag)unsigned int(8) icc_data[icc_data_size_minus1 + 1];if (hdr_flag && gainmap_flag && tmap_icc_flag)unsigned int(8) tmap_icc_data[tmap_icc_data_size_minus1 + 1];if (hdr_flag && gainmap_flag && gainmap_metadata_size > 0)unsigned int(8) gainmap jrietadata[gainmap_metadata_size];if (alpha_flag && alpha_item_data_size > 0)unsigned int(8) alpha_item_data[alpha_item_data_size];if (hdr_flag && gainmap_flag && gainmap Jtem_data_size > 0)unsigned int(8) gainmap Jtem_data[gainmap_item_data_size];unsigned int(8) main_item_data[main_item_data_size_minus1 + 1];if (exif_flag)unsigned int(8) exif_data[exif_data_size_minus1 + 1];if (xmp_flag)unsigned int(8) xmp_data[xmp_data_size_minus1 + 1];}

[0320] 0.3.3 Semantics

[0321] version: specifies the version of the MinimizedlmageBox. The version shall be set to 0 in this version of this document / description.

[0322] large_dimensions_flag: if set to 0, the length of the fields signaled among width_minus1, height_minus1 , gainmap_width_minus1 and gainmap_height_minus1 is 7 bits; otherwise, it is 15 bits.

[0323] width_minus1 : plus 1 specifies the width of the reconstructed image in pixels.

[0324] height_minus1 : plus 1 specifies the height of the reconstructed image in pixels.

[0325] orientation_mi nus 1 : plus 1 specifies the Exif orientation value as defined in Japan Electronics and Information Technology Industries Association, JEITA, CP-3451 E section 4.6.4.A "Orientation".

[0326] iccjlag: equal to 1 indicates that the main image is associated with an International Color Consortium (ICC) profile as defined in ISO 15076-1 or ICC.1.

[0327] exifjlag: equal to 1 indicates the presence of exchangeable image file format (Exif) metadata.

[0328] xmpjlag: equal to 1 indicates the presence of extensible metadata platform, XMP, metadata.

[0329] full_range_flag: is a binary value representing the VideoFullRangeFlag as defined in Recommendation, Rec., ITU-T H.273 | ISO / IEC 23091-2. This signaling applies exclusively to the main image.

[0330] chroma_subsampling: Specifies the number of color channels and the subsampling of the chroma channels, as defined in Table 3, of the main image.Table 3 - Chroma subsampling

[0331] chroma_is_horizontally_centered: 0 specifies that the chroma samples of the main image are colocated horizontally with the luma samples of the main image, otherwise they are horizontally centered between the luma samples of the main image. 0 unless chroma_subsampling is 1 or 2. chroma_is_vertically_centered: 0 specifies that the chroma samples of the main image are co-located vertically with the luma samples of the main image, otherwise they are vertically centered between the luma samples of the main image. 0 unless chroma_subsampling is 1. floatjlag: specifies the format of the pixel values of the reconstructed main and alpha image items as the component Jormat 0 and 1 values, as specified in PixellnformationProperty with px_flags&1!=0 in clause 6.5.6. bit_depth_log2_minus4: specifies the format of floating-point numbers used for the pixel values of the reconstructed main and alpha image items. The values 0, 1, and 2 respectively correspond to the bits_per_channel values 16, 32 and 64, as specified in PixellnformationProperty in clause 6.5.6. Other values are reserved. When floatjlag is set to 0, the value is undefined. high_bit_depth_flag: 0 specifies that the number of bits per channel for the pixel values of the reconstructed main and alpha image items, as specified in PixellnformationProperty in clause 6.5.6, is 8. Otherwise bit_depth_minus9 is signaled. When floatjlag is set to 1, the value is undefined. bit_depth_minus9: specifies the number of bits, minus nine, per channel for the pixel values of the reconstructed main and alpha image items, as specified in PixellnformationProperty in clause 6.5.6. When high_bit_depth_flag is set to 0 or floatjlag is set to 1 , the value is undefined, alpha Jlag: 0 specifies that the image is opaque. Otherwise the image has an alpha layer, whether the codec has native translucency support or an alpha auxiliary image item is used. alpha_is_premultiplied: when set to 1 specifies that the color channels are pre-multiplied by the alpha channel, otherwise the color channels are not pre-multiplied. Ignored if alphajlag is 0. explicit_cicpjlag: equal to 0 indicates the values of ColourPrimaries, Transfercharacteristics and Matrixcoefficients, as defined in Rec. ITU-T H.273 | ISO / IEC 23091-2, are set to the values defined in Table 4. When the value is equal to 1 it indicates that these values are signaled explicitly, as defined in Table 4.Table 4 - Values of Coding Independent Code Point (CICP) fields associated with the main image

[0332] sRGB is an abbreviation for standard red green blue.

[0333] colour_primaries: carries a ColourPrimaries value as defined in Rec. ITU-T H.273 | ISO / IEC 23091-2 for the main image. transfer_characteristics: carries a Transfercharacteristics value as defined in Rec. ITU-T H.273 | ISO / IEC 23091-2 for the main image. matrix_coefficients: carries a Matrixcoefficients value as defined in Rec. ITU-T H.273 | ISO / IEC 23091-2 for the main image. explicit_codec_types_flag: 0 specifies that the minor_version of the FileTypeBox carries a brand defining a single coded image item type and a single codec configuration property box type. 1 specifies that the minor_version of the FileTypeBox carries no such brand, or that the coded image item type and the codec configuration property box type defined by such a brand are overridden by the fields infe_type and codec_config_type, respectively. infe_type: carries the coded image item type. Corresponds to the item_type field of the version 2 of the ItemlnfoEntry box. Defined by the brand carried by the minor_version of the FileTypeBox if explicit_codec_types_flag is 0. codec_config_type: carries the codec configuration property box type. Defined by the brand carried by the minor_version of the FileTypeBox if explicit_codec_types_flag is 0. hdr_flag: 0 specifies that the image is standard dynamic range (SDR) and has no associated HDR-related signaling. Otherwise the image is either SDR with a SDR-to-HDR gain map, or HDR with an optional HDR-to-SDR gain map. gainmap_flag: 0 specifies that the file has no tone-mapped image and no associated HDR-related ISO 21496-1 gain map. Otherwise the file contains a tone-mapped image and is associated with a gain map, whether the codec has native gain map support or a separate gain map image item is used.0 if hdr_flag is 0. gainmap_width_minus1 : carries the width minus one of the gain map image in pixels.

[0334] gainmap_height_minus1 : carries the height minus one of the gain map image in pixels.

[0335] gainmap_matrix_coefficients: carries a Matrixcoefficients value as defined in Rec. ITU-T H.273 1 ISO / IEC 23091-2 for the gain map image.

[0336] gainmap_full_range_flag: carries a VideoFullRangeFlag as defined in Rec. ITU-T H.2731 ISO / IEC 23091-2 for the gain map image.

[0337] gainmap_chroma_subsampling: Specifies the number of colour channels and the subsampling of the chroma channels, as defined in Table 3, of the gain map image.

[0338] gainmap_chroma_is_horizontally_centered: 0 specifies that the chroma samples of the gain map image are co-located horizontally with the luma samples of the gain map image, otherwise they are horizontally centered between the luma samples of the gain map image. Ignored unless gainmap_chroma_subsampling is 1 or 2.

[0339] gainmap_chroma Js_vertically_centered: 0 specifies that the chroma samples of the gain map image are co-located vertically with the luma samples of the gain map image, otherwise they are vertically centered between the luma samples of the gain map image. Ignored unless gainmap_chroma_subsampling is 1.

[0340] gai nmap_float_flag : specifies the format of the pixel values of the reconstructed gain map image item as the component_format values, as specified in PixellnformationProperty with px_flags&1!=0 in clause 6.5.6.

[0341] gainmap_bit_depth_log2_minus4: specifies the format of floating-point numbers used for the pixel values of the reconstructed gain map image item. The values 0, 1, and 2 respectively correspond to the bits_per_channel values 16, 32 and 64, as specified in PixellnformationProperty in clause 6.5.6. Other values are reserved. When float_flag is set to 0, the value is undefined.

[0342] gai nmap_hig h_bit_depth_flag : 0 specifies that the number of bits per channel for the pixel values of the reconstructed gain map image item, as specified in PixellnformationProperty in clause 6.5.6, is 8. Otherwise gainmap J)it_depth_minus9 is signaled. When gainmap_float_flag is set to 1, the value is undefined.

[0343] gainmap_bit_depth_minus9: specifies the number of bits, minus nine, per channel for the pixel values of the reconstructed gain map image item, as specified in PixellnformationProperty in clause 6.5.6. When high_bit_depth_flag is set to 0 or float_flag is set to 1, the value is undefined.

[0344] tmap_icc_flag: if 1, specifies that the tone-mapped image is associated with an ICC profile as defined in ISO 15076-1 or ICC.1. 0 if gainmap_flag is 0.

[0345] tmap_explicit_cicp_flag: 0 specifies sRGB on-screen colors as the values of ColourPrimaries, Transfercharacteristics and Matrixcoefficients, as defined in Rec. ITU-T H.273 | ISO / IEC 23091-2, associated with the tone-mapped image, set to the values defined in Table 5. Otherwise these values are signaled explicitly, as defined in Table 5.Table 5 - Values of CICP fields associated with the tone-mapped image

[0346] tmap_colour_primaries: carries a ColourPrimaries value as defined in Rec. ITU-T H.2731 ISO / IEC 23091-2 for the tone-mapped image.

[0347] tmap_transfer_characteristics: carries a Transfercharacteristics value as defined in Rec. ITU-T H.273 1 ISO / IEC 23091 -2 for the tone-mapped image.

[0348] tmap_matrix_coefficients: carries a Matrixcoefficients value as defined in Rec. ITU-T H.273 | ISO / IEC 23091-2 for the tone-mapped image.

[0349] tmap_full_range_flag: carries a VideoFullRangeFlag as defined in Rec. ITU-T H.273 | ISO / IEC 23091 -2 for the tone-mapped image. Set to 1 if tmap_explicit_cicp_flag is O.clli_flaggainmap_flag

[0350] clli_flag: 1 specifies that there is signaling for ContentLightLevel attached to the main image. Otherwise no such signaling is present. 0 if hdr_flag is 0.

[0351] mdcv_flag: 1 specifies that there is signaling for MasteringDisplayColourVolume attached to the main image. Otherwise no such signaling is present. 0 if hdr_flag is 0.

[0352] ccl v_flag : 1 specifies that there is signaling for ContentColourVolume attached to the main image. Otherwise no such signaling is present. 0 if hdr_flag is 0.

[0353] amve_flag: 1 specifies that there is signaling for AmbientViewingEnvironment attached to the main image. Otherwise no such signaling is present. 0 if hdr_flag is 0.

[0354] reve_flag: 1 specifies that there is signaling for ReferenceViewingEnvironment attached to the main image. Otherwise no such signaling is present. 0 if hdr_flag is 0.

[0355] ndwt_flag: 1 specifies that there is signaling for NominalDiffuseWhite attached to the main image. Otherwise no such signaling is present. 0 if hdr_flag is 0.

[0356] tmap_clli_flag: 1 specifies that there is signaling for ContentLightLevel attached to the tonemapped image. Otherwise no such signaling is present. 0 if gainmap_flag is 0.

[0357] tmap_mdcv_flag: 1 specifies that there is signaling for MasteringDisplayColourVolume attached to the tone-mapped image. Otherwise no such signaling is present. 0 if gainmap_flag is 0.

[0358] tmap_cclv_flag: 1 specifies that there is signaling for ContentColourVolume attached to the tonemapped image. Otherwise no such signaling is present. 0 if gainmap_flag is 0.

[0359] tmap_amve_flag: 1 specifies that there is signaling for AmbientViewingEnvironment attached to the tone-mapped image. Otherwise no such signaling is present. 0 if gainmap_flag is 0.

[0360] tmap_reve_flag: 1 specifies that there is signaling for ReferenceViewingEnvironment attached to the tone-mapped image. Otherwise no such signaling is present. 0 if gainmap_flag is 0.

[0361] tmap_ndwt_flag: 1 specifies that there is signaling for NominalDiffuseWhite attached to the tonemapped image. Otherwise no such signaling is present. 0 if gainmap_flag is 0.

[0362] clli: The box body of the ContentLightLevelBox as defined in ISO / IEC 14496-12 attached to the main image. Only present if clli_flag is 1.

[0363] mdcv: The box body of the MasteringDisplayColourVolumeBox as defined in ISO / IEC 14496-12 attached to the main image. Only present if mdcv_flag is 1.

[0364] cclv: The box body of the ContentColourVolumeBox as defined in ISO / IEC 14496-12 attached to the main image. Only present if ccl v_flag is 1.

[0365] amve: The box body of the AmbientViewingEnvironmentBox as defined in ISO / IEC 14496-12 attached to the main image. Only present if amve_flag is 1.

[0366] reve: The box body of the ReferenceViewingEnvironmentBox attached to the main image. Only present if reve_flag is 1.

[0367] ndwt: The box body of the NominalDiffuseWhiteBox attached to the main image. Only present if ndwt_flag is 1.

[0368] tmap_clli: The box body of the ContentLightLevelBox as defined in ISO / IEC 14496-12 attached to the tone-mapped image. Only present if tmap_clli_flag is 1.

[0369] tmapjndcv: The box body of the MasteringDisplayColourVolumeBox as defined in ISO / IEC 14496-12 attached to the tone-mapped image. Only present if tmap_mdcv_flagis 1.

[0370] tmap_cclv: The box body of the ContentColourVolumeBox as defined in ISO / IEC 14496-12 attached to the tone-mapped image. Only present if tmap_cclv_flag is 1.

[0371] tmap_amve: The box body of the AmbientViewingEnvironmentBox as defined in ISO / IEC 14496-12 attached to the tone-mapped image. Only present if tmap_amve_flag is 1.

[0372] tmap_reve: The box body of the ReferenceViewingEnvironmentBox attached to the tone-mapped image. Only present if tmap_reve_flag is 1.

[0373] tmapjidwt: The box body of the NominalDiffuseWhiteBox attached to the tone-mapped image. Only present if tmap_ndwt_flag is 1.

[0374] large_metadata_flag: 0 specifies that the length of the signaled fields among icc_data_size_minus1, tmap_icc_data_size_minus1, gainmap_metadata_size, exif_data_size_minus1 and xmp_data_size_minus1 is 10 bits, otherwise 20 bits. Undefined unless one of these fields is signaled.

[0375] large_codec_config_flag: 0 specifies that the length of the signaled fields among gainmap Jtem_codec_config_size, main_item_codec_config_size and alpha_item_codec_config_size is 3 bits, otherwise 12 bits.

[0376] large Jtem_data_flag: 0 specifies that the length of the signaled fields among gainmap_item_data_size, main_item_data_size_minus1 and alpha_item_data_size is 15 bits, otherwise 28 bits.

[0377] icc_data_size_minus1: carries the size minus one in bytes of the ICC profile as defined in ISO 15076-1 or ICC.1 , associated with the main image. Undefined if icc_flag is 0.

[0378] tmap Jcc_data_size_minus1 : carries the size minus one in bytes of the ICC profile as defined in ISO 15076-1 or ICC.1 , associated with the tone-mapped image. Undefined if tmap_icc_flag is 0.

[0379] gainmap_metadata_size: carries the size of the gain map metadata. 0 if gainmap_flag is 0.

[0380] gainmap_item_data_size: carries the size of the coded sample data for the HDR-related gain map image item in bytes. If gainmap_flag is set to 1, a size of 0 is reserved for future use. 0 if gainmap_flag is 0.

[0381] gainmap Jtem_codec_config_size: carries the size of the codec configuration for the gain map auxiliary image item in bytes. The value 0 specifies that the codec does not need any configuration data for the gain map. 0 if gainmap Jtem_data_size is 0.

[0382] main Jtem_codec_config_size: carries the size of the codec configuration for the main image item in bytes.

[0383] main Jtem_data_size_minus1 : carries the size minus one of the coded sample data for the main image item in bytes.

[0384] alpha_item_data_size: carries the size of the coded sample data for the alpha auxiliary image item in bytes. If alpha_flag is set to 1, the value 0 specifies that the codec has native translucency support and that the alpha samples are coded alongside the color samples in the main Jtem_data chunk. 0 if alp ha_flag is 0.

[0385] alpha_item_codec_config_size: carries the size of the codec configuration for the alpha auxiliary image item in bytes. The value 0 specifies that the codec does not need any configuration data for alpha. 0 if alpha_item_data_size is 0.

[0386] exif_data_size_minus1 : specifies the size minus one of the Exit metadata in bytes. -1 if exif_flag is 0.

[0387] xmp_data_size_minus1 : specifies the size minus one of the XMP metadata in bytes. -1 if xmp_flag is 0.

[0388] trailing_bits: padding bits to ensure payloads are 8-bit aligned. Shall be 0.

[0389] alpha_item_codec_config: carries the optional alpha image codec configuration data. When alpha_item_codec_config_size is 0, alpha_item_codec_config is not present.

[0390] gainmap Jtem_codec_config: carries the HDR-related gain map image item codec configuration data. When gainmap Jtem_codec_config_size is 0, gainmap Jtem_codec_config is not present.

[0391] main_item_codec_config: carries the main image item codec configuration data. When main_item_codec_config_size is 0, main_item_codec_config is not present.

[0392] icc_data: carries the ICC profile data of the main image as defined in ISO 15076-1 or ICC.1. When icc_flag is 0, icc_data is not present.

[0393] tmap Jcc_data: carries the ICC profile data of the optional HDR-related tone-mapped image as defined in ISO 15076-1 or ICC.1. When tmap_icc_flag is 0, tmap_icc_data is not present.

[0394] gainmapjnetadata: Gain map metadata as defined by the GainMapMetadata struct in ISO 21496-1. Not present if gainmap_metadata_size is 0.

[0395] alpha_item_data: carries the coded sample data of the optional alpha image. When alpha_item_data_size is 0, alpha_item_data is not present.

[0396] gainmap Jtem_data: carries the coded sample data of the optional gain map image. When gainmap_item_data_size is 0, gainmap Jtem_data is not present.

[0397] main Jtem_data: carries the coded sample data of the main image.

[0398] exif_d ata: specifies the optional Exif metadata. When exif_flag is set to 0, exif_d ata is not present.

[0399] xmp_data: specifies the optional XMP metadata. When xmp_flag is set to 0, xmp_data is not present.

[0400] O.4 Equivalence with MetaBox

[0401] A MinimizedlmageBox has a one-to-one mapping to a MetaBox. Readers shall treat a MinimizedlmageBox as if it were the equivalent MetaBox that is transformed from MinimizedlmageBox as specified in this subclause. When a reader encounters the MinimizedlmageBox, it can create the equivalent MetaBox in memory and populate its contents based on the parsed contents of the MinimizedlmageBox to reuse regular MetaBox handling implementation.

[0402] File writers can either write an image file based on the MetaBox or write an image file based on the MinimizedlmageBox. File writers can write an image file based on the MinimizedlmageBox when the MetaBox would result in considerable overhead compared to the image data payload. Files writers can strip metadata such as Exif and XMP, entirely or to its strict minimum, to avoid bloating the MinimizedlmageBox.

[0403] The equivalent MetaBox has its version and flags set to 0 and the sub-boxes described by the following subclauses.

[0404] 0.4.1 HandlerBox

[0405] The equivalent MetaBox shall have a HandlerBox with handler_type equal to 'pict'.

[0406] 0.4.2 PrimaryltemBox

[0407] The equivalent MetaBox shall have a PrimaryltemBox with item_l D set to 1.

[0408] 0.4.3 ItemlnfoBox

[0409] The equivalent MetaBox shall have an ItemlnfoBox containing the following entries: Iteml nfoEntry of version 2 with flags set to 0, item_ID set to 1 and item_type set to infe_type. All other fields are set to null or 0 as appropriate. If alpha_item_data_size is not 0, Iteml nfoEntry of version 2 with flags set to 1 , itemJD set to 2 and item Jype set to infe Jype. All other fields are set to null or 0 as appropriate. If gainmapjlag is 1 , Iteml nfoEntry of version 2 with flags set to 0, item_l D set to 3 and item_type set to 'tmap'. All other fields are set to null or 0 as appropriate. If gainmap Jtem_data_size is not 0, Iteml nfoEntry of version 2 with flags set to 1 , item_l D set to 4 and item Jype set to infe Jype. All other fields are set to null or 0 as appropriate. If exit Jlag is 1 , Iteml nfoEntry of version 2 with flags set to 1 , item_l D set to 6 and item_type set to 'Exif . All other fields are set to null or 0 as appropriate. If xmp_flag is 1 , Iteml nfoEntry of version 2 with flags set to 1 , item_ID set to 7 and item Jype set to 'mime' and content Jype set to 'application / rdf+xml'. All other fields are set to null or 0 as appropriate.

[0410] 0.4.4 ItemReferenceBox

[0411] The ItemReferenceBox is populated with the following entries:

[0412] - If ItemlnfoBox has an entry for itemJD 2: Item type reference with referenceType set to 'auxl', fromJtemJD set to 2, reference_count set to 1 and toJtemJD set to 1.

[0413] - If ItemlnfoBox has an entry for itemj D 2 and alpha_is_premulti plied is set to 1 : Item type reference with referenceType set to 'prem', fromJtemJD set to 1, reference_count set to 1 and to_item_ID set to 2.

[0414] - If gainmapjlag is 1 and ItemlnfoBox has an entry for itemJD 4: item type reference with referenceType set to 'dimg', fromJtemJD set to 3, reference_count set to 2 and the following toJtemJD entries: 1, 4.

[0415] - If gainmapjlag is 1 and ItemlnfoBox has zero entry for itemJD 4: item type reference with referenceType set to 'dimg', fromJtemJD set to 3, reference_count set to 1 and toJtemJD set to 1.

[0416] - If ItemlnfoBox has an entry for itemJD 6: item type reference with referenceType set to 'cdsc', fromJtemJD set to 6, reference_count set to 1 and toJtemJD set to 1.

[0417] - If ItemlnfoBox has an entry for itemJD 7: item type reference with referenceType set to 'cdsc', fromJtemJD set to 7, reference_count set to 1 and toJtemJD set to 1.

[0418] If the resulting ItemReferenceBox contains at least one entry, it shall be added to the equivalent MetaBox. An empty ItemReferenceBox shall be ignored.

[0419] 0.4.5 EntityToGroupBox

[0420] If gainmapjlag is 1, the equivalent MetaBox shall have a GroupsListBox containing a single subbox. That sub-box is an EntityToGroupBox with groupingjype set to 'altr', version set 0, flags set to 0, groupjd set to 5, num_entitiesjn_group set to 2 and the following entityjd entries: 3, 1.

[0421] 0.4.6 ItemPropertiesBox

[0422] The equivalent MetaBox shall have an ItemPropertiesBox containing an ItemPropertyContainerBox and an ItemPropertyAssociationBox.

[0423] The ItemPropertyContainerBox shall have 32 entries as listed below. Any entry for which the condition is not true is replaced with a FreeSpaceBox.

[0424] The ItemPropertyAssociationBox shall have the entries below. Any association to a FreeSpaceBox shall be dropped.

[0425] Item 1 shall be associated with ItemPropertyContainerBox entries: 1, essential; 2, non-essential; 3, non-essential; 4, essential; 5, essential; 9, essential; 10, essential.

[0426] If hdrjlag is 1 , item 1 shall be associated with ItemPropertyContainerBox entries: 11 , non-essential; 12, non-essential; 13, non-essential; 14, non-essential; 15, non-essential; 16, non-essential.

[0427] If alpha_item_data_size is not 0, item 2 shall be associated with ItemPropertyContainerBox entries: 6, essential; 2, non-essential; 7, essential; 8, non-essential; 9, essential; 10, essential.

[0428] If gainmapjlag is 1, item 3 shall be associated with ItemPropertyContainerBox entries: 21, non-essential; 22, essential; 23, essential.

[0429] If hdrjlag is 1 and gainmapjlag is 1, item 3 shall be associated with ItemPropertyContainerBox entries: 24, non-essential; 25, non-essential; 26, non-essential; 27, non-essential; 28, non-essential; 29, non-essential.

[0430] If gainmap_item_data_size is not 0, item 4 shall be associated with ItemPropertyContainerBox entries: 17, essential; 18, non-essential; 19, non-essential; 20, non-essential; 9, essential; 10, essential.

[0431] If alpha Jlag is 1 and alphajtem_data_size is 0, item 1 shall be associated with ItemPropertyContainerBox entry: 30, essential.

[0432] 0.4.7 PixellnformationProperty

[0433] 0.4.7.1 Reconstruction

[0434] The various PixellnformationProperty boxes associated with the image items in the file are reconstructed given the arguments: main_components, alphajlag, subsampling, chroma_is_horizontally_centered, chroma_is_vertically_centered, float_flag, bit_depthjog2_minus4, high_bit_depth_flag, bit_depth_minus9.

[0435] Reconstruction happens as follows: version set to 0; px_flags set to 3; num_channels set to: main_components if alphajlag is false; main_components + 1 if alpha lag is true; each bits_per_channel entry set to 2bit_depth_log2_minus4+4 if float Jlag is 1, or to 8 if high Joit_depth Jlag is 0, or to bit_depth_minus9+9 otherwise; channel JabelJIag is set to 0 for all entries; the following entries for channel subsampling and formats; the following entry if alphajlag is true (skipped if false): channel Jdc set to 1, component_format set to float Jlag, subsampling Jype set to 0, subsamplingjocation set to 0; the following entry if main_components > 0 (skipped otherwise): channel Jdc set to 0, component_format set to float Jlag, subsamplingjype set to 0, subsamplingjocation set to 0; the following entry, repeated twice, if main_components > 1 (skipped otherwise): channeljdc set to 0, component_format set to float Jlag, subsampling_type set to 2 if subsampling is 1, 1 if subsampling is 2, 0 if subsampling is 3; subsamplingjocation set to 1 if chromajs_horizontally_centered and chromajs_vertically_centered are both true, 3 if chromajs_horizontally_centered is true and chromajs_vertically_centered is false, 0 if chromajs_horizontally_centered is false and chromajs_vertically_centered is true, 2 if chromajs_horizontally_centered and chromajs_vertically_centered are both false.

[0436] O.4.7.2 Main image PixellnformationProperty

[0437] The PixellnformationProperty associated with the main image item is reconstructed as described in section O.4.7.1 with the following arguments: main_components = 1 if chroma_subsampling is 0, else 3; alphajlag = alphajlag is 1 and alphajtem_data_size is 0; subsampling = chroma_subsampling; chromajs_horizontally_centered = chromajs_horizontally_centered; chromajs_vertically_centered = chromajs_vertically_centered; float Jlag = float Jlag; bit_depthjog2_minus4 = bit_depthjog2_minus4; high_bit_depth_flag = high Joit_depth Jlag; bit_depth_minus9 = bit_depth_minus9

[0438] O.4.7.3 Alpha auxiliary image PixellnformationProperty

[0439] If alphajtem_data_size is not 0, the PixellnformationProperty associated with the alpha auxiliary image item is reconstructed as described in section O.4.7.1 with the following arguments: main_components = 0; alphajlag = 1; subsampling = 0; chromajs_horizontally_centered = 0; chromajs_vertically_centered = 0; float Jlag = float Jlag; bit_depthjog2_minus4 = bit_depthjog2_minus4; high_bit_depth Jlag = high_bit_depth Jlag; bit_depth_minus9 = bit_depth_minus9.

[0440] O.4.7.4 Gain map image PixellnformationProperty

[0441] If gainmap_flag is not 0, the PixellnformationProperty associated with the gain map image item is reconstructed as described in section 0.4.7.1 with the following arguments: main_components = 1 if gainmap_chroma_subsampling is 0, else 3; alpha_flag = 0; subsampling = gainmap_chroma_subsampling; chroma_is_horizontally_centered = gainmap_chroma_is_horizontally_centered; chroma_is_vertically_centered; gainmap_chroma_is_vertically_centered; float_flag = gainmap_float_flag; bit_depth_log2_minus4 = gainmap_bit_depth_log2_minus4; high_bit_depth_flag = gainmap Jiigh_bitjtepth_flag; bit_depth_minus9 = gainmap J)it_depth_minus9.

[0442] 0.4.8 ToneMapImage metadata

[0443] If gainmap_flag is 1 , tmap Jtem_data is defined as a data chunk of tmap Jtem_data_size bytes, being gainmap_metadata_size+1 bytes, containing the ToneMapImage metadata as defined in section 6.6.2.4.2, with the following: version set to 0; GainMapMetadata set to gainmapjnetadata.

[0444] If gainmap_flag is 0, tmap Jtem_data is defined as an empty chunk and tmap Jtem_data_size as 0 byte.

[0445] 0.4.9 ItemLocationBox

[0446] The equivalent MetaBox shall have an ItemLocationBox of version 1 or version 2 containing the following entries:

[0447] - item_ID 1, with constructionjnethod set to 1, offset set to alpha_item_data_size+tmap_item_data_size+gainmap_item_data_size and length set to main Jtem_data_size_minus1 +1.

[0448] - Optional item_ID 2, with constructionjnethod set to 1, offset set to 0 and length set to alpha_item_data_size.

[0449] - Optional item J D 3, with construction nethod set to 1 , offset set to alpha_item_data_size, and length set to tmap Jtem_data_size

[0450] - Optional item J D 4, with constructionjnethod set to 1, offset set to alpha_item_data_size+tmap_item_data_size, and length set to gainmap Jtem_data_size

[0451] - Optional itemJD 6, with constructionjnethod set to 1, offset set to main_item_data_size_minus1+1+alpha_item_data_size+tmap_item_data_size+gainmap_item_data_size, and length set to exif_data_size_minus1 +1.

[0452] - Optional itemJD 7, with constructionjnethod set to 1, offset set to main_item_data_size_minus1+1+alpha_item_data_size+tmap_item_data_size+gainmap_item_data_size+ exif_data_size_minus1 +1 , and length setto xmp_data_size_minus1+1

[0453] 0.4.10 ItemDataBox

[0454] The equivalent MetaBox shall have an ItemDataBox containing the non-empty chunks among alpha_item_data, tmap_item_data, gainmap Jtem_data, main_item_data, exif_data and xmp_data, concatenated in that order.

[0455] The PixellnformationProperty descriptive item property indicates the number and bit depth of colour and alpha / depth components, if present, in the reconstructed image of the associated image item.aligned(8) class PixellnformationPropertyextends ltemFullProperty('pixi', version = 0, px_flags){unsigned int(8) num_channels;for (i=0; i<num_channels; i++) {unsigned int(8) bits_per_channel;}if((px_flags & 1) != 0) {for (i=0; i<num_channels; i++) {unsigned int(3) channeljdc;unsigned int(2) reserved = 0;unsigned int(2) component_format;unsigned int(1) channel JabeLflag;if((px_flags & 2) != 0) {unsigned int(4) subsampling_type;unsigned int(4) subsamplingjocation;}if(channel JabeLflag) {utf8string channeljabel;}}}}

[0456] utf is an abbreviation for universal coded character set, DCS, transmission format.

[0457] bits_per_chan nel : This field indicates the bits per channel for the pixels of the reconstructed image of the associated image item. The value of this field shall not be 0.

[0458] px_flags&1: If not 0, indicates that the channeljdc, component_format, and channel JabeLflag fields are present.

[0459] px_flags&2: If not 0, indicates that subsampling information is present. Only applicable when px_flags&1 is not 0.

[0460] channeljdc: This field indicates the contents of the channel. A value of 0 indicates colour / grayscale. A value of 1 indicates alpha. A value of 2 indicates depth. Values 3-7 are reserved for future use. At most one channel shall have a channeljdc of 1.

[0461] component_format: This field indicates the data type of the channel as defined by the component_format values in ISO / IEC 23001-17 where component J)it_depth is considered to be equal to bits_per_channel.

[0462] channel JabelJIag: This flag indicates the presence of the channel Jabel field .subsampli ng_type: This field indicates the subsampling type as specified by GenericSubsamplingType in Rec. ITU-T H.273 | ISO / IEC 23091-2.

[0463] subsamplingjocation: This field indicates the subsampling sample location as specified by GenericSubsamplingSampleLocType in Rec. ITU-T H.273 ISO / IEC 23091-2.

[0464] channel Jabel: The human readable description of the channel.

[0465] Embodiments related to Pixel Information Property

[0466] In an embodiment the new parameters added to version=0 of the Pixell nformationProperty may be moved under a new version. For example. Version=1 of the Pixell nformationProperty.

[0467] In an embodiment, not all the channels may be subsampled for example the luma channel in a YUV image, the alpha channel (if present), the depth channel (if present). Hence instead of indicating the presence of subsampling information (subsampling_type and subsamplingjocation) using the box flags (px_flags & 2) indicate the presence of subsampling information on a per-channel basis. YUV is a color space that uses one luma (Y) component and two chrominance (U, V) components.

[0468] In an embodiment, the example syntax of PixellnformationProperty with the above changes is defined below.aligned(8) class PixellnformationPropertyextends ltemFullProperty('pixi', version = 0, px_flags){unsigned int(8) num_channels;for (i=0; i<num_channels; i++) {unsigned int(8) bits_per_channel;}if((channeljnfo_present_flag) {for (i=0; i<num_channels; i++) {unsigned int(3) channeljdc;unsigned int(1) reserved = 0;unsigned int(2) component ormat;unsigned int(1) channel_subsampling_info_flag;unsigned int(1) channel JabeLflag;if(channel_subsampling_info_flag) {unsigned int(4) subsampling_type;unsigned int(4) subsamplingjocation;}if(channel JabeLflag) {utf8string channeljabel;}}}}

[0469] The flags field is used to signal channel related information. The following flags values are defined.

[0470] px_flags & 0x000001 : channel Jnfo_present_flag: when set to 1 , specifies that the image channel information (the channeljdc, component_format, channel_subsampling_info_flag and channel_label_flag) is present. When set to 0, the image channel information is not present.

[0471] In an alternate embodiment, the PixellnformationProperty may be extended with a new version with value equal to one and the box flag pxjlag is set to zero. When PixellnformationProperty is with version one then it contains image channel information (the channeljdc, component-format, channel_subsamplingjnfo_flag and channeljabel_flag) in addition to the number of channels and bits per channel information.

[0472] channel_subsamplingjnfo_flag: when set to 1, this flag indicates the presence of the subsampling_type and subsamplingjocation fields. channel_subsamplingjnfo_flag: when set to 0, this flag indicates the absence of the subsampling Jype and subsamplingjocation fields.

[0473] In an alternate embodiment, the PixellnformationProperty may be extended with version equal to zero and with setting the box flag pxjlag as above, but an indication indicates through the brand definition if the box flag px lag has the corresponding bits set or not.

[0474] Embodiments related to 'mif3'-compliant WC image and image collection brand

[0475] In an embodiment, files that include 'mif3' as the major Jjrand and the brand 'vvi3' as the minor_version (wherein the 'vvi3' brand defines the coded image item type to be 'vvd ' as defined in L.2.2.1.2 of the HEIF specification) in the FileTypeBox comply with the specifications in L.2 (HEIF specification) and additionally be constrained as follows: An item of type 'vvd ' consists of the NAL units (in case of multiple layers) or only NAL unit payload (in case of single layer) of a VVC bitstream that are length-delimited andthe WC bitstream contains an entire WC access unit, the VVC picture in the access unit does not contain any "extractable" WC subpictures.

[0476] In an embodiment, the syntax of the VVCItemData in files that include 'mif3' as the major_brand and the brand 'vvi3' as the minor_version (wherein the 'vvi3' brand defines the coded image item type to be 'vvd' as defined in L.2.2.1.2 of the HEIF specification) in the FileTypeBox is defined below.

[0477] Syntaxaligned(8) class VVCItemData{ / / temporary variableunsigned int(8) item_size = MinimizedlmageBox.main_item_data_size_minus1 + 1; if(CompactVvcDecoderConfigurationRecord.multi_layer_flag){for (i=0; i<item_size; ){unsigned int((CompactVvcDecoderConfigurationRecord.LengthSizeMinusOne+1)*8)NALUnitLength;bit(NALUnitl_ength * 8) NALUnit;i ■+= (DecoderConfigurationRecord.LengthSizeMinusOne+1) +NALUnitLength;}}else{bit(item_size*8) NALUnitPayload;}}

[0478] In the syntax above, the following applies:

[0479] The value of item_size is equal to main_item_data_size_minus1 + 1 as specified in the MinimizedlmageBox.

[0480] CompactVvcDecoderConfigurationRecord indicates the compact decoder configuration information for ISO / IEC 23090-3 video content.

[0481] NALUnitLength indicates the size of a NAL unit measured in bytes. The length field includes the size of both the two-byte NAL header and the NAL unit payload but does not include the length field itself.

[0482] NALUnit contains a single NAL unit. The syntax of a NAL unit shall conform to the syntax specified in ISO / IEC 23090-3 and includes both the two-byte NAL unit header and the variable-length NAL unit payload.

[0483] NALUnitPayload contains a single variable-length NAL unit payload. The NALUnitPayload conform to the syntax specified in ISO / IEC 23090-3 and does not include the two-byte NAL unit header and includes only the variable-length NAL unit payload the size of which is equal to main_item_data_size_minus1 + 1 as defined in the MinimizedlmageBox. The NALUnitPayload is present only when the CompactVvcDecoderConfigurationRecord.multiJayer_flag is set to 0. The NALUnitPayload is byte aligned. =

[0484] In an embodiment, as a response to 'mif3' as the major_brand and the brand 'vvi3' as the minor_version in the FileTypeBox, the readers reconstruct a VVC bitstream from the VVCItemData syntax as defined above.

[0485] In an embodiment, as a response to 'mif3' as the major_brand and the brand 'vvi3' as the minor_version in the FileTypeBox, and when the CompactVvcDecoderConfigurationRecord.multi_layer_flag is set to 1 the readers reconstruct a WC bitstream by concatenating all the NAL units present in the CompactVvcDecoderConfigurationRecord together with all the NAL units in the VVCItemData in the order as defined in in H.266| ISO / IEC 23090-3.

[0486] In an embodiment, as a response to 'mif3' as the major_brand and the brand 'vvi3' as the minor_version in the FileTypeBox, and when the CompactVvcDecoderConfigurationRecord.multi_layer_flag is set to 0 the readers reconstruct a WC bitstream by concatenating all the NAL units present in the CompactVvcDecoderConfigurationRecord together with concatenation of the reconstructed NAL unit header and the NAL unit payload in the VVCItemData in the order as defined in in H.266| ISO / IEC 23090-3.

[0487] Wherein, the NAL unit header is reconstructed by setting the following values.

[0488] The NAL unit header is the WC NAL unit header as defined in H.266| ISO / IEC 23090-3 with values as set below.

[0489] IDR_W_RADL corresponds to I DR with RADL.

[0490] In an embodiment, files that include 'mif3' as the major_brand and the brand 'vvi3' as the minor_version (wherein the 'vvi3' brand defines the coded image item type to be 'vvd ' as defined in L.2.2.1.2 of the HEIF specification) in the FileTypeBox comply with the specifications in L.2 (HEIF specification) and additionally be constrained as follows: image items of type 'vvd ' containing multiple layers do not have an associated item property VvcOperatingPointsInformationProperty. The information present in the VvcOperatingPointsInformationProperty is embedded within the CompactVvcDecoderConfigurationRecord.

[0491] In an example embodiment, the information present in the VvcOperatingPointsInformationProperty embedded within the CompactVvcDecoderConfigurationRecord is defined below.aligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;} / / other information within the CompactVvcDecoderConfigurationRecordif (multi_layer_flag) {VvcOperati ngPoi ntsRecord op J nfo; / / specified in ISO / I EC 14496-15

[0492] The semantics of VvcOperatingPointsRecord are specified in ISO / IEC 14496-15.

[0493] When included in VvcOperatingPointsinformationProperty, the values of the syntax elements of VvcOperatingPointsRecord are constrained as follows:

[0494] - frame_rate_info_flag shall be equal to 0. Consequently, avg_frame_rate and constant_frame_rate are not present and their semantics are not specified.

[0495] - bit_rate_info_flag shall be equal to 0. Consequently, max_bit_rate and avg_bit_rate are not present and their semantics are not specified.

[0496] - ptl_max_temporal_id[i] for each value of i in the range of 0 to num_profile_tier_level_minus1, inclusive, shall be equal to 0.

[0497] max_temporal_id shall be equal to 0

[0498] In an alternate example embodiment, the information present in the VvcOperatingPointsInformationProperty is conditionally present within the CompactVvcDecoderConfigurationRecord by using a new flag as defined below.aligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;} / / other information within the CompactVvcDecoderConfigurationRecordif (multi_layer_flag) {unsigned int(1) operating j)ointjnfo_present_flag;bit(7) reserved = 0;if(operating_point_info_present_flag){VvcOperati ngPoi ntsRecord op J nfo; / / specified in ISO / I EC 14496-15}}}

[0499] In an embodiment, when operating_point_info_present_flag is set to 1, operating_point_info_present_flag indicates that the VvcOperatingPointsRecord is present in the CompactVvcDecoderConfigurationRecord. When operating_point_info_present_flag is set to 0, operating_point_info_present_flag indicates that the VvcOperatingPointsRecord is not present in the CompactVvcDecoderConfigurationRecord.

[0500] In an embodiment, when multi Jayer_flag is equal to 1 and if the VvcOperatingPointsRecord is not present in the CompactVvcDecoderConfigurationRecord then the information in the VvcOperatingPointsRecord is reconstructed from the VPS NAL unit present in the CompactVvcDecoderConfigurationRecord.

[0501] In an embodiment, when multi Jayer_flag is equal to 1 and if the VvcOperatingPointsRecord is not present in the CompactVvcDecoderConfigurationRecord then the information in the VvcOperatingPointsRecord is reconstructed from the VPS NAL unit present in the CompactVvcDecoderConfigurationRecord with some examples as follows:- VvcOperatingPointsRecord. num_profile_tier_level_minus1 = VPS.vps_num_ptls_minus1- VvcOperatingPointsRecord.ptl_max_temporal_id[i] = VPS. vps_ptl_max_tid[ i ]- VvcOperatingPointsRecord.ptl[i] = VPS. profile_tier_level( vps_pt_present_flag[ i ], vps_ptl_max_tid[ i ] )- VvcOperatingPointsRecord.allJndependent_layers_flag = VPS.vps_all_independent_layers_flag - VvcOperatingPointsRecord.each_layer_is_an_ols_flag = VPS.vps_each_layer_is_an_ols_flag - VvcOperatingPointsRecord.oIsjnodeJdc = VPS.olsModeldc- VvcOperatingPointsRecord.incomplete_operating_points_flag = VPS:VvcOperatingPointsRecord. num_ref_sublayer_for_ols_present_flag;VvcOperatingPointsRecord.num_olss = VPS.TotalNumOlssVvcOperatingPointsRecord. ptl Jdx[i] = VPS. vps_ols_ptl_idx[ i ]VvcOperatingPointsRecord. VvcPTLRecord.num_bytes_constraint_info;- If p rofi leTierP rese ntFlag is set to 1 , then VvcOperatingPointsRecord.VvcPTLRecord.generaLprofileJdc =VPS.profile_tier_level.general_profileJdc- VvcOperatingPointsRecord.VvcPTLRecord. general Jier_flag = VPS.profile_tier_level.general_tier_flag- VvcOperatingPointsRecord.VvcPTLRecord.generalJevelJdc = VPS.profile_tier_level.generaljeveljdc- VvcOperatingPointsRecord.VvcPTLRecord.pt! Jrame_only_constraint_flag = VPS.profile_tier_level. ptl_frame_only_constraint_flag- VvcOperatingPointsRecord.VvcPTLRecord.ptl_multi_layer_enabled_flag = VPS.profile_tier_level.ptl_multilayer_enabled_flag- VvcOperatingPointsRecord.VvcPTLRecord.general_constraint_info = VPS.profile_tier Jevel. general_constraints_info( )- for (i=num_sublayers - 2; i >= 0; i--) VvcOperatingPointsRecord.VvcPTLRecord.ptl_sublayer_level_present_flag[i] = VPS.profile_tierJevel.ptl_sublayerJevel_present_flag[ i ]- for (j=num_sublayers; j<=8 && num_sublayers > 1 ; j++) set,- VvcOperatingPointsRecord.VvcPTLRecord.ptl_reserved_zero_bit = 0;- for (i=num_sublayers-2; i >= 0; i--) and if (ptl_sublayer _level_present_fl ag [i] is set to 1 VvcOperatingPointsRecord.VvcPTLRecord.sublayer_level_idc[i] = VPS.profile_tier_level. sublayerjevel Jdc[ i ]- VvcOperatingPointsRecord.VvcPTLRecord.ptl_num_sub_profiles = VPS.profile_tier_level.ptl_num_sub_profiles. ptl_num_sub_profiles- for 0=0; j < ptl_num_sub_profiles; j++) set VvcOperatingPointsRecord.VvcPTLRecord.general_sub_profile_idc[j] = VPS.profile_tier_level. general_sub_profile_idc[ i ]

[0502] In an embodiment, files that include 'mif3' as the major_brand and the brand 'vvi3' as the minor_version (wherein the 'vvi3' brand defines the coded image item type to be 'vvd ' as defined in L.2.2.1.2 of the HEIF specification) in the FileTypeBox comply with the specifications in L.2 (HEIF specification) and additionally be constrained as follows: image items of type 'vvd ' containing multiple layers do not have an associated item property TargetOlsProperty. The information present in the TargetOlsProperty is embedded within the CompactVvcDecoderConfigurationRecord.

[0503] In an example embodiment, the information present in the TargetOlsProperty embedded within the CompactVvcDecoderConfigurationRecord is defined below.aligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;} / / other information within the CompactVvcDecoderConfigurationRecordif (multi_layer_flag) {unsigned int(16) target_ols_idx;}}

[0504] In an embodiment, when multi Jayer_flag is equal to 1 , the target_ols_idx provides the output layer set index to be used as input for the decoding process of WC coded image item. The target_ols_idx is used as the value of the TargetOlsIdx variable and is specified in the same coding format as in ISO / IEC 23090-3. The 'vvd ' image item includes the layers that are included in the output layer set identified by the associated target_ols_idx in the CompactVvcDecoderConfigurationRecord and not include other layers.

[0505] The target_ols_idx in the CompactVvcDecoderConfigurationRecord, when present, is same as the olsjdx value (when ptl_present_flag is equal to 1) of the CompactVvcDecoderConfigurationRecord.

[0506] In an alternate embodiment, when multi Jayer_flag is equal to 1 , the target_olsjdx is not present in the CompactVvcDecoderConfigurationRecord and is infered to be equal to the olsjdx value (when ptl_present_flag is equal to 1) of the CompactVvcDecoderConfigurationRecord.

[0507] In an alternate example embodiment, the information present in the TargetOlsProperty is conditionally present within the CompactVvcDecoderConfigurationRecord by using a new flag as defined below.aligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;} / / other information within the CompactVvcDecoderConfigurationRecordif (multi_layer_flag) {unsigned int(1) target_olx_idx_present_flag;bit(7) reserved = 0;if(target_olx_idx_present_flag){unsigned int(16) target_ols_idx;}}}

[0508] In an embodiment, when target_olx_idx_present_flag is equal to 1, the target_ols_idx value is present in the CompactVvcDecoderConfigurationRecord. When target_olx_idx_present_flag is equal to 0, the target_ols_idx value is not present in the CompactVvcDecoderConfigurationRecord.

[0509] In an embodiment, the target_ols_idx in the CompactVvcDecoderConfigurationRecord, when not present is set to be equal to the lowest output layer set, OLS, index that contains the largest number of layers among all OLSs specified by the VPS and the largest number of output layers among the OLSs with the largest number of layers. In an embodiment, files that include 'mif3' as the major_brand and the brand 'vvi3' as the minor_version (wherein the 'vvi3' brand defines the coded image item type to be 'vvd ' as defined in L.2.2.1.2 of the HEIF specification) in the FileTypeBox comply with the specifications in L.2 (HEIFspecification) and additionally be constrained as follows: image items of type 'vvd ' containing multiple layers do not have an associated item property LayerSelectorProperty. The information present in the LayerSelectorProperty is embedded within the CompactVvcDecoderConfigurationRecord.

[0510] In an example embodiment, the information present in the LayerSelectorProperty embedded within the CompactVvcDecoderConfigurationRecord is defined below.aligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;} / / other information within the CompactVvcDecoderConfigurationRecordif (multi_layer_flag) {unsigned int(16) layerjd;}}

[0511] In an embodiment, when multi Jayer_flag is equal to 1, the layerjd provides the layer ID that is among the nuh JayerJd values of the output layers of the output layer set identified by target_ols Jdx or the olsjdx value (when ptl_present_flag is equal to 1) of the CompactVvcDecoderConfigurationRecord.

[0512] In an alternate example embodiment, when multi Jayer_flag is equal to 1 , the information present in the LayerSelectorProperty is conditionally present within the CompactVvcDecoderConfigurationRecord by the presence of a new flag as defined below.aligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;} / / other information within the CompactVvcDecoderConfigurationRecordif (multi_layer_flag) {unsigned int(1) layerselector_info_present_flag;bit(7) reserved = 0;if(layerselector_info_present_flag)unsigned int(16) layerjd;}}

[0513] In an embodiment, when layerselector_info_present_flag is equal to 1, layerselector_info_present_flag indicates that the layerjd value for layer selection is present in the CompactVvcDecoderConfigurationRecord. When layerselectorjnfo_present_flag is equal to 0, layerselectorjnfo_present_flag indicates that the layerjd value for layer selection is not present in the CompactVvcDecoderConfigurationRecord.

[0514] In an embodiment, when pictures from multiple layers are present in the files that include 'mif3' as the major_brand and the brand 'vvi3' as the minor_version (wherein the 'vvi3' brand defines the coded image item type to be 'vvd ' as defined in L.2.2.1.2 of the HEIF specification) in the FileTypeBox comply with the specifications in L.2 (HEIF specification) then multiple SPS and PPS NAL units may be present in the CompactVvcDecoderConfigurationRecord.

[0515] In an embodiment, when multi Jayer_flag is equal to 1, and multiple SPS and PPS NAL units are present in the CompactVvcDecoderConfigurationRecord, the syntax is defined as follows.aligned(8) class CompactVvcDecoderConfigurationRecord { unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native_ptl.ptl_multilayer_enabled_flag == 1) unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;}if (multi Jayer_flag == 0 1| nal_units_present_flag) { unsigned int(1) additional_nal_unit_flag;if (additional_nal_unit_flag) {unsigned int(3) num_aps_nal_unit;unsigned int(3) num_sei_nal_unit;}if (multi Jayer_flag){unsigned int(1) array_completeness;unsigned int(3) num_sps_nal_unit;unsigned int(3) num_pps_nal_unit;}else{num_sps_nal_unit = 1;num_pps_nal_unit = 1;}trailing_bits();if (multi _layer_flag) {unsigned int(8) vps_nal_unit_length; bit(8*vps_nal_unit_length) vps_nal_unit;}for (i=0; i< num_sps_nal_unit; i++) {unsigned int(8) sps_nal_unit_length;bit(8*sps_nal_unit_length) sps_nal_unit;}for (i=0; i< num_pps_nal_unit; i++) {unsigned int(8) pps_nal_unit_length;bit(8*pps_nal_unit_length) pps_nal_unit;}if (additional_nal_unit_flag) {for (i=0; i< num_aps_nalus; i++) {unsigned int(8) aps_nal_unit_length;bit(8*aps_nal_unit_length) aps_nal_unit;}for (i=0; i< num_sei_nalus; i++) {unsigned int(8) sei_nal_unit_length;bit(8*sei_nal_unit_length) sei_nal_unit;}}}}

[0516] num_sps_nal_unit: indicates the number of SPS NAL units included in the configuration record.

[0517] num_pps_nal_unit: indicates the number of PPS NAL units included in the configuration record.

[0518] In an alternate embodiment, the CompactVvcDecoderConfigurationRecord does not contain explicit indication of different NAL units present in it but only contains the indication of number of NAL units present in it as follows:aligned(8) class CompactVvcDecoderConfigurationRecord {unsigned int(1) multi Jayer_flag;unsigned int(2) lengthSizeMinusOne;if (multi_layer_flag) {unsigned int(1) ptl_present_flag;if (ptl_present_flag) {VvcPTLRecord(l) native_ptl;if (native j)tLptl_multilayer_enabled_flag == 1)unsigned int(3) olsjdx;}unsigned int(1) nal_units_present_flag;}if (multi Jayer_flag == 0 1| nal_units_present_flag) {unsigned int(8) num_nal_units;for (i=0; i< num_nal_units; i++) {unsigned int(1) array_completeness;bit(7) reserved = 0;unsigned int(8) nal_unit_length;bit(8*nal_unit_length) nal_unit;}}}

[0519] In an embodiment, num_nal_units indicates the number of NAL units present in the CompactVvcDecoderConfigurationRecord. When array_completeness is set to 0, array_completeness indicates that the NAL unit may be present in item data. array_completeness when set to 1 indicates that the NAL unit is only present in the CompactVvcDecoderConfigurationRecord and not present in the item data. In an embodiment, as a response to 'mif3' as the major_brand and the brand 'vvi3' as the minor_version in the FileTypeBox, the readers treat the file as if an equivalent file were created containing FileTypeBox with 'vvic' as the equivalent major_brand and the equivalent MetaBox derived with the following additional operations: infer infe_type to be equal to 'vvcT; infer codec_config_type to be equal to 'vvcC; expand the data of each codec configuration property from the compact syntax to the full structure; when multi_layer_flag is equal to 1, expand the data of VvcOperatingPointsInformationProperty property from the compact syntax to the full structure as defined below; when multi_layer_flag is equal to 1, expand the data of TargetOlsProperty property from the compact syntax to the full structure as defined below; when multi _layer_flag is equal to 1, expand the data of LayerSelectorProperty property from the compact syntax to the full structure as defined below.

[0520] When multi_layer_flag is equal to 0, the readers reconstruct a VVC bitstream by concatenating all the NAL units present in the CompactVvcDecoderConfigurationRecord together with concatenation of the reconstructed NAL unit header and the NAL unit payload in the WCItemData in the order as defined in in H.266| ISO / IEC 23090-3. Wherein, the NAL unit header is reconstructed by setting the following values.

[0521] The NAL unit header is the WC NAL unit header as defined in H.266| ISO / IEC 23090-3 with values as set below.

[0522] When multi _layer_flag is equal to 1 , the readers reconstruct a VVC bitstream by concatenating all the NAL units present in the CompactVvcDecoderConfigurationRecord together with all the NAL units in the WCItemData in the order as defined in in H.266| ISO / IEC 23090-3.

[0523] Equivalence with the VVC decoder configuration

[0524] In an embodiment, CompactVvcDecoderConfigurationRecord is considered equivalent to VvcDecoderConfigurationRecord as defined in ISO / IEC 14496-15 with the following fields:

[0525] - When ptl_present_flag is present and set to 1 and if the codec configuration property is associated with the gainmap image item: chroma_format_idc is set to the value of the gainmap_chroma_subsampling field from the MinimizedlmageBox.

[0526] In an embodiment, when multi _layer_flag is set to 1 and nal_units_present_flag is set to 0 then all the NAL units needed to reconstruct and decode the WC bitstream is present in the alpha_item_data, if the codec configuration property is associated with the alpha image item, or is present in the gainmap_item_data, if the codec configuration property is associated with the gainmap image item or is present in the main_item_data if the codec configuration property is associated with the main image item.

[0527] In an embodiment, when multi _layer_flag is set to 1 and nal_units_present_flag is set to 1 and if array_completeness is set to 1, then it indicates that the following NAL units may be in the CompactVvcDecoderConfigurationRecord and none are present in the item data field of the corresponding associated item in the MinimizedlmageBox; when equal to 0 indicates that all or a subset of the following NAL units may be present in the item data field of the corresponding associated item in the MinimizedlmageBox; NAL_unit_type set to 14 (VPS_NUT as defined in ISO / IEC 23090-3), NAL_unit_type set to 15 (SPS_NUT as defined in ISO / IEC 23090-3), NAL_unit_type set to 16 (PPS_NUT as defined in ISO / IEC 23090-3), NAL_unit_type set to 17 (PREFIX_APS_NUT as defined in ISO / IEC 23090-3), NAL_unit_type set to 23 (PREFIX_SEI_NUT as defined in ISO / IEC 23090-3).

[0528] In an embodiment, when multi _layer_flag is set to 1 and nal_units_present_flag is set to 1, the number of SPS and PPS NAL units in the reconstructed WC bitstream may be more than:

[0529] - The number of SPS and PPS NAL units in the CompactVvcDecoderConfigurationRecord may be only one additionally the array_completeness is set to 0 indicating that the additional SPS and PPS NAL unit may be present in the item data field of the corresponding associated item in the MinimizedlmageBox or

[0530] - The number of SPS and PPS NAL units in the CompactVvcDecoderConfigurationRecord may be more than one additionally the array_completeness is set to 1 indicating that the additional SPS and PPS NAL unit is not present in the item data field of the corresponding associated item in the MinimizedlmageBox or

[0531] - The number of SPS and PPS NAL units in the CompactVvcDecoderConfigurationRecord may be more than one additionally the array_completeness is set to 0 indicating that the additional SPS and PPS NAL unit is present in the item data field of the corresponding associated item in the MinimizedlmageBox however additional SPS and PPS NAL unit are not present in the item data field of the corresponding associated item in the MinimizedlmageBox.

[0532] Equivalence with the WC Operating Points Information

[0533] In an embodiment, the VvcOperatingPointsInformationProperty as defined in ISO / IEC 23008-12 is derived from CompactVvcDecoderConfigurationRecord as follows:

[0534] - If nal_units_present_flag is set to 1 and multi _layer_flag is set to 1, the CompactVvcDecoderConfigurationRecord contains the VvcOperatingPointsRecord which forms the contents / payload of the VvcOperatingPointsInformationProperty.

[0535] - The values of the syntax elements of VvcOperatingPointsRecord are constrained as follows: frame_rate_info_flag shall be equal to 0. Consequently, avg_frame_rate and constant_frame_rate are not present and their semantics are not specified; bit_rate_info_flag shall be equal to 0. Consequently, max_bit_rate and avg_bit_rate are not present and their semantics are not specified; ptl_max_temporal_id [i] for each value of i in the range of 0 to num_profile_tier_level_minus1, inclusive, shall be equal to 0; max_temporal_id shall be equal to 0.

[0536] Equivalence with the WC Target Output Laver Set

[0537] In an embodiment, the TargetOlsProperty as defined in ISO / IEC 23008-12 s derived from CompactVvcDecoderConfigurationRecord as follows:

[0538] - If nal_units_present_flag is set to 1 and multi _layer_flag is set to 1, the CompactVvcDecoderConfigurationRecord contains the target_ols_idx which forms the payload of the TargetOlsProperty

[0539] - The target_ols_idx in the CompactVvcDecoderConfigurationRecord, when present, is same as the olsjdx value (when ptl_present_flag is equal to 1) of the CompactVvcDecoderConfigurationRecord.

[0540] - In an alternate embodiment, nal_units_present_flag is set to 1 and multi _layer_flag is set to 1 , the target_ols_idx is not present in the CompactVvcDecoderConfigurationRecord and is inferred to be equal to the olsjdx value (when ptl_present_flag is equal to 1) of the CompactVvcDecoderConfigurationRecord which forms the payload of the TargetOlsProperty.

[0541] Equivalence with the WC Laver Selection

[0542] In an embodiment, the LayerSelectorProperty as defined in ISO / IEC 23008-12 s derived from CompactVvcDecoderConfigurationRecord as follows:

[0543] - If nal_units_present_flag is set to 1 and multi Jayer_flag is set to 1, the CompactVvcDecoderConfigurationRecord contains the layerjd which forms the payload of the LayerSelectorProperty

[0544] - In an alternate embodiment, nal_units_present_flag is set to 1 and multi Jayer_flag is set to 1 , the the layerjd is not present in the CompactVvcDecoderConfigurationRecord and is infered to be equal to the 0 value indicating the base layer which forms the payload of the LayerSelectorProperty.

[0545] In an embodiment, If nal_units_present_flag is set to 1 and multi Jayer_flag is set to 1 the presence of target_olsjdx in the CompactVvcDecoderConfigurationRecord may be gated by the presence of a flag for example, target_olsjdx_presence_flag. When target_olsjdx_presence_flag is set to 1 then target_olsjdx is present in the CompactVvcDecoderConfigurationRecord. When target_olsjdx_presence_flag is set to 0 then target_olsjdx is not present in the CompactVvcDecoderConfigurationRecord.

[0546] In an embodiment, If nal_units_present_flag is set to 1 and multi Jayer_flag is set to 1 the presence of layerjd in the CompactVvcDecoderConfigurationRecord may be gated by the presence of a flag for example, layerjd_presence_flag. When layerjd_presence_flag is set to 1 then layerjd is present in the CompactVvcDecoderConfigurationRecord. When layerjd_presence_flag is set to 0 then layerjd is not present in the CompactVvcDecoderConfigurationRecord.

[0547] Minimized Image Box changes

[0548] In an embodiment, the MinimizedlmageBox may contain all or a subset of the following items: the main image item or main item, the gain map item, the alpha item, wherein the dimension (width x height) of the main image item is equal to the dimension of the gain map item (if present) or wherein the dimension (width x height) of the main image item is equal to the dimension of the alpha map item (if present) or wherein the dimension (width x height) of the gain map item is equal to the dimension of the alpha map item (if present).

[0549] In an embodiment, in addition to the dimension (width x height) the codec configuration between the items may be same. In an embodiment, it is indicated in a MinimizedlmageBox, for example, using a flag for a second or later item whether its dimensions (width x height) are the same as for a first item, which may be inferred (e.g. be the main item) or may be indicated (e.g. for a third item, it may be indicated if the dimensions are the same as for the first item, the second item, or neither of them).

[0550] In an embodiment, the MinimizedlmageBox may contain all or a subset of the following items: the main image item or main item, the gain map item, the alpha item, wherein the codec configuration data needed to decode the main image item is same as the codec configuration data needed to decode the gain map item (if present) or wherein the codec configuration data needed to decode the main image item is same as the codec configuration data needed to decode the alpha map item (if present) or wherein the codec configuration data needed to decode the gain map item is same as the codec configuration data needed to decode the alpha map item (if present). In an embodiment, it is indicated in a MinimizedlmageBox, for example, using a flag for a second or later item whether its codec configuration data are the same as for a first item, which may be inferred (e.g. be the main item) or may be indicated (e.g. for a third item, it may be indicated if the codec configuration data are the same as for the first item, the second item, or neither of them).

[0551] In an embodiment, the chroma subsampling between the main image item and the gainmap item may be same. In an embodiment, the MinimizedlmageBox may contain all or a subset of the following items: the main image item or main item, the gain map item, wherein the chroma subsampling of the main image item is same as the chroma subsampling of the gain map item (if present). In an example embodiment, the syntax of the MinimizedlmageBox is modified as below to indicate the dimension (width x height), the codec configuration data and the chroma subsampling shared between the item(s) when present. aligned(8) class MinimizedlmageBox extends Box('mini') {bit(2) version = 0; / / flagsbit(1 ) exp I icit_codec_types_fl ag ;bit(1) float_flag;bit(1) full_range_flag;bit(1) alpha_flag;bit(1) explicit_cicp_flag;bit(1) hdr_flag;bit(1) icc_flag;bit(1) exif_flag;bit(1) xmp_flag;bit(2) chroma_subsampling;bit(3) orientation_minus1 ; / / Spatial extentsbit(1) large_dimensions_flag;unsigned int(large_dimensions_flag ? 15 : 7) width_minus1; unsigned int(large_dimensions_flag ? 15 : 7) height_minus1; / / Pixel informationif (chroma_subsampling == 1 || chroma_subsampling == 2) bit(1 ) chroma_is_horizontally_centered;if (chroma_subsampling == 1)bit(1 ) chroma_is_vertically_centered;if (float_flag)bit(2) bit_depth Jog2_minus4;else {bit(1) high_bit_depth_flag;if (high_bit_depth_flag)bit(3) bit_depth_minus9;}if (alpha_flag)bit(1) alpha_is_premultiplied; / / Colour propertiesif (explicit_cicp_flag) {bit(8) colour_primaries;bit(8) transfer_characteristics;if (chroma_subsampling != 0)bit(8) matrix_coefficients;elsematrix_coefficients = 2;}else {colour_primaries = icc_flag ? 2 : 1 ;transfer_ch aracteristics = icc_flag ? 2 : 13;matrix_coefficients = chroma_subsampling == 0 ? 2 : 6;}if (explicit_codec_types_flag) {bit(32) infe_type;bit(32) codec_config_type;} else { / / codec information is defined by the brand in the minor_version field of FileTypeBox } / / High Dynamic Range propertiesif (hdr_flag) {bit(1) gainmap_flag;if (gainmap_flag) {bit(1 ) gainmap dimension sameas main item flag; if(gainmap_dimension_sameas_main_item_flag){unsigned int(large_dimensions_flag ? 15 : 7) gainmap_width_minus1 = width_minus1;unsigned int(large_dimensions_flag ? 15 : 7) gainmap_height_minus1= height minusl ;}else{unsigned int(large_dimensions_flag ? 15 : 7) gainmap_width_minus1; unsigned int(large_dimensions_flag ? 15 : 7) gainmap_height_minus1 ;}bit(8) gainmap_matrix_coefficients;bit(1) gainmap_full_range_flag;bit(1) gainmap chroma subsampling sameas main item flag; if(gainmap_chroma_subsampling_sameas_main_item_flag){bit(2) gainmap chroma subsampling = chroma subsampling;if (gainmap_chroma_subsampling == 1|| gainmap_chroma_subsampling == 2)bit(1 ) gainmap_chroma_is_horizontally_centered= chroma_is_horizontally_centered;if (gainmap_chroma_subsampling == 1)bit(1 ) gainmap_chroma_is_vertically_centered = chroma_is_vertically_centered;}else{bit(2) gainmap chroma subsampling;if (gainmap_chroma_subsampling == 1|| gainmap_chroma_subsampling == 2)bit(1) gainmap_chroma_is_horizontally_centered; if (gainmap_chroma_subsampling == 1)bit(1 ) gainmap_chroma_is_vertically_centered; }bit(1) gainmap_float_flag;if (gainmap_float_flag)bit(2) gainmap_bit_depth_log2_minus4;else {bit(1 ) g ai nmap_h ig h_bit_depth_fl ag ;if (gainmap Jiigh_bit_depth_flag)bit(3) gainmap_bit_depth_minus9;}bit(1) tmap_icc_flag;bit(1) tmap_explicit_cicp_flag;if (tmap_explicit_cicp_flag) {bit(8) tmap_colour_primaries;bit(8) tmap_transfer_characteristics;bit(8) tmap_matrix_coefficients;bit(1) tmap_full_range_flag;}else {tmap_colour_primaries = 1 ; tmap_transfer_characteristics = 13; tmap_matrix_coefficients = 6;tmapjull angejlag = 1;}}bit(1) cllijlag;bit(1) mdcv_flag;bit(1) cclvjlag;bit(1) amve_flag;bit(1) revejlag;bit(1) ndwt_flag;if (cllijlag)ContentLightLevel clli;if (mdcvjlag)MasteringDisplayColourVolume mdcv;if (ccl vjlag)ContentColourVolume cclv;if (amve Jlag)AmbientViewingEnvironment amve;if (revejlag)ReferenceViewingEnvironment reve;if (ndwtjlag)NominalDiffuseWhite ndwt;if (gainmap Jlag) {bit(1) tmap_clli_flag;bit(1) tmapjndcv Iag;bit(1) tmap_cclv_flag;bit(1) tmap_amve lag;bit(1) tmap_reve_flag;bit(1) tmap_ndwt_flag;if (tmap_clli Jlag)ContentLightLevel tmap_clli;if (tmapjndcvJIag) MasteringDisplayColourVolume tmapjndcv; if (tmap_cclv_flag)ContentColourVolume tmap_cclv;if (tmap_amve_flag)AmbientViewingEnvironment tmap_amve;if (tmap_reve_flag)ReferenceViewingEnvironmenttmap_reve;if (tmap_ndwt_flag)NominalDiffuseWhite tmap_ndwt;}} / / Chunk sizesif (icc_flag || exif_flag || xmp_flag || (hdr_flag && gainmap_flag))bit(1) large_metadata_flag;bit(1) large_codec_config_flag;bit(1) large_item_data_flag;if (icc_flag)unsigned int(large_metadata_flag ? 20 : 10) icc_data_size_minus1;if (hdr_flag && gainmap_flag && tmap_icc_flag)unsigned int(large_metadata_flag ? 20 : 10) tmap_icc_data_size_minus1; unsigned int(large_codec_config_flag ? 12 : 3) main_item_codec_config_size; unsigned int(large_item_data_flag ? 28 : 15) main_item_data_size_minus1; if (hdr_flag && gainmap_flag)unsigned int(large_metadata_flag ? 20 : 10) gainmap_metadata_size;if (hdr_flag && gainmap_flag)unsigned int(large_item_data_flag ? 28 : 15) gainmap Jtem_data_size;if (hdr_flag && gainmap_flag && gainmap Jtem_data_size > 0){bit(1) gainmap_codec_config_sameas_main_item_flag; if(gainmap_codec_config_sameas_main_item_flag){unsigned int(large_codec_config_flag ? 12 : 3)gainmap Jtem_codec_config_size = main_item_codec_config_size;}else{unsigned int(large_codec_config_flag ? 12 : 3)gainmap Jtem_codec_config_size;}}if (alpha_flag)unsigned int(large_item_data_flag ? 28 : 15) alpha_item_data_size;if (alpha_flag && alpha_item_data_size > 0){bit(1 ) alpha_item_codec_config_sameas_gainmap_flag; if(alpha_item_codec_config_sameas_gainmap_flag){unsigned int(large_codec_config_flag ? 12 : 3) alpha_item_codec_config_size = gainmap Jtem_codec_config_size;else{unsigned int(large_codec_config_flag ? 12 : 3) alpha_item_codec_config_size; }}if (exif_flag)unsigned int(large_metadata_flag ? 20 : 10) exif_data_size_minus1;if (xmp_flag)unsigned int(large_metadata_flag ? 20 : 10) xmp_data_size_minus1;trailing_bits(); / / bit padding till byte alignment / / Chunksif (main_item_codec_config_size > 0)unsigned int(8) main_item_codec_config[main_item_codec_config_size];if (hdr_flag && gainmap_flag && gainmap Jtem_codec_config_size > 0 && !gainmap_codec_config_sameas_main_item_flag){unsigned int(8) gainmap Jtem_codec_config[gainmap_item_codec_config_size];}else if(hdr_flag && gainmap_flag && gainmap Jtem_codec_config_size > 0 && gainmap_codec_config_sameas_main_item_flag){unsigned int(8) gainmap Jtem_codec_config[gainmap_item_codec_config_size] main_item_codec_config;}if (alpha_flag && alpha_item_data_size > 0 && alpha_item_codec_config_size > 0 && !alpha_item_codec_config_sameas_gainmap_flag){unsigned int(8) alpha_item_codec_config[alpha_item_codec_config_size];}else if(alpha_f lag && alpha_item_data_size > 0 && alpha_item_codec_config_size > 0 && alpha_item_codec_config_sameas_gainmap_flag){unsigned int(8) alpha_item_codec_config[alpha_item_codec_config_size] = gainmap Jtem_codec_config;}if (icc_flag)unsigned int(8) icc_data[icc_data_size_minus1 + 1];if (hdr_flag && gainmap_flag && tmap_icc_flag)unsigned int(8) tmap_icc_data[tmap_icc_data_size_minus1 + 1];if (hdr_flag && gainmap_flag && gainmap_metadata_size > 0)unsigned int(8) gainmap jrietadata[gainmap_metadata_size];if (alpha_flag && alpha_item_data_size > 0)unsigned int(8) alpha_item_data[alpha_item_data_size];if (hdr_flag && gainmap_flag && gainmap Jtem_data_size > 0)unsigned int(8) gainmap Jtem_data[gainmap_item_data_size];unsigned int(8) main_item_data[main_item_data_size_minus1 + 1];if (exif_flag)unsigned int(8) exif_data[exif_data_size_minus1 + 1];if (xmp_flag)unsigned int(8) xmp_data[xmp_data_size_minus1 + 1];}

[0552] In an embodiment, the semantics of the following parameters are defined below.

[0553] gainmap_dimension_sameas_main Jtem_flag when set to 1 indicates that the dimensions of the gainmap item is same as the dimensions of the main image item. The parameters gainmap_width_minus1 and gainmap_height_minus1 are infered and set equal to widthjninusl and heightjninusl, respectively. When gainmap jfimension_sameas_main_item_flag is set to 0, indicates that the dimensions of the gainmap item is not same as the dimensions of the main image item. The parameters gainmap_width_minus1 and gainmap_height_minus1 are signaled in the MinimizedlmageBox.

[0554] gainmap_chroma_subsampling_sameas_main Jtem_flag when set to 1 indicates that the chroma subsampling of the gainmap item is same as the chroma subsampling of the main image item. The parameters gainmap_chroma_subsampling, gainmap_chroma_is_horizontally_centered andgainmap_chroma_is_vertically_centered are infered and set equal to chroma_subsampling, chroma_is_horizontally_centered and chroma_is_vertically_centered, respectively. When gainmap_chroma_subsampling_sameas_main Jtem_flag is set to 0, indicates that the chroma subsampling of the gainmap item is not same as the chroma subsampling of the main image item. The parameters gainmap_chroma_subsampling, gainmap_chroma_is_horizontally_centered and gainmap_chroma_is_vertically_centered are signaled in the MinimizedlmageBox.

[0555] gainmap_codec_config_sameas_main_item_flag when set to 1, indicates that the codec configuration data of the gainmap item is same as the codec configuration of the main image item. The parameters gainmap Jtem_codec_config_size and gainmap Jtem_codec_config are inferred and set equal to main_item_codec_config_size and main_item_codec_config, respectively. gainmap_codec_config_sameas_main_item_flag when set to 0, indicates that the codec configuration data of the gainmap item is not same as the codec configuration of the main image item. The parameters gainmap Jtem_codec_config_size and gainmap Jtem_codec_config are signalled in the MinimizedlmageBox.

[0556] alpha_item_codec_config_sameas_gainmap_flag when set to 1, indicates that the codec configuration data of the alpha item is same as the codec configuration of the gainmap item The parameters alpha_item_codec_config_size and alpha_item_codec_config are inferred and set equal to gainmap Jtem_codec_config_size and gainmap Jtem_codec_config, respectively. alpha_item_codec_config_sameas_gainmap_flag when set to 0, indicates that the codec configuration data of the alpha item is not same as the codec configuration of the gainmap item The parameters alpha_item_codec_config_size and alpha_item_codec_config are signalled in the MinimizedlmageBox.

[0557] In an alternate example embodiment, the syntax of the MinimizedlmageBox is modified as below to indicate the dimension (width x height) shared between the item(s) when present, and the codec configuration data shared between the item(s) when present.aligned(8) class MinimizedlmageBox extends Box('mini') {bit(2) version = 0; / / flagsbit(1 ) exp I icit_codec_types_fl ag ;bit(1) float_flag;bit(1) full_range_flag;bit(1) alpha_flag;bit(1) explicit_cicp_flag;bit(1) hdr_flag;bit(1) icc_flag;bit(1) exif_flag;bit(1) xmp_flag;bit(2) chroma_subsampling;bit(3) orientation_minus1 ; / / Spatial extentsbit(1) large_dimensions_flag;unsigned int(large_dimensions_flag ? 15 : 7) width_minus1; unsigned int(large_dimensions_flag ? 15 : 7) height_minus1; / / Pixel informationif (chroma_subsampling == 1 || chroma_subsampling == 2) bit(1 ) chroma_is_horizontally_centered;if (chroma_subsampling == 1)bit(1 ) chroma_is_vertically_centered;if (float_flag)bit(2) bit_depth Jog2_minus4;else {bit(1) high_bit_depth_flag;if (high_bit_depth_flag)bit(3) bit_depth_minus9;}if (alpha_flag)bit(1) alpha_is_premultiplied; / / Colour propertiesif (explicit_cicp_flag) {bit(8) colour_primaries;bit(8) transfer_characteristics;if (chroma_subsampling != 0)bit(8) matrix_coefficients;elsematrix_coefficients = 2;}else {colour_primaries = icc_flag ? 2 : 1 ;transfer_ch aracteristics = icc_flag ? 2 : 13;matrix_coefficients = chroma_subsampling == 0 ? 2 : 6;}if (explicit_codec_types_flag) {bit(32) infe_type;bit(32) codec_config_type;} else { / / codec information is defined by the brand in the minor_version field of FileTypeBox } / / High Dynamic Range propertiesif (hdr_flag) {bit(1) gainmap_flag;if (gainmap_flag) {bit(1 ) gainmap dimension sameas main item flag; if(gainmap_dimension_sameas_main_item_flag){unsigned int(large_dimensions_flag ? 15 : 7) gainmap_width_minus1 = width_minus1;unsigned int(large_dimensions_flag ? 15 : 7)gainmap_height_minus1= height minusl ;}else{unsigned int(large_dimensions_flag ? 15 : 7) gainmap_width_minus1; unsigned int(large_dimensions_flag ? 15 : 7) gainmap_height_minus1 ;}bit(8) gainmap_matrix_coefficients;bit(1) gainmap_full_range_flag;bit(1) gainmap chroma subsampling sameas main item flag; if(gainmap_chroma_subsampling_sameas_main_item_flag){bit(2) gainmap chroma subsampling = chroma subsampling;if (gainmap_chroma_subsampling == 1|| gainmap_chroma_subsampling == 2)bit(1 ) gainmap_chroma_is_horizontally_centered = chroma_is_horizontally_centered;if (gainmap_chroma_subsampling == 1)bit(1 ) gainmap_chroma_is_vertically_centered = chroma_is_vertically_centered;}else{bit(2) gainmap chroma subsampling;if (gainmap_chroma_subsampling == 1|| gainmap_chroma_subsampling == 2)bit(1) gainmap_chroma_is_horizontally_centered; if (gainmap_chroma_subsampling == 1) bit(1 ) gainmap_chroma_is_vertically_centered; }bit(1) gainmap_float_flag;if (gainmap_float_flag)bit(2) gainmap_bit_depth_log2_minus4;else {bit(1 ) g ai nmap_h ig h_bit_depth_fl ag ;if (gainmap Jiigh_bit_depth_flag)bit(3) gainmap_bit_depth_minus9;}bit(1) tmap_icc_flag;bit(1) tmap_explicit_cicp_flag;if (tmap_explicit_cicp_flag) {bit(8) tmap_colour_primaries;bit(8) tmap_transfer_characteristics;bit(8) tmap_matrix_coefficients;bit(1) tmap_full_range_flag;}else {tmap_colour_primaries = 1 ; tmap_transfer_characteristics = 13; tmap_matrix_coefficients = 6; tmap_full_range_flag = 1;}}bit(1) cllLflag;bit(1) mdcv_flag;bit(1) cclv_flag;bit(1) amve_flag;bit(1) reve_flag;bit(1) ndwt_flag;if (clli_flag)ContentLightLevel clli;if (mdcv_flag) MasteringDisplayColourVolume mdcv; if (ccl v_flag)ContentColourVolume cclv;if (amve_flag) AmbientViewingEnvironment amve; if (reve_flag) ReferenceViewingEnvironment reve; if (ndwt_flag)NominalDiffuseWhite ndwt;if (gainmap_flag) {bit(1) tmap_clli_flag;bit(1) tmap_mdcv_flag;bit(1) tmap_cclv_flag;bit(1) tmap_amve_flag;bit(1) tmap_reve_flag;bit(1) tmap_ndwt_flag;if (tmap_clli_flag)ContentLightLevel tmap_clli;if (tmap_mdcv_flag)MasteringDisplayColourVolume tmapjndcv;if (tmap_cclv_flag)ContentColourVolume tmap_cclv;if (tmap_amve_flag)AmbientViewingEnvironment tmap_amve;if (tmap_reve_flag)ReferenceViewingEnvironmenttmap_reve;if (tmap_ndwt_flag)NominalDiffuseWhite tmap_ndwt;}} / / Chunk sizesif (icc_flag || exif_flag || xmp_flag || (hdr_flag && gainmap_flag))bit(1) large_metadata_flag;bit(1) large_codec_config_flag;bit(1) large_item_data_flag;if (icc_flag)unsigned int(large_metadata_flag ? 20 : 10) icc_data_size_minus1;if (hdr_flag && gainmap_flag && tmap_icc_flag)unsigned int(large_metadata_flag ? 20 : 10) tmap_icc_data_size_minus1; if (hdr_flag && gainmap_flag)unsigned int(large_metadata_flag ? 20 : 10) gainmap_metadata_size;if (hdr_flag && gainmap_flag)unsigned int(large_item_data_flag ? 28 : 15) gainmap Jtem_data_size;if (hdr_flag && gainmap_flag && gainmap Jtem_data_size > 0){bit(1) gainmap_codec_config_sameas_main_item_flag;unsigned int(large_codec_config_flag ? 12 : 3)gainmap Jtem_codec_config_size;}if (gainmap_codec_config_sameas_main_item_flag){unsigned int(large_codec_config_flag ? 12 : 3) main_item_codec_config_size = gainmap_item_codec_config_size;}else{unsigned int(large_codec_config_flag ? 12 : 3) main_item_codec_config_size;}unsigned int(large_item_data_flag ? 28 : 15) main_item_data_size_minus1;if (alpha_flag)unsigned int(large_item_data_flag ? 28 : 15) alpha_item_data_size;if (alpha_flag && alpha_item_data_size > 0){bit(1 ) alpha_item_codec_config_sameas_gainmap_flag;bit(1 ) alpha_item_codec_config_sameas_main Jtem_flag; if(alpha_item_codec_config_sameas_gainmap_flag){unsigned int(large_codec_config_flag ? 12 : 3) alpha_item_codec_config_size = gainmap Jtem_codec_config_size;elseif (alpha_item_codec_config_sameas_main_item_flag){unsigned int(large_codec_config_flag ? 12 : 3) alpha_item_codec_config_size = main_item_codec_config_size;}else{unsigned int(large_codec_config_flag ? 12 : 3) alpha_item_codec_config_size; }}if (exif_flag)unsigned int(large_metadata_flag ? 20 : 10) exif_data_size_minus1;if (xmp_flag)unsigned int(large_metadata_flag ? 20 : 10) xmp_data_size_minus1; trailing_bits(); / / bit padding till byte alignment / / Chunksif (alpha_flag && alpha_item_data_size > 0 && alpha_item_codec_config_size > 0) unsigned int(8) alpha_item_codec_config[alpha_item_codec_config_size];if (hdr_flag && gainmap_flag && gainmap Jtem_codec_config_size > 0 && !alpha_item_codec_config_sameas_gainmap_flag){unsigned int(8) gainmap Jtem_codec_config[gainmap_item_codec_config_size]; }else if(hdr_flag && gainmap_flag && gainmap Jtem_codec_config_size > 0 && alpha_item_codec_config_sameas_gainmap_flag){unsigned int(8) gainmap Jtem_codec_config[gainmap_item_codec_config_size] alpha_item_codec_config;}if (main_item_codec_config_size > 0 && !alpha_item_codec_config_sameas_main_item_flag && I gainmap_codec_config_sameas_main_item_flag){unsigned int(8) main_item_codec_config[main_item_codec_config_size];} else if(main_item_codec_config_size > 0 && gainmap_codec_config_sameas_main_item_flag){ unsigned int(8) main_item_codec_config[main_item_codec_config_size] =gainmap Jtem_codec_config;} else if(main_item_codec_config_size > 0 && alpha Jtem_codec_config_sameas_main_item_flag){ unsigned int(8) main_item_codec_config[main_item_codec_config_size] = alpha_item_codec_config;}if (icc_flag)unsigned int(8) icc_data[icc_data_size_minus1 +1];if (hdr_flag && gainmap_flag && tmap_icc_flag)unsigned int(8) tmap_icc_data[tmap_icc_data_size_minus1 + 1];if (hdr_flag && gainmap_flag && gainmap_metadata_size > 0)unsigned int(8) gainmap jrietadata[gainmap_metadata_size];if (alpha_flag && alpha_item_data_size > 0)unsigned int(8) alpha_item_data[alpha_item_data_size];if (hdr_flag && gainmap_flag && gainmap Jtem_data_size > 0)unsigned int(8) gainmap Jtem_data[gainmap_item_data_size];unsigned int(8) main_item_data[main_item_data_size_minus1 +1];if (exif_flag)unsigned int(8) exif_data[exif_data_size_minus1 + 1];if (xmp_flag)unsigned int(8) xmp_data[xmp_data_size_minus1 + 1];}

[0558] alpha_item_codec_config_sameas_main_item_flag when set to 1, indicates that the alpha item codec configuration data size (alpha_item_codec_config_size) is equal to the main item configuration data size(main_item_codec_config_size) and the alpha item codec configuration data (alpha_item_codec_config ) is equal to the main item codec configuration data (main_item_codec_config). When alpha_item_codec_config_sameas_main_item_flag when set to 0, indicates that the alpha item codec configuration data size (alpha_item_codec_config_size) is not equal to the main item configuration data size(main_item_codec_config_size) and the alpha item codec configuration data (alpha_item_codec_config ) is not equal to the main item codec configuration data ( main_item_codec_config).

[0559] FIG. 1 shows an encoder 100 according to an embodiment. FIG. 1 illustrates an image to be encoded (In), a predicted representation of an image block (P'n), a prediction error signal (Dn), a reconstructed prediction error signal (D'n), a preliminary reconstructed image (l'n), a final reconstructed image (R'n), a transform (T) and inverse transform (T~1), a quantization (Q) and inverse quantization (Q~1), entropy encoding (E), a reference frame memory (RFM), inter prediction (Pinter), intra prediction (Pmtra), mode selection (MS) and filtering (F). Minimized Image Box Signaling 102 implements the examples described herein related to a method for a low overhead image file format. FIG. 2 shows a decoder 200 according to an embodiment. FIG. 2 illustrates a predicted representation of an image block (P'n), a reconstructed prediction error signal (D'n), a preliminary reconstructed image (l'n), a final reconstructed image (R'n), an inverse transform (T-1), an inverse quantization (Q-1), an entropy decoding (E1), a reference frame memory (RFM), a prediction (either inter or intra) (P), and filtering (F). Minimized Image Box 202 implements the examples described herein related to a method for a low overhead image file format.

[0560] Referring to FIG. 1 , the output of E comprises coded video data. Minimized Image Box signaling 102 is encoded and multiplexed with the coded video data E by a multiplexer 118 to form the final bitstream 120, respecting a proper order. Referring to FIG. 2, one or more file formats are decoded first with file format processing 210, and the one or more decoded file format(s) include a file format comprising the Minimized Image Box 202. R'nis the one or more reconstructed frame(s), that may also be processed together with or by the Minimized Image Box 202. Minimized Image Box 202 is used to generate one or more images 212.

[0561] A video encoder transforms the input video into a compressed representation suited for storage / transmission and a video decoder decompresses the compressed video representation back into a viewable form. Typically, an encoder discards some information in the original video sequence in order to represent the video in a more compact form (that is, at lower bitrate).

[0562] A video encoder may encode the video information in two phases. Firstly, pixel values in a certain picture area (or “block”) are predicted for example by motion compensation means (finding and indicating anarea in one of the previously coded video frames that corresponds closely to the block being coded) or by spatial means (using the pixel values around the block to be coded in a specified manner). Secondly the prediction error, e.g., the difference between the predicted block of pixels and the original block of pixels, is coded. This is typically done by transforming the difference in pixel values using a specified transform (e.g., Discrete Cosine Transform (DCT) or a variant of it), quantizing the coefficients and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (picture quality) and size of the resulting coded video representation (file size or transmission bitrate).

[0563] Inter prediction, which may also be referred to as temporal prediction, motion compensation, or motion-compensated prediction, exploits temporal redundancy. In inter prediction the sources of prediction are previously decoded pictures (a.k.a. reference pictures).

[0564] Intra prediction utilizes the fact that adjacent pixels within the same picture are likely to be correlated. Intra prediction can be performed in spatial or transform domain, e.g., either sample values or transform coefficients can be predicted. Intra prediction is typically exploited in intra coding, where no inter prediction is applied.

[0565] An intra picture may be defined as a coded picture that is decoded using intra prediction only, or in other words, does not make use of inter prediction in decoding. An intra picture may be interchangeably called an intra frame.

[0566] An inter picture may be defined as a coded picture whose decoding may include intra prediction and inter prediction. An inter picture may be interchangeably called an inter frame.

[0567] FIG. 3 is a block diagram illustrating a system 300 in accordance with several examples. In an example, the encoder 330 is used to encode an image or video from the scene 315, which scene 315 may be captured with one or more cameras, and the encoder 330 is implemented in a transmitting apparatus 380. The encoder 330 produces a bitstream 310 comprising signaling that is received by the receiving apparatus 382, which implements a decoder 340. The encoder 330 sends the bitstream 310 that comprises the herein described signaling. The decoder 340 forms the image or video for the scene 315-1, and the receiving apparatus 382 would present this to the user, e.g., via a smartphone, television, or projector among many other options.

[0568] In some examples, the transmitting apparatus 380 and the receiving apparatus 382 are at least partially within a common apparatus, and for example are located within a common housing 350. In other examples the transmitting apparatus 380 and the receiving apparatus 382 are at least partially not within a common apparatus and have at least partially different housings. Therefore in some examples, the encoder 330 and the decoder 340 are at least partially within a common apparatus, and for example are locatedwithin a common housing 350. For example the common apparatus comprising the encoder 330 and decoder 340 implements a codec. In other examples the encoder 330 and the decoder 340 are at least partially not within a common apparatus and have at least partially different housings, but when together still implement a codec.

[0569] In some examples, 3D media from the capture (e.g., volumetric capture) at a viewpoint 312 of the scene 315, which includes a person 313) is converted via projection to a series of two-dimensional, 2D, representations with occupancy, geometry, attributes and / or displacements. Additional atlas information is also included in the bitstream to enable inverse reconstruction. For decoding, the received bitstream 310 is separated into its components with atlas information; occupancy, geometry, displacement, and attribute 2D representations. A 3D reconstruction is performed to reconstruct the scene 315-1 created looking at the viewpoint 312-1 with a “reconstructed” person 313-1. The “-1” are used to indicate that these are reconstructions of the original.

[0570] As indicated at 320, the decoder 340 performs an action or actions based on the received Minimized Image Box signaling.

[0571] In some examples, the scene 315 and reconstructed scene 315-1 are not limited to volumetric outside-in video as indicated by the eye representing viewpoint 312 circling the person 313 and the eye representing viewpoint 312-1 circling the reconstructed person 313-1. For example, the scene 315 may be a 2D frame or an image, and the reconstructed scene 315-1 may be a 2D frame or an image.

[0572] Encoding 390 performs the examples described herein related to encoding of multiple inferences signaling. Decoding 392 performs the examples described herein related to decoding multiple inferences signaling to generate reconstructed scene 315-1.

[0573] FIG. 4 is an example apparatus 400, which may be implemented in hardware, configured to implement the examples described herein. The apparatus 400 comprises at least one processor 402 (e.g., an FPGA and / or central processing unit (CPU) and / or graphics processing unit (GPU)), one or more memories 404 including computer program code 405, the computer program code 405 having instructions to carry out the methods described herein, wherein the at least one memory 404 and the computer program code 405 are configured to, with the at least one processor 402, cause the apparatus 400 to implement circuitry, a process, component, module, or function (implemented with control module 406) to implement the examples described herein.

[0574] Apparatus 400 may be a smartphone, personal digital device or assistant, smart television, laptop, pad, tablet, head-mounted display (HMD), or other user device or terminal device. The memory 404 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a non-volatile memory (e.g., ROM).

[0575] Minimized Image Box 430 implements the examples described herein related to a method for a low overhead image file format.

[0576] The apparatus 400 includes a display and / or input / output (I / O) interface 408, which includes user interface (Ul) circuitry and elements, that may be used to display features or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, etc. The apparatus 400 includes one or more communication e.g. network (N / W) interfaces (l / F(s)) 410. The communication l / F(s) 410 may be wired and / or wireless and communicate over the I nternet / other network(s) via any communication technique including via one or more links 424. The communication l / F(s) 410 may comprise one or more transmitters or one or more receivers.

[0577] The transceiver 416 comprises one or more transmitters 418 and one or more receivers 420. The transceiver 416 and / or communication l / F(s) 410 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator, and encoder / decoder circuitries and one or more antennas, such as antennas 414 used for communication over wireless link 426.

[0578] The control module 406 of the apparatus 400 comprises one of or both parts 406-1 and / or 406-2, which may be implemented in a number of ways. The control module 406 may be implemented in hardware as control module 406-1 , such as being implemented as part of the one or more processors 402. The control module 406-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 406 may be implemented as control module 406-2, which is implemented as computer program code (having corresponding instructions) 405 and is executed by the one or more processors 402. For instance, the one or more memories 404 store instructions that, when executed by the one or more processors 402, cause the apparatus 400 to perform one or more of the operations as described herein. Furthermore, the one or more processors 402, one or more memories 404, and example algorithms (e.g., as flowcharts and / or signaling diagrams), encoded as instructions, programs, or code, are means for causing performance of the operations described herein.

[0579] The apparatus 400 to implement the functionality of control 406 may correspond to any apparatus depicted herein. Alternatively, apparatus 400 and its elements may not correspond to any of the other apparatuses depicted herein. For example apparatus 400 may be part of a self-organizing / optimizing network (SON) node or other node, such as a node in a cloud.

[0580] The apparatus 400 may also be distributed throughout the network including within and between apparatus 400 and any network element (such as a base station and / or terminal device and / or user equipment).

[0581] Interface 412 enables data communication and signaling between the various items of apparatus 400, as shown in FIG. 4. For example, the interface 412 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. Computer program code (e.g. instructions) 405, including control 406 may comprise object-oriented software configured to pass data or messages between objects within computer program code 405. Computer program code (e.g. instructions) 405, including control 406 may comprise procedural, functional, or scripting code. The apparatus 400 need not comprise each of the features mentioned, or may comprise other features as well. The various components of apparatus 400 may at least partially reside in a common housing 428, or a subset of the various components of apparatus 400 may at least partially be located in different housings, which different housings may include housing 428.

[0582] FIG. 5 shows a schematic representation of non-volatile memory media 500a (e.g. computer / compact disc (CD) or digital versatile disc (DVD)) and 500b (e.g. universal serial bus (USB) memory stick) and 500c (e.g. cloud storage for downloading instructions and / or parameters 502 or receiving emailed instructions and / or parameters 1302) storing instructions and / or parameters 502 which when executed by a processor allows the processor to perform one or more of the operations of the methods described herein. Instructions and / or parameters 502 may represent or correspond to a non-transitory computer readable medium.

[0583] FIG. 6 is an example method 600 based on the examples described herein. At 610, the method includes signaling, into a bitstream, information with a file format comprising a minimized image box. At 620, the method includes wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item. At 630, the method includes wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box. Method 600 may be performed with encoder 100, transmitting apparatus 380 with encoder 330, or apparatus 400.

[0584] FIG. 7 is an example method 700 based on the examples described herein. At 710, the method includes decoding, from a bitstream, information from a file format comprising a minimized image box. At 720, the method includes wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item. At 730, the method includes wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha mapitem when present within the minimized image box. Method 700 may be performed with decoder 200, receiving apparatus 382 with decoder 340, or apparatus 400.

[0585] In an embodiment, an apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: signal, into a bitstream, information with a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.In an embodiment, when the main image item or main item and the gain map item are present within the minimized image box, a dimension of the main image item or main item comprising a width of the main image item or main item multiplied by a height of the main image item or main item is equal to a dimension of the gain map item comprising a width of the gain map item multiplied by a height of the gain map item.

[0586] In an embodiment, when the main image item or main item and the alpha map item are present within the minimized image box, a dimension of the main image item or main item comprising a width of the main image item or main item multiplied by a height of the main image item or main item is equal to a dimension of the alpha map item comprising a width of the alpha map item multiplied by a height of the alpha map item.

[0587] In an embodiment, when the gain map item and the alpha map item are present within the minimized image box, a dimension of the gain map item comprising a width of the gain map item multiplied by a height of the gain map item is equal to a dimension of the alpha map item comprising a width of the alpha map item multiplied by a height of the alpha map item.

[0588] A codec configuration of the main image item or main item may be the same as a codec configuration of the gain map item. A codec configuration of the main image item or main item may be the same as a codec configuration of the alpha map item. A codec configuration of the gain map item may be the same as a codec configuration of the alpha map item.

[0589] In an embodiment, when the main image item or main item and the gain map item are present within the minimized image box, codec configuration data used to decode the main image item or main item is the same as codec configuration data used to decode the gain map item.

[0590] In an embodiment, when the main image item or main item and the alpha map item are present within the minimized image box, codec configuration data used to decode the main image item or main item is the same as codec configuration data used to decode the alpha map item.

[0591] In an embodiment, when the gain map item and the alpha map item are present within the minimized image box, codec configuration data used to decode the gain map item is the same as codec configuration data used to decode the alpha map item.

[0592] Parameters added to an original version of a pixel information property may be moved to a new version of the pixel information property. Versatile video coding operating points may be derived from a compact versatile video coding decoder configuration record. A target output layer property may be derived from a compact versatile video coding decoder configuration record. A layer selector property may be derived from a compact versatile video coding decoder configuration record.

[0593] In an embodiment, an apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: decode, from a bitstream, information from a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box. In an embodiment, the apparatus is further caused to: derive versatile video coding operating points from a compact versatile video coding decoder configuration record. In an embodiment, the apparatus is further caused to: derive a target output layer property from a compact versatile video coding decoder configuration record. In an embodiment, the apparatus is further caused to: derive a layer selector property from a compact versatile video coding decoder configuration record.

[0594] In an embodiment, a method includes signaling, into a bitstream, information with a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

[0595] In an embodiment, a method includes decoding, from a bitstream, information from a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

[0596] In an embodiment, an apparatus includes means for signaling, into a bitstream, information with a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

[0597] In an embodiment, an apparatus includes means for decoding, from a bitstream, information from a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

[0598] In an embodiment, a computer readable medium includes instructions stored thereon for performing at least the following: signaling, into a bitstream, information with a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

[0599] In an embodiment, a computer readable medium includes instructions stored thereon for performing at least the following: decoding, from a bitstream, information from a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item; wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

[0600] Embodiments have been described in relation to specific syntax. It is to be understood that embodiments may be similarly realized with any syntax achieving functionality described in embodiments.

[0601] Embodiments have been described in relation to specific identifiers, such as four-character codes. It is to be understood that embodiments may be similarly realized with any identifier values, which are not necessarily the same as those presented in examples and embodiments.

[0602] Embodiments have been described in relation to WC. It is to be understood that embodiments may be similarly realized with any codec.

[0603] Embodiments have been described in relation to syntax. It is to be understood that respective embodiments may be realized for a file writer that generates a file according to the syntax described in embodiments. Similarly, it is to be understood that embodiments may be realized with a file reader that parses or interprets a file according to the syntax described in embodiments.

[0604] References to a ‘computer’, ‘processor’, etc. should be understood to encompass not only computers having different architectures such as single / multi-processor architectures and sequential / parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGAs), application specific circuits (ASICs), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device such as instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device, etc.

[0605] The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., random access memory (RAM) vs. read only memory (ROM)).

[0606] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0607] As used herein, the term ‘circuitry’, ‘circuit’ and variants may refer to any of the following: (a) hardware circuit implementations, such as implementations in analog and / or digital circuitry, and (b) combinations of circuits and software (and / or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s) / software including digital signal processor(s), software, and one or more memories that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even when the software or firmware is not physically present. As a further example, as used herein, the term ‘circuitry’ would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ would also cover, for example and when applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device. Circuitry or circuit may also be used to mean a function or a process used to execute a method.

[0608] It should be understood that the foregoing description is only illustrative. Various alternatives and modifications may be devised by those skilled in the art. For example, features recited in the various dependent claims could be combined with each other in any suitable combination(s). In addition, features from different embodiments described above could be selectively combined into a new embodiment. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

Claims

CLAIMSWhat is claimed is:

1. An apparatus comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:signal, into a bitstream, information with a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item;wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

2. The apparatus of claim 1, wherein when the main image item or main item and the gain map item are present within the minimized image box, a dimension of the main image item or main item comprising a width of the main image item or main item multiplied by a height of the main image item or main item is equal to a dimension of the gain map item comprising a width of the gain map item multiplied by a height of the gain map item.

3. The apparatus of any of claims 1 to 2, wherein when the main image item or main item and the alpha map item are present within the minimized image box, a dimension of the main image item or main item comprising a width of the main image item or main item multiplied by a height of the main image item or main item is equal to a dimension of the alpha map item comprising a width of the alpha map item multiplied by a height of the alpha map item.

4. The apparatus of any of claims 1 to 3, wherein when the gain map item and the alpha map item are present within the minimized image box, a dimension of the gain map item comprising a width of the gain map item multiplied by a height of the gain map item is equal to a dimension of the alpha map item comprising a width of the alpha map item multiplied by a height of the alpha map item.

5. The apparatus of any of claims 1 to 4, wherein a codec configuration of the main image item or main itemis the same as a codec configuration of the gain map item.

6. The apparatus of any of claims 1 to 5, wherein a codec configuration of the main image item or main item is the same as a codec configuration of the alpha map item.

7. The apparatus of any of claims 1 to 6, wherein a codec configuration of the gain map item is the same as a codec configuration of the alpha map item.

8. The apparatus of any of claims 1 to 7, wherein when the main image item or main item and the gain map item are present within the minimized image box, codec configuration data used to decode the main image item or main item is the same as codec configuration data used to decode the gain map item.

9. The apparatus of any of claims 1 to 8, wherein when the main image item or main item and the alpha map item are present within the minimized image box, codec configuration data used to decode the main image item or main item is the same as codec configuration data used to decode the alpha map item.

10. The apparatus of any of claims 1 to 9, wherein when the gain map item and the alpha map item are present within the minimized image box, codec configuration data used to decode the gain map item is the same as codec configuration data used to decode the alpha map item.

11. The apparatus of any of claims 1 to 10, wherein parameters added to an original version of a pixel information property are moved to a new version of the pixel information property.

12. The apparatus of any of claims 1 to 11 , wherein versatile video coding operating points are derived from a compact versatile video coding decoder configuration record.

13. The apparatus of any of claims 1 to 12, wherein a target output layer property is derived from a compact versatile video coding decoder configuration record.

14. The apparatus of any of claims 1 to 13, wherein a layer selector property is derived from a compact versatile video coding decoder configuration record.

15. An apparatus comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to:decode, from a bitstream, information from a file format comprising a minimized image box; wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item;wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

16. The apparatus of claim 15, wherein when the main image item or main item and the gain map item are present within the minimized image box, a dimension of the main image item or main item comprising a width of the main image item or main item multiplied by a height of the main image item or main item is equal to a dimension of the gain map item comprising a width of the gain map item multiplied by a height of the gain map item.

17. The apparatus of any of claims 15 to 16, wherein when the main image item or main item and the alpha map item are present within the minimized image box, a dimension of the main image item or main item comprising a width of the main image item or main item multiplied by a height of the main image item or main item is equal to a dimension of the alpha map item comprising a width of the alpha map item multiplied by a height of the alpha map item.

18. The apparatus of any of claims 15 to 17, wherein when the gain map item and the alpha map item are present within the minimized image box, a dimension of the gain map item comprising a width of the gain map item multiplied by a height of the gain map item is equal to a dimension of the alpha map item comprising a width of the alpha map item multiplied by a height of the alpha map item.

19. The apparatus of any of claims 15 to 18, wherein a codec configuration of the main image item or main item is the same as a codec configuration of the gain map item.

20. The apparatus of any of claims 15 to 19, wherein a codec configuration of the main image item or main item is the same as a codec configuration of the alpha map item.12121. The apparatus of any of claims 15 to 20, wherein a codec configuration of the gain map item is the same as a codec configuration of the alpha map item.

22. The apparatus of any of claims 15 to 21 , wherein when the main image item or main item and the gain map item are present within the minimized image box, codec configuration data used to decode the main image item or main item is the same as codec configuration data used to decode the gain map item.

23. The apparatus of any of claims 15 to 22, wherein when the main image item or main item and the alpha map item are present within the minimized image box, codec configuration data used to decode the main image item or main item is the same as codec configuration data used to decode the alpha map item.

24. The apparatus of any of claims 15 to 23, wherein when the gain map item and the alpha map item are present within the minimized image box, codec configuration data used to decode the gain map item is the same as codec configuration data used to decode the alpha map item.

25. The apparatus of any of claims 15 to 24, wherein parameters added to an original version of a pixel information property are moved to a new version of the pixel information property.

26. The apparatus of any of claims 15 to 25, wherein the apparatus is further caused to:derive versatile video coding operating points from a compact versatile video coding decoder configuration record.

27. The apparatus of any of claims 15 to 26, wherein the apparatus is further caused to:derive a target output layer property from a compact versatile video coding decoder configuration record.

28. The apparatus of any of claims 15 to 27, wherein the apparatus is further caused to:derive a layer selector property from a compact versatile video coding decoder configuration record.

29. A method comprising:signaling, into a bitstream, information with a file format comprising a minimized image box;wherein the minimized image box contains one or more of: a main image item, or a main item, or again map item, or an alpha map item;wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

30. A method comprising:decoding, from a bitstream, information from a file format comprising a minimized image box;wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item;wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

31. An apparatus comprising:means for signaling, into a bitstream, information with a file format comprising a minimized image box;wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item;wherein syntax of the minimized image box indicates at least one of a dimension, codec configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

32. An apparatus comprising:means for decoding, from a bitstream, information from a file format comprising a minimized image box;wherein the minimized image box contains one or more of: a main image item, or a main item, or a gain map item, or an alpha map item;wherein syntax of the minimized image box indicates at least one of a dimension, codec123configuration data, or chroma subsampling shared between at least two of the main image item or the main item, the gain map item, or the alpha map item when present within the minimized image box.

Images