Message reference

Abstract

A referencing mechanism for messages (e.g., SEI messages) is provided to reduce bit costs when different versions of a message (e.g., SEI message) are used in an arbitrary manner in the bitstream, e.g., in an alternating manner in the bitstream. In an embodiment, two SEI messages are used to represent a function. In a first SEI message, a syntax element is provided that specifies the function. The first SEI message may also have an identifier value, which uniquely identifies an instance of a particular SEI message. A second SEI message, which is generally much smaller and sent more frequently than the first SEI message, is used to reference the first SEI message to apply the function of the first SEI message to a portion of the bitstream determined by the persistence range of the second SEI message, e.g., for a picture or sub-picture sent together with the second SEI message.

Description

Technical Field

[0001] The present disclosure relates to the coding and decoding of video sequences and / or still images, and more particularly, to message references.

Background Art

[0002] VVC and HEVC

[0003] Versatile Video Coding (VVC) and High Efficiency Video Coding (HEVC) are block-based video codecs jointly standardized and developed by the International Telecommunication Union - Telecommunication (ITU-T) and the Moving Picture Experts Group (MPEG). The codec utilizes both temporal prediction and spatial prediction. The first version of HEVC was finalized in April 2013, and the first version of VVC was finalized in July 2020. The current versions of the two codec specifications as of writing are HEVC version 7 and VVC version 1.

[0004] Spatial prediction is achieved using Intra (I) prediction within the current picture. Temporal prediction is achieved using unidirectional (P) prediction or bidirectional Inter (B) prediction at the block level from previously decoded reference pictures. At the encoder, the difference between the original pixel data and the predicted pixel data, called the residual, is transformed into the frequency domain, quantized, and then entropy-coded along with the necessary prediction parameters such as the prediction mode and motion vectors before being transmitted. The decoder performs entropy decoding, inverse quantization, and inverse transform to obtain the residual, and then adds the residual to the Intra prediction or Inter prediction to reconstruct the picture.

[0005] Component

[0006] A video sequence has a series of images, where each image consists of one or more components. Each component can be described as a two-dimensional rectangular array of sample values. In a video sequence, the images typically have three components: one luminance component Y with sample values being luminance values, and two chrominance components Cb and Cr with sample values being chrominance values. Also, it is common for the dimensions of the chrominance components to be half as small as those of the luminance component in each dimension. For example, for an HD image, the size of the luminance component is 1920×1080, and the chrominance components each have dimensions of 960×540. The components may be called color components.

[0007] NAL unit

[0008] Both VVC and HEVC define a Network Abstraction Layer (NAL). All data in HEVC and VVC, that is, both Video Coding Layer (VCL) data or non-VCL data, are encapsulated in NAL units. VCL NAL units contain data representing picture sample values. Non-VCL NAL units contain additional relevant data such as parameter sets and Supplemental Enhancement Information (SEI) messages. NAL units in VVC and HEVC start with a header called the NAL unit header. The syntax for the NAL unit header for HEVC starts with a forbidden_zero_bit which is always equal to 0 to prevent start code emulation. Without it, some MPEG systems may confuse the HEVC video bitstream with other data, but the 0 bit in the NAL unit header makes all possible HEVC bitstreams uniquely identifiable as HEVC bitstreams. The NAL unit header in VVC shown in Table 1 is exactly the same as the NAL unit header in HEVC, but uses 1 bit less for nal_unit_type and reserves this bit for future use instead. The nal_unit_type, nuh_layer_id, and nuh_temporal_id_plus1 codewords specify the NAL unit type, the scalability layer ID to which the NAL unit belongs, and the temporal layer ID of the NAL unit, which identify what type of data is being carried in the NAL unit. The NAL unit type indicates and specifies how the NAL unit should be parsed and decoded. The rest of the bytes of the NAL unit are the payload of the type indicated by the NAL unit type. The bitstream consists of a series of concatenated NAL units. The syntax for the NAL unit header in VVC is shown in Table 1.

[0009] TIFF0007686080000001.tif46170

[0010] After seeing the NAL unit header, the decoder or bitstream parser can conclude how the NAL unit should be handled, for example parsed and decoded. The remaining bytes of the NAL unit are the payload of the type indicated by the NAL unit type. All VVC or HEVC bitstreams consist of a series of concatenated NAL units.

[0011] The decoding order is the order in which the NAL units are to be decoded, which is the same as the order of the NAL units in the bitstream. The decoding order may differ from the output order, which is the order in which the decoded pictures are to be output by the decoder for display etc.

[0012] TIFF0007686080000002.tif255170

[0013] Scalability layer

[0014] In VVC and HEVC, the value of the nuh_layer_id syntax element in the NAL unit header specifies the scalability layer ID to which the NAL unit belongs. This enables the association of NAL units and pictures to scalability layers that can be used for scalable coding.

[0015] Picture unit, access unit and access unit delimiter

[0016] In VVC, a picture unit (PU) is defined as a set of NAL units where all VCL NAL units belong to the same layer. These are associated with each other according to specified classification rules, are consecutive in decoding order, and contain exactly one coded picture. In previous versions of VVC, the PU was called a layer access unit. In HEVC, the PU is called an access unit (AU).

[0017] In VVC, an access unit is a set of PUs that belong to different scalability layers and contain coded pictures that are associated with the output from the decoded picture buffer (DPB) at the same time, i.e., have the same POC value.

[0018] An access unit in VVC can start from an access unit delimiter (AUD) NAL unit, which indicates the start of the access unit, the type of slices allowed in the coded picture, i.e., I, I-P, or I-P-B, and whether the access unit is an IRAP access unit or a GDR access unit.

[0019] Layer, dependent and independent layers

[0020] In VVC, a layer is defined as a set of VCL NAL units that all have a specific value of nuh_layer_id and associated non-VCL NAL units. In this disclosure, a layer such as a VVC layer is called a scalability layer.

[0021] The coded layer video sequence (CLVS) in VVC is defined as a sequence of PUs. The sequence of PUs consists of zero or more PUs that are not the CLVS start (CLVSS) PU and all subsequent PUs after the CLVSS PU, excluding the subsequent PUs that are the CLVSS PU itself, in decoding order.

[0022] The relationship among PU, AU, and CLVS is shown in FIG. 3.

[0023] In VVC, scalability layers can be coded independently of each other or dependently on each other. When scalability layers are coded independently, for example, a scalability layer with nuh_layer_id 0 may not predict video data from another scalability layer with, for example, nuh_layer_id 1. In VVC, dependent coding between scalability layers that enables support for scalable coding using SNR, spatial, and view scalability can be used.

[0024] In the present disclosure, the term "scalability layer" is used when referring to scalability layers such as SNR, spatial, and view scalability, which are identified by layer ID values such as nuh_layer_id values in HEVC and VVC.

[0025] Temporal layer

[0026] In VVC and HEVC, all pictures are associated with a TemporalId value that specifies which temporal layer the picture belongs to. The TemporalId value is decoded from the nuh_temporal_id_plus1 syntax element in the NAL unit header. The encoder is required to set the TemporalId value such that pictures belonging to lower temporal layers are fully decodable when higher temporal layers are discarded. For example, assume that the encoder outputs a bitstream using temporal layers 0, 1, and 2. In that case, removing all NAL units of temporal layer 2, or removing all NAL units of layers 1 and 2, will result in a bitstream that can be decoded without problems. This is guaranteed by a limitation in the HEVC specification that the encoder must comply with. For example, it is not allowed for pictures in a temporal layer to reference pictures in a higher temporal layer.

[0027] In the present disclosure, the term "temporal layer" is used when referring to the temporal layers that have been conventionally used in HEVC. The term "layer" in the present disclosure may refer to a temporal layer, or a scalability layer, or a combination of a temporal layer and a scalability layer.

[0028] Slice

[0029] The concept of a slice in HEVC is to divide a picture into independently coded slices, where the decoding of one slice in a picture is independent of other slices in the same picture. Different coding types can be used for slices of the same picture, i.e., a slice can be either an I slice, a P slice, or a B slice. One purpose of a slice is to enable resynchronization in case of data loss. In HEVC, a slice is a set of CTUs.

[0030] In VVC, a picture can be partitioned into either raster scan slices or rectangular slices. A raster scan slice consists of several complete tiles in raster scan order. A rectangular slice consists of a rectangular region in the picture, or a group of tiles that together occupy consecutive numbers of CTU rows inside one tile. Each slice has a slice header with syntax elements. The decoded slice header values from these syntax elements are used when decoding the slice. Each slice is carried in one VCL NAL unit.

[0031] In previous versions of the VVC draft specification, a slice was called a tile group.

[0032] Picture header

[0033] The VVC includes a picture header, which is a NAL unit having a nal_unit_type equal to PH_NUT. The picture header is similar to the slice header, but the values of the syntax elements in the picture header are used to decode all slices of one picture. Each picture in VVC consists of one picture header NAL unit and all the coded slices of the picture, where each coded slice is transmitted in one coded slice NAL unit that comes after the picture header NAL unit.

[0034] Intra Random Access Point (IRAP) pictures and Coded Video Sequence (CVS)

[0035] In single-scalability layer coding in HEVC, an access unit (AU) is the coded representation of a single picture. An AU can consist of several Video Coding Layer (VCL) NAL units as well as non-VCL NAL units.

[0036] An Intra Random Access Point (IRAP) picture in HEVC is a picture that does not reference any other picture for prediction in its decoding process. The first picture in the bitstream in the decoding order in HEVC must be an IRAP picture, but an IRAP picture can appear later in the bitstream as well. HEVC specifies three types of IRAP pictures, namely, Broken Link Access (BLA) pictures, Instantaneous Decoder Refresh (IDR) pictures, and Clean Random Access (CRA) pictures.

[0037] A Coded Video Sequence (CVS) in HEVC is a sequence of access units that starts with an IRAP access unit and then comes zero or more AUs up to the next IRAP access unit in the decoding order, but does not include the next IRAP access unit.

[0038] An IDR picture always starts a new CVS. An IDR picture may have an associated Random Access Decodable Leading (RADL) picture. An IDR picture does not have an associated Random Access Skipped Leading (RASL) picture.

[0039] A BLA picture in HEVC also starts a new CVS and has the same effect on the decoding process as an IDR picture. However, a BLA picture in HEVC may contain a syntax element that specifies a non-empty set of reference pictures. A BLA picture may have an associated RASL picture, and when the associated RASL picture contains a reference to a picture that may not be present in the bitstream, it may not be output by the decoder and may not be decodable. A BLA picture may also have an associated RADL picture to be decoded. A BLA picture is not included in VVC.

[0040] A CRA picture may have an associated RADL picture or RASL picture. Similar to the case of a BLA picture, a CRA picture may contain a syntax element that specifies a non-empty set of reference pictures. In a CRA picture, when the associated RASL picture contains a reference to a picture that may not be present in the bitstream, a flag may be set to specify that the associated RASL picture is not output by the decoder because it may not be decodable. A CRA may or may not start a CVS.

[0041] In VVC, a CVS starts at a CVS start (CVSS) access unit and then is a sequence of access units where there are zero or more AUs up to but not including the next CVSS access unit in decoding order. A CVSS access unit may contain an IRAP picture, i.e., an IDR picture or a CRA picture, or a gradual decoding refresh (GDR) picture. A CVS may contain one or more CLVSs.

[0042] A GDR picture is essentially used for random access in a bitstream encoded for low latency coding where a full IRAP picture would cause too much latency. A GDR picture may use a progressive intra refresh that updates per video picture where each picture is only partially intra-coded. Assuming the bitstream is tuned at a GDR picture, the recovery POC count is signaled by the GDR picture that specifies when the video is fully refreshed and ready for output. A GDR picture in VVC may start a CVS or a CLVS. A GDR picture is included as a normative feature in VVC but not as a normative part of the HEVC standard where a GDR picture may instead be indicated using an SEI message.

[0043] Parameter set

[0044] VVC and HEVC specify three types of parameter sets, namely, Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS). PPS contains data that is common to the entire picture, SPS contains data that is common to the coded video sequence (CVS), and VPS contains data that is common to multiple CVSs, e.g., data for multiple scalability layers in the bitstream.

[0045] VVC also specifies one additional parameter set, namely, Adaptive Parameter Set (APS). APS conveys the parameters required for the Adaptive Loop Filter (ALF) tool, the Luma Mapping and Chroma Scaling (LMCS) tool, and the Scaling List tool. APS may contain information that can be used for multiple slices, and two slices of the same picture can use different APSs. Both VVC and HEVC allow for certain information (e.g., parameter sets) to be provided by external means. By "external means" it is to be interpreted that the information is not provided in the coded video bitstream but by some other means not specified in the video codec specification, e.g., in some cases in a different data channel or via metadata provided as constants in the decoder.

[0046] Decoding Capability Information (DCI)

[0047] DCI may not change during a decoding session and specifies information, such as the maximum number of allowed sublayers, that may be good for the decoder to know. The information in DCI is not necessary for the operation of the decoding process. In an early draft of the VVC specification, DCI was called Decoding Parameter Set (DPS).

[0048] The decoding capability information includes a set of general constraints on the bitstream that gives decoder information on what to expect from the bitstream regarding coding tools, types of NAL units, etc. In VVC, the general constraint information can also be signaled in the VPS or SPS.

[0049] Picture Order Count (POC)

[0050] Pictures in HEVC are identified by their Picture Order Count (POC) values, also known as full POC values. Both the encoder and the decoder track the POC and assign POC values to each picture being encoded / decoded. The decoded pictures are output in ascending order of POC, which means that the POC value represents the output order. The picture order count value of a picture is called PicOrderCntVal in HEVC. Usually, the PicOrderCntVal for the current picture is simply called PicOrderCntVal.

[0051] Reference Picture Resampling (RPR)

[0052] RPR is a new feature in VVC that does not exist in HEVC. In HEVC, all pictures of a layer have the same spatial resolution. However, in VVC, pictures belonging to the same layer can have different spatial resolutions. The spatial resolution (width and height) of a picture is signaled in the PPS in VVC. When the current picture and a reference picture have different spatial resolutions, RPR enables the reference picture to be used for the prediction of the current picture by scaling the reference picture to the same spatial resolution as the current picture before prediction. RPR can be used for pictures belonging to the same layer or different layers.

[0053] SEI Message

[0054] The supplementary enhancement information (SEI) message is a code point in the coded bitstream that does not affect the decoding process of the coded picture from the VCL NAL unit. SEI messages typically address issues in the representation / rendering of the decoded bitstream. The overall concept of SEI messages, and many of the messages themselves, are inherited from the H.264 and HEVC specifications into the VVC specification. In VVC, the SEI RBSP contains one or more SEI messages.

[0055] The SEI message syntax table, which describes the general structure of SEI messages in VVC, is shown in Table 3. The type of each SEI message is identified by the payload type of each SEI message.

[0056] Table 3 SEI message syntax table in VVC

[0057] TIFF0007686080000003.tif78170

[0058] Appendix D in the VVC specification specifies the syntax and semantics for the SEI message payload for some SEI messages, and specifies the use of SEI messages and VUI parameters for which the syntax and semantics are specified in ITU-T H.SEI|ISO / IEC 23002-7. The SEI payload structure in Appendix D, which lists the SEI messages supported in VVC version 1, is shown in Table 4.

[0059] TIFF0007686080000004.tif255170TIFF0007686080000005.tif255170

[0060] SEI messages assist processes related to decoding, display, or other purposes. However, SEI messages are not required to construct luma or chroma samples by the decoding process. Some SEI messages are required for bitstream compliance checking and for output timing decoder compliance. Other SEI messages are not required for bitstream compliance checking. The decoder does not necessarily need to support all SEI messages. Usually, when the decoder encounters an unsupported SEI message, that SEI message is discarded.

[0061] ITU-T H.274|ISO / IEC 23002-7, also known as VSEI, specifies the syntax and semantics of SEI messages, and in particular, is for use with coded video bitstreams, but is written in a manner intended to be general enough to be used with other types of coded video bitstreams as well. The first version of ITU-T H.274|ISO / IEC 23002-7 was finalized in July 2020. Version 2 is under development as this is being written. JVET-U2006-v1 is the current draft for version 2, which specifies additional SEI messages for use with coded video bitstreams.

[0062] The persistence of an SEI message indicates the pictures to which the values signaled in an instance of the SEI message can apply. The portion of the bitstream to which the values of an SEI message can apply is called the persistence scope of the SEI message.

[0063] Scalable nesting SEI message

[0064] The scalable nesting SEI message in VVC provides a mechanism for associating an SEI message with a specific OLS, a specific layer, or a specific set of sub-pictures. The scalable nesting SEI message contains one or more SEI messages. The SEI messages contained in the scalable nesting SEI message are also called scalable-nested SEI messages.

[0065] The scalable nesting SEI message syntax in VVC is shown in Table 5.

[0066] TIFF0007686080000006.tif158170

[0067] Film grain

[0068] Adding noise or film grain after decoding a picture

[0069] Noise in video originates from different sources. This noise can be suppressed by the encoder at the earliest stages of the process. When a picture is reconstructed in the decoder before display, modeled or unmodeled noise can be added to the decoded frame in one way or another. Different purposes have been introduced to clarify the subjective quality improvement by adding noise, which has become more apparent as a result of the increase in picture resolution. The first reason for adding noise can be, for example, to achieve an artistic effect, capture reality, or obtain the "cinematic effect of reality" in the case of a movie while shooting a documentary, portrait, or black-and-white scene. The second reason is to hide coding artifacts such as blurring, blocking, and banding effects that appear due to heavy encoding procedures in the encoder.

[0070] The film grain characteristic SEI message in VVC

[0071] The film grain process is specified in the VSEI specification and is supported by VVC. This process is essentially equivalent to the film grain process specified in the H.264 / AVC and HEVC video coding standards. The process includes an SEI message that conveys a parameterized model for film grain synthesis in the decoder.

[0072] The film grain characteristics SEI message includes a cancellation flag, film_grain_characteristics_cancel_flag, and the cancellation flag enables the film grain process when the cancellation flag is set equal to 0. Also, when the flag is set to 0, the film grain parameter syntax element follows the flag. Finally, the film_grain_characteristics_persistence_flag specifies the persistence of the film grain characteristics SEI message for the current layer. A simplified version of the syntax is shown in Table 6 below.

[0073] TIFF0007686080000007.tif67170

[0074] Normalized film grain

[0075] In JVET-Q0424, a normalized film grain generation process was proposed for VVC. The contribution document proposed signaling film grain parameters in the APS and including a film grain seed in the picture header to make the film grain process pseudo-random and deterministic.

[0076] LMCS

[0077] Luma mapping with chroma scaling (LMCS) is an in-loop filter processing process in VVC that allows signals to be dynamically adapted according to their codeword statistics. Parameters for in-loop LMCS are signaled in an APS with a specific APS ID. The LMCS APS ID is referenced from the picture header. In-loop LMCS allows pictures to be stored in the DPB in their original (unmapped) format. LMCS can be made adaptable or can be made possible according to picture type.

[0078] In JVET-U0078, it was proposed to signal out-of-loop LMCS using either canonical signaling in the parameter set or signaling as an SEI message. SUMMARY OF THE INVENTION

[0079] Parameter sets in VVC and HEVC provide a flexible and highly compression-efficient way of referencing syntax elements that apply to a certain part of the bitstream. For example, each picture references a PPS using a specific PPS ID, and each picture and / or slice may further reference one or more APSs using their APS IDs. The parameter set ID provides flexibility that allows the syntax elements of a parameter set not to be repeated when multiple parameter sets are used in the bitstream.

[0080] The same flexibility does not exist for SEI messages. The persistence scope for SEI generally pertains to the current picture or all subsequent pictures until the current picture is replaced by a new version of the SEI message. In the case of SEI messages, which are not always necessarily applicable to each picture in a set of consecutive pictures, the current persistence scheme is not very coding efficient. For example, as shown in Figure 4, when two versions of an SEI message (i.e., two SEI messages of the same type with different values for syntax elements) are applied alternately, each being applicable for every second picture respectively, a complete SEI message has to be sent for each picture, although the SEI messages for each of those two versions are equivalent.

[0081] Generally, a parameter set comprises syntax elements that are essential for a decoder compliant with the specifications to support. On the other hand, SEI messages are optional for a decoder to support, and thus the content of SEI messages may not be suitable for signaling in the parameter set. Moreover, it is not possible to add syntax elements specifying functionality to the normative parameter set without creating a new profile for the codec. Creating a new profile may cause market fragmentation and should not be done unless there is a strong market need for the new profile.

[0082] One aspect of the embodiment is for providing a reference mechanism for a message (e.g., SEI message) to reduce bit cost when different versions of the message (e.g., SEI message) are used in an arbitrary manner in a bitstream, for example, in an alternating manner. In the embodiment, two SEI messages are used to represent a function. In the first SEI message, a syntax element for specifying the function is provided. The first SEI message may also have an identifier value, and the identifier value uniquely identifies an instance of a specific SEI message. The second SEI message, which is generally much smaller and more frequently sent than the first SEI message, is used to reference the first SEI message to apply the function of the first SEI message to a portion of the bitstream determined by the persistence range of the second SEI message, for example, for a picture or sub-picture sent together with the second SEI message.

[0083] The advantage of some embodiments is that they provide a mechanism for referring to an instance of a first message (e.g., a first SEI message) that specifies a function from a second, and generally significantly smaller, message (e.g., a second SEI message) so that the first message does not need to be repeated when different versions of the message are applied in an arbitrary manner (e.g., in an alternating manner) in the bitstream.

[0084] In prior art solutions, it may be necessary to send an SEI message with a certain function for every picture. The benefit in some embodiments disclosed herein is that it is not always necessary to constantly repeat the function of a message (e.g., an SEI message). It is also possible to have several first messages to link to two or more versions of a function.

[0085] In conventional SEI messages, if the SEI message is to be used for many, but not all, segments in the bitstream, the SEI message may have to be repeated several times, which can have a high cost in bits when the SEI message carries a lot of information. For example, the content color volume SEI message specified in the VSEI specification may need to send 37 bytes per picture in the worst case.

[0086] Embodiments enable the function information of a message (e.g., an SEI message) not to be repeated when not needed. This saves bits and also provides flexibility as to where the function should be applied. The function can be applied, for example, to any subset of pictures or sub-pictures in the bitstream in a flexible manner.

[0087] The information transmitted in the SEI message does not affect the decoding process and can generally be ignored by the core decoder. The information transmitted in the parameter set is normative and generally affects the decoding process. Thus, information suitable for the SEI message may not be suitable for the parameter set. One benefit of having an SEI message compared to a parameter set is that functions can be added to the existing specification without the need to add a new profile. The SEI message can be added to the specification without the need to add a new profile. A decoder compliant with the profile described in the first version of the specification can use the SEI message specified in the updated version of the specification. The benefit of the general-purpose version of the solution is that it will also work for previously specified SEI messages that do not have a unique identifier.

[0088] Another benefit of having SEI messages compared to a parameter set is that new features can be added later to an already finalized specification. For example, in the case of VVC version 1, there may be no further normative additions such as additional signaling in the parameter set, but a VVC version 1 decoder can decide to support SEI messages added to the VVC specification after the VVC version 1 specification has been finalized.

[0089] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various embodiments.

Brief Description of the Drawings

[0090]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Modes for Carrying Out the Invention

[0091] It should be understood by those skilled in the art that the following embodiments may be combined to form solutions that are not explicitly defined but are still covered by the present invention.

[0092] The embodiments may be applicable to both the encoder and the decoder, as well as to the components of the encoder or decoder that can be deployed locally or remotely from each respective encoder or decoder. An exemplary encoder is shown in FIG. 1. An exemplary decoder is shown in FIG. 2.

[0093] Embodiment 1 - General Method

[0094] FIG. 5 shows an exemplary bitstream according to one embodiment. An encoder may encode a bitstream such as that shown in FIG. 5, and a decoder may decode a bitstream such as that shown in FIG. 5. The first embodiment described herein is generalized to be applicable to different types of packets, including where the packet type is an SEI message or other packet type. In this embodiment, video bitstream 1 comprises a set of coded pictures. Two packets, a first packet 11 and a second packet 21, are used to apply a certain function. The first packet 11 comprises one or more syntax elements that provide information about the function (i.e., function information 32), and an identifier 12 that identifies the particular packet and / or function information. Generally, for example for each picture and further for sub-pictures or slices, the second packet 21, which is sent more frequently than the first packet, comprises a reference identifier 22 (or, reference ID 22) to the first packet and / or function information, and specifies one or more segments 41 that are signaled in a third packet 43 to which a particular function is to be applied. In some embodiments, the second packet 21 may also include additional information 23, such as light-weight control regarding the particular function to be applied. In one embodiment, one or more segments 41 to which the function information 32 is to be applied are signaled in the third packet 43. In another embodiment, only a part of the segments 41 are signaled in the third packet 43.

[0095] One aspect of the embodiment is for the need not to repeat the function information of the first packet (which may potentially be many bits) when not needed, but to be able to reference that function information in later packets. This saves bits and provides flexibility as to where the function is to be applied. The function may be applied, for example, in a flexible manner to any subset of pictures or sub-pictures in the bitstream.

[0096] Description of Packets, Packet Types, Segments, and Functional Information

[0097] Any or all of the packets (e.g., the first, second, and third packets) in this embodiment may be NAL units. The first packet and the second packet are, in some embodiments, non-VCL NAL units, and the packet type of the first packet and the packet type of the second packet may both be, for example, SEI messages. In the following text, the packets are referred to as SEI messages, but it should be understood that the packets can be of any type, and the following explanations referring to SEI messages can also apply to any type of packet. In some embodiments, the first packet and the second packet may be of the same type, and in other embodiments, they may be of different types. For example, the type of the first packet may be a parameter set such as SPS, PPS, or APS, and the type of the second packet may be an SEI message. In the following text, the second packet is often referred to as a reference SEI message, and the first packet is often referred to as a functional information SEI message that is referred to by the reference SEI message, but the explanations are generally applicable to other packet types.

[0098] In some embodiments, the third packet is a VCL NAL unit, and the packet type of the third packet is a slice type. The segment can be, for example, a picture, a sub-picture, a slice, a tile, or a CTU.

[0099] Function information can be, for example, film grain parameters, LMCS parameters, mastering display color volume parameters, content light level parameters, ambient viewing environment parameters, content color volume parameters, spherical rotation parameters, region-wise packing parameters, omnidirectional viewport parameters, sample aspect ratio parameters, annotated region parameters, scalability dimension parameters, multiview acquisition parameters, depth representation parameters, etc., and can be a syntax element that specifies any type of function that can be carried in an SEI message or parameter set. These parameters can include model descriptions such as filter model type, filter strength, etc.

[0100] Applying a function to a segment based on function information can include using parameter values to modify the segment according to specifications or to attach information to the segment. This can include, for example, applying film grain noise to the segment, applying outside the LMCS loop to the segment, or modifying the display color volume, content light level, ambient viewing, or content color volume for the segment, applying geometric transformations or displacements of the segment such as spherical rotation, region-wise packing, sample aspect ratio conversion, etc., or attaching information such as omnidirectional viewport information, annotated region information, scalability dimension information, multiview acquisition information, and depth representation information to the segment.

[0101] Functional information may, in the present disclosure, be interpreted as parameter values or syntax element values, alternatively or additionally. The use of such functional information in the present disclosure may be regarded as parameter referencing, where, instead of decoding a parameter or syntax element from a segment, a reference identifier is used to reference a parameter.

[0102] Using two types of SEI messages

[0103] The following table shows an exemplary syntax for using two different types of SEI messages for the first embodiment. TIFF0007686080000008.tif30170TIFF0007686080000009.tif25170

[0104] The first SEI message is an SEI message that conveys functional information. An instance of the SEI message is uniquely identified using functionality_information_id. functionality_information() in the syntax table represents the syntax elements used to specify a function. The details of the syntax elements in the functional information are outside the scope of the present disclosure. Two functional information SEI messages with different functionality_information_id may have different values for the syntax elements in the functional information. The second SEI message is an SEI message that references a functional information SEI message using referenced_functionality_information_id, where referenced_functionality_information_id specifies one or more segments to which the function should apply. The reference SEI message may also include one or several syntax elements for lightweight control of the function specified in the functional information SEI message.

[0105] FIG. 6 shows an exemplary bitstream according to one embodiment. An encoder may encode a bitstream such as that shown in FIG. 6, and a decoder may decode a bitstream such as that shown in FIG. 6. As shown in FIG. 6, the bitstream may include two functional information SEI messages with different identifier values and several reference SEI messages (four are shown) each associated with a segment, and each reference SEI message refers to one of the functional information SEI messages so that each unique functional information is not repeated in the bitstream.

[0106] As shown in FIG. 6, at the beginning of the bitstream, the encoder may encode a functional information SEI message. The encoder may also select to encode a functional information SEI message when needed. The encoder may, for example, follow these steps. a. Determine the function to be used for the segment b. If a suitable functional information SEI message with a first identifier value (e.g., 0) specifying the function has already been signaled in the bitstream, i. Encode a reference SEI message with a reference ID value corresponding to the first identifier value into the bitstream ii. Encode the segment into the bitstream c. Otherwise, i. Encode a suitable functional information SEI message with a new identifier value (e.g., 1) into the bitstream ii. Encode a reference SEI message with a reference ID value corresponding to the new identifier value into the bitstream iii. Encode the segment into the bitstream

[0107] In one version of the embodiment, the identifier value can be signaled with a fixed number of bits, such as 8, as in the above exemplary syntax table. In another version of the embodiment, the number of bits used to signal the identifier is signaled in a syntax element in the SEI message. This is shown in the following exemplary syntax, where the syntax element id_len_minus_1+1 specifies the number of bits used to signal the functionality_information_id syntax element representing the identifier value. In another version of the embodiment, the identifier value is signaled with a non-fixed number of bits, such as an unsigned integer zero-order exponential Golomb-coded (Exp-Golomb-coded) syntax element denoted as ue(v). TIFF0007686080000010.tif35170TIFF0007686080000011.tif30170

[0108] Referenced SEI message persistence range

[0109] The persistence range of the referenced SEI message, i.e., to which set of segments the functionality of the functional information SEI message should apply, can be clearly defined in the specification. For example, the specification can state that the persistence of the referenced SEI message is the access unit (AU) containing the SEI message. The specification can also state that the persistence of the referenced SEI message is the picture unit (PU) containing the SEI message, or alternatively, that the persistence of the referenced SEI message is the slice following the SEI message, or alternatively, that the persistence of the referenced SEI message is the sub-picture to which the slice following the SEI message belongs.

[0110] The persistence range of a reference SEI message may also be specified using one or more syntax elements in the SEI message. For example, a reference SEI may follow the persistence structure commonly used in the VSEI specification that has a persistence flag and a cancellation flag. The following exemplary syntax and semantics show the signaling of the persistence range of a reference SEI message below. TIFF0007686080000012.tif46170

[0111] A referencing_sei_cancel_flag equal to 1 indicates that the SEI message cancels the persistence of the previous reference SEI message in the output order that has a referenced_functionality_information_id equal to the referenced_functionality_information_id of the current reference SEI message. A referencing_sei_cancel_flag equal to 0 indicates that the reference SEI message information comes later.

[0112] The referencing_sei_persistence_flag specifies the persistence of the reference SEI message for the current layer.

[0113] A referencing_sei_persistence_flag equal to 0 specifies that the reference SEI message applies only to the current decoded picture.

[0114] A referencing_sei_persistence_flag equal to 1 specifies that the reference SEI message applies to the current decoded picture and persists for all subsequent pictures of the current layer in the output order until one or more of the following conditions become true. · A new CLVS of the current layer starts. · The bitstream ends. · The picture in the current layer within the AU associated with a reference SEI message having a referenced_functionality_information_id equal to the referenced_functionality_information_id of the current reference SEI message is an output that comes after the current picture in the output order.

[0115] Another example of how to signal the range of a reference SEI message is shown below, where the range of the SEI message can be for a picture associated with the reference SEI message or for a set of sub-pictures of that picture. In one version of the embodiment, as shown in the following exemplary syntax, the previous reference SEI message is erased / overwritten only if the referenced_functionality_information_id has the same value in the current reference SEI message as in the previous reference SEI message. In another version of the embodiment, the referenced_functionality_information_id of the current reference SEI message and the referenced_functionality_information_id of the previous reference SEI message do not need to have the same value for the previous reference SEI message to be erased / overwritten. TIFF0007686080000013.tif63170

[0116] A subpic_flag equal to 1 specifies that the reference SEI message applies only to a specific sub-picture of the picture associated with the SEI message. A subpic_flag equal to 0 specifies that the reference SEI message applies to all sub-pictures of the picture.

[0117] num_subpics_minus1 + 1 specifies the number of sub-pictures in the picture.

[0118] subpic_id_len_minus1 + 1 specifies the number of bits used to represent the syntax element subpic_id[i]. The value of subpic_id_len_minus1 shall be in the range of 0 to 15, inclusive of both end values.

[0119] subpic_id[i] specifies the subpicture ID of the i-th subpicture in the picture. The length of the subpic_id[i] syntax element is subpic_id_len_minus1 + 1 bits.

[0120] In another version of the embodiment, the pictures to which the function information is applied are listed in the reference SEI message, for example, as absolute POC values or delta POC values.

[0121] Duration range of the function information SEI message

[0122] In yet another version of the embodiment, the function information SEI message is specified as being active or not active, and can be specified, for example, using a syntax element (such as an indicator like a flag) in the function information SEI message. The function information can be applied to a segment without the first packet (function information SEI message) being referenced by the second packet (reference SEI message) when the function information SEI is active. Thus, the indicator value indicates whether the function information can be applied to a segment without the first packet being referenced by the second packet. This is shown using the sei_active_flag syntax element in the following exemplary syntax table.

[0123] When the sei_active_flag has a certain value, for example 1, the functional information SEI is specified to be active by itself according to its defined persistence range. For example, the persistence range for the functional information SEI message can be defined to have a certain persistence, such as the rest of the CVLS, unless it is referenced by a reference SEI message having a referenced_functionality_information_id equal to the functionality_information_id of the functional information SEI. When so referenced, the persistence range of the reference SEI replaces the original persistence range of the functional information SEI message. When the sei_active_flag has another value, for example 0, the functional information SEI is specified to be inactive unless it is referenced by a reference SEI message. When so referenced, the functional information SEI becomes active and the persistence range of the reference SEI message is applied to the functional information SEI message. TIFF0007686080000014.tif36170

[0124] Option to disable reference SEI messages

[0125] Since there is an extra burden of signaling many small SEI messages when not needed, the functional information SEI message may contain an indicator value as to whether a reference SEI message can be expected in the bitstream. When the indicator has a first value, e.g., 1, the reference is enabled and the functional information SEI is specified to be inactive by itself. When the indicator has a second value, e.g., 0, the reference is not enabled and the functional information SEI message is active and acts as a conventional SEI message. In that case, a reference SEI message that references the functional information SEI message may not be present in the bitstream following the functional information SEI message. One possible exception is when another functional information SEI with the first value for the indicator comes after the first functional information SEI message in the bitstream, in which case the other functional information SEI overwrites the first functional information SEI message. When the reference SEI message is not enabled, there is no need to send an identifier in the functional information SEI message. This is shown in the following exemplary syntax where the referencing_allowed_flag has the indicator value. TIFF0007686080000015.tif41170

[0126] Combining the functional information SEI message and the reference SEI message

[0127] In another version of this embodiment, the functional information SEI message and the reference SEI message are specified within the same type of SEI message having the same SEI payload type value. When the SEI message is acting as a reference SEI message, the SEI message does not include the syntax elements for the functional information, excluding the reference to the instance of the SEI message acting as the functional SEI message, and optionally one or several syntax elements for the lightweight control of the function. The following syntax table illustrates specifying the functional information SEI message and the reference SEI message within the same type of SEI message. In this example, the referencing_flag is used to specify whether the SEI message acts as a functional information SEI message or a reference SEI message. TIFF0007686080000016.tif67170

[0128] The main advantages of the functional information SEI message and the reference SEI message being specified within the same type of SEI message are that this saves one SEI payload type value and that this increases the readability of the specification.

[0129] Order in the bitstream

[0130] In some embodiments, the reference SEI message comes after its referenced functional information SEI message in the decoding order in the bitstream. In another embodiment, the reference SEI message may precede its referenced functional information SEI message in the decoding order. In that case, the reference indicator can be parsed, but the process of applying the functional SEI message to one or more segments cannot start until the functional information SEI message is received. In yet another embodiment, the functional SEI message and the reference SEI message are provided by external means.

[0131] Comparing the solution with the parameter set

[0132] As described above, reference SEI messages can generally be sent more frequently than functional information SEI messages, for example, for each picture, and even for sub-pictures or slices. For example, if the functional information SEI message is sent once per CVLS, the functional information SEI message can be referenced by a reference SEI message in the same way as how SPS is referenced in VVC. When the functional information SEI message is sent more frequently, such as for a set of pictures or even for a single picture, the functional information SEI message can be referenced by a reference SEI message for each picture in the same way as how PPS or APS is referenced in VVC. When the functional information SEI message is sent for a set of pictures or a single picture, the functional information SEI message can be referenced by a reference SEI message for each slice in the same way as how APS is referenced in VVC.

[0133] One advantage of referencing a specific SEI message as in this embodiment compared to referencing a parameter set is that new features can be added later to an already finalized specification. For example, in the case of VVC version 1, there may be no further normative additions such as additional signaling in the parameter set, but a VVC version 1 decoder can support SEI messages added to the VVC specification after the VVC version 1 specification was finalized.

[0134] Encoder Steps and Decoder Steps

[0135] The encoder may apply all or a subset of the following steps to encode segments into the bitstream according to this embodiment.

[0136] (1) Determine an identifier value for a first packet of a first packet type, where the identifier value represents an identifier of the first packet. The packet can here be an NAL unit. The first packet is, in some embodiments, a non-VCL NAL unit, and the first packet type can be an SEI message. The first packet type can also be of another non-VCL NAL unit type such as a parameter set like SPS, PPS, or APS.

[0137] (2) Encode the identifier value into one or more syntax elements in a first set of syntax elements in the first packet, and encode the first packet into a bitstream.

[0138] (3) Determine a function to be applied to the segment. The function can be specified by function information. The function information can be, for example, syntax elements that specify any type of function that can be carried in an SEI message, such as film grain parameters, LMCS parameters, mastering display color volume parameters, content light level parameters, ambient viewing environment parameters, content color volume parameters, spherical rotation parameters, region packing parameters, omnidirectional viewport parameters, sample aspect ratio parameters, annotated region parameters, scalability dimension parameters, multi-view acquisition parameters, depth representation parameters, etc. These parameters can include model descriptions, filter intensities, etc. The function information can, in the present disclosure, alternatively be interpreted as a parameter value or a syntax element value.

[0139] (4) Encode the function information into one or more syntax elements of a set of syntax elements. The set of syntax elements can be a subset of the first set of syntax elements encoded in the first packet.

[0140] (5)Encode a reference ID value corresponding to the identifier value into one or more syntax elements in a second packet of the second packet type (e.g., the reference ID value may be equal to the identifier value). The second packet type may be an SEI message. If both the first packet type and the second packet type are SEI messages, the payload type of the first SEI message may or may not be the same as the payload type of the second SEI message.

[0141] (6)Encode a segment into a third packet in the bitstream. The packet may here be a VCL NAL unit and the packet type may be a slice type. The segment may be, for example, a picture, a sub-picture, a slice, a tile, or a CTU. The second packet is associated with the segment such that the functional information is to be applied to the segment. For example, the second packet may be the closest preceding packet of the second packet type, and the closest preceding packet of the second packet type is a packet having a second packet type that precedes the third packet in decode order, and there is no other packet having a second packet type that follows the closest preceding packet and precedes the third packet in decode order. In some embodiments, the closest preceding packet is a packet included in the same AU as the third packet.

[0142] The decoder may perform all or a subset of the following steps to decode a segment from the bitstream according to this embodiment.

[0143] (1) Receive a first packet of a first packet type in a bitstream, where the first packet comprises a first set of syntax elements. The packet can be a NAL unit. The first packet can be, in some embodiments, a non-VCL NAL unit, and the first packet type can be an SEI message. The first packet type can also be of another non-VCL NAL unit type such as a parameter set like SPS, PPS, or APS.

[0144] (2) Decode an identifier value from one or more syntax elements in the first set of syntax elements of the first packet, where the identifier value represents a unique identifier of the first packet.

[0145] (3) Decode function information from a set of syntax elements in the bitstream, where the function information specifies a function. The set of syntax elements can be a subset of the first set of syntax elements of the first packet. The function information can be, for example, syntax elements that specify any type of function that can be carried in an SEI message, such as film grain parameters, LMCS parameters, mastering display color volume parameters, content light level parameters, ambient viewing environment parameters, content color volume parameters, spherical rotation parameters, regional packing parameters, omnidirectional viewport parameters, sample aspect ratio parameters, annotated region parameters, scalability dimension parameters, multi-view acquisition parameters, depth representation parameters, etc. These parameters can include model descriptions such as filter model type, filter strength, etc. In the present disclosure, the function information can alternatively be interpreted as a parameter value or a syntax element value.

[0146] (4) Receive a second packet of a second packet type comprising a second set of syntax elements from the bitstream. The second packet type may be an SEI message. If both the first packet type and the second packet type are SEI messages, the payload type of the first SEI message may or may not be the same as the payload type of the second SEI message.

[0147] (5) Decode a reference ID value from one or more syntax elements in the second set of syntax elements of the second packet.

[0148] (6) Receive a third packet of a third packet type in the bitstream, comprising a third set of syntax elements. The third packet may be a VCL NAL unit, and the third packet type may be a slice type.

[0149] (7) Determine that the second packet is associated with a segment such that the function information will be applied to the segment. For example, the second packet may be the closest preceding packet of the second packet type. The closest preceding packet of the second packet type may be defined as the packet having the second packet type that precedes the third packet in the decoding order, with no other packet having the second packet type that precedes the third packet in the decoding order coming after the closest preceding packet. In some embodiments, the closest preceding packet is a packet contained in the same AU as the third packet.

[0150] (8) Optionally, if the set of syntax elements is a subset of the first set of syntax elements of the first packet, determine that the reference ID value corresponds to (e.g., is equal to) an identifier value, and in response to determining that the reference ID value corresponds to the identifier value, select function information from the first set of syntax elements.

[0151] (9) Decode segments from a third set of syntax elements and apply functions to the segments based on the selected function information. The decoding of segments may or may not use the selected function information. A segment can be, for example, a picture, sub-picture, slice, tile, or CTU. Alternatively, the function information can be used in decoding such that the segment is decoded from a third set of syntax elements and the selected function information.

[0152] Embodiment 2 - Film Grain SEI Message Using a Second SEI Message with References and Seeds

[0153] Applying different sets of film grain model parameters to different parts of a video can be advantageous. To achieve this in the current film grain SEI message in the VSEI specification, every time the set of film grain model parameters is changed, a complete film grain SEI message has to be sent even if the set of film grain model parameters has been used previously in the bitstream. This is uneconomical as film grain SEI messages contain many bits.

[0154] In a second embodiment based on the first embodiment, the function information SEI message is a film grain SEI message used to apply film grains to the decoded video in a post-processing step that comes after decoding. An example of the film grain SEI message syntax is shown below. TIFF0007686080000017.tif29170

[0155] The film grain SEI message includes a film_grain_sei_id that uniquely identifies an instance of the film grain SEI message in the bitstream, and a set of syntax elements, film_grain_model_syntax_elements(), for specifying the film grain model.

[0156] The embodiment also includes a film grain reference SEI message based on a reference SEI message from the first embodiment. An exemplary syntax of the film grain reference SEI message is shown below. TIFF0007686080000018.tif35170

[0157] The film grain reference SEI has a referenced_film_grain_sei_id syntax element that is used to reference a specific instance of a film grain SEI message having a film_grain_sei_id value corresponding to the value of referenced_film_grain_sei_id. The film grain reference SEI message may also include a syntax element film_grain_seed that is used to determine a seed value that is used as a starting value for generating film grains using the film grain model parameters in the film grain SEI message. The film grain reference SEI message may also include syntax elements from the film grain SEI message that may be frequently modified, such as a scale factor or a mixing mode.

[0158] Embodiment 3 - LMCS out-of-loop SEI message using a second SEI message with a reference

[0159] In a third embodiment based on the first embodiment, the functional information SEI message is an LMCS out-of-loop SEI message that specifies an LMCS model to be used out of the loop. The reference SEI message references the LMCS out-of-loop SEI message to identify an LMCS model to be applied to one or more segments. Exemplary syntaxes for the two SEI messages are shown below. TIFF0007686080000019.tif30170TIFF0007686080000020.tif30170

[0160] In another version of the embodiment according to the first embodiment, the first packet is a parameter set, such as SPS, PPS or APS, and the second packet is a reference SEI message. The parameter set comprises a syntax element that specifies the LMCS model to be applied to one or more segments. In one embodiment, the parameter set is APS. The reference SEI message comprises a syntax element that identifies a parameter set identifier, such as an APS ID, of a parameter set that includes the LMCS model to be applied to one or more segments. The following syntax table shows an example of a reference SEI message. TIFF0007686080000021.tif30170

[0161] Embodiment 4 - Generic wrapper SEI message

[0162] In a fourth embodiment according to the first embodiment, both the SEI message with an identifier value and the reference SEI message are generic SEI messages. As in the case of the first embodiment, the SEI message with an identifier is referred to by a generic reference SEI message. The difference from the first embodiment is that the functional information of the SEI message with an identifier value is not specified for the first SEI message, but instead, the first SEI message encloses or refers to another existing SEI message. In this embodiment, the enclosed or referred SEI message is called a functional information SEI message, and the SEI message with an identifier is called a generic wrapper SEI message because the SEI message encloses or refers to the functional information SEI message.

[0163] The advantage of this solution is that the reference function of the present invention can be applied to any SEI message, even to already specified SEI messages, without making changes to those SEI messages.

[0164] In the first version of this embodiment, the general wrapper SEI message comprises a general wrapper identifier value that uniquely identifies an instance of the general wrapper SEI message and a payload of the enclosed functional information SEI message. The general reference SEI message includes a syntax element for identifying the referenced general wrapper SEI message. The general reference SEI message may also include the SEI payload type of the functional information SEI message enclosed in the general wrapper SEI message in order to more easily identify the SEI message that should be applied to one or more segments within the scope of the reference SEI message. This can also be useful for error resilience reasons, as it can become easier to identify whether the SEI message was lost or removed from the bitstream by other means. An example of the first version of the embodiment where sei_message() includes the enclosed functional information SEI message is shown below. TIFF0007686080000022.tif30170TIFF0007686080000023.tif30170

[0165] In another version of the embodiment, the general wrapper SEI message does not enclose the functional information SEI message, but instead refers to the functional information SEI message as shown below by including the payload for the functional information SEI message. TIFF0007686080000024.tif31170

[0166] In yet another embodiment, the general wrapper SEI message neither encloses nor directly refers to the functional information SEI message to be used. Instead, the SEI message that comes immediately after the general wrapper SEI message in the decoding order in the bitstream is the functional information SEI message to be used. This is shown in the bitstream of FIG. 7.

[0167] FIG. 7 shows an exemplary bitstream according to one embodiment. An encoder may encode a bitstream such as that shown in FIG. 7, and a decoder may decode a bitstream such as that shown in FIG. 7. The bitstream in FIG. 7 is the same as that shown in FIG. 5, except that in FIG. 5, the second packet 11 contains both the reference identifier 12 and the function information 32, and in FIG. 7, the second packet 11 (e.g., a general wrapper SEI message) contains the reference identifier 12, and a function information message 31 (e.g., a function information SEI message) containing the function information 32 comes immediately after.

[0168] In other embodiments, the general wrapper encapsulates or refers to two or more SEI messages. The following exemplary syntax shows a general wrapper SEI message that encapsulates two or more function information SEI messages. TIFF0007686080000025.tif51170

[0169] In another embodiment, the general wrapper SEI message and the general reference SEI message are combined in one SEI message type having one value for the SEI payload type, similar to how the SEI payload type was described in the first embodiment.

[0170] The specification may list which SEI messages may be used with the general wrapper SEI message.

[0171] The specification may also list which SEI messages may not be used with the general wrapper SEI message.

[0172] Embodiment 5 - Signal identifier in the SEI payload extension.

[0173] In the fifth embodiment based on the first embodiment, the identifier value of the functional information SEI message is signaled in the extension part of the packet. In one version of the embodiment, the reserved bits in the SEI payload structure, sei_reserved_payload_extension_data, are used to signal the identifier value for the functional information SEI message. The solution is shown in the following syntax table, where the syntax element for the identifier value for the functional information SEI message in the first embodiment is called sei_unique_id. TIFF0007686080000026.tif105170

[0174] This solution is also general and does not require each SEI message to implement its own identifier. One disadvantage is that the identifier value will be signaled after the actual payload of the SEI message.

[0175] Embodiment 6 - Signaling in the second SEI message an identifier for the first SEI message and a modification to the function of the first SEI message

[0176] In the sixth embodiment, the identifier value of the functional information SEI message is signaled in the reference SEI message, together with at least one syntax element in the reference SEI message that overwrites the value before at least one syntax element in the functional information SEI message. The following table shows an exemplary syntax for using two different types of SEI messages for the embodiment. TIFF0007686080000027.tif40170TIFF0007686080000028.tif30170

[0177] The first SEI message is an SEI message that conveys functionality information. An instance of the SEI message is uniquely identified using functionality_information_id. functionality_information_syntax_1 to functionality_information_syntax_N in the syntax table represent syntax elements used to specify functionality. Two functionality information SEI messages with different functionality_information_id may have different values for the functionality information syntax elements. The second SEI message is an SEI message that references a functionality information SEI message using referenced_functionality_information_id. The reference SEI message includes one or several syntax elements such as functionality_information_syntax_I (1 ≤ I ≤ N) that overwrite at least one syntax element specified in the functionality information SEI message.

[0178] Figure 8 shows a flowchart according to an embodiment. Process 800 is a method for decoding segments from a bitstream. The method may begin at step s802.

[0179] Step s802 includes receiving a first packet of a first packet type in the bitstream, the first packet including a first set of syntax elements.

[0180] Step s804 includes decoding an identifier value from one or more of the syntax elements in the first set of syntax elements of the first packet, the identifier value representing the identifier of the first packet.

[0181] Step s806 involves decoding function information from a set of syntax elements in the bitstream, including decoding the function information where the function information specifies a function.

[0182] Step s808 involves receiving a second packet of a second packet type in the bitstream, including receiving the second packet of the second packet type where the second packet comprises a second set of syntax elements.

[0183] Step s810 includes decoding a reference ID value from one or more syntax elements in the second set of syntax elements of the second packet.

[0184] Step s812 involves receiving a third packet of a third packet type in the bitstream, including receiving the third packet of the third packet type where the third packet comprises a third set of syntax elements and the third set of syntax elements encodes a segment. Encoding a segment can include fully encoding the segment or encoding a part of the segment. In some embodiments, the third packet comprises a coded representation of at least 95% of the segment.

[0185] Step s814 includes determining that the segment is associated with the second packet.

[0186] Step s816 includes determining that the reference ID value corresponds to an identifier value.

[0187] Step s818 includes selecting function information from the first set of syntax elements in response to determining that the reference ID value corresponds to an identifier value.

[0188] Step s820 includes decoding the segment from the third set of syntax elements.

[0189] Step s822 includes applying the function to the segment based on the selected function information.

[0190] In some embodiments, decoding function information from a set of syntax elements in a bitstream includes decoding function information from a first set of syntax elements in a first packet. In some embodiments, each of the first packet, the second packet, and the third packet includes a network abstraction layer (NAL) unit, respectively. In some embodiments, one or both of the first packet type and the second packet type include a SEI message. In some embodiments, the third packet type includes a slice type. In some embodiments, the first packet is of the same type and subtype (e.g., the type of the SEI message) as the second packet. If a given packet type itself can have different types, the different types of the packet type may be referred to as subtypes of the packet type. In an example for the VVC specification, the type is a NAL unit type such as PREFIX_SEI_NUT nal_unit_type that specifies that the NAL unit is a prefix SEI message. The subtype can here be the payload type of the SEI message carried in the prefix SEI message NAL unit. The payload type is decoded from the payload_type_byte syntax element in the NAL unit, and the payloadType variable is assigned the value of the payload type. An example of a subtype is the buffering period SEI message, which is signaled using payload type 0 and setting payloadType to 0.

[0191] In some embodiments, the first packet is of the same type as the second packet, and the first packet is of a different subtype than the second packet. In some embodiments, a segment comprises one or more of a picture, sub-picture, slice, tile, and coding tree unit (CTU). In some embodiments, the method further comprises decoding an indicator value from a first set of syntax elements, the indicator value indicating whether both an identifier value and a second set of syntax elements are present in the bitstream. In some embodiments, determining that a segment is associated with a second packet comprises (i) determining that the segment is in the same access unit (AU) or picture unit (PU) as the second packet, (ii) determining that a third packet comprising the segment comes immediately after the second packet in the bitstream, (iii) determining that the second packet is the closest preceding packet of the second packet type, the closest preceding packet of the second packet type being a packet having the second packet type that precedes a third packet comprising the segment in decoding order and having no other packet of the second packet type that comes after the closest preceding packet and precedes a third packet comprising the segment in decoding order, (iv) determining that the second packet comprises a persistence flag that specifies whether the second packet is associated with the segment or a third packet comprising the segment, and (v) decoding a segment identifier value representing an identifier of the segment from one or more syntax elements in the bitstream, decoding a reference segment ID value from one or more syntax elements in the second packet, and determining that the segment identifier value corresponds to the reference segment ID value, including one or more of.

[0192] In some embodiments, the method is to decode an indicator value from a first set of syntax elements, the indicator value indicating whether functional information can be applied to a segment without the first packet being referenced by a second packet, and further comprising decoding the indicator value. In some embodiments, the first packet precedes the second packet in the decoding order in the bitstream. In some embodiments, the functional information comprises one or more of film grain parameters, luma mapping with chroma scaling (LMCS) parameters, mastering display color volume parameters, content light level parameters, ambient viewing environment parameters, content color volume parameters, sphere rotation parameters, regional packing parameters, omnidirectional viewport parameters, sample aspect ratio parameters, annotated region parameters, scalability dimension parameters, multi-view acquisition parameters, depth representation parameters, filter model type, filter strength, and model description. In some embodiments, the functional information comprises information specifying a film grain model, and the method further comprises decoding a seed value from one or more syntax elements in a second set of syntax elements and using the seed value when applying film grains having the film grain model to a segment associated with the second packet.

[0193] In some embodiments, the method comprises receiving a fourth packet of a fourth packet type in a bitstream, the fourth packet comprising a fourth set of syntax elements, the fourth packet preceding a second packet in decoding order, further comprising receiving a fourth packet of the fourth packet type, and decoding functional information from a set of syntax elements in the bitstream comprises decoding functional information from the fourth set of syntax elements in the fourth packet. In some embodiments, the fourth packet follows the first packet in decoding order. In some embodiments, the fourth packet is included in the first packet. In some embodiments, the fourth packet type is an SEI message. In some embodiments, the method further comprises decoding an SEI payload type value from syntax elements in the first packet, the SEI payload type value indicating the SEI payload type value of the fourth packet. In some embodiments, the identifier value is decoded from an extension in the first packet. In some embodiments, at least one syntax element of the fourth set of syntax elements overwrites functional information from one or more syntax elements of the fourth set of syntax elements.

[0194] Figure 9 shows a flowchart according to one embodiment. Process 900 is a method for encoding segments in a bitstream. The method can begin at step s902.

[0195] Step s902 comprises determining an identifier value for a first packet of a first packet type, the identifier value representing an identifier of the first packet, the first packet comprising a first set of syntax elements in the bitstream.

[0196] Step s904 includes encoding an identifier value to one or more syntax elements in the first set of syntax elements.

[0197] Step s906 is to determine a function to be applied to the segment, including determining the function where the function is specified by function information.

[0198] Step s908 includes encoding function information to one or more syntax elements among the set of syntax elements in the bitstream.

[0199] Step s910 is to encode a reference ID value corresponding to the identifier value to one or more syntax elements in the second set of syntax elements, including encoding the reference ID value where the second packet of the second packet type comprises the second set of syntax elements in the bitstream.

[0200] Step s912 is to encode the segment to one or more syntax elements in the third set of syntax elements, including encoding the segment where the third packet of the third packet type comprises the third set of syntax elements in the bitstream and the segment is associated with the second packet.

[0201] In some embodiments, encoding functional information to one or more syntax elements of a set of syntax elements in a bitstream includes encoding the functional information to a first set of syntax elements in a first packet. In some embodiments, each of the first packet, the second packet, and the third packet comprises a network abstraction layer (NAL) unit, respectively. In some embodiments, one or both of the first packet type and the second packet type comprise a SEI message. In some embodiments, the third packet type comprises a slice type. In some embodiments, the first packet is of the same type and subtype (e.g., the type of the SEI message) as the second packet. In some embodiments, the first packet is of the same type as the second packet, and the first packet is of a different subtype from the second packet. In some embodiments, a segment comprises one or more of a picture, a subpicture, a slice, a tile, and a coding tree unit (CTU).

[0202] In some embodiments, the method comprises encoding an indicator value into a first set of syntax elements, the indicator value indicating whether both an identifier value and a second set of syntax elements are present in the bitstream. In some embodiments, a segment is associated with a second packet in at least one of the following ways: (i) the segment is in the same access unit (AU) or picture unit (PU) as the second packet; (ii) a third packet comprising the segment comes immediately after the second packet in the bitstream; (iii) the second packet is the closest preceding packet of a second packet type, where the closest preceding packet of the second packet type is a packet having a second packet type that precedes a third packet comprising the segment in decoding order and there is no other packet of the second packet type that comes after the closest preceding packet and precedes a third packet comprising the segment in decoding order; (iv) the second packet comprises a persistence flag that specifies whether the second packet is associated with the segment or with a third packet comprising the segment; and (v) the segment identifier value corresponds to a reference segment ID value.

[0203] In some embodiments, the method comprises encoding an indicator value in a first set of syntax elements, the indicator value indicating whether functional information can be applied to a segment without the first packet being referenced by a second packet. In some embodiments, the first packet precedes the second packet in the decoding order in the bitstream. In some embodiments, the functional information comprises one or more of film grain parameters, luma mapping with chroma scaling (LMCS) parameters, mastering display color volume parameters, content light level parameters, ambient viewing environment parameters, content color volume parameters, sphere rotation parameters, regional packing parameters, omnidirectional viewport parameters, sample aspect ratio parameters, annotated region parameters, scalability dimension parameters, multi-view acquisition parameters, depth representation parameters, filter model type, filter strength, and model description. In some embodiments, the functional information comprises information specifying a film grain model, and the method further comprises encoding a seed value in one or more syntax elements in a second set of syntax elements.

[0204] In some embodiments, the method is encoding a fourth packet of a fourth packet type in a bitstream, the fourth packet comprising a fourth set of syntax elements and the fourth packet preceding a second packet in decode order, the method further comprising encoding functional information in a set of syntax elements in the bitstream, which includes encoding the functional information in the fourth set of syntax elements in the fourth packet. In some embodiments, the fourth packet comes after the first packet in decode order. In some embodiments, the fourth packet is included in the first packet. In some embodiments, the fourth packet type is an SEI message. In some embodiments, the method further comprises encoding an SEI payload type value in a syntax element in the first packet, the SEI payload type value indicating the SEI payload type value of the fourth packet. In some embodiments, the identifier value is encoded in an extension in the first packet. In some embodiments, at least one syntax element of the fourth set of syntax elements overwrites functional information from one or more syntax elements of the fourth set of syntax elements.

[0205] FIG. 10 is a block diagram of a node 1000 (e.g., an encoder or a decoder) according to some embodiments. As shown in FIG. 10, the node 1000 includes a processing circuit (PC) 1002 that can include one or more processors (P) 1055 (e.g., one or more general-purpose microprocessors, and / or one or more other processors such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), where the processors can be co-located in a single housing or in a single data center or can be geographically distributed (i.e., the node 1000 can be a distributed computing device), a processing circuit (PC) 1002, at least one network interface 1048 (e.g., a physical interface or an air interface), where the node 1000 includes a transmitter (Tx) 1045 and a receiver (Rx) 1047 to enable the node 1000 to send data to and receive data from other nodes connected to a network 1010 (e.g., an Internet Protocol (IP) network) to which the network interface 1048 is (physically or wirelessly) connected (e.g., the network interface 1048 can be coupled to an antenna configuration that includes one or more antennas to enable the node 1000 to send / receive data wirelessly), at least one network interface 1048, and a local memory unit (also known as a "data storage system") 1008 that can include one or more non-volatile memory devices and / or one or more volatile memory devices. In embodiments where the PC 1002 includes a programmable processor, a computer program product (CPP) 1041 can be provided. The CPP 1041 includes a computer-readable medium (CRM) 1042 that stores a computer program (CP) 1043 comprising computer-readable instructions (CRI) 744. The CRM 1042 can be a non-transitory computer-readable medium such as a magnetic medium (e.g., a hard disk), an optical medium, a memory device (e.g., a random access memory, a flash memory), etc.In some embodiments, when the CRI 1044 of the computer program 1043 is executed by the PC 1002, the CRI is configured to cause the node 1000 to perform the steps described herein (e.g., the steps described herein with reference to the flowchart). In other embodiments, the node 1000 may be configured to perform the steps described herein without the need for code. That is, for example, the PC 1002 may simply consist of one or more ASICs. Thus, the features of the embodiments described herein may be implemented in hardware and / or software.

[0206] As used herein, a network element, node, or subsystem (e.g., an encoder or decoder) can consist of one or more service network devices, including hardware and software, which communicatively interconnect other devices on the network (e.g., other network elements, end stations, etc.) and are operable to receive / consume content in a media distribution network where media content assets can be distributed and delivered using stream-based or file-based mechanisms. With respect to a plurality of subscribers and associated user equipment (UE) nodes that are operable in either a virtualized environment or a non-virtualized environment, it is adapted to host one or more applications or services. Thus, some network elements can be deployed in a wireless network environment, and other network elements can be deployed in a public packet-switched network infrastructure, including or otherwise accompanied by a suitable CDN infrastructure, such as a public content delivery network (CDN), a private CDN, or a hybrid CDN. Further, suitable network elements, including one or more embodiments described herein, can be associated with terrestrial and / or satellite broadband delivery infrastructure, such as a digital subscriber line (DSL) network architecture, a data over cable service interface specification (DOCSIS)-compliant cable modem termination system (CMTS) architecture, a switched digital video (SDV) network architecture, a hybrid fiber coaxial (HFC) network architecture, a suitable satellite access network architecture, or a broadband wireless access network architecture via cellular and / or WiFi connectivity.Accordingly, some network elements may be equipped with "multi-service network elements" that, in addition to supporting multiple application services (e.g., data and multimedia applications including 360° immersive video assets (also referred to as 360-degree video assets or simply 360 video assets) in various qualities or specifications), also support multiple network-based functions (e.g., 360° immersive A / V media ready-delivery policy management, session control, QoS policy enforcement, bandwidth scheduling management, content provider priority policy management, streaming policy management, etc.). Exemplary subscriber end stations or client devices may, in some embodiments, include various devices that may be tethered or untethered and that can consume or distribute media content assets using streaming and / or file-based download techniques that may involve a certain type of rate adaptation. Accordingly, an exemplary client device or UE device may include any device configured to execute one or more client applications to receive, record, store, and / or decrypt / render 360 video content, live media, and / or static / on-demand media, including virtual reality (VR) media, augmented reality (AR) media, mixed reality (MR) media, from one or more content providers, especially via a broadband access network, using, for example, HTTP, HTTPS, RTP, etc.Accordingly, such client devices may include next-generation IP-based STBs, networked TVs, personal / digital video recorders (PVR / DVRs), networked media projectors, portable laptop computers, netbooks, palmtops, tablets, smartphones, multimedia / video phones, mobile / wireless user equipment, portable media players, portable gaming systems or consoles (such as Wii (registered trademark), Play Station3 (registered trademark), etc.) that operate in cooperation with 3D display devices, etc., which may access or consume 360-degree content / services provided via a suitable media distribution network that may provide bandwidth and quality of experience (QoE) schemes according to one or more embodiments described herein.

[0207] One or more embodiments of the present disclosure may be implemented using different combinations of software, firmware, and / or hardware. Thus, one or more of the techniques shown in the figures (e.g., flowcharts) may be implemented using code and data stored on and executed on one or more electronic devices or nodes (e.g., subscriber client devices or end stations, network elements, etc.). Such electronic devices may use computer-readable media such as non-transitory computer-readable storage media (e.g., magnetic disks, optical disks, random access memory, read-only memory, flash memory devices, phase change memory, etc.), transient computer-readable transmission media (e.g., carrier waves, infrared signals, digital signals, etc., electrical, optical, acoustic, or other forms of propagating signals) to store and communicate the code and data (internally and / or with other electronic devices via a network). Further, such network elements may generally include a set of one or more processors coupled to one or more other components such as one or more storage devices (e.g., non-transitory machine-readable storage media) as well as (one or more) storage databases, user input / output devices (e.g., keyboards, touchscreens, pointing devices, and / or displays), and network connections for realizing signaling and / or bearer media transmission. The coupling of the set of processors to the other components may generally be through one or more buses and bridges (also called bus controllers) configured in any known (e.g., symmetric / shared multiprocessing) or hitherto unknown architecture. Thus, the storage devices or components of a given electronic device or network element may be set to store code and / or data for execution on one or more processors of that element, node, or electronic device for the purpose of implementing one or more techniques of the present disclosure.

[0208] Those skilled in the art will recognize that the above-described generalized exemplary network environment can be implemented in a hierarchical network architecture, along with various aspects of media capture and preparation, including, for example, source stream stitching, projection mapping, source media compression, tiled / ABR encoding / transcoding, packaging, etc., as well as distributed / uploading and edge node processes that occur in different network portions deployed at different hierarchical levels, and involving one or more operators, content delivery networks (CDNs), edge networks, etc. Further, in some implementations, at least some of the above-described apparatuses and processes can be cloud-based. In some configurations, a CDN can be a large distributed system of servers deployed in multiple data centers connected to the Internet or other public / private communication networks. A CDN can be a managed or unmanaged network, or even a federation of managed or unmanaged networks.

[0209] An exemplary embodiment of a media server / source system operably associated within the exemplary network environment described above can, therefore, be configured, for example, as a global headend, to receive media content from live sources and / or static file sources such as online content providers such as Hulu®, Netflix®, YouTube®, or Amazon® Prime, as well as VOD catalogs or content providers, or studios such as Disney, Warner, Sony, etc. Media content from live sources can include live programming captured with respect to cable broadcaster channels such as Time Warner channels, including any type of event such as sports / entertainment / gaming events, concerts, live TV shows such as live news broadcast sources such as national broadcasters (e.g., NBC, ABC, etc.), as well as any secondary media insertions such as advertising media channels, and including CNN, ESPN, CNBC, etc., and local broadcasters.

[0210] Abbreviations Abbreviation Explanation APS Adaptive Parameter Set AU Access Unit AUD Access Unit Delimiter ALF Adaptive Loop Filter BLA Broken Link Access CRA Clean Random Access CVS Coded Layer Video Sequence CLVS Coded Layer Video Sequence CLVSS Coded Layer Video Sequence Start CTU Coding Tree Unit CU Coding Unit DCI Decoding Capability Information DPB Decoded Picture Buffer GDR Gradual Decoding Refresh HEVC High Efficiency Video Coding IRAP Intra Random Access Point IDR Instantaneous Decoding Refresh LMCS Luma Mapping and Chroma Scaling NAL Network Abstraction Layer OLS Output Layer Set POC Picture Order Count PPS Picture Parameter Set PU Picture Unit RADL Random Access Decodable Reading RASL Random Access Skip Reading RPR Reference Picture Resampling SEI Supplemental Enhancement Information SPS Sequence Parameter Set VCL Video Coding Layer VPS Video Parameter Set VVC Versatile Video Coding

[0211] Although various embodiments are described herein (and in any appendices), it should be understood that those embodiments are presented by way of example only and not by way of limitation. Accordingly, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described above. Moreover, unless otherwise indicated herein or clearly contradicted by context, any combination of all possible variations of the elements described above in all conceivable forms thereof is encompassed by the present disclosure.

[0212] Furthermore, the processes described above and shown in the drawings are presented as a sequence of steps, but this was done for illustrative purposes only. Accordingly, it is contemplated that some steps may be added, some steps may be omitted, the order of steps may be rearranged, and some steps may be performed in parallel.

Claims

1. A method for decoding a segment from a bitstream, the method comprising: Receiving a first packet of a first packet type in the bitstream, the first packet comprising a first set of syntax elements; Decoding an identifier value from one or more syntax elements in the first set of syntax elements of the first packet, the identifier value representing an identifier of the first packet; Decoding function information from a set of syntax elements in the bitstream, the function information specifying a function; Receiving a second packet of a second packet type in the bitstream, the second packet comprising a second set of syntax elements; Decoding a reference ID value from one or more syntax elements in the second set of syntax elements of the second packet; Receiving a third packet of a third packet type in the bitstream, the third packet comprising a third set of syntax elements, the third set of syntax elements encoding the segment; Determining that the segment is associated with the second packet; Determining that the reference ID value corresponds to the identifier value; In response to determining that the reference ID value corresponds to the identifier value, selecting the function information from the first set of syntax elements; Decoding the segment from the third set of syntax elements; Applying the function to the segment based on the selected function information A method comprising:

2. The method according to claim 1, wherein decoding function information from a set of syntax elements in the bitstream comprises decoding the function information from the first set of syntax elements in the first packet.

3. Receiving a fourth packet of a fourth packet type in the bitstream, wherein the fourth packet comprises a fourth set of syntax elements and the fourth packet precedes the second packet in decoding order, further comprising receiving a fourth packet of the fourth packet type, and decoding functional information from a set of syntax elements in the bitstream includes decoding the functional information from the fourth set of syntax elements in the fourth packet, the method of claim 1.

4. The method of claim 3, wherein at least one of the first packet type, the second packet type, and the fourth packet type comprises an SEI message.

5. The method of claim 1, wherein the first packet is of the same type as the second packet and the first packet is of a different subtype than the second packet.

6. The method of claim 1, wherein the segment comprises one or more of a picture, a subpicture, a slice, a tile, and a coding tree unit (CTU).

7. Further comprising decoding an indicator value from the first set of syntax elements, wherein the indicator value indicates whether both the identifier value and the second set of syntax elements are present in the bitstream, the method of claim 1.

8. Determining that the segment is associated with the second packet is (i) determining that the segment is in the same access unit (AU) or picture unit (PU) as the second packet, (ii) determining that the third packet comes immediately after the second packet in the bitstream, (iii) determining that the second packet is the closest preceding packet of the second packet type, wherein the closest preceding packet of the second packet type is a packet having the second packet type that precedes the third packet in decoding order, and there is no other packet having the second packet type that comes after the closest preceding packet and precedes the third packet in decoding order. (iv) determining that the second packet comprises a persistence flag that specifies whether the second packet is associated with the segment or with the third packet; (v) decoding a segment identifier value representing an identifier of the segment from one or more syntax elements in the bitstream, decoding a reference segment ID value from one or more syntax elements in the second packet, and determining that the segment identifier value corresponds to the reference segment ID value; (vi) one or more of (i) through (v), the method of claim 1. (vii) Claim 9 (viii) further comprising decoding an indicator value from the first set of syntax elements, the indicator value indicating whether the functional information can be applied to the segment without the first packet being referenced by the second packet, the method of claim 1. (ix) Claim 10 (x) the functional information comprises one or more of film grain parameters, luma mapping with chroma scaling (LMCS) parameters, mastering display color volume parameters, content light level parameters, ambient viewing environment parameters, content color volume parameters, spherical rotation parameters, regional packing parameters, omnidirectional viewport parameters, sample aspect ratio parameters, annotated region parameters, scalability dimension parameters, multi-view acquisition parameters, depth representation parameters, filter model type, filter strength, and model description, the method of claim 1. (xi) Claim 11 (xii) A method for encoding segments in a bitstream, the method comprising: (xiii) determining an identifier value for a first packet of a first packet type, the identifier value representing an identifier of the first packet, the first packet comprising a first set of syntax elements in the bitstream; (xiv) encoding the identifier value in one or more of the syntax elements in the first set of syntax elements; (xv) determining a function to be applied to the segment, the function being specified by function information; encoding the functional information in one or more syntax elements of a set of syntax elements in the bitstream; encoding a reference ID value corresponding to the identifier value in one or more syntax elements of a second set of syntax elements, wherein a second packet of a second packet type comprises the second set of syntax elements in the bitstream; encoding the segment in one or more syntax elements of a third set of syntax elements, wherein a third packet of a third packet type comprises the third set of syntax elements in the bitstream and the segment is associated with the second packet; A method comprising: **Claim 12** The method according to claim 11, wherein encoding the functional information in one or more syntax elements of a set of syntax elements in the bitstream comprises encoding the functional information in the first set of syntax elements in the first packet. **Claim 13** The method according to claim 11, wherein the segment comprises one or more of a picture, a sub-picture, a slice, a tile, and a coding tree unit (CTU). **Claim 14** The segment is in the following manner: (i) the segment is in the same access unit (AU) or picture unit (PU) as the second packet; (ii) the third packet comprising the segment comes immediately after the second packet in the bitstream; (iii) the second packet is the closest preceding packet of the second packet type, and the closest preceding packet of the second packet type has a second packet type that precedes the third packet comprising the segment in decoding order, and there is no other packet of the second packet type that comes after the closest preceding packet and precedes the third packet comprising the segment in decoding order. (iv) the second packet comprising a persistence flag specifying whether the second packet is associated with the segment or is associated with a third packet comprising the segment; (v) encoding a segment identifier value in the bitstream and encoding a reference segment ID value in the second packet, the segment identifier value corresponding to the reference segment ID value; The method according to claim 11, wherein the second packet is associated with at least one of the above.

15. The method according to claim 11, wherein the function information comprises information specifying a film particle model, and the method further comprises encoding a seed value in one or more syntax elements of the second set of syntax elements.

16. A computer program comprising instructions which, when executed by a processing circuit of a node, cause the node to perform the method according to claim 1.

17. A decoder configured to decode segments from a bitstream, the decoder further configured to perform the method according to claim 1.

18. An encoder configured to encode segments in a bitstream, the encoder configured to perform the method according to claim 11.

Images