Restriction on time identifiers for SEI messages

Abstract

To provide a video coding mechanism.SOLUTION: A mechanism includes encoding a coded picture in one or more video coding layer (VCL) network abstraction layer (NAL) units in a bitstream. A non-VCL NAL unit is encoded into the bitstream such that a temporal identifier (TemporalId) for the non-VCL NAL unit is constrained to be equal to a TemporalId of an access unit (AU) containing the non-VCL NAL unit when a NAL unit type (nal_unit_type) of the non-VCL NAL indicates a supplemental enhancement information (SEI) message. A set of bitstream conformance tests is performed on the bitstream based on the SEI message. The bitstream is stored for communication toward a decoder.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application is a joint publication of "Video Coding Impr This application claims priority to U.S. Provisional Patent Application No. 62 / 905,236, entitled "Microchip Ovens."

[0002] FIELD OF THE DISCLOSURE This disclosure relates generally to video coding, and more particularly to multi-layer bitstreaming. Refining signaling parameters to support coding of multiple streams do. [Background technology]

[0003] Even for a relatively short video, the amount of video data required to represent this is significant. This makes it difficult to stream data over a communications network that has limited bandwidth capacity. Difficulties can arise when transactions are made or data is otherwise communicated. For this reason, video data is typically compressed before being transmitted over today's communications networks. If the video is stored on a storage device, memory resources may be scarce, The size of the video can also be an issue. Video compression devices often reduce the size of the video for transmission or storage. The supplier uses software and / or hardware to encode the video data prior to The digital video encoder reduces the amount of data required to represent a digital video image. The compressed data is then sent to a video decompressor which decodes the video data at the destination. Network resources are limited and higher video quality is required. With the increasing demand for high-quality images, compression is becoming more and more common. Improved compression and decompression techniques that increase the data transfer rate are desirable. Summary of the Invention [Means for solving the problem]

[0004] In one embodiment, the present disclosure includes a method implemented by a decoder, the method comprising: The receiver of the coder implements one or more Video Coding Layer (VCL) network abstractions. Includes coded pictures in VCL layer (NAL) units and non-VCL NAL units receiving a bitstream, the bitstream including a NAL unit type (nal_u nit_type) is a prefix extension information (SEI) NAL unit type (PREFIX_SEI_ NUT) or suffix SEI NAL unit type (SUFFIX_SEI_NUT), The temporal identifier (TemporalId) of a non-VCL NAL unit is the same as that of the access unit that contains the non-VCL NAL unit. The step and the decoder process are constrained to be equal to the TemporalId of the unit (AU). The NAL unit header time indicator plus one (nuh _temporal_id_plus1) TemporalId of non-VCL NAL unit based on syntax element and a processor of the decoder for generating a decoded picture. decoding a coded picture from a VCL NAL unit by Includes.

[0005] A video sequence can contain many pictures. As shown in the introduction, the video coding system assigns a TemporalId to a picture. Some video coding systems use layers of pictures, where each layer Includes videos that are essentially the same but with different resolutions, picture sizes, frame rates, etc. Pictures in the layer that have been selected can be selectively displayed according to conditions on the decoder side. Thus, pictures in different layers that are located at the same point in the video sequence are In addition, pictures in different layers that share the same TemporalId For example, a decoder may perform the following operations in each AU to display a video sequence: Some video coding schemes may use the same image to display a single picture selected from a single layer. The encoding system uses the SEI message, which is used to indicate the sample in the decoded picture. It contains information that is not required by the decryption process to determine the value of the SEI message. Sage runs the code on the encoder to verify the bitstream for compliance with the standard. It may also contain parameters used by a hypothetical reference decoder (HRD) that operates the The decoding system encodes the video sequence into a bitstream as layers of pictures. The SEI message may be encoded into a frame of a changing picture and / or layer. Thus, a complex multi-layer bit stream can be associated with The system ensures that the correct SEI messages are associated with the correct picture / layer. It can be difficult to ensure that SEI messages are associated with the correct layer / picture. If the layer / picture compatibility is not properly verified, the HRD may not be able to properly verify the layer / picture compatibility. This may lead to encoding errors.

[0006] This example shows the mechanism to correctly associate SEI messages with their corresponding pictures / layers. A multi-layer bitstream contains pictures and associated parameters. The AU can be organized into AUs. An AU is a set of coded pictures contained in different layers. and are associated with the same output time. In addition, the SEI message may include the TemporalId in The TemporalId is the relative position of the NAL unit in the video sequence. The TemporalId of an SEI message is an identifier that indicates the Temporal In other words, a picture is a VCL NAL unit. The non-VCL NAL unit is included in the SEI message, and the parameters are included in the non-VCL NAL unit. For a SEI NAL unit that contains a non-VCL NAL unit, the TemporalId of the non-VCL NAL unit is This technique restricts the SEI message to be equal to the TemporalId of the AU that contains the message. This ensures that the image is correctly associated with the corresponding picture in the AU. Errors can be avoided, resulting in improved performance of the encoder and decoder. Furthermore, coding efficiency has been improved, and the processing time is reduced both in the encoder and the decoder. Reduce utilization of processor, memory, and / or network signaling resources .

[0007] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein a non-VCL The nal_unit_type of the NAL is equal to PREFIX_SEI_NUT.

[0008] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein a non-VCL The nal_unit_type of the NAL is equal to SUFFIX_SEI_NUT.

[0009] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein the code The inserted picture is assigned to the VCL NAL unit based on the SEI message in the non-VCL NAL unit. The bit is decoded from the

[0010] Optionally, in any of the preceding aspects, a further implementation of the aspect is provided, further comprising: Derive the TemporalId of non-VCL NAL units according to nuh_temporal_id_plus1-1. Further includes:

[0011] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein nuh_t The value of emporal_id_plus1 is not zero.

[0012] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein the VCL NA The TemporalId of an L-unit is constrained to be the same for all VCL NAL units in the same AU. .

[0013] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, a second bitstream including one or more second VCL NAL units and a second non-VCL NAL unit; receiving the stream, wherein the nal_unit_type of the second non-VCL NAL is equal to or greater than the SEI message; In the case of a message, the TemporalId of the second non-VCL NAL unit is the TemporalId of the first message that contains the second non-VCL NAL unit. and in response to the step of receiving, the TemporalId of the second AU is not equal to the TemporalId of the second AU. Before decoding the coded picture from the VCL NAL unit, a second bitstream is generated. Any other modifications to ensure that a bitstream conforming to the protocol is received. The method further includes the steps of:

[0014] In one embodiment, the present disclosure includes a method implemented by an encoder, the method comprising: , by the encoder's processor in one or more VCL NAL units in the bitstream. and encoding the coded picture by a processor. encoding the VCL NAL units into a bitstream, such that When the nal_unit_type of a non-VCL NAL is an SEI message, the nuh_tempo of the non-VCL NAL unit ral_id_plus1 is equal to nuh_temporal_id_plus1 of the AU that contains the non-VCL NAL unit. and a processor for determining whether to perform a process based on the SEI message. performing a set of bitstream conformance tests on the bitstream; a bitstream for communication to a decoder by a memory coupled to the processor; and storing the result.

[0015] A video sequence can contain many pictures. As shown in the introduction, the video coding system assigns a TemporalId to a picture. Some video coding systems use layers of pictures, where each layer Includes videos that are essentially the same but with different resolutions, picture sizes, frame rates, etc. Pictures in the layer that have been selected can be selectively displayed according to conditions on the decoder side. Thus, pictures in different layers that are located at the same point in the video sequence are In addition, pictures in different layers that share the same TemporalId For example, a decoder may perform the following operations in each AU to display a video sequence: Some video coding schemes may use the same image to display a single picture selected from a single layer. The encoding system uses the SEI message, which is used to indicate the sample in the decoded picture. It contains information that is not required by the decryption process to determine the value of the SEI. The message is generated by the encoder to verify the bitstream for conformance with the standard. It may contain parameters used by the operating hypothetical reference decoder (HRD). A video coding system encodes a video sequence into a bitstream in layers of pictures. The SEI message may be encoded into a stream of changing pictures and / or layers. Thus, complex multi-layer bitstreams can be associated with varying combinations. The frame ensures that the correct SEI messages are associated with the correct picture / layer. It can be difficult to ensure that SEI messages are associated with the correct layer / picture. If not, the HRD may not be able to properly verify layer / picture compatibility. This can lead to encoding errors.

[0016] This example shows the mechanism to correctly associate SEI messages with their corresponding pictures / layers. A multi-layer bitstream contains pictures and associated parameters. The AU can be organized into AUs. An AU is a set of coded pictures contained in different layers. and are associated with the same output time. In addition, the SEI message may include the TemporalId in The TemporalId indicates the relative position of the NAL unit in the video sequence. The TemporalId of an SEI message is the TemporalId of the AU that contains the SEI message. In other words, a picture is contained in a VCL NAL unit. Rarely, parameters are included in non-VCL NAL units. Non-VCL NAL units are included in SEI messages. For a SEI NAL unit that contains This approach restricts the TemporalId of the SEI message to be equal to the TemporalId of the AU that contains the SEI message. This ensures that each picture is correctly associated with its corresponding picture in the AU. As a result, the performance of the encoder and the decoder is improved. In addition, coding efficiency is improved, and the processor is The present invention reduces utilization of network, memory, and / or network signaling resources.

[0017] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein a non-VCL The nal_unit_type of the NAL is equal to PREFIX_SEI_NUT.

[0018] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein a non-VCL The nal_unit_type of the NAL is equal to SUFFIX_SEI_NUT.

[0019] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein nuh_t The value of emporal_id_plus1 is not zero.

[0020] Optionally, in any of the preceding aspects, there is provided another implementation of the aspect, wherein the VCL NA The nuh_temporal_id_plus1 of an L unit is the same for all VCL NAL units in the same AU. The number of users is limited to the following.

[0021] In one embodiment, the present disclosure provides a method for detecting a signal from a processor, a receiver coupled to the processor, and a method for detecting a signal from the processor. a memory coupled to the processor; and a transmitter coupled to the processor, A video copier, comprising: a receiver, a memory, and a transmitter configured to perform the method of the aforementioned aspect. Includes a loading device.

[0022] In one embodiment, the present disclosure provides a computer program for use by a video coding device. A non-transitory computer readable medium containing a computer program product, The system article includes computer-executable instructions stored on a non-transitory computer-readable medium, When executed by a processor, the video coding apparatus is adapted to perform a method according to any of the preceding aspects. Make the law happen.

[0023] In one embodiment, the present disclosure provides a method for determining whether a coded picture in one or more VCL NAL units is A receiving means for receiving a bitstream including a non-VCL NAL unit and a non-VCL chunk. When the nal_unit_type of a CL NAL is an SEI message, the TemporalId of a non-VCL NAL unit is TemporalId of the AU containing the non-VCL NAL unit, , the coded picture is extracted from the VCL NAL unit to generate the decoded picture. and displaying the decoded video sequence as part of the video sequence. and a transfer means for transferring the decoded pictures to the decoder.

[0024] A video sequence can contain many pictures. As shown in the introduction, the video coding system assigns a TemporalId to a picture. Some video coding systems use layers of pictures, where each layer Includes videos that are essentially the same but with different resolutions, picture sizes, frame rates, etc. Pictures in the layer that have been selected can be selectively displayed according to conditions on the decoder side. Thus, pictures in different layers that are located at the same point in the video sequence are In addition, pictures in different layers that share the same TemporalId For example, a decoder may perform the following operations in each AU to display a video sequence: Some video coding schemes may use the same image to display a single picture selected from a single layer. The encoding system uses the SEI message, which is used to indicate the sample in the decoded picture. It contains information that is not required by the decryption process to determine the value of the SEI. The message is run by the encoder to verify the bitstream for conformance with the standard. It may also contain parameters used by a hypothetical reference decoder (HRD) that operates the The decoding system encodes the video sequence into a bitstream as layers of pictures. The SEI message may be encoded into a frame of a changing picture and / or layer. Thus, a complex multi-layer bit stream can be associated with The system ensures that the correct SEI messages are associated with the correct picture / layer. It can be difficult to ensure that SEI messages are associated with the correct layer / picture. If the layer / picture compatibility is not properly verified, the HRD may not be able to properly verify the layer / picture compatibility. This may lead to encoding errors.

[0025] This example shows the mechanism to correctly associate SEI messages with their corresponding pictures / layers. A multi-layer bitstream contains pictures and associated parameters. The AU can be organized into AUs. An AU is a set of coded pictures contained in different layers. and are associated with the same output time. In addition, the SEI message may include the TemporalId in The TemporalId is the relative position of the NAL unit in the video sequence. The TemporalId of an SEI message is an identifier that indicates the Temporal In other words, a picture is a VCL NAL unit. The non-VCL NAL unit is included in the SEI message, and the parameters are included in the non-VCL NAL unit. For a SEI NAL unit that contains a non-VCL NAL unit, the TemporalId of the non-VCL NAL unit is This technique restricts the SEI message to be equal to the TemporalId of the AU that contains the message. This ensures that the image is correctly associated with the corresponding picture in the AU. Errors can be avoided, resulting in improved performance of the encoder and decoder. Furthermore, coding efficiency has been improved, and the processing time is reduced both in the encoder and the decoder. Reduce utilization of processor, memory, and / or network signaling resources .

[0026] Optionally, in any of the preceding aspects, the decoder comprises: Another implementation of the aspect is provided that is further configured to perform the method.

[0027] In one embodiment, the present disclosure provides a method for determining whether a VCL NAL unit in one or more VCL NAL units in a bitstream is Encodes a coded picture and adds the nal_unit_type of non-VCL NAL to the SEI message. In the case of a non-VCL NAL unit, the TemporalId of the non-VCL NAL unit is the Te mporalId to be constrained to be equal to the non-VCL NAL unit and encoding the SEI message into a bitstream based on the encoding means. a bitstream conformance test unit for performing a set of bitstream conformance tests on the bitstream and communicating the bitstream conformance test unit to the decoder; and storage means for storing a bitstream for encoding the encoded bitstream.

[0028] A video sequence can contain many pictures. As shown in the introduction, the video coding system assigns a TemporalId to a picture. Some video coding systems use layers of pictures, where each layer Includes videos that are essentially the same but with different resolutions, picture sizes, frame rates, etc. Pictures in the layer that have been selected can be selectively displayed according to conditions on the decoder side. Thus, pictures in different layers that are located at the same point in the video sequence are In addition, pictures in different layers that share the same TemporalId For example, a decoder may perform the following operations in each AU to display a video sequence: Some video coding schemes may use the same image to display a single picture selected from a single layer. The encoding system uses the SEI message, which is used to indicate the sample in the decoded picture. It contains information that is not required by the decryption process to determine the value of the SEI. The message is generated by the encoder to verify the bitstream for conformance with the standard. It may contain parameters used by the operating hypothetical reference decoder (HRD). A video coding system encodes a video sequence into a bitstream in layers of pictures. The SEI message may be encoded into a stream of changing pictures and / or layers. Thus, complex multi-layer bitstreams can be associated with varying combinations. The frame ensures that the correct SEI messages are associated with the correct picture / layer. It can be difficult to ensure that SEI messages are associated with the correct layer / picture. If not, the HRD may not be able to properly verify layer / picture compatibility. This can lead to encoding errors.

[0029] This example shows the mechanism to correctly associate SEI messages with their corresponding pictures / layers. A multi-layer bitstream contains pictures and associated parameters. The AU can be organized into AUs. An AU is a set of coded pictures contained in different layers. and are associated with the same output time. In addition, the SEI message may include the TemporalId in The TemporalId is the relative position of the NAL unit in the video sequence. The TemporalId of an SEI message is an identifier that indicates the Temporal In other words, a picture is a VCL NAL unit. The non-VCL NAL unit is included in the SEI message, and the parameters are included in the non-VCL NAL unit. For a SEI NAL unit that contains a non-VCL NAL unit, the TemporalId of the non-VCL NAL unit is This technique restricts the SEI message to be equal to the TemporalId of the AU that contains the message. This ensures that the image is correctly associated with the corresponding picture in the AU. Errors can be avoided, resulting in improved performance of the encoder and decoder. Furthermore, coding efficiency has been improved, and the processing time is reduced both in the encoder and the decoder. Reduce utilization of processor, memory, and / or network signaling resources .

[0030] Optionally, in any of the preceding aspects, the encoder comprises: Another implementation of the aspect is provided that is further configured to perform a method.

[0031] For clarity, any one of the above-mentioned embodiments may be combined with any other of the above-mentioned embodiments. may be combined with any one or more of these to create new embodiments within the scope of the present disclosure. This may be the case.

[0032] These and other features are apparent from the following detailed description, taken in conjunction with the accompanying drawings and claims. This will be more clearly understood.

[0033] For a more complete understanding of the present disclosure, reference should now be made to the accompanying drawings and detailed description. Reference is made to the following brief description, which is to be understood as including similar references in the accompanying drawings and detailed description. Reference numbers represent similar parts. [Brief description of the drawings]

[0034] [Figure 1] 1 is a flow diagram of an exemplary method for encoding a video signal. [Diagram 2] 1 is a schematic diagram of an exemplary encoding-decoding (codec) system for video encoding. [Diagram 3] FIG. 1 is a schematic diagram illustrating an exemplary video encoder. [Figure 4] FIG. 2 is a schematic diagram illustrating an exemplary video decoder. [Diagram 5] 1 is a schematic diagram illustrating an exemplary hypothetical reference decoder (HRD). [Figure 6] FIG. 2 is a schematic diagram illustrating an exemplary multi-layer video sequence. [Figure 7] FIG. 2 is a schematic diagram illustrating an exemplary bitstream. [Figure 8] 1 is a schematic diagram of an exemplary video coding device; [Figure 9] 1 is a flow diagram of an example method for encoding a video sequence into a bitstream by restricting a temporal identifier (TemporalId) of a supplemental enhancement information (SEI) message in the bitstream. [Figure 10] 1 is a flow diagram of an example method for decoding a video sequence from a bitstream in which the TemporalId of an SEI message in the bitstream is restricted. [Figure 11] 1 is a schematic diagram of an example system for encoding a video sequence using a bitstream, in which the TemporalId of an SEI message in the bitstream is restricted. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] First, exemplary implementations of one or more embodiments are provided below, but the disclosed The systems and / or methods described herein are not intended to be limiting or in any way restrictive, whether currently known or in existence. It should be understood that the present disclosure may be implemented using a number of techniques. Exemplary implementations shown below, including exemplary designs and implementations shown or described The present invention should in no way be limited to the embodiments, drawings, and techniques described herein, together with their full scope of equivalents. Modifications may be made within the scope of the appended claims.

[0036] The following terms are defined as follows, unless used herein to the contrary: Specifically, the following definitions are intended to further clarify the present disclosure. However, terms may be described differently in different contexts. Therefore, the following definitions are used: should be considered auxiliary and in accordance with the explanation given to such terms herein. It should not be considered as limiting other definitions.

[0037] The bitstream is a compressed video stream for transmission between the encoder and decoder. The encoder compresses the video data into a bitstream. A decoder is a device configured to use the encoding process to generate a stream of A decoding process is then used to recover the video data from the bitstream for display. A picture is generated for a frame or a field thereof. An array of luma samples and / or an array of chroma samples. A slice is a single An integer number of complete pictures contained exclusively in a Network Abstraction Layer (NAL) unit. A tile, or an integer number of consecutive complete coding tree units (e.g., within a tile) For clarity, the picture being encoded or decoded is A coded picture is a part of an access unit ( A video having a specific value of the NAL unit header layer identifier (nuh_layer_id) in the NAL unit header layer identifier (nuh_layer_id) Coding Layer (VCL) is a coded representation of a picture that contains NAL units. A decoded picture contains the entire coding tree unit (CTU) of the picture. A picture is generated by applying a decoding process to a loaded picture.

[0038] An AU is a set of coded pictures contained in different layers, and a decoded picture is The NAL units are associated with the same time for output from the Decoder Buffer (DPB). It is a syntax structure that contains the data in the Byte Sequence Payload (RBSP) format. This indicates the type of data, interspersed with emulation prevention bytes if necessary. A VCL NAL unit contains video data, such as a coded slice of a picture. Non-VCL NAL units are NAL units that are coded to include the Syntax and / or parameters that support performing conformance checks and other operations NAL unit type (nal_unit_type ) is a syntax statement contained in a NAL unit that indicates the type of data contained in the NAL unit. A layer is indicated by a layer ID and an associated non-VCL NAL unit. Share the same characteristics (for example, a common resolution, frame rate, image size) as specified The set of VCL NAL units that represent the layer identifier (nuh_layer_id) of the NAL unit header. is a syntax element that specifies the identifier of the layer that contains the NAL unit. (TemporalId) is a derived Identifier that indicates the relative position of the NAL unit within a video sequence. The NAL unit header temporal identifier plus one (nuh_temporal_id_plus1 ) is a signaled identifier that indicates the relative position of a NAL unit within a video sequence. It is Besshi.

[0039] A hypothetical reference decoder (HRD) performs code generation to verify conformance to certain constraints. A data processor that runs on the encoder to check for variability in the bitstream produced by the encoding process. The bitstream conformance test is based on the encoded bitstream. Testing to determine whether a system complies with standards such as Universal Video Coding (VVC) The HRD parameters are syntax expressions that initialize and / or define the operating conditions of the HRD. The HRD parameters are included in the Supplemental Enhancement Information (SEI) message and / or in the video The SEI message can be included in the parameter set (VPS). A specific semaphore is used to convey information that is not required by the decryption process to determine the value of the A SEI NAL unit is a syntax structure with semantics. A specific SEI NAL unit is now called a SEI NAL unit. A scalable nesting SEI message may contain one or more output layer sets ( A message that contains multiple SEI messages, each corresponding to one or more layers. The Buffering Period (BP) SEI message is used to inform the In the SEI message containing the HRD parameters to initialize the HRD to manage the CPB. The Picture Timing (PT) SEI message is used to inform the CPB and / or the decoded picture buffer. This is an SEI message that includes HRD parameters for managing AU delivery information in the DPB. The Decoding Unit Information (DUI) SEI message manages the delivery information of DUs in the CPB and / or DPB. Scalable nesting SEI is an SEI message that contains HRD parameters for managing the Scalable Nesting SEI. A message is a set of scalable nested SEI messages. A nested SEI message is nested inside a scalable nesting SEI message. A prefix SEI message is a prefix SEI message that is a part of one or more subsequent NAL units. A suffix SEI message is an SEI message that applies to one or more preceding This is an SEI message that applies to the NAL unit being processed.

[0040] The Picture Parameter Set (PPS) is a set of syntax elements found in each picture header. Syntax applied to the entire coded picture as determined by the The picture header is a syntax structure that contains the picture information elements. A syntax structure that contains syntax elements that apply to all slices of a slice. The header contains data elements for the entire tile or for the CTU rows in the tile represented by the slice. A coded video sequence that includes a slice that is part of the coded slice. is a set of one or more coded pictures. A picture sequence is a set of one or more decoded pictures.

[0041] In this specification, the terms access unit (AU), coding tree block (CTB), and coding coding tree unit (CTU), coding unit (CU), coding layer video sequence Video Sequence Start (CLVS), Coding Layer Video Sequence Start (CLVSS), Video sequence (CVS), coded video sequence start (CVSS), joint video Joint Video Expert Team (JVET), Virtual Reference Decoder HRD, Motion Constraint Tile Set (MCTS), Large Transmission Unit (MTU), Network Abstraction Layer (NAL), Output Layer Set (OLS), Point of Order Count (POC), Random Access Point (RAP), Raw Byte Sequence Payload (RBSP), sequence parameter set (SPS), video parameter set (V Acronyms such as Video Standards Track (GSTC) and Video Codec Generalized (VVC) are used.

[0042] Many video formats are supported to reduce the size of video files with minimal data loss. Compression techniques can be used. For example, video compression techniques can reduce the amount of In order to reduce or remove data redundancy, spatial (e.g., intra-picture) prediction and and / or performing temporal (e.g., inter-picture) prediction. In block-based video coding, a video slice (e.g., a video picture or A video picture (part of a video picture) may be partitioned into video blocks, The blocks are tree blocks, coding tree blocks (CTBs), coding tree units, Also referred to as coding units (CTUs), coding units (CUs), and / or coding nodes. Video blocks within an intra-coded (I) slice of a picture may be The image is coded using spatial prediction with respect to reference samples in adjacent blocks in the same frame. The video in an inter-coded unidirectionally predicted (P) or bidirectionally predicted (B) slice of a picture Of-blocks are spatially related to reference samples in adjacent blocks in the same picture. prediction, or coding using temporal prediction with respect to reference samples in other reference pictures A picture may be referred to as a frame and / or an image, and a reference picture may be used. The spatial prediction or temporal prediction is performed by multiplying the image block by the number of frames. The residual data is the difference between the original image block and the predicted block. Therefore, inter-coded blocks are the pixel differences between the reference sub-blocks that form the prediction block. The motion vector that points to the sample block and the difference between the coding block and the prediction block are The intra-coded block is encoded according to the intra-coded residual data. The residual data is encoded according to the encoding mode and the residual data. may be transformed from the pixel domain to the transform domain. These become the residual transform coefficients and are quantized. The quantized transform coefficients may be initially arranged in a two-dimensional array. The coefficients may be scanned to generate a one-dimensional vector of transform coefficients. To achieve this, entropy coding may be applied. , which are described in more detail below.

[0043] To ensure that the encoded video can be decoded correctly, the video must be Video coding standards include the International Telecommunications Union (ITU) Standardization Sector (ITU-T) H.261, International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Moving Picture Experts Group (MPEG)-1 Part 2, ITU-T H.262 or ISO / IEC MP EG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or ISO / IE C Advanced Video Coding (AVC), also known as MPEG-4 Part 10, and ITU-T H.265 AVC includes High Efficiency Video Coding (HEVC), also known as MPEG-H Part 2. Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding extensions such as Multimedia Video Coding Plus Depth (MVC+D) and Three-Dimensional (3D) AVC (3D-AVC) HEVC is divided into Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D HEVC. (3D-HEVC) and other extensions. ITU-T and ISO / IEC Joint Video Experts Team (JVET ) began developing a video coding standard called Versatile Video Coding (VVC). It is included in the draft (WD) and includes JVET-O2001-v14.

[0044] A video sequence can contain many pictures. As shown in the introduction, video coding systems use temporal identifiers (TRIs) in pictures. Some video coding systems may assign a layered image to a picture. Each layer is essentially the same image, with different resolutions, picture sizes, frame rates, etc. The pictures in different layers are selectively displayed according to the decoder's requirements. Thus, pictures in different layers that are located at the same point in the video sequence can be In addition, different layers that share the same TemporalId The pictures in a stream constitute an access unit (AU). To display a sequence, each AU can display a single picture selected from a single layer. do.

[0045] Some video coding systems use the SEI message. are needed in the decoding process to determine the values ​​of the samples in the decoded picture. For example, the SEI message verifies the bitstream's conformance to the standard. The parameter may include parameters used by the HRD operating in the encoder to In the video coding system, the video sequence is divided into layers of pictures. The SEI messages may be encoded into a stream of various pictures and / or layers. Various combinations can be associated with this, thus resulting in a complex multi-layer bit stream. The system ensures that the correct SEI messages are associated with the correct picture / layer. It can be difficult to ensure that SEI messages are associated with the correct layer / picture. If the layer / picture compatibility is not properly verified, the HRD may not be able to properly verify the layer / picture compatibility. This may lead to encoding errors.

[0046] This specification discloses a method to correctly associate SEI messages with the corresponding pictures / layers. A multi-layer bitstream is a mechanism for storing pictures and associated The parameters obtained can be organized into AUs. An AU is a set of coding units contained in different layers. The SEI message is a sequence of pictures that are associated with the same output time. The SEI message may be placed in the same AU as the first picture associated with the message. , TemporalId. TemporalId is the network ID of the video sequence. It is an identifier that indicates the relative position of a NAL unit. The oralId is constrained to be equal to the TemporalId of the AU that contains the SEI message. In other words, pictures are contained in video coding layer (VCL) NAL units and are The non-VCL NAL unit contains the SEI message. For non-VCL NAL units, the TemporalId of the non-VCL NAL unit is the Te of the AU that contains the non-VCL NAL unit. This technique ensures that SEI messages are consistent with their corresponding AUs. This ensures that the image is correctly associated with the picture it is attached to, thus avoiding various errors. As a result, the performance of the encoder and decoder can be improved. This improves the coding efficiency and reduces the processor, memory, and and / or reduce utilization of network signaling resources.

[0047] FIG 1 is a flow diagram of an exemplary operational method 100 of encoding a video signal. The video signal is then encoded by an encoder. The encoding process creates a video file. It compresses the video signal using various techniques to reduce the file size. ,Transmission of compressed video files to users, and the associated bandwidth overhead The decoder then decodes the compressed video file and The decoder restores the original video signal for display to the end user. To allow for robust reconstruction, the decoding process usually mirrors the encoding process. .

[0048] In step 101, a video signal is input to an encoder. As another example, the video may be an uncompressed video file stored in a Video files are captured by a video capture device such as a video camera, Video files may be encoded to support live streaming of video. A file may contain both audio and video components. The video component is a sequence of image frames. A frame contains a series of pixels which, when viewed in succession, give the visual impression of motion. A pixel contains light, referred to here as the luma component (or luma sample), and a chroma component ( In some cases, a three-dimensional view is used to represent the color, called a color sample. To support this, a frame may also contain a depth value.

[0049] In step 103, the video is partitioned into blocks. The coding sub-divides the pixels in each frame into square and / or rectangular blocks for compression. For example, High Efficiency Video Coding (HEVC) (combined with H.265 and MPEG-H Part 2) In JPEG2000 (also known as JPEG2000), a frame is first subdivided into coding tree units (CTUs). The CTU can be divided into blocks of a given size (for example, 64 pixels x 64 pixels). Contains both chroma and matrices. The coding tree is used to split the CTU into blocks. Then, split the blocks until a configuration is achieved that supports further encoding. For example, the luma component of a frame can be subdivided recursively into individual blocks. The chrominance components of the frame may be further subdivided until they contain relatively uniform illumination values. The divisions may be subdivided until each block contains relatively uniform color values. Thus, the partitioning scheme varies depending on the content of the video frame.

[0050] In step 105, the image blocks partitioned in step 103 are compressed in various ways. For example, inter-prediction and / or intra-prediction may be used. Inter prediction is useful because objects in a typical scene tend to appear in consecutive frames. Therefore, the block representing the object in the reference frame is A block does not need to be repeated in adjacent frames. An object may remain in a certain position for multiple frames. Therefore, the desk may be The neighboring frames can refer to the reference frame. By using the same image, it is possible to align objects across multiple frames. Due to object motion and camera motion, moving objects may be shown over multiple frames. As a specific example, a video shows a car moving across the screen over multiple frames. Such movements can be described using motion vectors. A motion vector is The offset from the coordinates of an object in a frame to the coordinates of that object in a reference frame Therefore, inter prediction is a two-dimensional vector that provides the image block in the current frame. The block is then converted into a set of motion vectors that indicate the offset from the corresponding block in the reference frame. can be encoded as

[0051] Intra prediction encodes blocks within a given frame. This exploits the fact that ma and chroma components tend to cluster together in a frame. For example, Green patches in one part of a tree tend to be located near similar green patches. The directional prediction supports multiple directional prediction modes (e.g. 33 in HEVC), planar modes, and direct conversion. The current block uses the current (DC) mode. The directional mode uses the current block in the corresponding direction. Planar mode indicates that the samples in the block are similar / same. ) can be interpolated based on the neighboring blocks at the edges of the row. The planar mode is actually achieved by using a relatively constant slope for the varying values. , which shows a smooth transition of light / color across rows / columns. DC mode is used for boundary smoothing, and block The block is related to the samples of all neighboring blocks related to the angular direction of the directional prediction mode. Therefore, the intra predicted block is actually Instead of the value of , the image block can be represented as various related prediction mode values. Inter-prediction blocks represent image blocks as motion vector values ​​instead of actual values. In either case, the prediction block may not accurately represent the image block in some cases. The difference is stored in the residual block. Therefore, a transform can be applied to the residual block.

[0052] In step 107, various filtering techniques are applied. The filter is applied according to the filtering scheme described above. In addition, block-based prediction methods can create blocky images. The block is then encoded and the encoded block is used later as a reference block. The in-loop filtering method is a noise suppression filter, a deblocking filter, and Filtering, adaptive loop filter, and sample adaptive offset (SAO) filter These filters are applied repeatedly to blocks / frames. Reduces coding artifacts so encoded files can be restored accurately In addition, these filters are used to reduce artifacts in the reconstructed reference blocks. , further arching in subsequent blocks that are encoded based on the reconstructed reference block. The likelihood of a fact occurring becomes smaller.

[0053] Once the video signal has been partitioned, compressed and filtered, the resulting digital The data is encoded into a bitstream in step 109. In addition to containing the above data, it also contains the following additional information to support proper video signal recovery at the decoder: For example, such data may include partition data. , prediction data, residual blocks, and various flags that provide coding instructions to the decoder. The bitstream is stored in memory for transmission to a decoder on demand. The bitstream may also be broadcast and / or distributed to multiple decoders. Or it may be multicast. Creating a bitstream is an iterative process. Thus, steps 101, 103, 105, 107, and 109 process a number of frames and blocks. The steps may be performed sequentially and / or simultaneously over a period of time. The present invention is presented for clarity and simplicity and does not restrict the video encoding process to a specific order. It is not intended to.

[0054] The decoder receives the bitstream and begins the decoding process in step 111. Specifically, the decoder uses an entropy decoding method to convert the bitstream into a corresponding The decoder converts the bitstream data into syntax data and video data in step 111. Determines the partition of the frame using syntax data from the stream. The partitioning must be consistent with the block partitioning in step 103. This allows the entropy encoding / decoding used in step 111 to be The encoder chooses between several options based on the spatial arrangement of values ​​in the input image. Many choices are made during the compression process, including choosing a block partitioning scheme from the A number of bins may be used to convey different options. The bins used here are: A binary value that is treated as a variable (i.e., a bit value that can change depending on the situation). In loop coding, the encoder breaks options that obviously do not hold in certain situations. You can then discard all the options, leaving a set of acceptable options. Each possible option is assigned a codeword whose length depends on the number of allowable options. based on (e.g., 1 bin for 2 options, 2 bins for 3-4 options, etc. The encoder then encodes the selected optional codeword. A word uniquely indicates a choice from a potentially large set of all possible options. whereas a simple approximation uniquely indicates a choice among a small subset of acceptable options. This scheme reduces the size of the codeword so that it is the desired size when Similar to the encoder, it makes a selection by determining the set of acceptable options. The decoder decodes the codeword by determining the set of allowable options. can be read to determine the selection made by the encoder.

[0055] In step 113, the decoder performs block decoding. Specifically, the decoder performs inverse The decoder then uses the transform to generate a residual block. The predicted block is used to restore the image block based on the partitioning. The block is a combination of the intra-predicted block and the inter-predicted block generated by the encoder in step 105. The reconstructed image block may then include both the measured and unmeasured blocks. The partitioning data is used to determine the location of the frame in the restored video signal. The syntax of step 113 is also encoded into a bitstream by the entropy coding described above. The information may be transmitted in a radio

[0056] In step 115, the frame of the restored video signal is read out in the same manner as in step 107 in the encoder. Filtering is performed on the frames to remove blocking artifacts. , e.g., noise suppression filters, deblocking filters, adaptive loop filters, and You can apply the SAO filter to a frame. Once the frame is filtered, you can save the video in step 117. The audio signal can be output to a display for viewing by an end user.

[0057] FIG. 2 illustrates an example encoding and decoding (codec) system 200 for video encoding. Specifically, the codec system 200 supports the implementation of the operating method 100. It provides the ability to represent components used in both the encoder and the decoder. To this end, the codec system 200 is generally described. The codec system 200 is Receive the video signal as described above with respect to steps 101 and 103, and transmit it to the party. The result is a partitioned video signal 201. The network system 200 then performs the steps 105, 107, and 109 of the method 100. When functioning as an encoder, the partitioned video signal 201 is encoded. The codec system 200 acts as a decoder. In this case, as described with respect to steps 111, 113, 115, and 117 of the method of operation 100, The codec system 200 generates an output video signal from the bit stream. Control component 211, transform scaling quantization component 213, intra picture The estimation component 215, the intra-picture prediction component 217, the motion compensation component 219, motion estimation component 221, inverse scaling component 229, The data control analysis component 227, the in-loop filter component 225, the decoded picture buffer Buffer component 223, and Header Formatting Context-Aware Bidirectional The component includes a vectorial arithmetic algorithm coding (CABAC) component 231. Such a component includes, as shown In Figure 2, the black lines indicate the movement of data being encoded / decoded. The dashed lines indicate the movement of control data that controls the operation of other components. Any of the components of the network system 200 may reside in an encoder. A decoder may include a subset of the components of the decoder system 200. For example, a decoder may include Intra-picture prediction component 217, motion compensation component 219, inverse scaling The transform component 229, the in-loop filter component 225, and the decoded picture buffer Buffer component 223. Describe these components.

[0058] The partitioned video signal 201 is divided into blocks of pixels by a coding tree. A captured video sequence is partitioned into blocks. The split tree splits blocks of pixels into smaller blocks of pixels using various split modes. These blocks may then be further subdivided into smaller blocks. A block is sometimes called a node on the coding tree. A larger parent node is The number of times a node is subdivided is determined by the number of nodes / coding The divided blocks are sometimes called coating units. For example, a CU may be included in a CU along with the syntax instructions for that CU. Then, there is a luma block, (one or more) red-difference chroma (Cr) blocks, and (one or It may be a sub-portion of a CTU that contains multiple blue difference chrominance (Cb) blocks. The modes are binary tree (BT), triple tree (TT), and quad tree (QT). These modes can be split into two, three or four parts depending on the split mode used. is used to partition a node into four child nodes. The encoded video signal 201 is then passed through a general coder control component 211, a transformer and scaler 220, and a decoder 222 for compression. The ring quantization component 213, the intra-picture estimation component 215, the filter control The resulting image is forwarded to the control analysis component 227 and the motion estimation component 221 .

[0059] The general coder control component 211 controls the image coding of a video sequence according to application constraints. The image processing unit is configured to make decisions related to encoding the image into a bitstream. For example, the general coder control component 211 controls the bit rate / bit stream size and the reconstruction Such decisions are driven by available storage space / bandwidth and image quality. The general coder control component 211 also controls the buffer size. In order to reduce the underrun and overrun problems of the data, the buffer is adjusted based on the transmission speed. To manage these issues, the General Coder Control component 211 manages partitioning, prediction, and filtering by other components. For example, the general coder control component 211 may increase the resolution to increase bandwidth usage. Compression complexity may be dynamically increased to accommodate the needs of the image, or resolution and bandwidth usage may be decreased. Therefore, the general coder control component 211 controls other components of the codec system 200 to recover the video signal. Balancing quality and bitrate issues. The General Coder Control component 211 The control data is also used to generate control data for controlling the operation of the component. encoded in the bitstream to signal parameters for decoding at The header is then forwarded to the header formatting CABAC component 231 for processing.

[0060] The partitioned video signal 201 is divided into motion estimation components for inter prediction. The partitioned video is also sent to the video component 221 and the motion compensation component 219. A frame or slice of the audio signal 201 may be divided into multiple video blocks. Component 221 and motion compensation component 219 may include one or more The interval of a received video block relative to one or more blocks in a reference frame. The codec system 200 performs predictive coding, for example, by applying adaptive coding to each block of video data. In order to select the appropriate encoding mode, multiple encoding steps can be performed.

[0061] The motion estimation component 221 and the motion compensation component 219 may be highly integrated. are shown separately for conceptual purposes. Motion estimation, as used herein, is the process of generating motion vectors that estimate the movement of video blocks. A motion vector indicates, for example, the movement of a coded object relative to a prediction block. A predicted block is a block that closely matches the block to be coded in terms of pixel difference. A predicted block is a block for which we know the predicted block is a reference block. The pixel differences are calculated using the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference measures. HEVC divides CTU, coding tree block (CTB), and CU into For example, a CTU can be split into CTBs, and then can be split into CBs for inclusion in CUs. A CU is a prediction unit that contains prediction data. and / or as a transformation unit (TU) containing the transformed residual data of the CU. The motion estimation component 221 can be used as part of the rate-distortion optimization process. Generate motion vectors, prediction units, and TUs by using rate-distortion analysis. For example, the motion estimation component 221 may estimate multiple references for the current block / frame. It can determine the best rate-distortion performance by using the best possible block and multiple motion vectors. The best rate-distortion performance is achieved by selecting the best reference blocks, motion vectors, etc. for the video. The quality of the reconstruction (e.g., the amount of data lost due to compression) and the coding efficiency (e.g., the This is a balance between the size of the device and the size of the ring.

[0062] In some examples, the codec system 200 may include a decoded picture buffer component. For example, the sub-integer pixel position of the reference picture stored in the video buffer 223 can be calculated. The audio codec system 200 may select a quarter pixel position, an eighth pixel position, or The motion estimation component 221 can then interpolate values ​​for other fractional pixel positions. A motion search is performed based on the pixel position and the fractional pixel position, and a motion vector with fractional pixel accuracy is obtained. The motion estimation component 221 can output the location of the prediction unit based on the prediction of the reference picture. By comparing with the block position, the video Calculates motion vectors for prediction units of the of blocks. Motion Estimation Component 22 1 is a header formatting CABAC component that stores the calculated motion vector as motion data. Component 231 for encoding, and outputs motion compensation component 219 for motion compensation. do.

[0063] The motion compensation performed by the motion compensation component 219 includes a motion estimation component 2 A prediction block can be obtained or generated based on the motion vector determined by 21. Again, in some examples, the motion estimation component 221 and the motion compensation component The motion of the prediction unit of the current video block may be integrated. The motion compensation component 219 receives the motion vector and calculates the predicted block indicated by the motion vector. The position of the lock can then be determined from the pixel values ​​of the current video block to be coded. A residual video block is formed by subtracting the pixel values ​​of the block to form pixel difference values. Typically, the motion estimation component 221 performs motion estimation on the luma component, The compensation component 219 converts the motion vector calculated based on the luma component into the chroma component and the luma component. The prediction block and the residual block are used for both the transform scaling and quantization components. The call is then forwarded to component 213.

[0064] The partitioned video signal 201 is then fed to an intra-picture estimation component 215. It is also sent to the intra-picture prediction component 217. and motion compensation component 219, as well as intra-picture estimation component 215 and The trap picture prediction component 217 may also be highly integrated, but for conceptual purposes The intra-picture estimation component 215 and the intra-picture prediction component 216 are shown separately. Component 217 includes a motion estimation component 221 and a motion compensation component 222, as described above. Instead of the inter prediction performed by component 219 from frame to frame, The current block is intra-predicted based on a block in the current frame. Then, the intra-picture estimation component 215 estimates the current block to be used for encoding. In some examples, the intra-picture estimation component determines the intra-prediction mode. Component 215 selects the appropriate intra-prediction mode to encode the current block from among multiple proven intra-prediction modes. Then, the selected intra prediction mode is used for encoding. The header is then forwarded to the header formatting CABAC component 231 for formatting.

[0065] For example, the intra picture estimation component 215 can use various proven intra prediction modes. Rate-distortion analysis is used to calculate rate-distortion values ​​for each mode, and the best mode is selected from the proven modes. The rate-distortion analysis is usually used to select an intra prediction mode with the best rate-distortion characteristics. The coded block and the encoded block The amount of distortion (or error) between the original unencoded block and the encoded Determine the bitrate (i.e. number of bits) used to generate the block. The distortion estimation component 215 estimates the ratio from the distortion and rate of the various encoded blocks. The calculation determines which intra prediction mode gives the best rate-distortion value for the block. In addition, the intra picture estimation component 215 performs rate distortion optimization (RDO). The depth blocks of the depth map are coded using a depth modeling mode (DMM) based on The device may be configured to

[0066] When implemented on an encoder, the intra-picture prediction component 217 The selected intra-prediction mode determined by the trapicle estimation component 215. Alternatively, the residual block can be generated from the prediction block based on If so, read the residual block from the bitstream. The residual block is represented as a matrix. The residual block contains the difference between the values ​​of the predicted block and the original block, which is then transformed The image is forwarded to the scaling and quantization component 213. The Intrapicture Prediction component 215 and the Intrapicture Prediction component 217 operate on both the luma and chroma components. can.

[0067] The transform scaling and quantization component 213 further compresses the residual block. The transform scaling and quantization component 213 includes a discrete cosine transform (DCT), Applying a transformation, such as the discrete sine transform (DST), or a conceptually similar transformation, to the residual block and generating a video block containing residual transform coefficient values. Subband transformations, or other types of transformations, can also be used. The transformation goes from the pixel value domain to the frequency domain. The Transform Scaling Quantization Component 21 can transform the residual information into a transform domain such as the 3 is also configured to scale the transformed residual information, e.g., based on frequency. Such scaling applies a scale factor to the residual information, which results in the various frequency components being scaled. The information is quantized to different granularities, which affects the final visual quality of the restored video. The transform scaling quantization component 213 can also further increase the bit rate. The transform coefficients are configured to be quantized to reduce the noise. The quantization process reduces some or all of the coefficients. You can reduce the bit depth associated with all or part of the image. The degree of quantization is determined by the quantization parameter. In some examples, the transform scaling quantization The component 213 may then perform a scan of the matrix containing the quantized transform coefficients. The quantized transform coefficients are then stored in a header to be encoded into the bitstream. The data is forwarded to the formatting CABAC component 231.

[0068] The inverse scaling component 229 performs transformation scaling to support motion estimation. Apply the inverse operation of the ring quantization component 213. The processor 229 may apply inverse scaling, transformation, and / or quantization to, for example, The remaining block is used later as a reference block that can become a prediction block for another current block. The difference block is reconstructed in the pixel domain. Component 219 uses the residual block in motion estimation for subsequent blocks / frames. The reference block can be calculated by adding it to the corresponding predicted block. To reduce artifacts caused by quantization and transformation, the reconstructed reference block A filter is applied to such blocks, which would otherwise be overridden when subsequent blocks are predicted. Various artifacts can lead to inaccurate predictions (and further artifacts can occur) ) is possible.

[0069] The filter control analysis component 227 and the in-loop filter component 225 are Apply a filter to the block and / or to the reconstructed image block, e.g. To recover the original image block, we use the transformed image from the inverse scaling component 229. The predicted residual block is then fed to the intra-picture prediction component 217 and / or the motion compensation component 218. The corresponding prediction block from the component 219 can then be combined with the A filter can be applied to the original image block. In some examples, the filter Instead, it can be applied to the residual block. As with the other components in Figure 2, the filter control solution The analysis component 227 and the in-loop filter component 225 are highly integrated and Although both may be performed, they are depicted separately for conceptual purposes. The filters applied to a particular spatial region are applied to the The filter control analysis component 227 includes a number of parameters that adjust how much of the signal is restored. determining circumstances in which such a filter should be applied by analyzing the reference blocks provided; Set the relevant parameters. Such data is subject to filtering due to encoding. The header formatting CABAC component 231 then forwards the header to the CABAC component 231 as control data. The group filter component 225 applies such filters based on the filter control data. The filters are deblocking filter, noise suppression filter, SAO filter, and and an adaptive loop filter. Such filters may include spatial / pixel domain (e.g. It may be applied in the local domain (eg, on the reconstructed pixel blocks) or in the frequency domain.

[0070] When acting as an encoder, the filtered reconstructed image block, the residual block , and / or the predicted block is added to the decoded picture for later use in the motion estimation described above. The decoded picture is stored in the buffer component 223 when operating as a decoder. A buffer component 223 stores the reconstructed and filtered blocks and generates an output video signal. The decoded picture buffer component 223 transfers the decoded picture to the display as part of the signal. Any device capable of storing prediction blocks, residual blocks, and / or reconstructed image blocks. The memory device may be a

[0071] The header formatting CABAC component 231 is responsible for the various Receives data from a component and encodes the data for transmission to a decoder. Specifically, the header formatting The CABAC component 231 stores control data such as general control data and filter control data. It generates various headers for encoding, as well as intra prediction and motion data. The prediction data, including This is also encoded into the bitstream. The final bitstream is is all the information needed to recover the original partitioned video signal 201. Such information includes an intra-prediction mode index table (codeword mapping table). the encoding context of the various blocks, It may also include an indication of the most likely intra prediction mode, an indication of partition information, etc. Such data may be encoded using entropy coding. For example, the information is , Context-Adaptive Variable Length Coding (CAVLC), CABAC, Syntax-Based Context-Adaptive Adaptive Binary Arithmetic Coding (SBAC), Probabilistic Interval Partitioning Entropy (PIPE) code The data can be encoded using entropy coding, or other entropy coding techniques. After encoding, the encoded bitstream is transmitted to another device (e.g. a video decoder). or can be stored for later transmission or retrieval.

[0072] FIG. 3 is a block diagram illustrating an example video encoder 300. may be used to implement the encoding functions of the codec system 200 and / or to generate the motion vectors. used to perform steps 101, 103, 105, 107, and / or 109 of method 100. The encoder 300 partitions the input video signal and the resulting The partitioned video signal 301 is the same as the partitioned video signal 201. The partitioned video signal 301 is then fed to the encoder 300. The image is compressed by the image processing component and encoded into a bitstream.

[0073] Specifically, the partitioned video signal 301 is divided into two parts for intra prediction. The image is forwarded to the intra-picture prediction component 317. The component 317 is the intra-picture estimation component 215 and the intra-picture prediction component. The partitioned video signal 301 may be substantially similar to the component 217. For inter prediction based on reference blocks in the picture buffer component 323, The motion compensation component 321 also transfers the motion to the motion compensation component 321. The motion compensation component 219 may be substantially similar to the intra frame component 221 and the motion compensation component 219. Predicted blocks and residuals from the picture prediction component 317 and motion compensation component 321 The block is forwarded to the transform and quantization component 313 for transforming and quantizing the residual block. The transform and quantization component 313 is implemented in conjunction with the transform, scaling, and quantization component 213. The transformed and quantized residual block and the corresponding prediction block are Entropy coding (along with associated control data) for encoding into a bitstream The entropy coding component 331 then forwards the header format to the The matting may be substantially similar to the CABAC component 231 .

[0074] The transformed and quantized residual block and / or the corresponding prediction block are subjected to motion compensation. For the reconstruction to the reference block used by the component 321, the transform and quantization component The inverse transform and quantization component 329 also receives the The scaling inverse transform component 329 may be substantially similar to the scaling inverse transform component 229. In this case, the residual block and / or the reconstructed reference block are subjected to in-loop filter convolution. An in-loop filter in the in-loop filter component 325 is also applied. The filter control analysis component 227 and the in-loop filter component 22 5. The in-loop filter component 325 may be substantially similar to the in-loop filter The filtered block may include any of the filters described above with respect to the filter component 225. The block is then used as a reference block by the motion compensation component 321. The decoded picture buffer component 323 stores the decoded picture. The decoded picture buffer component 323 may be substantially similar to the decoded picture buffer component 223.

[0075] 4 is a block diagram illustrating an example video decoder 400. To implement the decoding function of the codec system 200 and / or the steps of the operating method 100 The decoder 400 may be used to implement the steps 111, 113, 115, and / or 117. For example, the bitstream is received from the encoder 300 and is then processed for display to the end user. To achieve this, a reconstructed output video signal is generated based on the bitstream.

[0076] The bitstream is received by the entropy decoding component 433. The encryption decoding component 433 can decode CAVLC, CABAC, SBAC, PIPE encoding, or other encoding. It is configured to implement an entropy decoding scheme, such as an entropy coding technique. The entropy decoding component 433 uses the header information to It can provide context for interpreting additional data that is encoded as a code word. The encoded information includes general control data, filter control data, partition information, and motion data. The decoding of the video signal includes the data, prediction data, and quantized transform coefficients from the residual block. The quantized transform coefficients are then inverse transformed to reconstruct the residual block. The inverse transform and quantization component 429 performs inverse transform and quantization. It may be similar to component 329.

[0077] The reconstructed residual block and / or the predicted block are based on an intra prediction operation. The resulting image is forwarded to the intra-picture prediction component 417 for reconstruction into an intra-picture block. The intra-picture prediction component 417 is a combination of the intra-picture estimation component 215 and the In particular, the intrapicture prediction component 217 may be similar to the intrapicture prediction component 217. The picture prediction component 417 uses the prediction mode to determine the location of the reference block within the frame. The residual block is then applied to the result to recover the intra-predicted image block. The reconstructed intra-predicted image block and / or the residual block and the corresponding The inter-prediction data is fed to the decoded picture via the in-loop filter component 425. The decoded picture is then transferred to the buffer component 423. The in-loop filter component 225 is substantially the same as the in-loop buffer component 223. The in-loop filter component 425 filters the reconstructed image blocks, residual The difference block and / or the prediction block are filtered, and such information is added to the decoded picture buffer. The decoded picture is stored in the decoded picture buffer component 423. The original image block is forwarded to the motion compensation component 421 for inter prediction. The motion compensation component 421 may be implemented as a motion estimation component 221 and / or a motion compensation component. Specifically, the motion compensation component 219 may be substantially similar to the motion compensation component 219. The method 421 uses the motion vector from the reference block to generate a prediction block, and the remaining blocks are added to the result. The difference block is applied to reconstruct the image block. The reconstructed block is then In addition, the decoded picture buffer component 425 is connected to the in-loop filter component 425. The decoded picture buffer component 423 may then forward the further reconstructed The restored image blocks are still stored in the partition information. Such frames may also be arranged in a sequence. This sequence is output to a display as a reconstructed output video signal.

[0078] FIG. 5 is a schematic diagram illustrating an example HRD 500. The HRD 500 includes the codec system 200 and The HRD 500 may be used in a bitstream and / or an encoder such as the encoder 300. Before the stream is transferred to a decoder, such as decoder 400, the stream generated in step 109 of method 100 is In some examples, the bitstream may be inspected. Once loaded, the bitstream can be continuously transferred through the HRD 500. If any part of the bitstream does not comply with the associated constraints, the HRD 500 determines whether the encoder should We re-encode such anomalies by using a different mechanism to re-encode the corresponding part of the stream. This can be shown on the data sheet.

[0079] The HRD 500 includes a virtual stream scheduler (HSS) 541. The HSS 541 schedules the virtual distribution media. The virtual delivery mechanism is a component configured to perform the HR Regarding the timing and data flow of the bit stream 551 input to the D 500, It is used to check the conformance of a stream or a decoder. For example, HSS 541 is 5, receives a bitstream 551 output from the encoder, and performs adaptation of the bitstream 551. In one particular example, the HSS 541 may manage the process of coding It controls the rate at which conforming pictures pass through the HRD 500 and the bitstream 551 filters out non-conforming data. It can be verified that it does not contain

[0080] The HSS 541 may transmit the bitstream 551 to the CPB 543 at a predetermined rate. The data may be managed in a decoding unit (DU) 553. The DU 553 may be divided into access units (AUs) or is a subset of AUs and is a non-Video Coding Layer (VCL) network abstraction layer. Specifically, an AU is an output time associated with a NAL unit. For example, an AU may contain one picture in one layer bitstream. , and may contain pictures for each layer in a multi-layer bitstream. Each picture in an AU is divided into slices, each of which is contained in a corresponding VCL NAL unit. Therefore, a DU 553 may receive one or more pictures, one or more slices of a picture, or Also, the AU, picture, and / or slice may be decoded using The parameters used may be included in non-VCL NAL units. Thus, DU 553 is a DU 55 3 contains data necessary to support decoding of VCL NAL units. The CPB 543 is a first-in, first-out buffer for the HRD 500. The CPB 543 is a The CPB 543 contains the DU 553 that contains the data in decoding order. The video data is stored for use.

[0081] The CPB 543 forwards the DU 553 to the decryption process component 545. Component 545 is a component that complies with the VVC standard. For example, the decoding process component The component 545 may emulate the decoder 400 used by the end user. The decoding process component 545 performs the decoding process that can be achieved in an exemplary end-user decoder. The decryption process component 545 detects an overflow in the CPB 543 and decrypts the DU 553. If DU 553 cannot be decoded fast enough to prevent bitstream 551 from being encoded in the standard and will need to be re-encoded.

[0082] The decryption process component 545 decrypts the DU 553 to generate a decrypted DU 555. The decoded DU 555 contains the decoded picture. The decoded DU 555 is forwarded to the DPB 547. The DPB 547 is connected to the decoded picture buffer components 223, 323, and / or 423. For the purpose of supporting inter prediction, the decoded DU 555 may be The retrieved picture marked for use as a reference picture 556 is further The DPB 547 is passed back to the decryption process component 545 to support further decryption. The decoded video sequence is then output as a series of pictures 557. Usually, it reflects the pictures encoded by the encoder into the bitstream 551. This is the restored picture.

[0083] The picture 557 is forwarded to the output cropping component 549. The component 549 is configured to apply an adaptive cropping window to the picture 557. This results in the output cropped picture 559. The cropped picture 559 is a fully reconstructed picture. The decoded picture 559 is what the end user will see when they decode the bitstream 551. This mimics what would happen if the encoder was to make sure that the encoding was good. To do this, we can consider the output cropped picture 559.

[0084] The HRD 500 is initialized based on the HRD parameters in the bitstream 551. For example, The HRD 500 may read HRD parameters from the VPS, SPS, and / or SEI messages. The HRD 500 then generates a bitstream 551 based on the information in the HRD parameters. As a specific example, the HRD 500 may select one of the HRD parameters. The delivery schedule may determine the above CPB delivery schedule. Specifies the timing of delivery to / from memory locations such as the DPB and / or DPB. Therefore, the CPB delivery schedule may include the delivery of AUs, DUs 553, and / or pictures to / from the CPB 543. The CPB 543 specifies the timing of delivery. The HRD 500 specifies the DPB delivery schedule for the DPB 547. Note that a CPB delivery schedule may be used, which is similar to the CPB delivery schedule.

[0085] The video is encoded in a variety of formats for use with decoders with different hardware capabilities and in a variety of formats. It may be coded into different layers and / or OLS to accommodate network conditions. The CPB delivery schedule is selected with these issues in mind. Due to various hardware and network conditions, the upper layer sub-bitstreams are Therefore, the upper layer may receive one or more CPB delivery schedules. The amount of memory used by the B 543 is large, and the delay in transferring the DU 553 to the DPB 547 is small. Similarly, if the decoder hardware capabilities are limited and / or network conditions are poor, In this case, the sub-bitstream of the lower layer is specified. Therefore, a lower layer can consist of one or more The amount of memory used by the CPB 543 is small, The delay in transferring DU 553 to DPB 547 is large. Next, the expected Verify that the resulting sub-bitstream can be correctly decoded under the conditions specified. To achieve this, the OLS layer, sublayer, or a combination of these, Therefore, the HRD parameters of the bitstream 551 can be tested according to the delivery schedule. The data indicates the CPB delivery schedule and allows the HRD 500 to determine the CPB delivery schedule. and correlate the CPB delivery schedule with the corresponding OLS, layer, and / or sublayer. The method may include sufficient data to allow the user to

[0086] 6 is a schematic diagram illustrating an exemplary multi-layer video sequence 600. The video sequence 600 may be processed by the codec system 200 and / or the codec 202, for example according to the method 100. is encoded by an encoder such as encoder 300, and the codec system 200 and / or or decoder 400. In addition, multi-layer video sequences The process 600 may be inspected for conformance by an HRD, such as the HRD 500. Video sequence 600 is an exemplary representation of layers in a coded video sequence. The multi-layer video sequence 600 includes layers N 631 and 632 and layer N+1 632.

[0087] In one example, the multi-layer video sequence 600 may use inter-layer prediction 621. Inter prediction 621 is performed between pictures 611, 612, 613, and 614, as well as between pictures of different layers. In the illustrated example, the image is applied between pictures 611, 612, and 613. , 613, and 614 are part of layer N+1 632, and pictures 615, 616, 617, and 618 are Layer N 631 and / or layer N+1 632 are part of the , all of which have similar characteristics such as size, quality, resolution, signal-to-noise ratio, and capacity. A layer is a group of pictures that are associated with the same layer ID and associated VCL can be formally defined as a set of VCL NAL units that share a set of non-VCL NAL units. A NAL unit contains video data, such as a coded slice of a picture. Non-VCL NAL units are NAL units coded as follows: , syntax and / or parameters to support conformance checking and other operations. A NAL unit that contains non-video data, such as video data.

[0088] In the illustrated example, layer N+1 632 is related to a larger image size than layer N 631. Thus, in this example, pictures 611, 612, 613 of layer N+1 632 , and 614 are pictures that are larger than pictures 615, 616, 617, and 618 of layer N 631. size (e.g., large height and width, and therefore large number of samples); however, However, such a picture is divided into layers N+1 632 and N 631 according to other characteristics. Although only two layers, layer N+1 632 and layer N 631, are shown, a series of A picture can be divided into any number of layers based on associated characteristics. Layer N+1 632 and Layer N 631 are also represented by a layer ID. An item of data associated with a picture and which is part of the layer in which the picture is shown. Therefore, it is possible to know which layer N+1 632 or layer N 631 corresponds to which picture. Each picture 611-618 may be associated with a corresponding layer ID to indicate whether it contains For example, the layer ID may include a NAL unit header layer identifier (nuh_layer_id). This means that the layer that contains the NAL unit (e.g., slices and / or pictures in the layer) It is a syntax element that specifies the identifier of the layer N 631 (including the parameters of the layer N). So, the lower quality / smaller image size / smaller bitstream size associated with A layer is usually assigned a lower layer ID and is called a lower layer. Like N+1 632, associated with higher quality / larger image size / larger bitstream size The layer that is assigned a higher layer ID is usually assigned and is referred to as the higher layer.

[0089] The pictures 611-618 in the different layers 631-632 are configured to be selectively displayed. As a specific example, if a small picture is desired, the decoder may select Picture 615 may be decoded and displayed, or, if a larger picture is desired, The decoder may decode and display picture 611 at the current display time. Pictures 611 to 614 in layer N+1 632 are the same as the pictures 611 to 614 in the lower layer (regardless of picture size). Each of the pictures 615-618 in the layer N 631 contains substantially the same image data as the corresponding pictures 615-618 in the layer N 631. Specifically, for example, picture 611 contains substantially the same image data as picture 615, and picture 612 contains substantially the same image data as picture 616 .

[0090] Pictures 611-618 refer to other pictures 611-618 in the same layer N 631 or N+1 632. A picture can be coded with reference to another picture in the same layer. The result of this is inter prediction 623. Inter prediction 623 is indicated by a solid arrow. For example, picture 613 is a composite of pictures 611, 612, and 613 in layer N+1 632. and / or 614 using inter prediction 623 that uses one or two of them as references. For unidirectional inter prediction, one picture may be referenced, and / or In addition, picture 617 is a reference picture for bidirectional inter prediction. Refer to one or two of pictures 615, 616, and / or 618 in Year N 631. For one-way inter prediction, One picture is referenced for each frame and / or two pictures are referenced for bidirectional inter prediction. When performing inter prediction 623, a picture references another picture in the same layer. When a picture is used as a reference, the picture may be called a reference picture. The decoder 612 uses a reference prediction 623 to code the picture 613. Inter prediction 623 is used in the context of multi-layers to predict the inter picture. In this way, inter prediction 623 is a prediction method for a different picture from the current picture. The sample of the current picture is calculated by referring to the sample indicated in the next reference picture. A reference picture and a current picture are coded in the same layer. is located.

[0091] Pictures 611-618 may also be coded with reference to other pictures 611-618 in different layers. This process is known as inter-layer prediction 621 and is represented by the dashed arrows. The inter-layer prediction 621 is performed by referring to a sample indicated in a reference picture. Therefore, it is a mechanism for coding samples of the current picture, and the current picture and It is in a different layer than the reference picture and therefore has a different layer ID. For example, , a picture in lower layer N 631 maps to a corresponding picture in upper layer N+1 632. As a specific example, picture 611 may be used as a reference picture for decoding. In this case, the image 615 may be coded according to the inter-image prediction 621. In this case, picture 615 is used as an inter-layer reference picture. The reference picture used for inter-layer prediction 621. In most cases, inter-layer prediction 621 is A current picture, such as picture 611, is a picture that is in the same AU 627 and is in a lower layer. 615, the layer is restricted to using only one or more inter-layer reference pictures. When multiple layers (e.g., more than two) are available, the inter-layer prediction 621 is Based on the cross-layer reference picture(s) below the current picture , the current picture can be encoded / decoded.

[0092] The video encoder may include many different combinations of inter prediction 623 and inter-layer prediction 621. A multi-layer video encoding system is used to encode the pictures 611 to 618 via encoding and / or substitution. For example, picture 615 may be generated using intra prediction. Next, pictures 616 to 618 can be coded using picture 615 as a reference picture. In addition, the picture 611 may be coded according to inter prediction 623 using The picture 615 is used as an inter-layer reference picture to perform coding according to inter-layer prediction 621. Then, pictures 612 to 614 can be processed using picture 611 as a reference picture. , may be coded according to inter prediction 623. In this way, the reference pictures may be different For the coding mechanism, single layer reference picture and inter-layer reference picture Based on the lower layer N 631, the upper layer N+1 By coding the 632 picture, the upper layer N+1 632 performs inter prediction 623 and using intra-layer prediction, which has much lower coding efficiency than inter-layer prediction. In this way, the coding efficiency of intra prediction can be avoided. pictures and therefore limited to coding a minimum amount of video data. The pictures used as reference pictures and / or inter-layer reference pictures are Reference Picture List Entry(s) contained in the Reference Picture List Structure can be shown in

[0093] The pictures 611 to 618 may also be included in an access unit (AU) 627. The AU 627 may include different A set of coded pictures contained in a layer that have the same output time during decoding. Therefore, coded pictures in the same AU 627 are related to each other. For example, pictures 614 and 615 are scheduled to be output from the DPB at the same time. Pictures 613 and 617 are in the same AU 627 as pictures 614 and 618. The pictures 614 and 618 in the same AU 627 may be selectively displayed. For example, picture 618 may be displayed when a small picture size is desired, and picture 620 may be displayed when a large picture size is desired. Picture 614 may be displayed when the larger picture size is desired. Picture 614 is output when desired, and picture 618 is only used for inter-layer prediction 621. In this case, the picture 618 is discarded without being output when the inter-layer prediction 621 is completed. will be done.

[0094] The AU 627 may be further divided into one or more picture units (PUs) 628. A PU 628 is a subset of an AU 627 that contains one coded picture. They are related to each other according to classification rules, are consecutive in decoding order, and are exactly one coded A PU 628 can be formally defined as a set of NAL units that contains a picture that is a part of a HRD and and / or associated conformance tests. Note that it is sometimes called the

[0095] Pictures 611 to 618, AU 627, and PU 628 each have a temporal identifier (TemporalId). Note that the TemporalId 629 is also associated with the Nth A derived identifier indicating the relative position of an AL unit. The TemporalId and / or PU 628 are associated with the same value of TemporalId 629. The first AU 627 of the TemporalId field contains a TemporalId of zero 629, and subsequent AUs 627 contain successively increasing TemporalIds. A non-VCL NAL unit may also be associated with a TemporalId 629. For example, a parameter set may be included in an AU 627 and associated with one or more pictures within the AU 627. In such a case, the TemporalId 629 of the parameter set is the Temporal oralId may be 629 or less.

[0096] 7 is a schematic diagram illustrating an example bitstream 700. 700 illustrates decoding by the codec system 200 and / or the decoder 400 according to the method 100. may be generated by the codec system 200 and / or the encoder 300 for In addition, the bitstream 700 may include a multi-layer video sequence 600. The bitstream 700 may be used to set various parameters to control the operation of an HRD, such as the HRD 500. Based on such parameters, the HRD may determine the transmission to the decoder for decoding. Prior to this, the bitstream 700 may be verified for compliance with the standard.

[0097] The bitstream 700 includes a VPS 711, one or more SPSs 713, and a number of picture parameters. A PPS 715, a plurality of adaptation parameter sets (APS) 716, and a plurality of picture headers. 718, a number of slice headers 717, image data 720, and an SEI message 719. 711 contains data about the entire bitstream 700. For example, the VPS 711 Contains data about the OLS, layers, and / or sublayers used in stream 700. The SPS 713 is a representation of the coded video sequence included in the bitstream 700. Each layer contains sequence data that is common to all pictures in the layer. For example, each layer contains one or more Each coded video sequence includes a plurality of coded video sequences, each of which includes: You may refer to SPS 713 for the corresponding parameters. The parameters in SPS 713 are Adjust the size of the image, bit depth, coding tool parameters, bit rate limit, etc. Each sequence may contain a reference to SPS 713, and in some cases, one SP Note that PPS 713 may contain data for multiple sequences. It contains parameters that apply to the entire picture. Therefore, each picture in a video sequence is , may refer to PPS 715. In some examples, while each picture may refer to PPS 715, Note that a single PPS 715 may contain data for multiple pictures. For example, Thus, multiple similar pictures may be coded according to similar parameters. In such cases, a single PPS 715 may contain data for such similar pictures. The available coding tools, quantization parameters, and offsets for the slices in the corresponding picture. It can indicate a set, etc.

[0098] The APS 716 is a syntax that applies to one or more slices 727 in one or more pictures 725. Such a correlation is expressed as a slice 72 7. The slice header 717 associated with the slice 7 is determined based on syntax elements found in the slice header 717. For example, the APS 716 may obtain at least one, but not all, of the first picture 721. a new slice 727, at least one, but not all, slices in the second picture 725 727. The APS 716 may be applied to multiple These types include the Adaptive Loop Filter (ALF), APS, Luma Mapping with Roma Scaling (LMCS) APS, and Scaling List (Scali ng) APS. The ALF contains a transfer function controlled by variable parameters and allows the transfer function to be precisely Based on an adaptive block that uses feedback from a feedback loop to refine Furthermore, ALF is a filter based on block-based coding. Coding artifacts such as smearing and ringing artifacts In this way, the ALF parameters included in the ALF APS are The parameter is used to reduce block-based coding artifacts during decoding at the decoder. The LMCS may include parameters selected by the encoder to cause F to be removed. It is applied as part of the decoding process to map luma samples to specific values, possibly It is the process of applying further scaling operations to the values ​​of the chroma samples. The rule is based on a mapping to the corresponding chroma components to reduce rate distortion. In this way, the LMCS APS can reshape the luma component in the LMCS tools. Contains the parameters selected by the encoder to shape the image. The list APS is associated with the quantization matrices used by the specified filter. Thus, APS 716 is a conformance test for HRD. Various modifications may be made to the coded slice 727 during testing and / or during decoding at the decoder. The parameter may include parameters used to apply a filter.

[0099] The picture header 718 applies to all slices 727 of a coded picture 725. A picture header is a syntax structure that contains syntax elements. For example, a picture header 718 Sequence count information, reference picture data, intra random access points (IRAP ) Data about pictures, such as data about applying filters to pictures 725 A PU may contain exactly one picture header 718 and exactly one picture 725. Thus, the bitstream 700 contains exactly one picture for each picture 725. The slice header 717 may include a unique header 718 for each slice 727 in the picture 725. Therefore, in a video sequence, one slice per slice 727 There may be a header 717. The slice header 717 includes slice type information, filtering information, , prediction weightings, tile entry points, deblocking parameters, etc. In some cases, syntax elements may be the same for all slices 727 in a picture 725. To reduce redundancy, the picture header 718 and slice header 717 include several types of For example, some parameters may be shared when applied to the whole picture 725. The picture header 718 may include other data (e.g., filtering parameters). When applied to a group of slices 727 that are a subset of the entire picture 725, the slices It may be included in the rice header 717.

[0100] The image data 720 is encoded according to inter prediction and / or intra prediction. The video data includes the converted video data, as well as the corresponding transformed and quantized residual data. The image data 720 may include layers 723, pictures 725, and / or slices 727. The layer 723 stores the specified properties as indicated by the layer ID, such as nuh_layer_id. VCL NAL Unit 745, which share a common resolution, frame rate, image size, etc. A layer 723 is a set of non-VCL NAL units 741 associated with the same nuh A layer 723 may include a set of pictures 725 that share a _layer_id. and / or may be substantially similar to 632. The nuh_layer_id is a member of at least one NAL unit. This is a syntax element that specifies the identifier of the layer 723 that contains the unit. For example, The lowest quality layer 723, known as the layer, may contain the lowest value of nuh_layer_id, The higher the quality of the layer 723, the higher the value of nuh_layer_id. Therefore, the lower layers are The layer with the smaller _layer_id value is layer 723, and the higher layer is the value of nuh_layer_id. becomes the larger layer 723.

[0101] A picture 725 is an array of luma samples that generate a frame or a field thereof, and For example, picture 725 is an array of chroma samples that are output for display. 725 to be encoded or output. A picture 725 is a coded image that can be used to A slice 727 is an integer number of segments of a picture 725 that are exclusively contained in a single NAL unit. complete tiles, or an integer number of consecutive complete codings (e.g., within a tile) A slice 727 may be defined in a tree unit (CTU) row. The CTU is then divided into coding tree blocks (CTBs). A set of samples of a given size that can be partitioned. A CTB is a subset of a CTU. The CTU / CTB is a block that contains the luma or chroma components of the CTU. The coding blocks are then further divided into coding blocks based on the prediction mechanism. It can be encoded / decoded according to the same algorithm.

[0102] The SEI message 719 is sent to the decoding processor to determine the values ​​of the samples in the decoded picture. Syntax constructs with specific semantics that convey information that is not required by the process. For example, SEI message 719 is a data or data structure that supports the HRD process. It may also include other supporting data that is not directly related to decoding the bitstream 700 by the coder. The set of SEI messages 719 is implemented as a scalable nesting SEI message. The scalable nesting SEI message can be used to nest the SEI message 719 in a specific layer 7 23. Scalable nesting SEI messages provide a mechanism for associating multiple A message that contains any number of scalable nested SEI messages. The stored SEI message is a message that corresponds to one or more OLSs or one or more layers 723. The OLS is a set of layers, at least one of which is an output layer. Therefore, the scalable nesting SEI message can be A set of scalable nested SEI messages may be said to contain a It may be said to include a set of EI messages 719. A nesting SEI message contains a set of homogenous scalable nested SEI messages. The SEI message 719 manages the CPB that tests the corresponding OLS and / or layer 723. For this purpose, the BP SEI message may include the HRD parameters for initializing the HRD. The I message 719 also sends the CPB and / or a PT SEI message including the HRD parameter that manages the delivery information of the AU in the DPB. The SEI message 719 may also include a test for the corresponding OLS and / or layer 723. DUI SEI, including HRD parameters that manage the delivery information of the DU in the CPB and / or DPB, It may contain a message.

[0103] The bitstream 700 may be coded as a sequence of NAL units. A unit is a container for video data and / or supporting syntax. An L unit can be a VCL NAL unit 745 or a non-VCL NAL unit 741. Bit 745 is a NAL unit coded to contain video data. In this example, a VCL NAL unit 745 contains a slice 727 and an associated slice header 717. Non-VCL NAL units 741 are used to decode video data, perform conformance checks, and perform other operations. NA, including non-video data, such as syntax and / or parameters that support Non-VCL NAL units 741 are VPS NAL units, SPS NAL units, PPS NAL units, AL unit, APS NAL unit, picture header (PH) NAL unit, and SEI NAL unit These may include VPS 711, SPS 713, PPS 715, APS 716, Picture Head The above list of NAL units is merely exemplary. Please note that this is a list and is not exhaustive.

[0104] Each NAL unit is formed by a NAL unit header temporal identifier plus one (nuh_temporal_id_plus s1) 731. nuh_temporal_id_plus1 731 is associated with the corresponding It is an identifier that is signaled to indicate the relative position of the corresponding NAL unit. and / or HRD determines the corresponding NAL unit based on the value of nuh_temporal_id_plus1 731 Specifically, the TemporalId of the set can be determined by using nuh_temporal_id_plus1 731 The TemporalId of a NAL unit is signaled in the header of the NAL unit. / HRD may determine this by subtracting 1 from the value of nuh_temporal_id_plus1 731. Thus, the value of nuh_temporal_id_plus1 731 must not be set to zero. This is because it would cause TemporalId to become negative.

[0105] Additionally, the SEI message 719 may include a prefix SEI message and / or a suffix SEI message. A prefix SEI message can be used as a prefix SEI message. The suffix SEI message is a SEI message 719 that applies to the following NAL units. The prefix SEI message 719 applies to one or more preceding NAL units. The message is contained in the prefix SEI NAL unit type (PREFIX_SEI_NUT) 742 and The fix SEI message has the suffix SEI NAL unit type (SUFFIX_SEI_NUT) 74 3. PREFIX_SEI_NUT 742 is a non-VCL NAL unit, and non-VCL NAL units Has its type value set to indicate that it contains a prefix SEI message SUFFIX_SEI_NUT 743 is a non-VCL NAL unit, and non-VCL NAL units are suffixed. The URI has a type value set to indicate that it contains a Service SEI message.

[0106] As mentioned above, the SEI message 719 verifies the bitstream's conformance to the standard. The SEI message may contain parameters used by the HRD running on the encoder to The page 719 is associated with a changing combination of pictures 725 and / or layers 723. Therefore, for complex multi-layer bitstreams, the appropriate SEI messages 7 19 to be associated with the appropriate picture 725 and / or layer 723. In addition, the prefix SEI message may be difficult to read. Any NAL unit included in the bitstream 700 prior to the first NAL unit associated with the EI message. The suffix SEI message must be associated with It must be included in the bitstream 700 immediately after the first NAL unit associated with it. The SEI message 719 is not correctly placed in the bitstream 700 and / or or is not associated with the correct layer 723 and / or picture 725, the HRD The conformance of layers 723 and / or pictures 725 may not be properly verified. This is , encoding errors caused by HRD, and / or errors when decoding by the decoder. For example, the HRD may filter the picture inappropriately or may violate the standard. In addition, the decoder may detect coding errors associated with the transmission. Failure to detect coding errors associated with a transmission or indication of coding errors when no such errors exist In some cases, the system may respond inappropriately.

[0107] The bitstream 700 includes an SEI message 719 that indicates the corresponding layers 723, pictures 725, and segments. Rice 727, and / or NAL units are modified to ensure that they are correctly associated with As explained with reference to FIG. 6, a multi-layer bitstream is a set of pictures and related We can organize the associated parameters into AUs. An AU is a set of parameters contained in different layers. The pictures are loaded into the bitstream and are associated with the same output time. At 0, each SEI message 719 includes a first picture 72 associated with the SEI message 719. 5. In addition, the SEI message 719 is assigned a TemporalId. The TemporalId of the SEI message 719 is equal to the TemporalId of the AU that contains the SEI message 719. In a particular example, each picture 725 in an AU has the same TemporalId They share the same value of nuh_temporal_id_plus1 731, and therefore share the same value of nuh_temporal_id_plus1 731. The nuh_temporal_id_plus1 731 in the SEI message 719 applies to The nuh_temporal_id_plus1 731 is the same as the nuh_temporal_id_plus1 731 of each of the corresponding pictures 725.

[0108] In other words, the picture 725 is contained in a VCL NAL unit 745, and the parameters are contained in a non-VCL NAL unit 745. The non-VCL NAL unit 741 is a type that contains the SEI message 719. When the SEI NAL unit is PREFIX_SEI_NUT 742 or SUFFIX_SEI_NUT 743, the non-VCL TemporalId / nuh_temporal_id_plus1 731 of NAL unit 741 is the same as that of non-VCL NAL unit 74 Constrained to be equal to TemporalId / nuh_temporal_id_plus1 of the AU containing 1 731 This technique ensures that the SEI message 719 is correctly associated with the corresponding picture 725 in the AU. Therefore, various errors can be avoided. In addition, the coding efficiency is improved and the encoder and decoder functions are improved. The processor, memory and / or network signaling may be implemented in both the decoder and the processor. The utilization of gre resources is reduced.

[0109] The above information will now be explained in more detail below. Scalable video coding, or video coding with scalability, is a Video coding scalability is achieved through multi-layer Multi-layer bitstreams can be supported using any of the following coding techniques: A scalable layer includes a layer (BL) and one or more extension layers (EL). Inter-view scalability, quality / signal-to-noise ratio (SNR) scalability, multi-view scalability This includes frame rate scalability, frame rate scalability, etc. Multi-layer coding When the technique is used, a picture or part of it can be coded without the use of a reference picture. A picture may be coded by referring to a reference picture in the same layer (intra prediction). The image may be fed to another layer (inter prediction) and / or to another layer(s). It may be coded with reference to a reference picture (inter-layer prediction). The reference picture used for inter-layer prediction is called an inter-layer reference picture (ILRP). Figure 6 shows an example of multi-layer coding for spatial scalability, with different The pictures within a layer have different resolutions.

[0110] Apart from the profile(s) for single layer coding, Some video coding families support scalability in multiple (multiple) profiles. Scalable Video Coding (SVC) provides support for spatial Advanced video coding, providing support for video, time, and quality scalability SVC is a scalable extension of AVC. Each macro in an EL picture is used to indicate whether the block is used to predict an EL MB. A flag is signaled in the block (MB). Prediction from the collocated block is The SVC implementation may include a feature, a motion vector, and / or a coding mode. may not directly reuse unmodified AVC implementations in their designs. The lock syntax and decoding process is different from the AVC syntax and decoding process. become.

[0111] Scalable HEVC (SHVC) supports spatial and quality scalability Multiview HEVC (MV-HEVC) is an extension of HEVC that provides multiview scalability. 3D HEVC (3D-HEVC) is an extension of HEVC that provides support for MV - HEVC, which provides support for more advanced and efficient 3D video coding. Temporal scalability is an integral part of the HEVC codec in a single layer. In the multi-layer extension of HEVC, the decoded Pictures are only taken from the same AU and are treated as long-term reference pictures (LTRPs). Such a picture, together with other temporal reference pictures in the current layer (one or more ) reference picture list. Inter-layer prediction (ILP) is , the inter-layer reference picture(s) in the reference picture list(s). The prediction unit level is calculated by setting the value of the reference index to refer to the Spatial scalability is achieved by using the ILRP to scale the current pixel being encoded or decoded. When a reference picture has a different spatial resolution from the image, the reference picture or a part of it is resampled. Reference picture resampling can be done at the picture level or at the coding block level. The above-mentioned configuration can be realized at any one of the following levels:

[0112] VVC may also support layered video coding. can contain multiple layers, all of which can be independent of each other. For example, each layer may be coded without using inter-layer prediction. In some cases, several layers use the ILP to A flag in the VPS indicates whether the layer is a broadcast layer or not. It can indicate whether some layers are using ILP. When SHVC and Unlike MV-HEVC and MV-HEVC, VVC does not require the OLS to be specified. OLS is a method to One or more layers in the set of layers are designated to be output layers. Layer is the layer of the output OLS. In some implementations of VVC, the layer is broadcast. When a distribution layer, only one layer can be selected for decoding and output. In some implementations, when any layer uses ILP, all layers are included. In addition, the entire bitstream including the layers is specified to be decoded. One or more layers are designated as output layers. The output layers can be the unique top layer, all layers, Or it may be displayed as a top layer plus a set of displayed lower layers. obtain.

[0113] The above-mentioned aspects have some problems. For example, the SPS, PPS, and APS NAL units In addition, the nuh_layer_id value of the SEI NAL unit may not be properly restricted. ralId values ​​may not be properly constrained. In addition, resampling of reference pictures is not possible. When enabled and the pictures in the CLVS have different spatial resolutions, the NoOutputOfPriorPicsFla g setting may not be specified properly. Also, some video coding systems may not be able to Suffix SEI messages MUST be included in scalable nesting SEI messages. As another example, the buffering period, picture timing, and decoding unit The SEI message for the packet information may contain analysis dependencies on the VPS and / or SPS. do.

[0114] Generally, this disclosure describes techniques for improving video coding. The techniques are based on VVC. However, this technique does not support layered video encoding based on other video codec specifications. It also applies to encoding.

[0115] One or more of the problems described above may be solved as follows: The nuh_layer_id value of the SEI NAL unit is now appropriately restricted. Here, however, it is appropriately restricted. Reference picture resampling is enabled and CLVS The NoOutputOfPriorPicsFlag setting is now correctly specified when pictures in a file have different spatial resolutions. The suffix SEI message is included in the scalable nesting SEI message. Parsing dependency of BP, PT, and DUI SEI messages for VPS or SPS The property is specified by the syntax element decoding_unit_hrd_param in the BP SEI message syntax. s_present_flag and the syntax element decoding_un in the PT SEI message syntax it_hrd_params_present_flag, and decoding_unit_cpb_params_in_pic_tim ng_sei_flag and the syntax element decoding_unit_cpb_param in the DUI SEI message s_in_pic_timing_sei_flag.

[0116] An exemplary implementation of the above mechanism is as follows: The semantics of the bit are as follows:

[0117] nuh_temporal_id_plus1 minus 1 specifies the temporal identifier of the NAL unit. The value of nuh_temporal_id_plus1 must not be zero. The variable TemporalId is It is derived from TemporalId=nuh_temporal_id_plus1-1 When nal_unit_type is within the range of IDR_W_RADL to RSV_IRAP_13, inclusive, , TemporalId must be zero. When nal_unit_type is STSA_NUT, Temporal alId must not be zero.

[0118] The value of TemporalId MUST be the same for all VCL NAL units of an access unit. The Te of a coded picture, layer access unit, or access unit. The value of mporalId is the VCL NAL unit of a coded picture, layer access unit, It may be the value of the TemporalId of the sublayer representation of the access unit, or the TemporalId of the access unit. The value of oralId may be the maximum of the TemporalId of all VCL NAL units in the sublayer representation. good.

[0119] The values ​​of TemporalId for non-VCL NAL units are restricted as follows: If it is equal to DPS_NUT, VPS_NUT, or SPS_NUT, then TemporalId is zero and the NAL unit The TemporalId of the access unit containing the bit must be zero. If t_type is equal to EOS_NUT or EOB_NUT, then TemporalId must be zero. Or, nal_unit_type is AUD_NUT, FD_NUT, PREFIX_SEI_NUT, or SUFFIX_ If it is equal to SEI_NUT, the TemporalId is the TemporalId of the access unit that contains the NAL unit. d or nal_unit_type must be equal to PPS_NUT or APS_NUT. If so, the TemporalId must be greater than or equal to the TemporalId of the access unit that contains the NAL unit. When the NAL unit is a non-VCL NAL unit, the value of TemporalId must be This must be the minimum of the TemporalId values ​​of all access units to which the unit applies. If l_unit_type is equal to PPS_NUT or APS_NUT, TemporalId is the This can be greater than or equal to the TemporalId of the data unit. In addition, the first coded picture is Has TemporalId equal to zero.

[0120] An exemplary sequence parameter set RBSP semantics is as follows: SPS RB The SP must be available to the decryption process before it can be referenced. It is included in at least one access unit with d equal to zero, or is provided by an external mechanism. An SPS NAL unit that contains an SPS may be provided by a PPS NAL unit that references the SPS. The layer may be restricted to have a nuh_layer_id equal to the lowest nuh_layer_id value of the

[0121] Exemplary picture parameter set RBSP semantics are as follows: PPS RBSP must be available to the decryption process before it can be referenced. is included in at least one access unit below the TemporalId of the PPS NAL unit, or or must be provided by an external mechanism. The nuh_layer_id is the lowest nuh_layer_id value of the coded slice NAL unit that references the PPS. They must have equal nuh_layer_id.

[0122] Exemplary adaptation parameter set semantics are as follows: Each APS RBSP The APS must also be available to the decryption process before the TemporalId is referenced. At least one TemporalId less than or equal to the TemporalId of the coded slice NAL unit that references the APS Must be contained in an access unit or provided by an external mechanism. An APS NAL unit cannot be shared by pictures / slices of multiple layers. The nuh_layer_id of an APS NAL unit is a codeword that references the APS NAL unit. It must be equal to the lowest nuh_layer_id value of the included slice NAL units. Alternatively, an APS NAL unit is not shared by pictures / slices of multiple layers. The nuh_layer_id of an APS NAL unit is the nuh_layer_id of the slice that references the APS. must be equal to

[0123] In one example, removing a picture from the DPB before decoding the current picture Before decoding the current picture (but not after the first sequence of the current picture), After parsing the slice headers of the slice, removing the picture from the DPB is This may occur at the CPB removal time of the first decoding unit of unit n (which contains the current picture). The process is as follows: The decoding process for constructing the reference picture list is invoked. , and the decoding process for reference picture marking is invoked.

[0124] The current picture is a coded layer video sequence start (CLVSS) picture that is not picture zero. When building a decoder, the following sequence of steps is applied: The NoOutputOfPriorPicsFlag is derived as follows: pic_width_ max_in_luma_samples, pic_height_max_in_luma_samples, chroma_format_idc , separate_colour_plane_flag, bit_depth_luma_minus8, bit_depth_chroma_m inus8, or the value of sps_max_dec_pic_buffering_minus1[ Htid ] is greater than the value of the preceding pic_width_in_luma_samples, pic_hei ght_in_luma_samples, chroma_format_idc, separate_colour_plane_flag, bit _depth_luma_minus8, bit_depth_chroma_minus8, or sps_max_dec_pic_buf When the value of fering_minus1[ Htid ] is different from that of NoOutputOfPriorPicsFlag, Regardless of the value of o_output_of_prior_pics_flag, the decoder under test Under such conditions, the NoOutputOfPriorPicsFlag may be set to no_output It may be preferable to set it to the same as t_of_prior_pics_flag, in which case The decoder under test is allowed to set NoOutputOfPriorPicsFlag to 1. Note that if this is not the case, NoOutputOfPriorPicsFlag is set to no_output_of May be set the same as _prior_pics_flag.

[0125] The derived value of NoOutputOfPriorPicsFlag for the decoder under test is applied to the HRD. As a result, when the value of NoOutputOfPriorPicsFlag is 1, all picture segments in the DPB are The storage buffer is emptied without outputting the picture it contains, and the DPB fullness is A DPB is set to zero if both of the following conditions hold for any picture k in the DPB: Any such picture k in B is removed from the DPB. Picture k is not used for reference. and picture k has a PictureOutputFlag equal to zero or the corresponding The DPB output time for the first decoding unit of the current picture n (represented by decoding unit m) is It is less than or equal to the CPB removal time. This means that DpbOutputTime[ k ] is less than or equal to DuCpbRemovalTime[ m ]. For each picture removed from the DPB, the DPB fullness is decremented by 1. will be displayed.

[0126] In one example, outputting or removing a picture from the DPB is described as follows: When the first decoding unit of an access unit containing a chunk is removed from the CPB, the current before decoding the slice (but after parsing the slice header of the first slice of the current picture) In this case, the output and removal of pictures from the DPB may occur, proceeding as follows: The decoding process for constructing the reference picture list and the decoding process for reference picture marking The process number is called.

[0127] If the current picture is a CLVSS picture that is not picture zero, the following steps are taken: If the variable NoOutputOfPriorPicsFlag is set to pic_width_max_in_luma_samples, pic_ height_max_in_luma_samples, chroma_format_idc, separate_colour_plane_fl ag, bit_depth_luma_minus8, bit_depth_chroma_minus8, or sps_max_dec_p The value of ic_buffering_minus1[ Htid ] is less than the SPS referenced by the preceding picture. Derived pic_width_in_luma_samples, pic_height_in_luma_samples, chroma _format_idc, separate_colour_plane_flag, bit_depth_luma_minus8, bit_dep th_chroma_minus8 or sps_max_dec_pic_buffering_minus1[ Htid ] value and When they are different, NoOutputOfPriorPicsFlag is no_output_of_prior_pics_fl Regardless of the value of ag, it may be set to 1 by the decoder under test. Under such conditions, NoOutputOfPriorPicsFlag is treated the same as no_output_of_prior_pics_flag. However, in this case the decoder under test must be set to NoOutputOfPriorPic Note that sFlag can be set to 1. Alternatively, NoOutputOfPriorPicsFlag can be set to no It can be set to the same as _output_of_prior_pics_flag.

[0128] The derived value of NoOutputOfPriorPicsFlag for the decoder under test is as follows: If the value of NoOutputOfPriorPicsFlag is 1, all pixels in the DPB The picture storage buffer is emptied without outputting the picture it contains, and the DPB file is The pixel count is set to zero, otherwise (when NoOutputOfPriorPicsFlag has the value 0). is a picture that does not need to be printed and is marked as not to be used for reference purposes. All picture storage buffers, including Non-empty picture storage in the DPB by repeatedly invoking the ping process All buffers are emptied and the DPB fullness is set to zero.

[0129] If not (if the current picture is not a CLVSS picture), there is no need to output it. All picture streams, including pictures marked as unused for reference The picture storage buffer is emptied (without output). For each buffer, DPB fullness is decremented by 1. If one or more of the following conditions are true: When this happens, the bumping process is repeatedly invoked while any of the following conditions are true: For each additional picture storage buffer that is emptied, For each, it also decrements the DPB fullness by 1. The condition is that the output must be marked as required. The number of pictures in the specified DPB is greater than sps_max_num_reorder_pics[Htid]. Other conditions are that sps_max_latency_increase_plus1[Htid] is not zero, and the associated variable PicLatencyCount is greater than or equal to SpsMaxLatencyPictures[ Htid ]. The requirement is that there is at least one picture in the DPB that is marked as needed for output. The condition is that the number of pictures in the DPB is greater than or equal to SubDpbSize[Htid].

[0130] The syntax of an example generic SEI message is as follows:

[0131] [Table 1]

[0132] The syntax of an example scalable nesting SEI message is as follows: do.

[0133] [Table 2]

[0134] The semantics of an exemplary scalable nesting SEI message are as follows: A scalable nesting SEI message, in the context of a particular OLS, The mechanism for associating SEI messages with specific layers, or contexts in OLS, is irrelevant. It provides a mechanism to associate a specific layer with a specific object regardless of its type. Scalable nesting An I-message contains one or more SEI messages. Scalable nesting SEI messages The SEI message included in the message is also referred to as a scalable nested SEI message. Bitstream conformance is determined by whether the SEI message is a scalable nesting SEI message. When included, the following restrictions may apply:

[0135] payloadType is 132 (decoded picture hash) or 133 (scalable nested Scalable nesting SEI messages must not be included in scalable nesting SEI messages. No. Scalable nesting SEI messages are not supported for buffering periods, picture timings, etc. When the SEI message of the decoding unit information is included, the scalable nesting is The buffering SEI message has a payloadType that is not zero (buffering period) or is 1 (picker). 130 (Decoder Unit Information), or any other SEI message in 131 (Decoder Timing). That must not be the case.

[0136] Bitstream conformance is achieved for SEI NAL units that contain scalable nesting SEI messages. The following further restrictions may be required to be applied to the value of nal_unit_type for a unit: The scalable nesting SEI message has a payloadType of zero (during buffering). (picture timing), 1 (picture timing), 130 (decoding unit information), 145 (RAP indication dependent) or When it contains a SEI message of 168 (frame field information), scalability SEI NAL units that contain a routing SEI message have nal_unit_type set to PREFIX_SEI_NUT Scalable nesting SEI message contains an SEI message with payloadType 132 (decoded picture hash). In this case, the SEI NAL unit that contains the scalable nesting SEI message is nal_unit_t. You must set type equal to SUFFIX_SEI_NUT.

[0137] In the context of a particular OLS, scalable nested SEI messages can be sent to a particular layer. To specify that the nesting_ols_flag applies to all nesting_ols, nesting_ols_flag may be set to 1. Scalable nested SEI messages (e.g., outside of the OLS context) To specify that the nesting_ols_flag should always be set to zero, can be set to

[0138] Bitstream conformance indicates that the following restrictions apply to the value of nesting_ols_flag: A scalable nesting SEI message may require a payloadType of zero. (buffering period), 1 (picture timing), or 130 (decoding unit information) When including an SEI message, the value of nesting_ols_flag must be 1. The nesting SEI message has a payloadType equal to a value in VclAssociatedSeiList. When including an SEI message with nesting_ols_flag, the value of nesting_ols_flag shall be zero. .

[0139] nesting_num_olss_minus1 plus 1 is the number of scalable nested SEI messages. The value of nesting_num_olss_minus1 is inclusive. nesting_ols_idx_delta_minus1 must be in the range 0 to TotalNumOlss-1. [ i ] is the scalable nested SEI message when nesting_ols_flag is 1. Derive the variable NestingOlsIdx[ i ] that specifies the OLS index of the ith OLS to apply. The value of nesting_ols_idx_delta_minus1[ i ] is inclusive. The variable NestingOlsIdx[ i ] must be within the range of 0 to TotalNumOlss-2. It is derived as follows. if(i==0) NestingOlsIdx[ i ]=nesting_ols_idx_delta_minus1[ i ] else NestingOlsIdx[ i ]=NestingOlsIdx[ i-1 ]+nesting_ols_idx_delta_m inus1[ i ]+1

[0140] nesting_num_ols_layers_minus1[ i ] plus 1 is NestingOlsIdx[ i Applying scalable nested SEI messages in the context of the ]th OLS The value of nesting_num_ols_layers_minus1[ i ] is the double bound. Must be in the range 0 to NumLayersInOls[ NestingOlsIdx[ i ] ]-1, inclusive.

[0141] nesting_ols_layer_idx_delta_minus1[ i ][ j ] means nesting_ols_flag is 1 In the context of NestingOlsIdx[ i ]th OLS, scalability is specifies the OLS layer index of the jth layer to which the generated SEI message applies, Used to derive the variable NestingOlsLayerIdx[ i ][ j ]. The value of yer_idx_delta_minus1[ i ] is between 0 and NumLayersInOls[ nestingO Must be in the range [lsIdx[ i ] ]-2.

[0142] The variable NestingOlsLayerIdx[ i ][ j ] is derived as follows: if(j==0) NestingOlsLayerIdx[ i ][ j ]=nesting_ols_layer_idx_delta_minus1 [ i ][ j ] else NestingOlsLayerIdx[ i ][ j ]=NestingOlsLayerIdx[ i ][ j-1 ]+ nesting_ols_layer_idx_delta_minus1[ i ][ j ]+1

[0143] For i in the range 0 to nesting_num_olss_minus1, inclusive, LayerIdInOls[ Among all values ​​of NestingOlsIdx[ i ] [NestingOlsLayerIdx[ i ]

[0000] ] The minimum value is the number of frames per second for the current SEI NAL unit (e.g., a scalable nesting SEI message). The nuh_layer_id must be equal to the nuh_layer_id of the SEI NAL unit that contains the A scalable nested SEI message whose nuh_layer_id is equal to or greater than the nuh_layer_id of the current SEI NAL unit nesting_all_layers to specify that the page normally applies to all layers. _flag may be set to 1. nuh_layer_id is the nuh_layer_id of the current SEI NAL unit. Scalable nested SEI messages that are above the above usually apply to all layers. To specify that nesting is optional, nesting_all_layers_flag is set to 0. good.

[0144] nesting_num_layers_minus1 plus 1 is the number of layers in a scalable nested SEI mesh. Specifies the number of layers that the effect normally applies to. The value of nesting_num_layers_minus1 is 0~vps_max_layers_minus1-GeneralLayerIdx[ nuh_layer_id ], inclusive and nuh_layer_id must be within the range of nuh_layer_i of the current SEI NAL unit. d. nesting_layer_id[ i ] is the same as nesting_all_layers_flag when it is set to zero. nuh_layer_ of the i-th layer to which a scalable nested SEI message is normally applied Specify the id value. The value of nesting_layer_id[ i ] must be greater than nuh_layer_id. Otherwise, nuh_layer_id is the nuh_layer_id of the current SEI NAL unit.

[0145] When nesting_ols_flag is 1, scalable nested SEI messages are normally The NestingNumLayers variable specifies the number of layers to use, and the scalable nested SE IMessage normally specifies a list of nuh_layer_id values ​​for the layers to which it applies, with both ends set to A list NestingLayerId[ i ] for i in the range 0 to NestingNumLayers-1, inclusive is derived as follows, where nuh_layer_id is the nuh_layer_id of the current SEI NAL unit: be. if(nesting_all_layers_flag) { NestingNumLayers= vps_max_layers_minus1+1-GeneralLayerIdx[ nuh_layer_id ] for(i=0;i < NestingNumLayers;i++) NestingLayerId[ i ]=vps_layer_id[ GeneralLayerIdx[ nuh_layer_id ]+i ](D-2) } else { NestingNumLayers=nesting_num_layers_minus1+1 for(i=0;i < NestingNumLayers;i++) NestingLayerId[ i ]=(i==0)? nuh_layer_id:nesting_layer_id[ i] }

[0146] nesting_num_seis_minus1 plus 1 is the number of scalable nested SEI messages. The value of nesting_num_seis_minus1 must be in the range 0 to 63, inclusive. The nesting_zero_bit must be set to zero. do not have.

[0147] 8 is a schematic diagram of an exemplary video coding device 800. 800 is suitable for implementing the disclosed examples / embodiments as described herein. The video coding device 800 includes a downstream port 820 and an upstream port 830. 850, and / or the network to stream data upstream and / or a transceiver unit including a transmitter and / or a receiver for transmitting downstream (Tx / Rx) 810. The video coding device 800 also includes logic for processing the data. A processor 830 including a device and / or a central processing unit (CPU), and a memory for storing data. The video coding device 800 also includes electrical and optical-electrical (OE) components. , electro-optical (EO) components, and / or electrical, optical or wireless communication networks Upstream port 850 and / or downstream port 850 for data communication over the network The wireless communication component may be coupled to the video port 820. The device 800 also includes input and / or output (I / O) devices 802 for communicating data with a user. The I / O device 860 may include a display for displaying video data, an audio The I / O device 860 may also include an output device such as a speaker for outputting audio data. Also, input devices such as keyboards, mice, trackballs, and / or such output devices. It may also include a corresponding interface for interacting with the force device.

[0148] The processor 830 is implemented in hardware and software. 30 may include one or more CPU chips, cores (e.g., as a multi-core processor), filters, etc. Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), and Devices The processor 830 may be implemented as a digital signal processor (DSP). 8, the upstream port 850, and the memory 832. The processor 830 includes an encoding module 814. The encoding module 814 performs the methods 100, 900, 1000 and 1001, which are multi-layer video The encoding module 81 may use the bitstream 600 and / or the bitstream 700. 4 may also implement any other method / mechanism described herein. The encoding module 814 includes the codec system 200, the encoder 300, the decoder 400, and / or For example, the encoding module 814 may implement a 32-bit 32-bit decoder (HRD) 500 as described herein. , may be used for signaling and / or reading out various parameters. The encoding module encodes and decodes the video sequence based on these parameters. In this manner, the signaling described herein may be used to read and / or decode the The ring modification may increase efficiency and / or reduce errors in the encoding module 814. Thus, the encoding module 814 may provide a mechanism to address one or more of the problems discussed above. Thus, the encoding module 814 may be configured to perform the following operations: The coding device 800 may include additional functions and / or codes when encoding the video data. In this manner, the encoding module 814 provides the video coding apparatus 800 with improved coding efficiency. It also improves the functionality of the video coding technology and addresses the problems inherent in the video coding technology. Module 814 performs transformation of video coding device 800 to a different state. The encoding module 814 may be implemented as instructions stored in a memory 832 (e.g., a non-transitory Executable by the processor 830 (as a computer program product stored on a programmable medium) Cut.

[0149] The memory 832 may be a disk, a tape drive, a solid-state drive, a read-only memory ROM, Random Access Memory (RAM), Flash Memory, Ternary Content Addressable Memory (TCAM) ), static random access memory (SRAM), and one or more memory types. The memory 832 stores such programs when such programs are selected for execution. for storing instructions and data to be read during the execution of a program. For this reason, it may be used as an overflow data storage device.

[0150] Figure 9 shows how to reduce the number of bits per second by restricting the TemporalId of SEI messages in the bitstream. 7, encoding the video sequence into a bitstream such as bitstream 700. 9 is a flow diagram of a typical method 900. The method 900 may be implemented by the codec system 200 when performing the method 100. , encoder 300, and / or video coding device 800. Additionally, the method 900 may operate on the HRD 500, thus enabling multi-layer video systems. A conformance test may be performed on sequence 600.

[0151] The method 900 may, for example, include a step of generating a video sequence based on an input from a user. and encoding the video sequence into a multi-layer bitstream. In step 901, the encoder generates one or more VCL NAL units. Encode the coded pictures in the unit into a bitstream. For example, , the coded picture may be included in an AU in a layer. One or more layers containing coded pictures into a multilayer bitstream A layer is a set of VCL NAL units with the same layer ID. For example, a set of VCL NAL units may include a set of non-VCL NAL units and associated non-VCL NAL units. A set of VCL NAL units is a layer when all of the NAL units have a particular value of nuh_layer_id. A layer is a VCL NAL unit that contains the video data of an encoded picture. Any parameter set used to encode such a picture, including a set of units One or more layers may be output layers. The layer is encoded to support reconstruction of the output layer(s). However, such support layers are not intended to be output by a decoder. The encoder then prepares various combinations of layers to send to the decoder on demand. Layers allow the decoder to adapt to network conditions, hardware requirements, and Depending on the software capabilities and / or user settings, different representations of the video sequence can be obtained. The information may be transmitted as needed.

[0152] In step 903, the encoder generates one or more non-VCL NAL units in a bitstream. For example, a layer and / or set of layers can be encoded into It also contains various non-VCL NAL units. All non-VCL NAL units have the nuh_layer_id Specifically, a set of non-VCL NAL units that have a particular value of The nal_unit_type of the AL unit indicates that a non-VCL NAL unit contains an SEI message. When the TemporalId of a non-VCL NAL unit indicates L NAL unit is encoded so that it is constrained to be equal to the TemporalId of the AU that contains it. In other words, an SEI message may not be included in the same AU as the picture to which it applies. Therefore, the TemporalId of an SEI message contained in a non-VCL NAL unit is The TemporalId is constrained to be equal to the TemporalId of the AU that contains the message / non-VCL NAL unit. In some examples, the SEI message is a prefix SEI message and therefore a non-VCL NAL. The nal_unit_type of the SEI message is equal to PREFIX_SEI_NUT. Fixed SEI message, therefore the nal_unit_type of the non-VCL NAL is SUFFIX_SEI_NU Equal to T.

[0153] The TemporalId of a non-VCL NAL unit is nuh_temporal_id_plus1 in the non-VCL NAL unit. Similarly, the TemporalId of an AU can be specified by the coding The nuh_temporal_id_plus1 sequence in the VCL NAL unit that contains the slice of the segmented picture. The TemporalId of a VCL NAL unit can be specified by the TemporalId of all VCL NAL units in the same AU. CL NAL unit. Therefore, the nuh_te of the VCL NAL unit The mporal_id_plus1 syntax element is intended to be the same for all VCL NAL units in the same AU. Therefore, the nuh_temporal_id in non-VCL NAL units that contain SEI messages is limited to The value of the _plus1 syntax element is any VCL NAL unit that is in the same AU as the SEI message. The value of this syntax element is the same as the value of the nuh_temporal_id_plus1 syntax element in the non-VCL NAL unit. The TemporalId of a set is derived as follows: TemporalId=nuh_temporal_id_plus1-1 In addition, nuh_temporal_id_plus1 for non-VCL NAL units and VCL NAL units in the AU The value of MUST NOT be set to zero, as TemporalId can become negative. / or requirements, the bitstream may be modified as shown in this specification to conform to VVC or similar. However, the encoder is not guaranteed to meet the standards, e.g. For example, other models may have fewer limitations when operating on a different standard or different versions of the same standard. It may be possible to operate in that mode.

[0154] In step 905, the encoder performs an encoding process on the bitstream based on the SEI message. The HRD is used to perform a set of bitstream conformance tests on one or more For example, the HRD may include a conformance test for correlating the SEI message with the picture. For this purpose, the TemporalId and / or nuh_temporal_id_plus1 values ​​can be used. Therefore, the HRD may not match one or more coded pictures in the same AU as the SEI message. To perform the integrity test, parameters from the SEI message can be used. The coder then, in step 907, prepares the bitstream for communication to the decoder. The encoder can then store the bitstream as needed for the decoder. It can also transmit rays.

[0155] Figure 10 shows the bitstream where the TemporalId of the SEI message is restricted. Exemplary Method 1 for Decoding a Video Sequence from a Bitstream, such as Stream 700 000. The method 1000 includes, when performing the method 100, the codec system 200, The decoder 400 and / or the video coding device 800 may be used in a decoder. Additionally, the method 1000 may be used with a multi-layer bilayer film that has been verified for suitability by an HRD, such as the HRD 500. The signal may be used in the video sequence 600.

[0156] The method 1000 may, for example, be performed as a result of the method 900 and / or in response to a request by a decoder. A decoder then generates a bitstream of encoded data representing a multi-layer video sequence. In step 1001, the decoder may start receiving one or more VCLs. Bits that contain coded pictures in NAL units and non-VCL NAL units For example, a coded picture may be included in an AU. , the bitstream may contain one or more layers that contain coded pictures. A layer is a set of VCL NAL units with the same layer ID, and associated non-VCL NAL units. For example, a set of VCL NAL units may all contain a particular When it has a value, the set of VCL NAL units is part of a layer. It contains a set of VCL NAL units that contain the video data of the input picture, The parameter set may also include any parameter set used to code a particular picture. may be an output layer. A layer that is not an output layer is The layers are encoded to support reconstruction, but such support layers are In this way, the decoder can adapt to network conditions, hardware characteristics, Depending on the functionality and / or user settings, you can obtain a different representation of the video sequence. The layers further contain various non-VCL NAL units. Each VCL NAL unit has a particular value of nuh_layer_id.

[0157] Specifically, the nal_unit_type of a non-VCL NAL unit is set to 0 if the non-VCL NAL unit does not have an SEI message. When the bitstream indicates that it contains a non-VCL NAL unit, , the TemporalId of the non-VCL NAL unit is equal to the TemporalId of the AU that contains the non-VCL NAL unit. In other words, the SEI message is encoded in such a way that it is restricted to It can be included in the same AU as the picture to which it applies. Therefore, the SEI included in a non-VCL NAL unit The TemporalId of the message is the same as the TemporalId of the AU that contains the SEI message / non-VCL NAL unit. In some cases, the SEI message is restricted to be equal to the prefix SEI message. message, and therefore the nal_unit_type of non-VCL NALs is equal to PREFIX_SEI_NUT. In some examples, the SEI message is a suffix SEI message and therefore the n of the non-VCL NAL. al_unit_type is equal to SUFFIX_SEI_NUT.

[0158] The TemporalId of a non-VCL NAL unit is nuh_temporal_id_plus1 in the non-VCL NAL unit. The decoder may then determine in step 1002 whether the Based on the nuh_temporal_id_plus1 syntax element in the non-VCL NAL unit, The TemporalId of a CL NAL unit can be derived. Similarly, the TemporalId of an AU can be derived from the nuh_temporal_ in a VCL NAL unit that contains a slice of a coded picture The TemporalId of a VCL NAL unit can be specified by the id_plus1 syntax element. Therefore, the VCL NAL unit is restricted to be the same as all VCL NAL units in the same AU. The nuh_temporal_id_plus1 syntax element of the Therefore, the nuh_t in a non-VCL NAL unit that contains a SEI message is restricted to be the same as The value of the emporal_id_plus1 syntax element is an identifier for any VCL that is in the same AU as the SEI message. It is the same as the value of the nuh_temporal_id_plus1 syntax element in the NAL unit. The TemporalId of an L NAL unit is derived as follows: TemporalId=nuh_temporal_id_plus1-1 In addition, nuh_temporal_id_plus1 for non-VCL NAL units and VCL NAL units in the AU The value of does not have to be set to zero since TemporalId can become negative.

[0159] In one embodiment, the video decoder may generate non-VCL NAL streams based on VVC or some other standard. If the unit's nal_unit_type is an SEI message as described above, then it is a non-VCL NAL unit. The TemporalId of the AU containing the non-VCL NAL unit is expected to be equal to the TemporalId of the AU containing the non-VCL NAL unit. However, if the decoder determines that this condition is not true, it ,detecting errors to ensure that a conforming bitstream is received; Signaling an error and requesting retransmission of a corrected bitstream (or part of it) or other corrective measures may be taken.

[0160] In step 1003, the decoder generates a decoded picture by decoding the VCL NAL The coded pictures from the unit can be decoded. For example, Use TemporalId and / or nuh_temp to correlate SEI messages with pictures. The decoder can then use the oral_id_plus1 value. When decoding the architecture, the SEI message can be used if necessary. In step 1005, the decoder generates the decoded video sequence for display as part of the decoded video sequence. The decoded picture can be transferred.

[0161] FIG. 11 shows the bitstream where the TemporalId of the SEI message is restricted. 1 is a schematic diagram of an example system 1100 for encoding a video sequence using a stream. The system 1100 includes a codec system 200, an encoder 300, a decoder 400, and / or a Or it may be implemented in an encoder and a decoder such as video coding device 800. Additionally, the system 1100 may include a multi-layer video sequence 600 and / or a bitstream. The HRD 500 may be used to perform compatibility testing on the ream 700. In addition, the system 1100 , may be used when implementing methods 100, 900, and / or 1000.

[0162] The system 1100 includes a video encoder 1102. The video encoder 1102 is a bitstream. Encodes a coded picture in one or more VCL NAL units in a stream. The encoding module 1103 further includes Furthermore, when the nal_unit_type of a non-VCL NAL unit is an SEI message, The TemporalId of an AU containing non-VCL NAL units is constrained to be equal to the TemporalId of the AU containing the non-VCL NAL unit. It is intended to encode non-VCL NAL units into the bitstream so that The video encoder 1102 generates a set of video streams for the bitstream based on the SEI message. The video end further includes an HRD module 1105 for performing a video stream conformance test. The encoder 1102 includes a storage module that stores the bitstream for communication to the decoder. The video encoder 1102 outputs the bits to the video decoder 1110. The video encoder 1102 further comprises a transmission module 1107 for transmitting the video stream. It may be further configured to perform any step of method 900.

[0163] The system 1100 further includes a video decoder 1110. The video decoder 1110 includes one or more The bits that contain the pictures coded in the VCL NAL units and non-VCL NAL units of a receiving module 1111 for receiving a non-VCL NAL unit TemporalId of the non-VCL NAL unit; If the nal_unit_type of the non-VCL NAL unit is an SEI message, The TemporalId of the AU that contains the decoded AU is restricted to be equal to the TemporalId of the AU that contains the decoded AU. Decodes coded pictures from VCL NAL units to generate the corresponding pictures. The video decoder 1110 further comprises a decoding module 1113 for decoding the decoded video sequence. a transfer module 1115 for transferring the decoded picture for display as part of the sequence; The video decoder 1110 may further comprise a step of performing any of the steps of the method 1000. It can be further configured as follows.

[0164] The first component is the line between the first component and the second component, If there are no intervening components except for a race or other intervening The first component is directly coupled to the second component. Any component other than the wires, traces, or other intermediaries between the two components If there is a component, it is indirectly coupled to the second component. The variation includes both direct and indirect binding. Use of the term "about" is intended to include any and all instances of the term "about" unless otherwise specified. Unless otherwise stated, it means a range including ±10% of the subsequent value.

[0165] The steps of the exemplary methods described herein may be performed in the order listed. It should also be understood that the order of steps in such methods need not necessarily be sequential. Likewise, in a manner consistent with various embodiments of the present disclosure, Therefore, such methods may include additional steps and / or omit some steps. They may be combined.

[0166] Although several embodiments are provided in this disclosure, the disclosed systems and methods It should be understood that the present disclosure may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. It will be appreciated that the examples in the present disclosure should be considered as illustrative and not limiting. The intent should not be limited to the details given in this specification. For example, various elements Or the components may be combined or integrated in another system or may be used for a specific function. Capabilities may be omitted or not implemented.

[0167] In addition, the techniques described and illustrated as separate or distinct in various embodiments may be used interchangeably. The methods, systems, subsystems, and methods may be implemented in other ways without departing from the scope of this disclosure. Any modifications, substitutions, and / or substitutions may be made without departing from the spirit or scope of the present invention. Other examples of modifications and variations will be apparent to those skilled in the art and are within the spirit and scope of the present disclosure. This may be done without departing from the scope of the present invention. [Explanation of symbols]

[0168] 100 How it works 200 Codec System 201 Partitioned Video Signal 211 General Coder Control Component 213 Transform Scaling Quantization Component 215 Intra-picture Estimation Component 217 Intra-picture Prediction Component 219 Motion Compensation Component 221 Motion Estimation Component 223 Decoded Picture Buffer Component 225 In-Loop Filter Components 227 Filter Control Analysis Component 229 Scaling Inverse Transformation Component 231 Header Formatting - Context-Adaptive Binary Arithmetic Coding (CABAC) component 300 Encoder 301 Partitioned Video Signal 313 Transform Quantization Component 317 Intra-picture Prediction Component 321 Motion Compensation Component 323 Decoded Picture Buffer Component 325 In-Loop Filter Components 329 Inverse Transform Quantization Component 331 Entropy Coding Components 400 Decoder 417 Intra-Picture Prediction Component 421 Motion Compensation Component 423 Decoded Picture Buffer Component 425 In-Loop Filter Components 429 Inverse Transform Quantization Component 433 Entropy Decoding Component 500 Hypothetical Reference Decoder (HRD) 541 Virtual Stream Scheduler (HSS) 543 Coded Picture Buffer (CPB) 545 Decryption Process Components 547 Decoded Picture Buffer (DPB) 549 Output Cropping Component 551 Bitstream 553 Decoding Unit (DU) 555 Decrypted DUs 556 Reference Pictures 557 Pictures 559 Output Cropped Picture 600 Multi-layer Video Sequences 611 Pictures 611 Pictures 612 Pictures 613 Pictures 614 Pictures 615 Pictures 616 Pictures 617 Pictures 618 Pictures 621 Inter-layer Prediction 623 Inter Prediction 627 Access Unit (AU) 628 Picture Unit (PU) 629 TemporalId 631 Layer N 632 Layer N+1 700 bitstream 711 VPS 713 SPS 715 PPS 716 APS 717 Slice Header 718 Picture Header 719 Supplemental Enhancement Information (SEI) Message 720 Image data 721 Pictures 723 Layer 725 Pictures 727 Slice 731 NAL unit header temporal identifier plus one (nuh_temporal_id_plus1) 741 Non-VCL NAL units 742 Prefix SEI NAL unit type (PREFIX_SEI_NUT) 743 Suffix SEI NAL unit type (SUFFIX_SEI_NUT) 745 VCL NAL Units 800 Video Coding Device 810 Transceiver Unit (Tx / Rx) 814 Encoding Module 820 downstream ports 830 Processor 832 Memory 850 Upstream Ports 860 Input and / or Output (I / O) Devices How to encode a 900 video sequence into a bitstream How to decode 1000 video sequences 1100 System for encoding a video sequence 1102 Video Encoder 1103 Encoding Module 1105 HRD Module 1106 Memory Module 1107 Transmitting Module 1110 Video Decoder 1111 Receiver Module 1113 Decryption module 1115 Transfer Module

Claims

1. A method implemented by a decoder, comprising: receiving a bitstream including coded pictures in one or more video coding layer (VCL) network abstraction layer (NAL) units and non-VCL NAL units, wherein when a NAL unit type (nal_unit_type) of the non-VCL NAL unit is equal to a prefix added extension information (SEI) NAL unit type (PREFIX_SEI_NUT) or a suffix SEI NAL unit type (SUFFIX_SEI_NUT), a temporal identifier (TemporalId) of the non-VCL NAL unit is equal to a TemporalId of an access unit (AU) that includes the non-VCL NAL unit, a NAL unit comprising a scalable nesting SEI message with payload type set to 133, the scalable nesting SEI message comprising one or more scalably nested messages, the scalable nesting SEI message comprising one or more output layer set (OLS) flags equal to 1 when the scalable nesting SEI message applies to an output layer set (OLS) and equal to 0 when the scalable nesting SEI message applies to a layer, the scalable nesting SEI message comprising a number of scalable nested SEI messages minus 1, inclusive, the number being in the range of 0 to 63, inclusive; deriving the TemporalId of the non-VCL NAL unit based on a NAL unit header temporal identifier plus one (nuh_temporal_id_plus1) syntax element in the non-VCL NAL unit; decoding the coded picture from the VCL NAL unit to generate a decoded picture; A method comprising:

2. The method of claim 1 , wherein the nal_unit_type of the non-VCL NAL is equal to the PREFIX_SEI_NUT.

3. The method of claim 1 , wherein the nal_unit_type of the non-VCL NAL is equal to the SUFFIX_SEI_NUT.

4. The method according to claim 1 , wherein the coded picture is decoded from the VCL NAL unit based on an SEI message in the non-VCL NAL unit.

5. The method of claim 1 , further comprising the step of deriving the TemporalId of the non-VCL NAL unit according to: TemporalId=nuh_temporal_id_plus1−1.

6. 6. The method according to claim 1, wherein the value of nuh_temporal_id_plus1 is not zero.

7. The method according to claim 1 , wherein the TemporalId of the VCL NAL unit is constrained to be the same for all VCL NAL units in the same AU.

8. A method for receiving a second bitstream including one or more second VCL NAL units and a second non-VCL NAL unit, wherein when a nal_unit_type of the second non-VCL NAL unit is an SEI message, a TemporalId of the second non-VCL NAL unit is not equal to a TemporalId of a second AU that includes the second non-VCL NAL unit; in response to the receiving step, taking corrective measures to ensure that a bitstream that matches the second bitstream is received before decoding the coded picture from the second VCL NAL unit; and 8. The method of claim 1, further comprising:

9. A computer readable storage medium having a program recorded thereon, the program causing the computer to carry out the method according to any one of claims 1 to 8.

10. 1. A decoding device, comprising:

1. A receiver configured to receive a bitstream including coded pictures in one or more video coding layer (VCL) network abstraction layer (NAL) units and non-VCL NAL units, wherein when a NAL unit type (nal_unit_type) of the non-VCL NAL unit is a supplemental enhancement information (SEI) message, a temporal identifier (TemporalId) of the non-VCL NAL unit is equal to a TemporalId of an access unit (AU) that includes the non-VCL NAL unit, and a receiver, wherein a NAL unit includes a scalable nesting SEI message with a payload type set to 133, the scalable nesting SEI message including one or more scalably nested messages, the scalable nesting SEI message including one or more output layer set (OLS) flags that are equal to 1 when the scalable nesting SEI message applies to an output layer set (OLS) and that are equal to 0 when the scalable nesting SEI message applies to a layer, the scalable nesting SEI message including a number of scalable nested SEI messages minus 1, inclusive, in a range of 0 to 63; a memory coupled to the receiver, the memory storing instructions; and a processor coupled to said memory, said processor being configured to execute said instructions to cause said decoding device to perform the method of any one of claims 1 to 8; A decoding device comprising:

11. A method implemented by an encoder, comprising: encoding a picture into one or more video coding layer (VCL) network abstraction layer (NAL) units in a bitstream; encoding a non-VCL NAL unit into the bitstream; when the NAL unit type (nal_unit_type) of the non-VCL NAL unit is equal to a prefix-added extension information (SEI) NAL unit type (PREFIX_SEI_NUT) or a suffix SEI NAL unit type (SUFFIX_SEI_NUT), a temporal identifier (TemporalId) of the non-VCL NAL unit is equal to the TemporalId of an access unit (AU) that contains the non-VCL NAL unit; 11. A method according to claim 1, wherein the non-VCL NAL unit comprises a scalable nesting SEI message with a payload type set to 133, the scalable nesting SEI message comprising one or more scalably nested messages, the scalable nesting SEI message comprising one or more output layer set (OLS) flags that are equal to 1 when the scalable nesting SEI message applies to an output layer set (OLS) and that are equal to 0 when the scalable nesting SEI message applies to a layer, and the scalable nesting SEI message comprises a number of scalable nested SEI messages minus 1, the number being in the range of 0 to 63, inclusive.

12. The method of claim 11, wherein the nal_unit_type of the non-VCL NAL is equal to the PREFIX_SEI_NUT.

13. The method of claim 11, wherein the nal_unit_type of the non-VCL NAL is equal to the SUFFIX_SEI_NUT.

14. The method of claim 11, wherein the TemporalId of the non-VCL NAL unit is derived according to TemporalId=nuh_temporal_id_plus1-1.

15. The method of claim 14, wherein the value of nuh_temporal_id_plus1 is not equal to zero.

16. A method according to claim 11, wherein the nuh_temporal_id_plus1 of the VCL NAL unit is constrained to be the same for all VCL NAL units within the same AU.

17. A non-transitory computer-readable medium having stored thereon computer instructions which, when executed by at least one hardware processor, cause the at least one hardware processor to perform the method of any one of claims 11 to 16.

18. An encoding device comprising: a memory and a processor, the memory coupled to the processor, the memory storing instructions; An encoding device, the processor executing the instructions to cause the encoding device to perform the method of any one of claims 11 to 16.

19. A program causing a computer to carry out the method according to any one of claims 1 to 8 or the method according to any one of claims 11 to 16.

20. A device for storing a bitstream, comprising at least one storage medium and at least one receiver, the at least one receiver configured to receive one or more bitstreams; the at least one storage medium is configured to store the one or more bitstreams; the bitstream includes coded pictures in one or more video coding layer (VCL) network abstraction layer (NAL) units and non-VCL NAL units, and when a NAL unit type (nal_unit_type) of the non-VCL NAL unit is equal to a prefix added extension information (SEI) NAL unit type (PREFIX_SEI_NUT) or a suffix SEI NAL unit type (SUFFIX_SEI_NUT), a temporal identifier (TemporalId) of the non-VCL NAL unit is equal to a TemporalId of an access unit (AU) that contains the non-VCL NAL unit, 11. A device comprising: a NAL unit comprising a scalable nesting SEI message with a payload type set to 133, the scalable nesting SEI message comprising one or more scalably nested messages; the scalable nesting SEI message comprising one or more output layer set (OLS) flags that are equal to 1 when the scalable nesting SEI message applies to an output layer set (OLS) and that are equal to 0 when the scalable nesting SEI message applies to a layer; and the scalable nesting SEI message comprising a number of scalable nested SEI messages minus 1, the number being in the range of 0 to 63, inclusive.

21. A method for storing a bitstream, comprising the steps of: receiving one or more bitstreams via at least one receiver; storing the one or more bitstreams in one or more memories; the bitstream includes coded pictures in one or more video coding layer (VCL) network abstraction layer (NAL) units and non-VCL NAL units, and when a NAL unit type (nal_unit_type) of the non-VCL NAL unit is equal to a prefix added extension information (SEI) NAL unit type (PREFIX_SEI_NUT) or a suffix SEI NAL unit type (SUFFIX_SEI_NUT), a temporal identifier (TemporalId) of the non-VCL NAL unit is equal to a TemporalId of an access unit (AU) that contains the non-VCL NAL unit, 11. A method according to claim 1, wherein a NAL unit includes a scalable nesting SEI message with a payload type set to 133, the scalable nesting SEI message including one or more scalably nested messages, the scalable nesting SEI message including one or more output layer set (OLS) flags that are equal to 1 when the scalable nesting SEI message applies to an output layer set (OLS) and that are equal to 0 when the scalable nesting SEI message applies to a layer, and the scalable nesting SEI message includes a number of scalable nested SEI messages minus 1, the number being in the range of 0 to 63, inclusive.

Images