Method and apparatus for image encoding and decoding

Abstract

The invention relates to a method for decoding, the method comprising: obtaining (501) a bitstream, the bitstream representing an encoded video sequence; obtaining (502) first information indicating whether there is no container of a first type providing at least one coding parameter in the bitstream; checking a value of a first syntax element indicating whether a coding tool using the at least one coding parameter is activated for a current block of a sample of an image of the video sequence; and, when the second syntax element indicates activation of the coding tool, adapting decoding of the current block in the absence of the container.

Description

[0001] This application is a divisional application of patent application No. 202180027261.0, filed on March 15, 2021, entitled "Method and apparatus for image encoding and decoding". 1. Technical Field

[0002] At least one embodiment of the present invention relates generally to methods and apparatus for image encoding and decoding, and more specifically to methods and apparatus for ensuring consistent interaction between some signaling tools and some encoding tools. 2. Background Technology

[0003] To achieve high compression efficiency, video coding schemes typically employ prediction and transform to utilize spatial and temporal redundancy in the video content. During encoding, images of the video content are divided into sample blocks (i.e., pixels), and these blocks are then partitioned into one or more sub-blocks, hereinafter referred to as original sub-blocks. Intra-frame or inter-frame prediction is then applied to each sub-block to utilize intra-frame or inter-frame image correlations. Regardless of the prediction method used (intra-frame or inter-frame), a predicted sub-block is determined for each original sub-block. The sub-blocks representing the difference between the original sub-blocks and the predicted sub-blocks (typically denoted as prediction error sub-blocks, prediction residual sub-blocks, or simply residual blocks) are then transformed, quantized, and entropy-coded to generate the encoded video stream. To reconstruct the video, the compressed data is decoded through the inverse process corresponding to the transform, quantization, and entropy coding.

[0004] Compared to earlier video compression methods such as MPEG-1 (ISO / CEI-11172), MPEG-2 (ISO / CEI 13818-2), or MPEG-4 / AVC (ISO / CEI 14496-10), the complexity of video compression methods has increased significantly. In fact, many new coding tools have emerged, or existing coding tools have been improved upon in previous generations of video compression standards (for example, in the international standard called Universal Video Coding (VVC), being developed by a joint collaborative group of ITU-T and ISO / IEC experts known as the Joint Video Experts Group (JVET), or in the standard HEVC (ISO / IEC 23008-2 – MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265)). Simultaneously, several signaling tools have been proposed, allowing signaling parameters of some coding tools to be shared, for example, by many sub-blocks along a video sequence. One such signaling tool is the Adaptive Parameter Set (APS). APS is a specific data container (called NAL (Network Abstraction Layer) unit) that provides parameters for the Adaptive Loop Filter (ALF) coding tool, the Luminance Mapping with Chroma Scaling (LMCS) coding tool, and the scaling matrix used for quantization.

[0005] In some cases, when an encoding tool using APS as a signaling parameter is activated, APS may not be present in the bitstream.

[0006] The goal is to propose a solution that allows for consistent behavior of the encoder or decoder when a signaling container is absent in the bitstream and an encoding tool referencing parameters provided by that signaling container is activated. 3. Summary of the Invention

[0007] In a first aspect, one or more embodiments of the present invention provide a method for decoding, the method comprising: obtaining a bitstream representing an encoded video sequence; obtaining first information indicating whether a container of a first type providing at least one encoding parameter is absent in the bitstream; checking the value of a first syntax element indicating whether an encoding tool using the at least one encoding parameter is activated for a current block of a sample of an image of the video sequence; and, when a second syntax element indicates activation of the encoding tool, adaptively decoding the current block in the absence of a container.

[0008] In one implementation, the first information is obtained from a second syntax element, which is obtained from a bitstream.

[0009] In one implementation, the first type of container is an adaptive parameter set, and the second syntax element indicates whether it is authorized that at least one adaptive parameter set exists in the bitstream.

[0010] In one implementation, adaptive decoding of the current block includes outputting second information representing non-conformities in the bitstream.

[0011] In one implementation, adaptive decoding of the current block includes disabling encoding tools for decoding the current block.

[0012] In one implementation, adaptive decoding includes obtaining at least one parameter from at least one container of at least one second type, and applying an encoding tool using the obtained parameter to decode the current block.

[0013] In one implementation, at least one container of at least one second type is a sequence parameter set and / or an image parameter set and / or an image header and / or a sequence header.

[0014] In a second aspect, one or more embodiments of the present invention provide a method for encoding, the method comprising: obtaining a video sequence for encoding in a bitstream; adaptively encoding sample blocks of images of the video sequence according to first information indicating whether a container of a first type is authorized to exist in the bitstream to provide at least one encoding parameter for the encoding tool.

[0015] In one implementation, the first information is encoded in a bitstream.

[0016] In one implementation, the first type of container is an adaptive parameter set, and the second syntax element indicates whether it is authorized that at least one adaptive parameter set exists in the bitstream.

[0017] In one implementation, adaptive encoding includes removing an encoding tool considered for encoding the current block from the list of encoding tools if an unauthorized container exists.

[0018] In one implementation, adaptive encoding includes: if an encoding tool is authorized to encode the current block, then encoding at least one encoding parameter in at least one container of at least one second type.

[0019] In one implementation, at least one container of at least one second type is a sequence parameter set and / or an image parameter set and / or an image header and / or a sequence header.

[0020] In a third aspect, one or more embodiments of the present invention provide an apparatus for decoding, the apparatus including electronic circuitry adapted to: obtain a bitstream representing an encoded video sequence; obtain first information indicating whether a container of a first type providing at least one encoding parameter is absent in the bitstream; check the value of a first syntax element indicating whether an encoding tool using the at least one encoding parameter is activated for the current block of a sample of an image in the video sequence; and, when a second syntax element indicates activation of the encoding tool, adaptively decode the current block in the absence of a container.

[0021] In one implementation, the first information is obtained from a second syntax element, which is obtained from a bitstream.

[0022] In one implementation, the first type of container is an adaptive parameter set, and the second syntax element indicates whether it is authorized that at least one adaptive parameter set exists in the bitstream.

[0023] In one implementation, adaptive decoding of the current block includes outputting second information representing non-conformities in the bitstream.

[0024] In one implementation, adaptive decoding of the current block includes disabling encoding tools for decoding the current block.

[0025] In one implementation, adaptive decoding includes obtaining at least one parameter from at least one container of at least one second type, and applying an encoding tool using the obtained parameter to decode the current block.

[0026] In one implementation, at least one container of at least one second type is a sequence parameter set and / or an image parameter set and / or an image header and / or a sequence header.

[0027] In a fourth aspect, one or more embodiments of the present invention provide an apparatus for encoding, the apparatus including electronic circuitry adapted to: acquire a video sequence for encoding in a bitstream; and adaptively encode sample blocks of images of the video sequence according to first information indicating whether a container of a first type is authorized to exist in the bitstream to provide at least one encoding parameter for the encoding tool.

[0028] In one implementation, the first information is encoded in a bitstream.

[0029] In one implementation, the first type of container is an adaptive parameter set, and the second syntax element indicates whether it is authorized that at least one adaptive parameter set exists in the bitstream.

[0030] In one implementation, adaptive encoding includes removing an encoding tool considered for encoding the current block from the list of encoding tools if an unauthorized container exists.

[0031] In one implementation, adaptive encoding includes: if an encoding tool is authorized to encode the current block, then encoding at least one encoding parameter in at least one container of at least one second type.

[0032] In one implementation, at least one container of at least one second type is a sequence parameter set and / or an image parameter set and / or an image header and / or a sequence header.

[0033] In a fifth aspect, one or more embodiments of the present invention provide a signal comprising data generated by the method for encoding according to the second aspect or by the apparatus for encoding according to the fourth aspect.

[0034] In a sixth aspect, one or more embodiments of the present invention provide a computer program comprising program code instructions for implementing the method according to the first or second aspect.

[0035] In a seventh aspect, one or more embodiments of the present invention provide an information storage device that stores program code instructions for implementing the method according to the first or second aspect. 4. Description of the attached drawings

[0036] Figure 1 An example of the partitions that the pixel images of the original video have gone through is shown;

[0037] Figure 2 The method for encoding a video stream, performed by the encoding module, is illustrated schematically.

[0038] Figure 3 A method for decoding encoded video streams (i.e., bitstreams) is illustrated schematically.

[0039] Figure 4A An example of a hardware architecture for a processing module capable of implementing an encoding or decoding module is schematically shown, in which various aspects and implementation schemes are implemented;

[0040] Figure 4B A block diagram of an example system is shown, in which various aspects and implementation schemes are implemented;

[0041] Figure 5 A schematic depiction of a solution for adaptive decoding when APS is unavailable on the decoder side; and,

[0042] Figure 6 This schematically depicts a solution for adapting the encoding process when APS cannot be used. 5. Detailed Implementation

[0043] In the following description, some implementations use tools developed in the context of VVC or HEVC. However, these implementations are not limited to video encoding / decoding methods corresponding to VVC or HEVC, and are applicable to other video encoding / decoding methods, as well as any method in which an image is predicted from another image.

[0044] Figure 1 An example of the partitioning experienced by pixel sample 11 of the original video 10 is shown. Here, a sample is considered to consist of three components: one luminance component and two chrominance components. In this case, the sample corresponds to a pixel. However, the following embodiments are applicable to images composed of samples including another number of components (e.g., where the sample includes a grayscale sample of one component), or images composed of samples including three color components and a transparency component and / or a depth component. The following embodiments are applicable to one component of the image. In this case, the sample corresponds to the value of one component.

[0045] The image is divided into multiple coded entities. First, as... Figure 1 As indicated by reference numeral 13, the image is divided into a grid of blocks called coding tree units (CTUs). A CTU consists of N×N luminance sample blocks and two corresponding chrominance sample blocks. N is typically a power of two, for example, a maximum of "128". Next, the image is divided into one or more groups of CTUs. For example, the image may be divided into one or more tile rows and tile columns, where a tile is a sequence of CTUs covering a rectangular area of ​​the image. In some cases, a tile may be divided into one or more bricks, each brick consisting of at least one row of CTUs within the tile. Above the concepts of tiles and bricks, there is another coding entity called a slice, which may contain at least one tile of the image or at least one brick of a tile.

[0046] exist Figure 1 In the example, as indicated by reference numeral 12, image 11 is divided into three slices S1, S2 and S3, each slice comprising multiple tiles (not shown).

[0047] like Figure 1 As indicated by reference numeral 14, a CTU can be partitioned into a hierarchical tree of one or more sub-blocks called coding units (CUs). The CTU is the root (i.e., parent node) of the hierarchical tree and can be partitioned into multiple CUs (i.e., child nodes). If each CU is not further partitioned into smaller CUs, then each CU becomes a leaf of the hierarchical tree; or if each CU is further partitioned into smaller CUs (i.e., child nodes), then each CU becomes a parent node of the smaller CUs. Several types of hierarchical trees can be applied, including, for example, quadtrees, binary trees, and ternary trees. In a quadtree, a CTU (or CU) can be partitioned into four equal-sized square CUs (i.e., it can be the parent node of four equal-sized square CUs). In a binary tree, a CTU (or CU) can be partitioned horizontally or vertically into two equal-sized rectangular CUs. In a ternary tree, a CTU (or CU) can be partitioned horizontally or vertically into three rectangular CUs. For example, a CU with a height of N and a width of M is vertically (or horizontally) divided into a first CU with a height of N (or N / 4) and a width of M / 4 (or M), a second CU with a height of N (or N / 2) and a width of M / 2 (or M), and a third CU with a height of N (or N / 4) and a width of M / 4 (or M).

[0048] exist Figure 1In the example, firstly, CTU 14 is partitioned into "4" square CUs using quadtree partitioning. The top-left CU is a leaf of the hierarchical tree because it is not further partitioned, meaning it is not the parent node of any other CU. The top-right CU is further partitioned into "4" smaller square CUs using quadtree partitioning again. The bottom-right CU is vertically partitioned into "2" rectangular CUs using binary tree partitioning. The bottom-left CU is vertically partitioned into "3" rectangular CUs using ternary tree partitioning.

[0049] During image encoding, partitioning is adaptive, with each CTU being partitioned to optimize the compression efficiency of the CTU criteria.

[0050] In some compression methods, the concepts of prediction units (PU) and transform units (TU) emerge. In this case, the coding entity used for prediction (i.e., PU) and the coding entity used for transform (i.e., TU) can be sub-partitions of the CU. For example, as... Figure 1 This indicates that a CU of size 2N×2N can be divided into PUs 1411 of size N×2N or size 2N×N. Additionally, the CU can be divided into “4” TUs 1412 of size N×N or “16” TUs of size (N / 2)×(N / 2).

[0051] In this application, the terms "block" or "image block" or "subblock" may be used to refer to any of CTU, CU, PU, ​​and TU. Additionally, the terms "block" or "image block" may be used to refer to macroblocks, partitions, and subblocks as specified in MPEG-4 / AVC or other video coding standards, and more generally to an array of samples of numerous sizes.

[0052] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture”, “subpicture”, “slice” and “frame” are used interchangeably.

[0053] Figure 2 A method for encoding a video stream, performed by an encoding module, is illustrated schematically. Variations of this encoding method are envisioned, but for clarity, they are described below. Figure 2 The method used for encoding is given, but not all expected variants are described.

[0054] The encoding of the current original image 201 during step 202 begins with a partition of the current original image 201, as per the relevant information. Figure 1 As described. Therefore, the current image is partitioned into 201 blocks: CTU, CU, PU, ​​TU, etc. For each block, the coding module determines the coding mode between intra-frame prediction and inter-frame prediction.

[0055] The intra-frame prediction, represented by step 203, includes predicting samples of the current block from a prediction block according to an intra-frame prediction method. This prediction block is derived from samples of a reconstructed block located near the causal relationship of the current block to be encoded. The result of the intra-frame prediction is a prediction direction indicating which samples from nearby blocks are used, and a residual block obtained by calculating the difference between the current block and the prediction block.

[0056] Inter-frame prediction involves predicting samples for the current block from sample blocks (called reference blocks) of images preceding or following the current image (referred to as the reference image). During encoding of the current block according to the inter-frame prediction method, the closest block to the current block in the reference image is determined by motion estimation step 204 based on a similarity criterion. During step 204, a motion vector indicating the location of the reference block in the reference image is determined. This motion vector is used during motion compensation step 205, during which a residual block is calculated as the difference between the current block and the reference block.

[0057] In the first video compression standard, the aforementioned one-way inter-frame prediction mode was the only available inter-frame mode. As video compression standards have evolved, the family of inter-frame modes has grown significantly and now includes many different inter-frame modes.

[0058] During the selection step 206, the coding module selects the prediction mode that optimizes compression performance from the tested prediction modes (intra-frame prediction mode, inter-frame prediction mode) according to the rate / distortion criterion (i.e., RDO criterion).

[0059] When a prediction mode is selected, the residual block is transformed during step 207 and quantized during step 209. During quantization, in the transform domain, the transform coefficients are weighted by a scaling matrix in addition to the quantization parameters. A scaling matrix is ​​a coding tool that allows some frequencies to be supported at the expense of others. Generally, low frequencies are advantageous. Some video compression methods allow the application of user-defined scaling matrices instead of the default scaling matrix. In this case, the parameters of the scaling matrix need to be emitted to the decoder. In some implementations, signaling tools such as signaling containers (i.e., NAL (Network Abstraction Layer) units) are used to specify the parameters of non-default scaling matrices. In some implementations, the NAL unit used to signal the scaling matrix parameters is called an Adaptive Parameter Set (APS).

[0060] It should be noted that the encoding module can skip the transformation and apply quantization directly to the untransformed residual signal.

[0061] When the current block is encoded according to the intra-frame prediction mode, during step 210, the prediction direction and the transformed and quantized residual block are encoded by the entropy encoder.

[0062] When encoding the current block according to the inter-frame prediction mode, motion data associated with the inter-frame prediction mode is encoded in step 208.

[0063] Generally speaking, two modes can be used to encode motion data, namely AMVP (Adaptive Motion Vector Prediction) mode and merge mode.

[0064] The AMVP model basically includes signaling a reference image for predicting the current block, a motion vector prediction index, and a motion vector difference (also known as a motion vector residual).

[0065] The merging pattern includes a signal indicating the index of some motion data collected in a list of motion data prediction values. This list consists of either "5" or "7" candidates and is constructed in the same way on both the decoder and encoder sides. Therefore, the merging pattern aims to derive some motion data taken from the merging list. The merging list typically contains motion data associated with some spatially and temporally adjacent blocks, which are available in their reconstructed state when processing the current block.

[0066] Once predicted, the motion information is then encoded by an entropy encoder along with the transformed and quantized residual blocks during step 210. It should be noted that the encoding module can bypass the transform and quantization; that is, entropy encoding can be applied to the residuals without applying the transform or quantization process. The result of the entropy encoding is inserted into the encoded video stream (i.e., the bitstream) 211.

[0067] It should be noted that entropy encoders can be implemented in the form of context-adaptive binary arithmetic encoders (CABAC). CABAC encodes binary symbols, which maintains low complexity and allows for probabilistic modeling of the more frequently used bits of any symbol.

[0068] After quantization step 209, the current block is reconstructed so that the pixels corresponding to that block are available for future prediction. This reconstruction stage is also called the prediction loop. Therefore, inverse quantization is applied to the transformed and quantized residual block during step 212, and inverse transform is applied during step 213. The prediction block of the current block is reconstructed based on the prediction mode used for the current block obtained during step 214. If the current block is encoded according to an inter-frame prediction mode, the encoding module applies motion compensation to a reference block using the motion information of the current block during step 216, where appropriate. If the current block is encoded according to an intra-frame prediction mode, the reference block of the current block is reconstructed during step 215 using the prediction direction corresponding to the current block. The reference block and the reconstructed residual block are added together to obtain the reconstructed current block.

[0069] Following reconstruction, during step 217, an in-loop post-filter designed to reduce coding artifacts is applied to the reconstruction block. This post-filter is called an in-loop post-filter because it occurs in the prediction loop to obtain the same reference image at the encoder as at the decoder, thereby avoiding drift between the encoding and decoding processes. Examples of in-loop post-filters include deblocking filtering, SAO (Sample Adaptive Shift) filtering, and adaptive loop filtering (ALF) with block-based filter adaptation.

[0070] In ALF, for the luminance component, a filter is selected from multiple filters for each 4×4 block of the image based on the direction and activity of the local gradient. Filter selection is based on the classification of the 4×4 blocks. The ALF filter parameters need to be transmitted to the decoder. In some implementations, the ALF filter parameters are signaled in an Adaptive Parameter Set (APS).

[0071] During entropy coding step 210, parameters representing the activation or deactivation of the in-loop deblocking filter and the characteristics of the in-loop deblocking filter when activated are introduced into the encoded video stream 211.

[0072] The new coding tool appearing in previous generation video compression methods has added a new processing block before the in-loop post-filtering. This coding tool, called Luminosity Mapping with Chroma Scaling (LMCS), has two main components: an in-loop mapping of the luminance component based on an adaptive piecewise linear model; and for the chrominance component, luminance-dependent chrominance residual scaling is applied. The in-loop mapping of the luminance component improves compression efficiency by reallocating codewords within the dynamic range of the input signal. Chroma residual scaling is designed to compensate for the interaction between the luminance signal and its corresponding chrominance signal. The parameters of the LMCS need to be transmitted to the decoder. In some implementations, the parameters of the LMCS are signaled in the Adaptive Parameter Set (APS).

[0073] Another new encoding tool that appeared in the previous generation of video compression methods is called Progressive Decode Refresh (GDR). GDR provides virtual boundaries for reconstructing the image, a portion of which cannot be used as a prediction reference. When GDR is activated for an image, its NAL cell type is signaled as GDR_NUT.

[0074] When reconstructing a block, the block is inserted into the reconstructed image stored in the decoded image buffer (DPB) 219 during step 218. The reconstructed image thus stored can then be used as a reference image for other images to be encoded.

[0075] Figure 3 The schematic depiction is used for the purpose of... Figure 2The described method is for decoding an encoded video stream (i.e., a bitstream) 211. The decoding method is performed by a decoding module. Variations of this decoding method are contemplated, but for clarity, the following description is preferred. Figure 3 The method used for decoding is described, but not all expected variations are described.

[0076] Decoding is performed block by block. For the current block, it begins with entropy decoding during step 310. Entropy decoding allows the acquisition of the prediction pattern for the current block.

[0077] If the current block has already been encoded according to the intra-prediction mode, entropy decoding allows for the acquisition of information representing the intra-prediction direction and the residual block.

[0078] If the current block has already been encoded according to the inter-frame prediction mode, entropy decoding allows for the acquisition of information representing motion data and residual blocks. Where appropriate, during step 308, motion data is reconstructed for the current block according to either the AMVP mode or the merge mode. In the merge mode, the motion data obtained through entropy decoding includes indices from a candidate list of motion vector prediction values. The decoding module applies the same process as the encoding module to reconstruct the candidate lists for both the regular merge mode and the sub-block merge mode. Using the reconstructed lists and indices, the decoding module is able to retrieve motion vectors for predicting the block's motion vectors.

[0079] The decoding method includes steps 312, 313, 315, 316, and 317, which are identical in all respects to steps 212, 213, 215, 216, and 217 of the encoding method. At the encoding module level, step 214 includes a mode selection process that evaluates each mode according to a rate distortion criterion and selects the optimal mode, while step 314 only includes reading information representing the selected mode from bitstream 211. In step 318, the decoded block is saved in the decoded image and the decoded image is stored in DPB 319. When the decoding module decodes a given image, the image stored in DPB 319 is the same as the image stored in DPB 219 by the encoding module during the encoding of the given image. The decoded image can also be output by the decoding module for, for example, display.

[0080] In specific cases where the current block is encoded using ALF, LMCS, or a non-default scaling matrix, the decoder needs to obtain the ALF, LMCS, and non-default scaling matrix parameters. As mentioned above, the ALF, LMCS, and non-default scaling matrix parameters are provided by the APS. However, in some cases, when the use of ALF and / or LMCS and / or a non-default scaling matrix is ​​activated for the current block, the bitstream received by the decoder may not include the APS. This may occur if the APS has been lost during transmission. This may also occur if the bitstream is locally encoded without an APS. Specifically, some video compression methods allow specifying that the encoded video stream does not include any APS. In some cases, this feature is specified in the encoded video stream, for example at the sequence header level, by a flag called no_aps_constraint_flag. If the flag no_aps_constraint_flag is equal to one, the provision of an APS in the encoded video stream is not authorized. Otherwise, if the flag no_aps_constraint_flag is equal to zero, there is no constraint on the presence of an APS in the encoded video stream. The flag `no_aps_constraint_flag` is encoded, for example, in a syntax element called `general_constraint_info()`, which in turn is encoded in the syntax element `profile_tier_level()`, which is embedded in a signal container (DPS, VPS, or SPS). The implementation described below proposes a solution that allows consistent behavior of the encoder or decoder when an APS is absent in the bitstream and an encoding tool referencing parameters provided by the APS is activated.

[0081] Similar to a tag no_aps_constraint_flag There is another constraint flag to disable the use of NAL cells of type GDR_NUT. This flag is named no_gdr_constraint_flag. It disables the use of GDR NAL cells, rather than disabling GDR itself.

[0082] Figure 4A The illustration schematically shows examples of hardware architectures for processing module 40, modified according to different aspects and implementation schemes, capable of implementing either an encoding module or a decoding module, which can respectively implement... Figure 2 Methods for encoding and Figure 3The method for decoding. As a non-limiting example, the processing module 40 includes the following items connected by a communication bus 405: a processor or CPU (central processing unit) 400 containing one or more microprocessors, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture; random access memory (RAM) 401; read-only memory (ROM) 402; a storage unit 403, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drive and / or optical disk drive, or storage media reader, such as SD (Secure Digital) card reader and / or hard disk drive (HDD) and / or network accessible storage device; at least one communication interface 404 for exchanging data with other modules, devices, or equipment. The communication interface 404 may include, but is not limited to, a transceiver configured to transmit and receive data through a communication channel. Communication interface 404 may include, but is not limited to, a modem or network card.

[0083] If processing module 40 implements a decoding module, then communication interface 404 enables, for example, processing module 40 to receive encoded video streams and provide decoded video streams. If processing module 40 implements an encoding module, then communication interface 404 enables, for example, processing module 40 to receive raw image data to be encoded and provide encoded video streams.

[0084] Processor 400 is capable of executing instructions loaded into RAM 401 from ROM 402, external memory (not shown), storage media, or a communication network. When processing module 40 is powered on, processor 400 is capable of reading instructions from RAM 401 and executing those instructions. These instructions form a computer program that enables, for example, processor 400 to implement... Figure 3 The described decoding method or about Figure 2 The encoding method described herein includes the following aspects and implementation schemes as described in this document.

[0085] All or part of the algorithms and steps of the encoding or decoding method may be implemented in software by executing a set of instructions by a programmable machine such as a DSP (Digital Signal Processor) or a microcontroller, or in hardware by a machine or dedicated component such as an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0086] Figure 4BA block diagram illustrating an example of a system 4 in which various aspects and embodiments are implemented is shown. System 4 may be embodied as a device including the various components described below and configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 4 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, system 4 includes a processing module 40 implementing a decoding module or an encoding module. However, in another embodiment, system 4 may include a first processing module 40 implementing a decoding module and a second processing module 40 implementing an encoding module, or a single processing module 40 implementing both a decoding module and an encoding module. In various embodiments, system 40 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 4 is configured to implement one or more aspects described in this document.

[0087] System 4 includes at least one processing module 40, which is capable of implementing one or both of an encoding module or a decoding module.

[0088] Inputs to processing module 40 may be provided by various input modules as shown in box 42. Such input modules include, but are not limited to: (i) a radio frequency (RF) module that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a component (COMP) input module (or a set of COMP input modules); (iii) a universal serial bus (USB) input module; and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 4B Other examples not shown include composite video.

[0089] In various embodiments, the input module of block 42 has associated corresponding input processing elements as known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF module of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box implementation, the RF module and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various implementations rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF module includes an antenna.

[0090] Additionally, the USB and / or HDMI modules may include corresponding interface processors for connecting System 4 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processing module 40. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processing module 40. Demodulation, error correction, and demultiplexing streams are provided to processing module 40.

[0091] Various components of System 4 can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards) and data can be transferred between these components. For example, in System 4, processing module 40 is interconnected with other components of System 4 via bus 405.

[0092] The communication interface 404 of the processing module 40 allows the system 4 to communicate over the communication channel 41. For example, the communication channel 41 can be implemented in a wired and / or wireless medium.

[0093] In various implementations, data is streamed or otherwise provided to system 4 using a wireless network such as Wi-Fi, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these implementations, the Wi-Fi signal is received via a communication channel 41 and a communication interface 404 suitable for Wi-Fi communication. The communication channel 41 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other implementations use a set-top box that transmits data via an HDMI connection to input block 42 to provide streaming data to system 4. Still other implementations use an RF connection to input block 42 to provide streaming data to system 4. As mentioned above, various implementations provide data in a non-streaming manner. Additionally, various implementations use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0094] System 4 can provide output signals to various output devices, including a display 46, a speaker 47, and other peripheral devices 48. The display 46 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 46 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. The display 46 can also be integrated with other components (e.g., as in a smartphone) or standalone (e.g., an external monitor for a laptop). In various examples of embodiments, other peripheral devices 46 include one or more of a standalone digital video disc (or digital universal disc, both terms being DVR), an optical disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 48 that provide functionality based on the output of System 4. For example, an optical disc player performs the function of playing the output of System 4.

[0095] In various embodiments, control signals are transmitted between system 4 and display 46, speaker 47, or other peripheral devices 48 using signaling protocols such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices are communicatively coupled to system 4 via dedicated connections through corresponding interfaces 43, 44, and 45. Alternatively, output devices can be connected to system 4 via communication interface 404 using communication channel 41. Display 46 and speaker 47 may be integrated into a single unit with other components of system 4 in electronic devices such as televisions. In various embodiments, display interface 43 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0096] For example, if the RF portion of input 42 is part of a separate set-top box, then display 46 and speaker 47 may optionally be separate from one or more other components. In various embodiments where display 46 and speaker 47 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0097] Various specific implementations participate in decoding. As used in this application, "decoding" may encompass all or part of a process performed, for example, on a received encoded video stream, to produce a final output suitable for display. In various implementations, such processes include one or more processes typically performed by the decoder, such as entropy decoding, inverse quantization, inverse transform, and prediction. In various implementations, such processes also include, or alternatively include, processes performed by the decoder of the various specific implementations or embodiments described in this application, such as for determining the presence of an APS in the bitstream, or for adaptive decoding of the current block when the use of an ALF and / or LMCS and / or a non-default scaling matrix is ​​activated for the current block while no APS is available to the decoder.

[0098] As a further example, in one implementation, "decoding" refers only to in-loop post-filtering ( Figure 3 Step 317) or inverse quantization Figure 3 (Step 312 in the text). Whether the phrase “decoding process” specifically refers to a subset of the operations or broadly refers to a wider decoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0099] Various specific implementations involve encoding. In a manner similar to the discussion above regarding “decoding,” the term “encoding,” as used herein, can encompass, for example, all or part of the process performed on an input video sequence to produce an encoded video stream. In various implementations, such processes include one or more processes typically performed by an encoder, such as partitioning, prediction, transform, quantization, in-loop post-filtering, and entropy coding. In various implementations, such processes also include, or alternatively include, processes performed by an encoder of the various specific implementations or embodiments described herein, such as adaptive encoding of blocks based on the presence or absence of an APS in the bitstream.

[0100] As a further example, in one implementation, "encoding" refers to quantization and dequantization ( Figure 2 Steps 209 and 212 in the loop and post-filtering ( Figure 2 (Step 217 in the text). Whether the phrase “encoding process” specifically refers to a subset of operations or broadly refers to a wider encoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0101] It should be noted that the names of syntax elements, tags, containers, and encoding tools used in this article are descriptive terms. Therefore, they do not preclude the use of other syntax element, tag, container, or encoding tool names.

[0102] When the accompanying drawings are presented as flowcharts, it should be understood that block diagrams of the corresponding devices are also provided. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that flowcharts of the corresponding methods / processes are also provided.

[0103] Various implementation schemes refer to rate distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered. Rate distortion optimization is generally formulated as minimizing a rate distortion function, which is a weighted sum of rate and distortion. Different approaches exist to solve the rate distortion optimization problem. For example, these methods may be based on extensive testing of all encoding options (including all considered modes or encoding parameter values) and a complete evaluation of their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to reduce encoding complexity, particularly for calculating approximate distortion based on prediction or prediction of the residual signal rather than the reconstructed residual signal. A hybrid of these two approaches can also be used, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other methods evaluate only a subset of the possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both encoding costs and associated distortion.

[0104] The specific embodiments and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific embodiment (e.g., discussed only as a method), specific embodiments of the discussed features may be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. Methods may be implemented, for example, in a processor, which generally refers to a processing device.

[0105] The processing device includes, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication devices, such as computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0106] The reference to "an implementation scheme" or "implementation scheme" or "a specific implementation" or "specific implementation," and other variations thereof, means that the specific features, structures, characteristics, etc., described in connection with the implementation scheme are included in at least one implementation scheme. Therefore, the appearance of the phrase "in an implementation scheme" or "in an implementation scheme" or "in a specific implementation" or "in a specific implementation," and any other variations appearing throughout this application, do not necessarily refer to the same implementation scheme.

[0107] In addition, this application may involve "determining" various types of information. Determining information may include, for example, estimated information, calculated information, predicted information, information inferred from other information, information retrieved from memory, or information obtained, for example, from another device, module, or user, one or more of these.

[0108] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, inferring information, or estimating information, or one or more of these.

[0109] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Moreover, "receiving" typically involves one or more of the following during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, inferring information, or estimating information.

[0110] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” “at least one of A and B,” and “one or more of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one,” “one or more” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C,” “at least one of A, B, and C,” and “one or more of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many of the listed items as possible.

[0111] Moreover, as used herein, the term "signaling" refers to (among other things) instructing the corresponding decoder to do something. For example, in some implementations, the encoder signals syntax elements or parameters associated with the ALF, LMCS, and scaling matrix. Thus, in one implementation, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters and others, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various implementations by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various implementations, information is signaled to the corresponding decoder using one or more syntax elements, tags, etc. Although the verb form of the term "signal" has been used above, the term "signal" can also be used as a noun herein.

[0112] It will be apparent to those skilled in the art that the embodiments can generate various signals formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the embodiments. For example, a signal may be formatted to carry an encoded video stream of the embodiment. Such signals may be formatted as electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the encoded video stream and modulating a carrier wave using the encoded video stream. The information carried by the signal may be, for example, analog or digital information. It is known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0113] Figure 5 A solution for adapting the decoding process is illustrated in the diagram when the APS is unavailable on the decoder side.

[0114] When the processing module 40 implements the decoding module, the processing module 40 executes... Figure 5 The process.

[0115] In step 501, processing module 40 obtains the encoded video stream 211. During step 501, processing module 40... Figure 3 The decoding method shown is applied to encoded video stream 211 to begin the decoding process. At the end of step 501, the processing module prepares to decode the current block.

[0116] In step 502, processing module 40 obtains information indicating whether an APS is absent in the encoded video stream. In a first embodiment of step 502, processing module 40 determines whether to authorize providing an APS in the encoded video stream 211 based on the flag no_aps_constraint_flag obtained from the encoded video stream 211. If providing an APS in the encoded video stream is authorized, processing module 40 performs a regular decoding process on the current block.

[0117] If no APS is available in the encoded video stream 211, then step 503 follows step 502. In the first embodiment of step 502, according to the flag no_aps_constraint_flag (no_aps_constraint_flag = 1), when it is not authorized to provide APS in the encoded video stream 211, no APS is available in the encoded video stream 211.

[0118] In step 503, processing module 40 checks the value of a syntax element that indicates whether an encoding tool with at least one encoding parameter provided by the APS is active for the current block. In a first embodiment of step 503, the syntax element indicating whether an encoding tool with at least one encoding parameter provided by the APS is active for the current block is one of the following: the flag `sps_alf_enabled_flag`, the flag `sps_lmcs_enabled_flag`, and the flag `sps_scaling_list_enabled_flag`. The flag `sps_alf_enabled_flag` specifies at the sequence level (Sequence Parameter Set (SPS)) that adaptive loop filtering is disabled when it is equal to zero. `sps_alf_enabled_flag` equal to one specifies that adaptive loop filtering is enabled. When equal to one, the flag `sps_lmcs_enabled_flag` specifies at the SPS level that LMCS is used in the encoded video stream. `sps_lmcs_enabled_flag` equal to zero specifies that LMCS is not used in the encoded video stream. When sps_scaling_list_enabled_flag equals one, it specifies a non-default scaling matrix for the scaling process of the transform coefficients at the SPS level. When sps_scaling_list_enabled_flag equals zero, it specifies that no non-default scaling matrix is ​​used for the scaling process of the transform coefficients.

[0119] In the case of GDR, a syntax element called gdr_enabled_flag is signaled at the SPS level and allows GDR to be disabled when it is equal to zero.

[0120] When no syntax element indicates that an encoding tool using at least one encoding parameter provided by APS is activated, step 503 is followed by step 504. Otherwise, if at least one syntax element indicates that an encoding tool using at least one parameter provided by APS is activated, the processing module 40 adaptively decodes the current block in step 505. In the first embodiment of step 503, step 505 is executed when at least one of the flags sps_alf_enabled_flag, sps_lmcs_enabled_flag, and sps_scaling_list_enabled_flag is equal to one.

[0121] In the first embodiment of step 505, even if the flags sps_alf_enabled_flag, sps_lmcs_enabled_flag, and sps_scaling_list_enabled_flag are equal to one, the processing module 40 considers them to be equal to zero. In this case, the adaptation of the decoding process includes ignoring the values ​​of each syntax element that indicate the encoding tool is activated using at least one parameter provided by APS.

[0122] In the second embodiment of step 505, when no APS is available and an encoding tool using at least one parameter provided by the APS is activated, the processing module 40 outputs non-compliance information. The non-compliance information is, for example, information indicating that the encoded video stream 211 is undecodeable. The non-compliance information is output to, for example, a display 46 so that it can be displayed to the user.

[0123] In the third embodiment of step 505, when no APS is available and an encoding tool using at least one parameter provided by the APS is activated, the processing module 40 obtains each parameter, typically provided by the APS, from at least one container of at least one second type, instead of searching for the parameter in the APS. In this case, the processing module 40 applies an encoding tool (ALF, LMCS, use of a non-default matrix) using the obtained parameter to decode the current block.

[0124] For example, in a first variant of the third embodiment of step 505, the ALF, LMCS, and scaling matrix parameters are signaled at the SPS level.

[0125] In this first variant of the third embodiment of step 505, the syntax of the SPS hierarchy is described in Table TAB1:

[0126]

[0127] Table TAB1

[0128] The bolded portion in Table TAB1 corresponds to the syntax element defined for the first variant of the third implementation of step 505.

[0129] Examples of semantic annotations for sps_alf_parameters_in_sps_flag, sps_lmcs_parameters_in_sps_flag, and sps_scaling_list_parameters_in_sps_flag are shown below:

[0130] A value of zero for `sps_alf_parameters_in_sps_flag` indicates that ALF parameters are not signaled at the SPS level. A value of one for `sps_alf_parameters_in_sps_flag` indicates that ALF parameters are signaled at the SPS level. If it does not exist, the value of `sps_alf_parameters_in_sps_flag` is assumed to be zero.

[0131] A value of zero for `sps_lmcs_parameters_in_sps_flag` indicates that LMCS parameters are not signaled at the SPS level. A value of one for `sps_lmcs_parameters_in_sps_flag` indicates that LMCS parameters are signaled at the SPS level. If it does not exist, the value of `sps_lmcs_parameters_in_sps_flag` is assumed to be zero.

[0132] A value of zero for `sps_scaling_list_parameters_in_sps_flag` indicates that scaling matrix parameters are not signaled at the SPS level. A value of `sps_scaling_list_parameters_in_sps_flag` indicates that scaling matrices are signaled at the SPS level. If it does not exist, the value of `sps_scaling_list_parameters_in_sps_flag` is assumed to be zero.

[0133] Tables TAB2 and TAB3 describe examples of the syntax for the picture header and slice header hierarchy of the first variant of the third implementation of step 505.

[0134]

[0135] Table TAB2

[0136]

[0137] Table TAB3

[0138] The bolded parts indicate modifications to the existing syntax caused by the proposed syntax at the SPS level, as shown in Table TAB1.

[0139] In a first variant of the third embodiment of step 505, if ALF is used when the flag no_aps_constraint_flag is equal to one, then the flag sps_alf_parameters_in_sps_flag is equal to one. Furthermore, in this case, the flag alf_info_in_ph_flag is equal to zero. Alf_info_in_ph_flag being equal to one indicates that the ALF information exists in the image header syntax structure and not in the slice header of a PPS that does not have an image header syntax structure. Alf_info_in_ph_flag being equal to zero indicates that the ALF information does not exist in the image header syntax structure and may exist in the slice header of a PPS that does not have an image header syntax structure.

[0140] In a first variant of the third implementation of step 505, if LMCS is used when the flag no_aps_constraint_flag is equal to one, then the flag sps_lmcs_parameters_in_sps_flag is equal to one.

[0141] In a first variant of the third implementation of step 505, if a non-default scaling matrix is ​​used when the flag no_aps_constraint_flag is equal to one, then the flag sps_scaling_list_parameters_in_sps_flag is equal to one.

[0142] In an alternative form of the first variant of the third embodiment of step 505, instead of adding three SPS level flags (i.e., sps_alf_parameters_in_sps_flag, sps_lmcs_parameters_in_sps_flag, and sps_scaling_list_parameters_in_sps_flag), a single SPS level flag, sps_aps_parameters_signaling, is used to signal the ALF, LMCS, and scaling list parameters at the SPS level. The flag sps_aps_parameters_signaling is encoded if the ALF, LMCS, or scaling list is activated by their SPS flags. In this alternative form of the first variant of the third embodiment of step 505, the syntax of the SPS level is described in table TAB1_Bis:

[0143]

[0144] Table TAB1_Bis

[0145] The bolded portion in Table TAB1_Bis corresponds to the syntax element defined for the alternative form of the first variant of the third implementation of step 505.

[0146] Tables TAB2_Bis and TAB3_Bis describe examples of the syntax for the picture header and slice header hierarchy of the first variant of the third implementation of step 505.

[0147]

[0148] Table TAB2_Bis

[0149]

[0150] Table TAB3_Bis

[0151] The bolded parts indicate modifications to the existing syntax caused by the proposed syntax at the SPS level, as shown in Table TAB1_Bis.

[0152] In a second variation of the third embodiment in step 505, the ALF, LMCS, and scaling matrix parameters are signaled at the PPS level. One advantage is that the PPS can be signaled more frequently than the SPS. That is, for a single sequence, one or more PPSs can be signaled.

[0153] In this second variant of the third embodiment of step 505, the syntax of the PPS hierarchy is described in Table TAB4:

[0154]

[0155] Table TAB4

[0156] The new syntax of the second variant of the third implementation applicable to step 505 is shown in bold in Table TAB4.

[0157] Examples of semantic markers for pps_alf_parameters_in_pps_flag, pps_lmcs_parameters_in_pps_flag, and pps_scaling_list_parameters_in_pps_flag are shown below:

[0158] A value of zero for pps_alf_parameters_in_pps_flag indicates that ALF parameters are not signaled at the PPS level. A value of one for pps_alf_parameters_in_pps_flag indicates that ALF parameters are signaled at the PPS level.

[0159] A value of zero for pps_lmcs_parameters_in_pps_flag indicates that LMCS parameters are not signaled at the PPS level. A value of one for pps_lmcs_parameters_in_pps_flag indicates that LMCS parameters are signaled at the PPS level.

[0160] A value of zero in `pps_scaling_list_parameters_in_pps_flag` indicates that scaling matrix parameters are not signaled at the PPS level. A value of one in `pps_scaling_list_parameters_in_pps_flag` indicates that scaling matrix parameters are signaled at the PPS level.

[0161] Tables TAB5 and TAB6 describe examples of the syntax for the picture header and slice header hierarchy of the second variant of the third implementation of step 505.

[0162]

[0163] Table TAB5

[0164]

[0165] Table TAB6

[0166] The bolded parts indicate modifications to the existing syntax caused by the proposed syntax at the PPS level, as shown in Table TAB4.

[0167] In a second variant of the third implementation in step 505, if ALF is used when the flag no_aps_constraint_flag is equal to one, then the flag pps_alf_parameters_in_pps_flag is equal to one.

[0168] In a second variant of the third implementation in step 505, if LMCS is used when the flag no_aps_constraint_flag is equal to one, then the flag pps_lmcs_parameters_in_pps_flag is equal to one.

[0169] In a second variant of the third implementation in step 505, if a non-default scaling matrix is ​​used when the flag no_aps_constraint_flag is equal to one, then the flag pps_scaling_list_parameters_in_pps_flag is equal to one.

[0170] In an alternative form of the second variant of the third embodiment of step 505, instead of defining three PPS level flags (pps_alf_parameters_in_pps_flag, pps_lmcs_parameters_in_pps_flag, and pps_scaling_list_parameters_in_pps_flag), a single flag, pps_aps_parameters_signaling, is used. In this alternative form of the second variant of the third embodiment of step 505, the syntax of the PPS levels is described in table TAB4_Bis:

[0171]

[0172] Table TAB4_Bis

[0173] Tables TAB5_Bis and TAB6_Bis describe examples of the syntax for the alternative forms of the picture header and slice header hierarchy of the second variant applicable to the third implementation of step 505.

[0174]

[0175] Table TAB5_Bis

[0176]

[0177] Table TAB6_Bis

[0178] In a third variation of the third embodiment in step 505, the ALF, LMCS, and scaling matrix parameters are signaled at the image header level. One advantage is that the image header is signaled more frequently than the SPS and PPS.

[0179] In this third variant of the third embodiment of step 505, the syntax for the image header (PH) level and slice level is described in tables TAB7 and TAB_7_Bis:

[0180]

[0181] Table TAB7

[0182]

[0183] Table TAB7_Bis

[0184] The new syntax for the third variant of the third implementation of step 505 is shown in bold in Table TAB7.

[0185] Examples of semantic markers for ph_alf_parameters_in_ph_flag, ph_lmcs_parameters_in_ph_flag, and ph_scaling_list_parameters_in_ph_flag are as follows:

[0186] A value of zero for `ph_alf_parameters_in_ph_flag` indicates that ALF parameters are not signaled at the PH level. A value of one for `ph_alf_parameters_in_ph_flag` indicates that ALF parameters are signaled at the PH level. If it does not exist, the value of `ph_alf_parameters_in_ph_flag` is assumed to be zero.

[0187] A value of 0 for ph_lmcs_parameters_in_ph_flag indicates that LMCS parameters are not signaled at the pH level. A value of 1 for ph_lmcs_parameters_in_ph_flag indicates that LMCS parameters are signaled at the pH level. If ph_lmcs_parameters_in_ph_flag does not exist, its value is assumed to be zero.

[0188] A value of zero for `ph_scaling_list_parameters_in_ph_flag` indicates that scaling matrix parameters are not signaled at the PH level. `ph_scaling_list_parameters_in_ph_flag` is equal to a list specifying the scaling parameters to be signaled at the PH level. If it does not exist, the value of `ph_scaling_list_parameters_in_ph_flag` is assumed to be zero.

[0189] In a third variant of the third implementation of step 505, if ALF is used when the flag no_aps_constraint_flag is equal to one, then the flag ph_alf_parameters_in_ph_flag is equal to one.

[0190] In a third variant of the third implementation of step 505, if LMCS is used when the flag no_aps_constraint_flag is equal to one, then the flag ph_lmcs_parameters_in_ph_flag is equal to one.

[0191] In a third variant of the third implementation of step 505, if a non-default scaling matrix is ​​used when the flag no_aps_constraint_flag is equal to one, then the flag ph_scaling_list_parameters_in_ph_flag is equal to one.

[0192] In the fourth variant of the third embodiment in step 505, the ALF, LMCS, and scaling matrix parameters are signaled at the slice header level.

[0193] In the fourth variant of the third embodiment in step 505, the syntax of the slice header (SH) level is described in Table TAB8:

[0194]

[0195]

[0196] Table TAB8

[0197] The new syntax for the fourth variant of the third implementation of step 505 is shown in bold in Table TAB7.

[0198] Examples of semantic markers for sh_alf_parameters_in_sh_flag, sh_lmcs_parameters_in_sh_flag, and sh_scaling_list_parameters_in_sh_flag are as follows:

[0199] A value of zero for `sh_alf_parameters_in_sh_flag` indicates that ALF parameters are not signaled at the SH level. A value of one for `sh_alf_parameters_in_sh_flag` indicates that ALF parameters are signaled at the SH level. If it does not exist, the value of `sh_alf_parameters_in_sh_flag` is assumed to be zero.

[0200] A value of zero for `sh_lmcs_parameters_in_sh_flag` indicates that LMCS parameters are not signaled at the SH level. A value of one for `sh_lmcs_parameters_in_sh_flag` indicates that LMCS parameters are signaled at the SH level. If it does not exist, the value of `sh_lmcs_parameters_in_sh_flag` is assumed to be zero.

[0201] A value of zero for `sh_scaling_list_parameters_in_sh_flag` indicates that scaling matrix parameters should not be signaled at the SH level. A value equal to `sh_scaling_list_parameters_in_sh_flag` indicates that scaling matrices should be signaled at the SH level. If it does not exist, the value of `sh_scaling_list_parameters_in_sh_flag` is assumed to be zero.

[0202] In a fourth variant of the third implementation in step 505, if ALF is used when the flag no_aps_constraint_flag is equal to one, then the flag sh_alf_parameters_in_sh_flag is equal to one.

[0203] In a fourth variant of the third implementation in step 505, if LMCS is used when the flag no_aps_constraint_flag is equal to one, then the flag sh_lmcs_parameters_in_sh_flag is equal to one.

[0204] In a third variant of the third implementation of step 505, if a non-default scaling matrix is ​​used when the flag no_aps_constraint_flag is equal to one, then the flag sh_scaling_list_parameters_in_sh_flag is equal to one.

[0205] In a fifth variant of the third embodiment of step 505, the ALF, LMCS, and scaling matrix parameters are signaled at any level via tags indicating their use. In this case, for example, there is one tag indicating the encoding of the SPS level in the SPS, another tag indicating the encoding of the PPS level in the PPS, another tag indicating the encoding of the PH level in the PH, and another tag indicating the encoding of the SH level in the SH. Parameters encoded at higher levels should not be encoded at lower levels. The corresponding syntax is described in Tables TAB9, TAB10, TAB11, and TAB12 (the semantics remain the same as in the previous variant of the third embodiment of step 505):

[0206]

[0207] Table TAB9

[0208]

[0209] Table TAB10

[0210]

[0211]

[0212] Table TAB11

[0213]

[0214]

[0215] Table TAB12

[0216] The new syntax for the fifth variant of the third implementation of step 505 is shown in bold in tables TAB9, TAB10, TAB11 and TAB12.

[0217] In the fifth variation of the third embodiment of step 505:

[0218] • If sps_alf_parameters_in_sps_flag equals one, then pps_alf_parameters_in_pps_flag equals zero;

[0219] • If sps_lmcs_parameters_in_sps_flag equals one, then pps_lmcs_parameters_in_pps_flag equals zero;

[0220] • If sps_scaling_list_parameters_in_sps_flag equals one, then pps_scaling_list_parameters_in_pps_flag equals zero;

[0221] • If no_aps_constraint_flag equals one, the ALF, LMCS, and scaling matrix parameters are encoded at the SPS and / or PPS and / or PH and / or SH levels (if activated).

[0222] In some cases, the parameters (ALF, LMCS, and non-default scaling matrix) of tools that typically signal at the APS level can be extended in other containers. For example, the parameters of the first encoding tool can be encoded at the SPS level, the parameters of the second encoding tool at the PPS level, and the parameters of the third encoding tool at the PH level.

[0223] In the second embodiment of step 502, the information indicating whether the APS is absent in the encoded video stream obtained by the processing module 40 is information indicating whether the APS was lost during the transmission of the encoded video stream 211.

[0224] In the third embodiment of step 502, the information indicating whether an APS is absent in the encoded video stream obtained by the processing module 40 is information indicating that the processing module 40 is not designed to consider APS. In this case, the APS ultimately present in the encoded video stream received by the processing module 40 is ignored by the processing module 40.

[0225] In the second embodiment of step 503, the syntax element indicating whether an encoding tool using at least one encoding parameter provided by the APS is activated for the current block checked by processing module 40 is a slice-level syntax element, such as the syntax element slice_alf_enabled_flag. slice_alf_enabled_flag equals one, indicating that ALF is enabled and can be applied to the Y, Cb, or Cr color components in the slice. slice_alf_enabled_flag equals zero, indicating that ALF is disabled for all color components in the slice. When it does not exist, the value of slice_alf_enabled_flag is inferred to be equal to ph_alf_enabled_flag.

[0226] In the third embodiment of step 503, the syntax element indicating whether to activate the encoding tool using at least one encoding parameter provided by the APS for the current block checked by the processing module 40 is a CTU-level syntax element, for example, the syntax element alf_ctb_flag[cIdx][xCtb][yCtb]. alf_ctb_flag[cIdx][xCtb][yCtb] equals one, specifying that the ALF is applied to the encoding unit of the color component indicated by cIdx of the CTU at the luma position (xCtb, yCtb). alf_ctb_flag[cIdx][xCtb][yCtb] equal to zero specifies that the ALF is not applied to the CU of the color component indicated by cIdx of the CTU at the luma position (xCtb, yCtb). When alf_ctb_flag[cIdx][xCtb][yCtb] does not exist, it is inferred to be equal to zero.

[0227] In a similar case to GDR, when no_gdr_constraint_flag equals one, the gdr_enabled_flag is equal to zero. In the first implementation involving GDR, the semantics of the constraint flags are as follows:

[0228] `no_gdr_constraint_flag` is equal to a value that specifies that NAL units of type `GDR_NUT` should not exist in the encoded video stream output by the encoder, and that `gdr_enabled_flag` should be zero. Equaling `no_gdr_constraint_flag` to zero does not impose this constraint.

[0229] In the second implementation involving GDR, the constraint flag no_gdr_constraint_flag is only the value of the constraint SPS level flag gdr_enabled_flag as follows:

[0230] `no_gdr_constraint_flag` is equal to specify that `gdr_enabled_flag` should be zero. Setting `no_gdr_constraint_flag` to zero does not impose this constraint.

[0231] In the third implementation involving GDR, a conformance constraint is added to ensure that the SPS level flag gdr_enabled_flag is set to zero when the constraint flag no_gdr_constraint_flag is one:

[0232] The requirement for the bitstream compliance item is that when no_gdr_constraint_flag equals one, the value of gdr_enabled_flag should be zero.

[0233] For all the above implementations, the encoding module and method are compatible with the decoding module and method. Specifically, in implementations that cause syntax modifications (e.g., the third implementation in step 505), the encoding module and method conform to the stated syntax.

[0234] Figure 6 This schematically depicts a solution for adapting the encoding process when APS cannot be used.

[0235] When the processing module 40 implements the encoding module, the processing module 40 executes... Figure 6 The process.

[0236] In step 601, the processing module 40 obtains the original video sequence and encodes it in the form of an encoded video stream 211.

[0237] In step 602, processing module 40 obtains information indicating whether it authorizes the use of APS to encode the original video sequence. This information may be provided by the user in the form of configuration parameters for processing module 40.

[0238] If APS is authorized, during step 602, processing module 40 signals this information in the encoded video stream 211 using the flag no_aps_constraint_flag. In this case, the flag no_aps_constraint_flag is set to zero. If APS is not authorized, the flag no_aps_constraint_flag is set to one.

[0239] If APS is authorized, the regular encoding process is applied by the processing module 40 in step 603.

[0240] Otherwise, the encoding process of the image blocks of the original video sequence is adapted to the impossibility of using APS during step 604.

[0241] In a first embodiment of step 604, when the APS is not authorized, each encoding tool for which the APS provides at least one parameter is removed from the list of encoding tools considered for encoding blocks of the video sequence. Therefore, when the flag no_aps_constraint_flag equals one, the ALF, LMCS, and non-default scaling matrix are not considered tools that can be used to encode blocks of the original video sequence. In this first embodiment of step 604, when no_aps_constraint_flag = 1, the SPS level flags sps_alf_enabled_flag, sps_lmcs_enabled_flag, and sps_scaling_list_enabled_flag are constrained to zero. In a variation of the first embodiment of step 604, when no_aps_constraint_flag = 1, the flag no_alf_constraint_flag is constrained to one. The flag no_alf_constraint_flag is encoded, for example, in a syntax element called general_constraint_info(). When equal to one, the flag `no_alf_constraint_flag` specifies that `sps_alf_enabled_flag` should be zero. `no_alf_constraint_flag` being zero does not impose this constraint. Similarly, the flags `no_lmcs_enabled_flag` and `no_scaling_list_enabled_flag` can be used to constrain the values ​​of `sps_lmcs_enabled_flag` and `sps_scaling_list_enabled_flag`.

[0242] In the second embodiment of step 604, adapting the encoding process includes: if an encoding tool utilizing these parameters is authorized to encode the current block, then the parameters typically encoded in the APS are encoded in the SPS and / or PPS and / or PH and / or SH. Therefore, the ALF parameters, LMCS parameters, and / or non-default scaling matrix parameters are encoded in the SPS and / or PPS and / or PH and / or SH, with the ALF, LMCS, and non-default scaling matrix authorized for the block of the original video sequence. This embodiment is compatible with a variant of the third embodiment of step 505 of the decoding process. Specifically, the second embodiment of step 604 uses the syntax described in conjunction with the third embodiment of step 505.

[0243] exist Figure 6 In an alternative form of the solution, a single SPS-level syntax element, `sps_no_aps_signaling`, is used to specify whether to signal the APS, LMCS, and scaling list parameters at levels other than the APS level. When this flag is set to one, signaling of these parameters is allowed at the slice level or picture level. This implementation allows for reduced signaling overhead, whereas in the previous implementation, at least three flags were used. The signaling flag `sps_no_aps_signaling` is signaled if at least the ALF, LMCS, or scaling list is activated by its SPS-level flag; otherwise, it is inferred to be zero. If `sps_no_aps_signaling` is equal to one, alternative encoding is allowed. Furthermore, this flag must also be one when the constraint flag used for APS (i.e., `no_aps_constraint_flag`) is set to one, to allow signaling of the APS, LMCS, and scaling list parameters, but not the APS.

[0244] The corresponding syntax is described in Tables TAB13, TAB14, and TAB15.

[0245]

[0246] Table TAB13

[0247]

[0248]

[0249] Table TAB14

[0250]

[0251] Table TAB15

[0252] Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, across various claim classes and types:

[0253] • Includes a bitstream or signal that transmits a syntax for information generated according to any of the embodiments described;

[0254] • Inserting syntax elements into the signaling allows the decoder to adapt the decoding process in a manner corresponding to that used by the encoder;

[0255] • Creating and / or transmitting and / or receiving and / or decoding bitstreams or signals comprising one or more of the syntax elements or variations thereof;

[0256] • Creation and / or transmission and / or reception and / or decoding according to any one of the embodiments described;

[0257] • The method, process, apparatus, medium for storing instructions, medium for storing data, or signal according to any one of the embodiments described above;

[0258] • Perform the encoding or decoding process according to any of the described implementation schemes.

[0259] An adaptive television, set-top box, mobile phone, tablet computer, or other electronic device; a television, set-top box, mobile phone, tablet computer, or other electronic device that performs an encoding or decoding process according to any of the described embodiments and displays the resulting image (e.g., using a monitor, screen, or other type of display).

[0260] • An adaptive television, set-top box, cellular phone, tablet computer, or other electronic device that selects (e.g., using a tuner) a channel to receive a signal including an encoded image and performs a decoding process according to any of the described embodiments;

[0261] • An adaptive television, set-top box, cellular phone, tablet or other electronic device that receives signals including encoded images over the air (e.g., using an antenna) and performs a decoding process according to any of the embodiments described.

Claims

1. A method for decoding, comprising: Get the video data representing the video sequence, represented by (501); (502) First information is obtained from the video data, the first information being indicated by a first value as allowing the existence of a first type container that provides at least one encoding parameter, and by a second value as disallowing the existence of the first type container; Check the value of the first syntax element (503), which indicates whether the encoding tool using the at least one encoding parameter is activated or deactivated for the current block of the image sample of the video sequence. as well as In response to the first information indicating that the first type of container is not allowed and the value of the first syntax element indicating that the encoding tool is activated, the decoding of the current block is adjusted (505), wherein the encoding tool is a luminance mapping and chroma scaling (LMCS) mode using the at least one encoding parameter, or the use of a non-default scaling matrix, or an adaptive loop filter.

2. The method of claim 1, wherein the first information is obtained from a second syntax element acquired from the video data.

3. The method of claim 1, wherein the first type container is an adaptive parameter set, and the second syntax element indicates whether it is authorized that at least one adaptive parameter set exists in the video data.

4. The method of claim 1, wherein the adjustment of decoding the current block comprises: Output second information indicating the non-compliance of the video data.

5. The method of claim 1, wherein the adjustment of decoding the current block comprises: The encoding tool is disabled for decoding the current block.

6. The method of claim 1, wherein the adjustment to the decoding comprises: The at least one parameter is obtained from at least one container of at least one second type, and the current block is decoded by applying the encoding tool that uses the obtained parameter.

7. The method of claim 6, wherein at least one container of the at least one second type is a sequence parameter set and / or an image parameter set and / or an image header and / or a sequence header.

8. A method for encoding, comprising: Acquire (601) video sequences to encode in video data; The video data is signaled to notify first information, which uses a first value to indicate that the existence of a first type of container is allowed, and a second value to indicate that the existence of the first type of container is not allowed. as well as Based on the value of the first information, the value of the second syntax element is set, the second syntax element indicating whether to activate or deactivate the use of an encoding tool that uses at least one parameter provided by the first type container, wherein the encoding tool is a Luminance Mapping and Chroma Scaling (LMCS) mode using the at least one encoding parameter, or the use of a non-default scaling matrix, or an adaptive loop filter.

9. The method of claim 8, wherein the first information is a first syntax element encoded in the video data.

10. The method of claim 8, wherein the first type container is an adaptive parameter set, and the second syntax element indicates whether it is authorized that at least one adaptive parameter set exists in the video data.

11. The method of claim 8, wherein the adjustment of the encoding comprises: If the existence of the first type of container is not authorized, the encoding tool is removed from the list of encoding tools considered for encoding the current block.

12. The method of claim 8, wherein adjusting the encoding comprises: If the encoding tool is authorized to be used to encode the current block, then the at least one encoding parameter is encoded in at least one container of at least one second type.

13. The method of claim 12, wherein at least one container of the at least one second type is a sequence parameter set and / or an image parameter set and / or an image header and / or a sequence header.

14. A device for decoding, comprising electronic circuitry adapted to perform: Get the video data representing the video sequence, represented by (501); (502) First information is obtained from the video data, the first information being indicated by a first value as allowing the existence of a first type container that provides at least one encoding parameter, and by a second value as disallowing the existence of the first type container; Check (503) the value of the first syntax element, which indicates whether the encoding tool using the at least one encoding parameter is activated or deactivated for the current block of the image samples of the video sequence; and In response to the first information indicating that the first type of container is not allowed and the value of the first syntax element indicating that the encoding tool is activated, the decoding of the current block is adjusted (505), wherein the encoding tool is a luminance mapping and chroma scaling (LMCS) mode using the at least one encoding parameter, or the use of a non-default scaling matrix, or an adaptive loop filter.

15. The device of claim 14, wherein the first information is obtained from a second syntax element acquired from the video data.

16. The device of claim 14, wherein the first type container is an adaptive parameter set, and the second syntax element indicates whether at least one adaptive parameter set is authorized in the video data.

17. The device of claim 14, wherein the adjustment of decoding the current block comprises: Output second information indicating the non-compliance of the video data.

18. A device for encoding, comprising electronic circuitry adapted to perform: In the video data, a signal is sent to notify first information, the first information using a first value to indicate that the existence of a first type of container is permitted, and using a second value to indicate that the existence of the first type of container is not permitted; and Based on the value of the first information, the value of the second syntax element is set, the second syntax element indicating whether to activate or deactivate the use of an encoding tool that uses at least one parameter provided by the first type container, wherein the encoding tool is a Luminance Mapping and Chroma Scaling (LMCS) mode using the at least one encoding parameter, or the use of a non-default scaling matrix, or an adaptive loop filter.

19. The device of claim 18, wherein the first information is a first syntax element encoded in the video data.

20. The device of claim 18, wherein the first type container is an adaptive parameter set, and the second syntax element indicates whether at least one adaptive parameter set is authorized to exist in the video data.

Images