Lossless mode for versatile video coding

Abstract

To provide a lossless coding mode in a video coding system that includes a plurality of coding tools.SOLUTION: A method for enabling a lossless mode include disabling a tool that is lossy by design and using only a lossless tool, adapting some tools to enable lossless coding, and adapting some tools to enable near-lossless coding such that secondary lossless coding is applied after residual coding. The method determines the type of residual coding by obtaining a flag representing a special mode when the information indicates that transform-skip residual coding is being used, and determines that normal residual coding is to be used instead of transform-skip residual coding to be used when the flag is true.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present disclosure is in the field of video compression, and at least one embodiment relates more specifically to a lossless mode of versatile video coding (VVC). [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes usually exploit spatial and temporal redundancy in the video content using prediction and transformation. In general, intra- or inter-prediction is used to exploit correlation within or between frames, and then the difference between the original and predicted image blocks, often denoted as prediction error or prediction residual, is transformed, quantized, and entropy coded. During encoding, the original image block is usually partitioned / divided into sub-blocks, possibly using, for example, quadtree partitioning. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to prediction, transformation, quantization, and entropy coding. Summary of the Invention

[0003] A lossless coding mode is proposed in a video coding system that includes a plurality of coding tools, some of which are lossy by design and some of which may be adapted to be lossless or near-lossless. To enable a lossless mode in such a video coding system, it is proposed to disable tools that are lossy by design and use only lossless tools, to adapt some tools to enable lossless coding, and to adapt some tools to enable near-lossless coding, thus allowing that secondary lossless coding may be applied after residual coding, thereby providing lossless coding.

[0004] In a particular embodiment, a method for determining a type of residual coding includes obtaining a flag representing a special mode when the information indicates that transform skip residual coding is being used, and determining that normal residual coding should be used instead of transform skip residual coding if the flag representing the special mode is true.

[0005] According to a first aspect, a method for determining a type of residual coding includes obtaining a flag representative of a special mode when information indicates that transform skip residual coding is being used, and determining that normal residual coding should be used instead of transform skip residual coding if the flag representative of the special mode is true.

[0006] According to a second aspect, a video encoding method includes determining, for a block of video, a type of residual coding according to the method of the first aspect.

[0007] According to a third aspect, a video decoding method includes determining, for a block of video, a type of residual coding according to the method of the first aspect.

[0008] According to a fourth aspect, a video encoding apparatus comprises an encoder configured to determine a type of residual coding according to the method of the first aspect.

[0009] According to a fifth aspect, a video decoding apparatus comprises a decoder configured to determine a type of residual coding according to the method of the first aspect.

[0010] One or more of the present embodiments also provide a non-transitory computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to at least a portion of any of the above methods. One or more embodiments also provide a computer program product including instructions for performing at least a portion of any of the above methods. [Brief description of the drawings]

[0011] [Figure 1A] 1 shows a block diagram of a video encoder according to one embodiment. [Figure 1B] 2 shows a block diagram of a video decoder according to one embodiment. [Diagram 2] 1 illustrates a block diagram of an example system in which various aspects and embodiments may be implemented. [Figure 3A-3B] 1 shows notations related to angular intra prediction. [Figure 4] The horizontal and vertical transformations of each SBT position are shown. [Diagram 5] 1 shows the LMCS architecture from the decoder perspective. [Figure 6] Illustrates the application of a quadratic transformation. [Figure 7] Denotes the reduced quadratic transformation (RST). [Figure 8] Illustrates forward and inverse contractive transforms. [Figure 9] Here is an example of a forward RST8x8 process using a 16x48 matrix. [Figure 10A-10B] 1 illustrates an exemplary flow chart in accordance with at least one embodiment. [Figure 11] 1 illustrates sequential application of forward and inverse functions for luma reshaping. [Figure 12] 13 shows an example flow chart of one embodiment including analysis of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is estimated to be true for a losslessly coded block. [Figure 13] 13 shows an example flow chart of one embodiment including analysis of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is estimated to be false for a losslessly coded block. [Figure 14]13 shows an example flow chart of one embodiment including cu_transquant_bypass_flag, transform_skip_flag analysis, and residual coding when transform skip is always analyzed for losslessly coded blocks. [Figure 15] 1 shows an example of a simplified block diagram of second-order lossless coding. [Figure 16] 13 shows an example flowchart of a parsing process for region_transquant_bypass_flag and split_cu_flag when region level signaling is used. [Figure 17] 1 shows the different partitioning of luma and chroma in the case of a binary tree. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Various embodiments relate to a post-processing method for a predicted value of a sample of a block of an image, the value of which is predicted according to an intra-prediction angle, where the value of the sample is modified after the prediction so that it is determined based on a weighting of the difference between the value of a left reference sample and the predicted value obtained for the sample, the left reference sample being determined based on the intra-prediction angle. An encoding method, a decoding method, an encoding device and a decoding device based on this post-processing method are proposed.

[0013] Additionally, although the present aspects describe principles related to particular drafts of the VVC (Versatile Video Coding) or HEVC (High Efficiency Video Coding) specifications, they are not limited to VVC or HEVC, and may be applied, for example, to other standards and recommendations, whether existing or developed in the future, and to extensions of any such standards and recommendations (including VVC and HEVC).Unless otherwise indicated or technically excluded, the aspects described in this application may be used individually or in any combination.

[0014] 1A shows a video encoder 100. Although variations of this encoder 100 are contemplated, the encoder 100 is described below for clarity without describing all possible variations. Before being encoded, the video sequence may undergo a pre-encoding process (101), such as applying a color transformation to the input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using a histogram equalization of one of the color components). Metadata may be associated with the pre-processing and may be attached to the bitstream.

[0015] In the encoder 100, a picture is coded by the encoder elements as described below. The picture to be coded is partitioned (102) and processed, for example, in units of CUs. Each unit is coded, for example, using either intra mode or inter mode. When a unit is coded in intra mode, intra prediction is performed (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder decides (105) whether to use one of the intra mode or inter mode for coding the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting the predicted block from the original image block (110).

[0016] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as the motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may bypass both the transform and quantization, i.e., the residual is coded directly without applying a transform or quantization process.

[0017] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined (155) with the predicted block to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / sample adaptive offset (SAO), adaptive loop filter (ALF) filtering, and reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0018] FIG. 1B shows a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by elements of the decoder as described below. The video decoder 200 generally performs a decoding pass that is an inverse of the encoding pass as described in FIG. 18. The encoder 100 also generally performs video decoding as part of encoding the video data. In particular, the input of the decoder includes a video bitstream that may be generated by the video encoder 100. First, the bitstream is entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. The picture partition information indicates how the picture is partitioned. Thus, the decoder can separate the picture according to the decoded picture partition information (235). The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the predicted block are combined (255) to reconstruct the image block. The predicted block (270) may be obtained from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0019] The decoded picture may further undergo post-decoding processing (285), such as an inverse color conversion (e.g., converting from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0020] FIG. 2 illustrates a block diagram of an example system in which various aspects and embodiments are implemented. System 1000 may be embodied as a device including various components described below and configured to implement one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 1000, either alone or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or separate components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or to other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more of the aspects described herein.

[0021] The system 1000 includes at least one processor 1010 configured to execute loaded instructions, for example, to implement various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits, as known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including, but not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drive, and / or optical disk drive. The storage device 1040 may include, as non-limiting examples, an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network-accessible storage device.

[0022] The system 1000 includes an encoder / decoder module 1030 configured to process data to provide, for example, encoded or decoded video, which may include its own processor and memory. The encoder / decoder module 1030 represents a module that may be included in a device that performs encoding and / or decoding functions. As is known, a device may include one or both of an encoding and decoding module. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0023] Program code loaded into the processor 1010 or the encoder / decoder 1030 to implement various aspects described herein may be stored in the storage device 1040 and subsequently loaded into the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, expressions, operations, and computational logic.

[0024] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be the memory 1020 and / or the storage device 1040, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, an external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a high-speed external dynamic volatile memory such as a RAM is used as working memory for video coding and decoding operations such as MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also referred to as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard developed by JVET, i.e. the Joint Video Experts Team).

[0025] Inputs to the elements of system 1000 may be provided through various input devices as shown in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, an RF signal transmitted over the air by a broadcast station, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high definition multimedia interface (HDMI) input terminal. Other examples not shown in FIG. 18 include composite video.

[0026] In various embodiments, the input devices of block 1130 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower frequency band to select a signal frequency band that may be referred to as a channel in a particular embodiment (for example), (iv) demodulating the down-converted, band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements that perform these functions, e.g., a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner that performs a variety of these functions, including down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to baseband) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements perform frequency selection by receiving, filtering, downconverting, and filtering again to the desired frequency band an RF signal transmitted over a wired (e.g., cable) medium. In various embodiments, the order of the above (and other) elements is rearranged, some of these elements are removed, and / or other elements that perform similar or different functions are added. Adding elements can include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0027] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or within the processor 1010, as desired. Similarly, aspects of the USB or HDMI interface processing may be implemented, for example, in a separate interface IC or within the processor 1010, as desired. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements including, for example, the processor 1010 and an encoder / decoder 1030 operating in combination with memory and storage elements to process the data stream as desired for display on an output device.

[0028] The various elements of the system 1000 may be provided within a unitary housing in which the various elements are interconnected and can transmit data between them using a suitable connection arrangement 1140, for example, an internal bus as is known in the art, including an Inter-IC (I2C) bus, hardwired, and printed circuit boards.

[0029] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented in a wired and / or wireless medium, for example.

[0030] Data is streamed or otherwise provided to the system 1000 in various embodiments using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received via a communication channel 1060 and a communication interface 1050 adapted for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, enabling streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 1000 using a set-top box that delivers data via an HDMI connection in the input block 1130. Still other embodiments provide streamed data to the system 1000 using an RF connection in the input block 1130. As noted above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.

[0031] The system 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, a tablet, a laptop, a mobile phone, or other device. The display 1100 can also be integrated with other components (e.g., as in a smartphone) or can be separate (e.g., an external monitor for a laptop). The other peripheral devices 1120 include, in various example embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (both terms DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of the system 1000. For example, a disc player performs the functionality of playing the output of the system 1000.

[0032] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow inter-device control with or without user intervention. The output devices may be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices may be connected to the system 1000 using a communication channel 1060 via the communication interface 1050. The display 1100 and speaker 1110 may be integrated into a single unit with other components of the system 1000 in an electronic device such as, for example, a television. In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0033] The display 1100 and speakers 1110 may alternatively be separate from one or more of the other components, for example, if the RF section of the input 1130 is part of a separate set-top box. In various embodiments in which the display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0034] The embodiments may be executed by the processor 1010 or by computer software implemented by hardware or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 1010 may be of any type suitable for the technical environment and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0035] A lossless mode is available in HEVC. In this mode, transform and quantization bypass is indicated at the CU level by a flag at the beginning of the CU syntax structure. When bypass is on, the transform and quantizer scaling operations are skipped and the residual signal is directly coded without degradation. This allows the mode to fully reconstruct a lossless representation of the coded block. Sample differences are coded as if they were at the quantized transform coefficient level, i.e., reusing transform block coding with transform sub-blocks, coefficient scanning, and last significant coefficient signaling. This mode can be useful, for example, for local coding of graphic content where quantization artifacts are very noticeable or not tolerated at all. The encoder may also switch to this mode if the rate-distortion cost of regular transform coding exceeds the rate-cost of bypass coding when using a normally low quantization parameter. For CUs coded in lossless mode, the postfilter is disabled.

[0036] PCM coding may be indicated at the CU level when activated in the sequence parameter set (SPS). When active for the considered CU, prediction, quantization, and transformation are not applied. Instead, the sample values ​​of the samples in the corresponding coding block are directly coded into the bitstream with the PCM sample bit depth configured in the SPS. The granularity of the application of PCM coding may be configured between a minimum of the size of the luma coding tree block and 32×32 at the high end and a minimum luma coding block size at the low end. If a coding unit is coded in PCM mode, the size of other coding units in the same coding tree unit must not be smaller than the size of the PCM unit. Since PCM coding by definition allows a lossless representation of the corresponding block, the bits spent on PCM coding of a coding unit can be considered as an upper limit on the amount of bits required to encode the CU. Thus, if the application of transform-based residual coding exceeds that limit, e.g., for very noisy content, the encoder can switch to PCM coding.

[0037] The lossless mode in HEVC is activated using the transquant_bypass_enabled_flag flag coded in the Picture Parameter Set (PPS) (see Table 1 below). This flag enables the coding of cu_transquant_bypass_flag at the CU level (Table 2). cu_transquant_bypass_flag specifies that quantization, transform processes, and loop filters are bypassed. [Table 1] [Table 2]

[0038] If cu_transquant_bypass_flag is true for all CUs, lossless coding may be used for the entire frame, but it is also possible that only regions are losslessly coded. This is typically the case for mixed content with natural video overlaid with text and graphics. Text and graphic regions may be losslessly coded to maximize readability, while the natural content may be coded in a lossy manner.

[0039] Furthermore, quantization is by definition lossy and is therefore disabled in lossless coding. The transform processes (DCT-2 and DST-4 are used in HEVC) are lossy in HEVC due to rounding operations. If the reconstructed signal is lossless, post-filters such as deblocking filters and sample adaptive offsets are not useful.

[0040] FIG. 3 shows an example of a simplified block diagram for the decoding process of PCM mode, lossless mode and transform skip mode. The input of the decoding process 300 is a bitstream. Data is decoded from the bitstream in step 310. Specifically, this results in quantized and transformed coefficients and a transform type. Inverse quantization is applied to the quantized and transformed coefficients in step 320. An inverse transform is applied to the resulting transformed coefficients in step 330. The resulting residual is added to the prediction signal coming from intra or inter prediction in step 340 in step 350. The result is a reconstructed block. If the I_PCM mode is selected in step 305, the sample values ​​are directly decoded without entropy decoding. If a lossless coding mode is selected, steps 320 and 330 are skipped. If a transform skip mode is selected, step 330 is skipped.

[0041] The recent Versatile Video Coding (VVC) Test Model 4 (VTM4) allows for larger transform sizes up to 64x64, which are primarily useful for higher resolution videos such as 1080p and 4K sequences. Since the high frequency transform coefficients are zeroed in a transform block whose size (width or height, or both width and height) is equal to 64, only the low frequency coefficients are kept and this information is lost. More precisely, this information is lost by not being coded in the resulting bitstream and is not available to the decoder. For example, for an MxN transform block, where M is the block width and N is the block height, only the left 32 columns of transform coefficients are kept when M is equal to 64. Similarly, only the top 32 rows of transform coefficients are kept when N is equal to 64. In both examples, the remaining 32 columns or rows of transform coefficients are lost. When using transform skip mode for large blocks, the entire block is used without zeroing any values. In practice, this consists of setting the maximum size of the transform to 5, as shown in the syntax excerpt from the current VVC specification (relevant lines are set in bold) in Table 3 below. Thus, the actual maximum transform size is used, which is set for example to 32 (2^5) in the current VVC specification. In the following, this parameter will be named "max_actual_transf_size". [Table 3]

[0042] In addition to DCT-2 used in HEVC, VVC adds a Multiple Transform Selection (MTS) scheme, which is used for residual coding of both inter- and intra-coded blocks. The newly introduced transform matrices are DST-7 and DCT-8. To control the MTS scheme, separate enable flags are specified for intra and inter at the SPS level, respectively. If MTS is enabled at the SPS level, a CU-level MTS index is signaled to indicate the pair of separable transforms to be used for the CU, among DCT-2, DST-7, and DCT-8. MTS applies only to luma. A CU-level MTS index is signaled if it meets the following conditions: both width and height are less than or equal to 32, and the CBF flag is equal to 1.

[0043] To reduce the complexity of large sizes of DST-7 and DCT-8, the high frequency transform coefficients of DST-7 and DCT-8 blocks whose size (width or height, or both width and height) is equal to 32 are zeroed. Only the coefficients within the 16x16 low frequency region are retained.

[0044] The block size restriction for transform skip is the same as the MTS restriction that transform skip is applicable to a CU if both the block width and height are less than or equal to 32.

[0045] For inter predicted CUs with cu_cbf equal to 1, cu_sbt_flag may be signaled to indicate whether the entire residual block or a sub-portion of the residual block is decoded. In the former case, the inter MTS information is further analyzed to determine the transform type of the CU. In the latter case, a portion of the residual block is coded with an estimated adaptive transform and other portions of the residual block are zeroed.

[0046] FIG. 4 shows the horizontal and vertical transforms of each SBT position. The sub-block transform is a position-dependent transform applied to the luma transform block. The two positions of SBT-H (horizontal) and SBT-V (vertical) are associated with different core transforms. More specifically, the horizontal and vertical transforms of each SBT position are shown in FIG. 4. For example, the horizontal and vertical transforms of SBT-V position 0 are DCT-8 and DST-7, respectively. If one side of the residual TU is larger than 32, the corresponding transform is set as DCT-2. Thus, the sub-block transform jointly specifies the TU tiling, cbf, and horizontal and vertical transforms of the residual block, which can be regarded as a syntax shortcut when the main residual of the block is on one side of the block.

[0047] In VTM4, a coding tool called Luma Mapping with Chroma Scaling (LMCS) is added as a new processing block before the loop filter. LMCS has two main components: 1) an in-loop mapping of the luma components based on an adaptive piecewise linear model, and 2) for the chroma components, a luma-dependent chroma residual scaling is applied.

[0048] Figure 5 shows the LMCS architecture from the decoder's perspective. The light grey blocks in Figure 5 show where processing is applied in the mapped domain. These include inverse quantization, inverse transform, luma intra prediction, and summation of luma residual and luma prediction. The unshaded blocks in Figure 5 show where processing is applied in the original (i.e., unmapped) domain. These include loop filters such as deblocking, ALF, and SAO, motion compensated prediction, chroma intra prediction, summation of chroma residual and chroma prediction, and storage of the decoded picture as a reference picture. The dark grey blocks in Figure 5 are the new LMCS functional blocks, which include forward and backward mapping of the luma signal, as well as the luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using SPS flags.

[0049] Secondary transforms, also called non-separable secondary transforms (NSST) or reduced secondary transforms (RST), are applied between the forward primary transform and quantization (at the encoder), and between dequantization and the inverse primary transform (at the decoder side).

[0050] Figure 6 shows the application of the quadratic transform. In JEM, for each 8x8 block, a 4x4 quadratic transform is applied to the smaller blocks (i.e., min(width,height)<8) and an 8x8 quadratic transform is applied to the larger blocks (i.e., min(width,height)>4).

[0051] The application of a non-separable transform is illustrated as follows using the following input as an example: A 4x4 input block is represented as the following matrix:

number

number

[0052] The non-separable transform is calculated as F=T·X, where F denotes the transform coefficient vector and T is a 16x16 transform matrix. The 16x1 coefficient vector F is then reorganized as a 4x4 block in the scan order of the block (horizontal, vertical, or diagonal). Coefficients with lower indices are placed at lower scan indices in the 4x4 coefficient block. There are a total of 35 transform sets, and three non-separable transform matrices (kernels) are used per transform set. The mapping from intra prediction modes to transform sets is predefined. For each transform set, the selected non-separable secondary transform candidates are further specified by an explicitly signaled secondary transform index. The index is signaled in the bitstream once per intra CU after the transform coefficients.

[0053] Figure 7 shows the reduced secondary transform (RST). In this method, matrices of 16x48 and 16x16 are used for 8x8 blocks and 4x4 blocks respectively. For notation convenience, the 16x48 transform is shown as RST8x8, and the 16x16 transform is shown as RST4x4. The main idea of the reduction transform (RT) is to map an N-dimensional vector to an R-dimensional vector in another space, where R / N (R < N) is the reduction factor.

[0054] Figure 8 shows the forward reduction transform and the inverse reduction transform. The RT matrix is an R×N matrix as follows.

Equation

[0055] Figure 9 shows an example of the forward RST8x8 process using a 16x48 matrix. In the adopted configuration, a 16x48 matrix is applied instead of a 16x64 with the same transform set configuration, and each obtains 48 input data from three 4x4 blocks excluding the bottom-right 4x4 block in the top-left 8x8 block (Figure 4). By reducing the dimension, the memory usage for storing all RST matrices is reduced from 10KB to 8KB, resulting in a moderate performance degradation.

[0056] Furthermore, in VTM5, a coding tool called joint coding of chrominance residuals is adopted. When this tool is activated, as shown in Equation 1, a single joint residual block is used to describe the residuals of both the Cb block and the Cr block within the same transform unit. res joint =(res Cb -res Cr ) / 2 Equation 1: Joint residual calculated from Cb residual and Cr residual

[0057] Then, the Cb and Cr signals are reconstructed by subtracting the combined residual for Cb and adding the combined residual for Cr, as shown in Equation 2.

number

[0058] The flag is coded at the TU level and enables joint coding of the chroma residual; if the flag is disabled, separate coding of the Cb and Cr residuals is used.

[0059] The embodiments described below are designed with the above in mind: the encoder 100 of Fig. 1A, the decoder 200 of Fig. 1B, and the system 1000 of Fig. 2 are adapted to implement at least one of the embodiments described below.

[0060] In at least one embodiment, the present application is directed to a lossless coding mode in a video coding system that includes multiple coding tools, some of which are lossy by design and some of which may be adapted to be lossless or near-lossless. To enable a lossless mode in a video coding system such as VVC, the following strategy is proposed. -Disable tools that are lossy by design and use only lossless tools. -Adapt some tools to enable lossless coding. - Adapting some tools to enable near-lossless coding (where the difference from the original signal is limited and small) and thus allowing a secondary lossless coding to be applied after the residual coding, thereby providing lossless coding.

[0061] Lossless coding can be performed both at the frame level or at the region level.

[0062] 10A and 10B show an example flow chart according to at least one embodiment. These figures show an example of a general process for determining whether a particular tool should be disabled at the frame level, block level, losslessly adapted, or whether a secondary lossless coding should be performed to use this tool adapted nearly losslessly. Such a process can be implemented, for example, in the encoder device 100 of FIG. 1A.

[0063] The first step (400 in FIG. 10A) is to evaluate whether the tool is lossy by design. If the tool is lossy, a second check (402 in FIG. 10B) tests whether the tool can be disabled at block level (in this case it can only be controlled at frame level). o If the tool cannot be disabled at block level, the value of the flag transquant_bypass_enabled_flag (usually signaled at PPS level) is checked in step 404 of Figure 10B. If transquant_bypass_enabled_flag is true, the tool is disabled in step 410 of Figure 10B (this applies for example to LMCS). If transquant_bypass_enabled_flag is false, the tool is enabled in step 411 of Figure 10B. If either step 410 or 411 applies, the process ends. o If the tool can be disabled at the block level, the value of the CU-level flag cu_transquant_bypass_flag is checked in step 405 of Fig. 10B. If cu_transquant_bypass_flag is true, the tool is disabled at the CU level in step 408 of Fig. 10B (this applies for SBT, MTS, LFNTS, for example). If cu_transquant_bypass_flag is false, the tool may be enabled at the CU level in step 409 of Fig. 10B. If either step 408 or 409 applies, the process ends. · If the tool is not lossy by design, then a test is performed in step 401 of FIG. 10A to check if it is lossless. o If the tool is lossless, nothing special is applied and the flow proceeds to the end of the process. o If the tool is not lossless, it is checked in step 403 of FIG. 10A whether the tool is near-lossless. ■ If the tool is not near-lossless, since it is not lossy by design either, this means that the tool can be adapted to become lossless (step 406 in FIG. 10A). This can be applied, for example, to an MTS that uses only lossless transforms in the MTS transform set. ■ If the tool is near-lossless, an additional (secondary) lossless coding step is applied (step 407 in FIG. 10A), for example a lossless transform without quantization may be applied. o If either step 306 or 307 applies, the process ends.

[0064] The remainder of this disclosure describes the application of this process to specific tools of the VVC specification.

[0065] Disabling tools incompatible with lossless coding This first case corresponds to steps 410 (picture level) and 408 (CU level) in FIG. 10A. The first element relates to the zero-out transform associated with large CUs. In one embodiment, for losslessly coded CUs or when transform skipping is used, the zero-out transform cannot be used. This is shown in the syntax specification in Table 4, where the proposed changes compared to the current VVC syntax are highlighted in italics. If a block is larger than max_actual_transf_size x max_actual_transf_size (actually 32x32 in the current VVC version) and is losslessly coded, all coefficients are coded, as opposed to the case of lossy coded blocks where only a portion of the coefficients are coded.

[0066] In fact, compared to the current VVC specification, the transform size limit only applies when transform_skip_flag is false and cu_transquant_bypass_flag is false. [Table 4]

[0067] In a variant, if the picture-level transquant_bypass_enabled_flag is enabled and a block is larger than max_actual_transf_size x max_actual_transf_size (e.g., 32x32 for current VVC), then quadtree partitioning is inferred or forced. In other words, if lossless coding is desired, blocks larger than 64x64 are systematically partitioned, resulting in a block size of 32x32, and are not subject to zeroing. Table 5 shows the modified syntax. [Table 5]

[0068] In another variant, cu_transquant_bypass_flag is coded only if the current CU is less than or equal to max_actual_transf_size x max_actual_transf_size (e.g., 32x32 for current VVC). This means that large blocks cannot be losslessly coded. Table 6 shows the associated syntax. [Table 6]

[0069] The second element is related to the sub-block transform (SBT). Since SBT uses transform tree subdivision where one of the transform units is presumed to have no residual, the tool cannot guarantee to reconstruct the CU losslessly. In one embodiment, when the lossless mode is activated (i.e., when transquant_bypass_enabled_flag is true), SBT is disabled at the CU level. The corresponding syntax changes are shown in Table 7. Decoding of SBT-related syntax and activation of SBT are possible only when transquant_bypass_enabled_flag is false. [Table 7]

[0070] In another implementation, SBT can be used only as TU tiling. In this embodiment, the residual can be coded for each sub-block, but in the initial design, some blocks are forced to have coefficients of value 0. The horizontal and vertical transforms are estimated as transform skips. The corresponding syntax changes are shown in Table 8. [Table 8]

[0071] FIG. 11 shows the sequential application of forward and inverse functions for luma reshaping. In fact, the third element is related to luma reshaping (LMCS). The reshaping is a lossy transformation, since the sequential application of the forward and inverse functions does not necessarily obtain the original signal due to the rounding operation. In LMCS, the reference picture is stored in the original domain. In the intra case, the prediction process is achieved in the "reshaped" domain, and once the samples are reconstructed, the inverse reshaping is applied just before the loop filtering step. In the inter case, after motion compensation, the predicted signal is reshaped forward. Then, once the samples are reconstructed, the inverse reshaping is applied just before the loop filtering step.

[0072] In one embodiment, if lossless coding is allowed in the PPS, LMCS is disabled at the slice level. The corresponding syntax changes are shown in Table 9. [Table 9]

[0073] The fourth element relates to multiple transform selection (MTS) and transform skip, and one embodiment relates to truly estimating transform skip. DCT and DST transforms are lossy because rounding errors lead to slight losses even when the quantization step is equal to 1. In the first embodiment, transform skip can only be used if the CU is losslessly coded (checked by the value of cu_transquant_bypass_flag in the CU level flags). transform_skip_flag is estimated to be 1 and tu_mts_idx is not coded. The corresponding syntax changes are shown in Table 10. If transform_skip_flag is estimated to be 1, transform skip residual coding is used for lossless blocks. [Table 10]

[0074] VTM-5.0 allows the use of two different residual coding processes, the first one used for non-transform skip residual coding, which is efficient for coding the residual of natural content blocks, and the second one used for transform skip residual coding, which is efficient for coding screen content blocks. The residual coding syntax choices are shown in Table 11. [Table 11]

[0075] 12 shows an exemplary flow chart of an embodiment including analysis of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is estimated to be true for a lossless coded block. In a first step 800, cu_transquant_bypass_flag is analyzed, and if the flag is true, transform_skip_flag is estimated to be true in step 801, otherwise transform_skip_flag is analyzed in step 802. If transform_skip_flag is equal to false, the normal residual is analyzed in step 803. Otherwise, if the flag is true, the transform skip residual is analyzed in step 804.

[0076] 13 shows an exemplary flow chart of an embodiment including analysis of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is estimated to be false for a lossless coded block. In a first step 900, cu_transquant_bypass_flag is analyzed, and if the flag is true, transform_skip_flag is estimated to be false in step 901 and normal residual coding is used in step 904, otherwise transform_skip_flag is analyzed in step 902. If transform_skip_flag is equal to false, normal residual is analyzed in step 904. Otherwise, if the flag is true, transform skip residual is analyzed in step 903.

[0077] Since transform skip residual coding is designed for coding residual from screen content coded blocks, it may be less efficient when coding natural content of lossless blocks. Therefore, in another embodiment, when the CU is losslessly coded, transform_skip_flag is estimated to be 0 and tu_mts_idx is not coded. Normal transform residual coding is used for losslessly coded blocks. Therefore, in this special mode (i.e., when cu_transquant_bypass_flag is true), normal coding is used even when transform_skip_flag is true, but normally, transform skip residual coding should be used when transform_skip_flag is true.

[0078] In other words, this embodiment described in Fig. 13 proposes to determine the type of residual coding by obtaining a flag representing a special mode when the information indicates that transform skip residual coding is used, and by selecting normal residual coding for the residual coding instead of transform skip residual coding to be used when this flag is true. The flag representing the special mode can be derived from other information, such as one indicating that the coding is lossless (e.g., cu_transquant_bypass_flag), one indicating that the quantization, transform process, and loop filter are bypassed (not used), or one indicating that the residual coding is forced to normal residual coding, etc.

[0079] FIG. 14 shows an example flow chart of one embodiment including parsing cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is always analyzed for losslessly coded blocks.

[0080] In such an embodiment, both regular residual coding and transform skip coding can be used to make the design of the lossless coded block closer to the lossy coded block. In this case, the transform_skip_flag is coded for the lossless coded block. This allows a competition between two different ways of coding the residual coefficients, allowing the residual coding to better adapt to the content.

[0081] In a first step 1400, cu_transquant_bypass_flag is analyzed, then transform_skip_flag is analyzed in step 1401. If transform_skip_flag is equal to false, the regular residual is analyzed in step 1403. Otherwise, if the flag is true, the transform skip residual is analyzed in step 1402.

[0082] In fact, the transform skip residual coding method is optimized for coding computer-generated content where there is a strong correlation between the values ​​of spatially adjacent coefficients. In general, in lossy coding, transform skip coefficient coding is mainly used when the content is computer-generated and is effective for this type of content.

[0083] However, in this embodiment, it is proposed to use a flag to select whether to select the normal coefficient coding method or the transform skip coefficient coding method. In fact, the transform skip coefficient coding can be used for computer-generated content, and the normal coefficient coding method can be used for normal content. This allows a competition between two different methods of coding the residual coefficients, allowing the residual coding method to be better adapted to the content. The flag only changes the coefficient coding method used to code the residual of the block.

[0084] In VVC, two methods are used to code the residual. - If transform and quantization are used, the usual coefficient coding method is used, inherited from HEVC with some further improvements, designed to code the residual after a transform that compresses the energy in the low frequency part of the signal. -When only quantization is used, the transform skip coefficient coding method is used. This method is designed for computer-generated content where the transform is often skipped due to a lot of spatial correlation between coefficients. The main features of the transform skip residual coding method are that it does not signal the last non-zero coefficient, it starts from the top-left of the block, it uses a reduced template of the previous coefficient to code the current coefficient, and the sign of the coefficient is Cabac context coded.

[0085] Figures 12, 13 and 14 show that, unlike the current VVC specification, the residual coding method can be decoupled from the fact that a transform is skipped or not. Indeed, in the current VVC specification, the transform_skip_flag syntax element equal to true means that no transform is applied to the residual of the block and the transform skip coefficient coding method is used. In the proposed method, by estimating transform_skip_flag as false, if the current block is lossless coded (transform and quantization are skipped), the normal coefficient coding method is used, and a better coefficient coding method is used when coding natural content. In a variant, even if cu_transquant_bypass_flag is true (in which case transform and quantization are skipped), a flag (here transform_skip_flag is reused for this purpose) is coded to indicate only which residual coding method is used.

[0086] One embodiment relates to a low frequency non-separable transform (LFNTS). In this embodiment, if the current CU is losslessly coded, LFNTS is disabled. The corresponding syntax changes are shown in Table 12. [Table 12]

[0087] One embodiment relates to joint Cb-Cr coding. The process of joint coding of Cb and Cr residuals is not lossless. Using Equation 1 and Equation 2 with resCb=resCr results in resjoint=0, recCb=predCb, recCr=predCr. In one embodiment, if cu_transquant_bypass_flag is true, joint Cb-Cr coding is disabled at the CU level. The corresponding syntax changes are shown in Table 13. [Table 13]

[0088] When resjoint_Cb and resJoint_Cr are separated in equation 1, equation 3 is obtained.

number

[0089] With this proposed modification, the process is reversible, but lossy due to rounding errors, as shown in Equation 4.

number

[0090] This variation can be utilized for lossless coding in implementing the secondary lossless coding process described later in this specification.

[0091] Adapting VVC tools for lossless coding Referring to FIG. 10A, this corresponds to step 406.

[0092] In one embodiment, lossless transforms are added to the MTS transform set. For lossless coding, these transforms need to be selected. Examples of lossless transforms are the unnormalized Walsh-Hadamard transform and the unnormalized Haar transform. Examples of near-lossless transforms are the normalized Walsh-Hadamard transform and the normalized Haar transform.

[0093] The lossless transform can be obtained by either the Walsh-Hadamard transform or the Haar transform. The Walsh-Hadamard and Haar unnormalized transforms consist of ±1 and zero, as shown in Equation 5 and Equation 6. For example, the unnormalized 4x4 Walsh-Hadamard matrix is:

number

number

number

number

number

number

number

number

number

[0094] The non-normalized Haar transform or non-normalized Walsh-Hadamard transform may increase the bit rate due to the increased dynamics of the coefficients. For example, when using a 4x4 non-normalized Walsh-Hadamard matrix, the coefficient energy can be up to twice the energy of the residual sample. A normalized matrix is ​​therefore achieved by dividing the non-normalized matrix by 2. However, doing so leads to a loss of integer representation. Consider the same example when transforming 4 residual samples using a "normalized" Walsh-Hadamard transform:

number

number

[0095] The remainders are then coded into the bitstream. First, we code a significant remainder bit indicating if the remainder is non-zero, then if it is significant, we code the sign of the remainder (0 is negative, 1 is positive). The syntax is shown in Table 15. Differential Pulse Code Modulation (DPCM) can also be used to exploit correlation between the remainders. Once the remainders are decoded, the decoder can compute the inverse transform as follows:

number

[0096] This technique can be used for other transform sizes, but normalization may not always be achievable. For example, a Walsh-Hadamard transform matrix of size 2x2 requires a division by the square root of 2 for normalization, which always introduces losses since the remainder parts are not simply +-0.5. However, dividing by 2, as in the 4x4 case, is beneficial as it reduces the increase in dynamics of the transform coefficients.

[0097] This technique can also be applied to the Haar transform, but because the norm of the Haar transform matrix is ​​non-uniform, this technique does not result in a normalized transform. For illustrative purposes, we consider the 4x4 Haar transform.

number

number

number

[0098] A simple way to take advantage of these transforms is to let the encoder select the optimal transform pair that reduces the rate, and then encode the index of the pair used so that the decoder can guess which pair was used to perform the inverse transform.

[0099] In the VTM, the current transform pair is either the core DCT2 transform for horizontal and vertical transforms, or an additional set of DST7 and DCT8 transforms known as Multiple Transform Selection (MTS). MTS can be switched on / off by a high-level flag. To fit this design, the default lossless transforms are no transforms in both horizontal and vertical directions, and the "multiple transforms" are the Walsh-Hadamard and Haar transforms. These can also be controlled by a high-level flag "Lossless_MTS_Flag". Thus, the transform selection table can be modified as follows: [Table 15]

[0100] Typically, the two non-normalized transforms are well suited to residuals of small dimensions, e.g., they may be used up to sizes of 8x8 or 4x4.

[0101] Block-level with second-order lossless coding Referring to FIG. 10A, this corresponds to step 407.

[0102] If a CU is lossy coded but the error is below a certain threshold, typically an additional coding stage may be introduced that enables lossless coding if the absolute value of the error is less than or equal to 1, as shown in Equation 20. The error is measured as the difference between the original pixel and the reconstructed sample. Equation 20: Boundary error of a lossy coding unit to which secondary lossless coding can be applied -1≦error≦1

[0103] For example, if a DCT or DST transform is used with a quantization step equal to 1, then a second order lossless coding may be applied after the lossy coding.

[0104] Figure 15 shows an example of a simplified block diagram of secondary lossless coding. The input of the process is a bitstream. Data is decoded from the bitstream in step 700. Specifically, this results in quantized and transformed coefficients, a transform type, and a newly introduced secondary coefficient. Inverse quantization is applied to the quantized transformed coefficients in step 701. An inverse transform is applied to the resulting transformed coefficients in step 702. The resulting residual is added in step 703 to the prediction signal coming from intra or inter prediction in step 704. The result is a reconstructed block. If an I_PCM mode is selected in step 710, the sample values ​​are directly decoded without entropy decoding. If a lossless coding mode is selected, steps 701 and 702 are skipped. If a transform skip mode is selected, step 702 is skipped. If a block level with a quadratic transform mode is selected, step 701 is skipped and an inverse transform is applied to obtain a first residual in step 720, and the quadratic lossless residual is added to the first residual. The resulting sum of residuals is then added to the predicted signal in step 703.

[0105] Since the error is below a given threshold, the syntax to code this small residual is very simple, essentially, when the threshold is 1, only the significance and sign flags may be needed to code the quadratic residual, and the syntax is shown in Table 16. [Table 16]

[0106] This second-order lossless coding can also be applied region-by-region instead of block-by-block.

[0107] Region-level signaling In lossless coding use cases, it is very likely that an entire region will be losslessly coded, and it is unlikely that some losslessly coded blocks will be confused with lossy coded blocks. To handle such cases, the actual syntax would require coding cu_transquant_bypass_flag per CU, which can be costly.

[0108] In the first embodiment, we propose to move the flag and code it before split_cu_flag, so that all child CUs of a parent CU can share the region_transquant_bypass_flag. The related syntax is shown in Table 17. [Table 17]

[0109] FIG. 16 shows an example flow chart of the parsing process of region_transquant_bypass_flag and split_cu_flag when region level signaling is used. When parsing the syntax of a coding tree unit, the first step (500) is to initialize two variables, namely isRegionTransquantCoded, which indicates whether a flag indicating lossless coding has already been coded for the current region, and isRegionTransquant, which indicates whether the current region is lossless coded, to false. Then, step 501 checks whether lossless coding is enabled at the picture level (Transquant_bypass_enabled_flag is true) and region_transquant_bypass_flag has not yet been coded for the current region (isRegionTransquantCoded is false). If these conditions are true, the flag region_transquant_bypass_flag is parsed in step 502. Otherwise, in step 504, split_cu_flag is parsed. After step 502, in step 503 it is checked whether region_transquant_bypass_flag is true. If region_transquant_bypass_flag is true, then in step 504 the variable isRegionTransquant is set to true. Then split_cu_flag is parsed in step 504. If split_cu_flag is true, the process returns to step 500. Otherwise, the tree is no longer split and the process ends (step 505). The syntax of the current coding unit may be parsed.

[0110] Lossless coding for binary trees Figure 17 shows the different partitioning of luma and chroma in the case of binary trees. In VVC, a coding structure called a binary tree allows separate coding of the luma tree and the chroma tree for intra slices.

[0111] In one embodiment, cu_transquant_bypass_flag is parsed separately for each tree, allowing greater flexibility for signaling lossless coding.

[0112] In one variant embodiment, cu_transquant_bypass_flag is not coded for the chroma tree, but is estimated from the luma tree. For a chroma CU, if one of the co-located luma CUs is lossless coded, the current chroma CU is also lossless coded. This figure shows different partitions of the luma tree and the chroma tree. For example, if luma CU L4 is lossless coded, chroma CUs C3 and C4 are lossless coded. In another example, if luma CU L3 is lossless coded, chroma CU C2 is also lossless coded, even if the co-located luma CU L2 is not lossless coded.

[0113] In this application, various aspects are described, including tools, features, embodiments, models, approaches, and the like. Many of these aspects are described to illustrate specificity and, at least, individual characteristics, and are described in a manner that may often sound limiting. However, this is for clarity of description and is not intended to limit the application or scope of these aspects. Indeed, all of the different aspects can be combined and substituted to provide further aspects. Additionally, these aspects can also be combined and substituted with aspects described in previous applications.

[0114] The aspects described and contemplated in this application can be implemented in many different forms. Figures 1A, 1B, and 2 provide some embodiments, but other embodiments are contemplated, and the description of these figures is not intended to limit the breadth of implementation. At least one of these aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0115] Various methods are described herein, each of which includes one or more steps or acts for achieving the described method. Unless a specific order of steps or acts is required for the correct operation of the method, the order and / or use of specific steps and / or acts may be varied or combined.

[0116] Various methods and other aspects described in this application can be used to modify modules such as those shown in Figures 1A and 1B, such as the motion compensation and motion estimation modules (170, 175, 275) of the video encoder 100 and the video decoder 200. Moreover, the aspects are not limited to VVC or HEVC, but can be applied to other standards and recommendations, whether existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this application can be used individually or in combination.

[0117] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes and the described aspects are not limited to these specific values.

[0118] Various implementations involve decoding. "Decoding" as used herein may encompass, for example, all or part of the processes performed on a received encoded sequence to generate a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder. In various embodiments, such processes also or alternatively include processes performed by the decoders of the various implementations described herein.

[0119] As a further example, in one embodiment, "decoding" refers to only entropy decoding, in another embodiment, "decoding" refers to only differential decoding, and in another embodiment, "decoding" refers to a combination of entropy and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process in general will be clear based on the context of the particular description and will be well understood by one of ordinary skill in the art.

[0120] Various implementations involve encoding. Similar to the above discussion regarding "decoding," "encoding" as used herein can encompass all or part of the processes performed on an input video sequence, for example to generate an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder. In various embodiments, such processes also or alternatively include processes performed by an encoder in the various implementations described herein.

[0121] As a further example, in one embodiment, "encoding" refers to only entropy encoding, in another embodiment, "encoding" refers to only differential encoding, and in another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process in general will be clear based on the context of the particular description and will be well understood by one of ordinary skill in the art.

[0122] It should be noted that the syntax elements used herein are descriptive terms, and therefore they do not preclude the use of other syntax element names.

[0123] Where a diagram is presented as a flow diagram, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, where a diagram is presented as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0124] Various embodiments refer to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, often given computational complexity constraints. Rate-distortion optimization is usually formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are various approaches to solving the rate-distortion optimization problem. For example, the approach may be based on an extensive test of all encoding options, including all modes or coding parameter values ​​considered, with a full evaluation of the coding cost and associated distortion of the reconstructed signal after coding and decoding. In particular, faster approaches may be used to reduce the coding complexity by calculating approximate distortion based on a prediction or prediction residual signal rather than a reconstructed signal. These two approaches may also be used in combination, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a full evaluation of both the coding cost and the associated distortion.

[0125] In this application, various aspects are described, including tools, features, embodiments, models, approaches, and the like. Many of these aspects are described to illustrate specificity and, at least, individual characteristics, and are described in a manner that may often sound limiting. However, this is for clarity of description and is not intended to limit the application or scope of these aspects. Indeed, all of the different aspects can be combined and substituted to provide further aspects. Additionally, these aspects can also be combined and substituted with aspects described in previous applications.

[0126] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed in the context of only a single implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., an apparatus or a program). The apparatus may be implemented, for example, in appropriate hardware, software, and firmware. The methods may be implemented, for example, in a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as computers, tablets, smartphones, mobile phones, portable / personal digital assistants, and other devices that facilitate communication of information between end users.

[0127] References to "one embodiment" or "one implementation" or "one implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in one implementation" or "in one implementation" in various places throughout this application, as well as any other variations, are not necessarily all referring to the same embodiment.

[0128] Additionally, the application may refer to "determining" various portions of information. Determining the information may include, for example, one or more of evaluating the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0129] Additionally, the application may refer to "accessing" various portions of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or evaluating information.

[0130] Additionally, the application may refer to "receiving" various portions of information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Additionally, "receiving" typically involves in some manner, for example, an operation such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or evaluating information.

[0131] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," "frame," "slice," and "tile" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side while "decoded" is used on the decoder side.

[0132] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," it should be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to cover the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to cover the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of only the third listed option (C), or the selection of only the first and second listed options (A and B), or the selection of only the first and third listed options (A and C), or the selection of only the second and third listed options (B and C), or the selection of all three options (A and B and C). This can be expanded to as many items as are listed, as would be apparent to one skilled in the art.

[0133] Also, as used herein, the word "signaling" refers to, among other things, instructing a corresponding decoder to do something. For example, in a particular embodiment, the encoder signals a particular one of the illumination compensation parameters. In this way, in one embodiment, the same parameters are used on both the encoder side and the decoder side. Thus, for example, the encoder can transmit a particular parameter to the decoder (explicit signaling), so that the decoder can use the same particular parameter. Conversely, if the decoder already has a particular parameter as well as other parameters, signaling can be used without transmission (implicit signaling) to allow the decoder to easily recognize and select the particular parameter. By avoiding the transmission of any actual function, bit savings are realized in various embodiments. It should be seen that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used in various embodiments to signal information to the corresponding decoder. Although the above is about the verb form of the word "signaling", the word "signaling" can also be used as a noun in this specification.

[0134] As will be apparent to one of ordinary skill in the art, implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bit stream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

Claims

1. 1. A method for determining a type of residual coding for a block of an image or video, comprising: - if the information indicates that transform skip residual coding is used, - obtaining a flag representing a special mode; - if the flag representing the special mode is true, determining that instead of transform skip residual coding to be used, normal residual coding should be used.

2. The method of claim 1 , wherein the flag representing the special mode indicates that the coding is lossless.

3. The method of claim 1 , wherein the flag representing the special mode indicates that quantization, a transform process, and a loop filter are bypassed.

4. The method of claim 1 , wherein the flag representing the special mode indicates that the residual coding is forced to normal residual coding.

5. A method for encoding coefficients for a block of pixels of an image or video, wherein a type of residual coding is determined by a method according to any of the claims 1 to 4.

6. A method for decoding coefficients for a block of pixels of an image or video, wherein the type of residual coding is determined by the method according to any of the claims 1 to 4.

7. An apparatus for encoding coefficients for a block of pixels of an image or a video, wherein a type of residual coding is determined by the method according to any of the claims 1 to 4.

8. An apparatus for decoding coefficients for a block of pixels of an image or a video, wherein a type of residual coding is determined by the method according to any of the claims 1 to 4.

9. A stream of data, for a block of pixels of an image or video, - coded information representative of said block of pixels; - information indicating that transform skip residual coding is used; - information representative of a special mode for determining a type of residual coding for a block of an image or video according to any one of claims 1 to 4.

10. A non-transitory computer readable medium comprising program code instructions which, when executed by a processor, implement the steps of the method according to at least one of claims 1 to 6.

11. A computer program comprising program code instructions which, when executed by a processor, implement the steps of the method according to at least one of the claims 1 to 6.

Images