Lossless mode for multi-purpose video coding

Abstract

A lossless coding mode is proposed in a video coding system comprising multiple coding tools, some of which are lossy by design and some of which can be adapted to be lossless or nearly lossless. To enable a lossless mode in such a system, it is proposed to disable tools that are lossy by design and use only lossless tools, adapt some tools to enable lossless coding, and adapt some tools to enable nearly lossless coding so that secondary lossless coding can be applied after residual coding. In a specific embodiment, it is proposed to determine the type of residual coding by acquiring a flag indicating a special mode when information indicates that transform skip residual coding is used, and determining that normal residual coding should be used instead of transform skip residual coding when this flag is true.

Description

Technology Field

[0001] The present disclosure relates to the field of video compression, and at least one embodiment relates more specifically to a lossless mode for Versatile Video Coding (VVC). Background Technology

[0002] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transformation to leverage spatial and temporal redundancy within video content. Generally, intra- or inter-prediction is used to utilize intra- or inter-frame correlation, and the differences between the original image block and the predicted image block—often referred to as prediction errors or prediction residuals—are transformed, quantized, and entropy-coded. During encoding, the original image block is usually partitioned / split into sub-blocks, if possible, using, for example, quad-tree partitioning. To reconstruct the video, the compressed data is decoded by inverse processes corresponding to prediction, transformation, quantization, and entropy coding. means of solving the problem

[0003] A lossless coding mode is proposed in a video coding system comprising multiple coding tools, some of which are lossy by design and some of which can be adapted to be lossless or nearly lossless. To enable a lossless mode in such a video coding system, it is proposed to disable lossy tools by design and use only lossless tools, adapt some tools to enable lossless coding and adapt some tools to enable nearly lossless coding, so that second-order lossless coding is applied after residual coding to provide lossless coding.

[0004] In a specific embodiment, a method for determining the type of residual coding comprises the steps of: acquiring a flag indicating a special mode when information indicates that transform skip residual coding is used; and determining that normal residual coding should be used instead of transform skip residual coding when the flag indicating the special mode is true.

[0005] According to a first embodiment, a method for determining the type of residual coding comprises the steps of: acquiring a flag indicating a special mode when information indicates that transform skip residual coding is used; and determining that normal residual coding should be used instead of transform skip residual coding when the flag indicating the special mode is true.

[0006] According to a second embodiment, the video encoding method includes the step of determining the type of residual coding for a block of video according to the method of a first embodiment.

[0007] According to a third embodiment, the video decoding method includes the step of determining the type of residual coding for a block of video according to the method of a first embodiment.

[0008] According to the fourth embodiment, the video encoding device includes an encoder configured to determine the type of residual coding according to the method of the first embodiment.

[0009] According to the fifth embodiment, the video decoding device includes a decoder configured to determine the type of residual coding according to the method of the first embodiment.

[0010] One or more of the embodiments also provide a non-transient computer-readable storage medium storing instructions for encoding or decoding video data according to at least a portion of any of the methods described above. One or more embodiments also provide a computer program product comprising instructions for performing at least a portion of any of the methods described above. Brief explanation of the drawing

[0011] FIG. 1a illustrates a block diagram of a video encoder according to an embodiment. FIG. 1b illustrates a block diagram of a video decoder according to an embodiment. FIG. 2 illustrates a block diagram of an example of a system in which various aspects and embodiments are implemented. Figures 3a and 3b illustrate notation related to angle intra prediction. Figure 4 illustrates the horizontal and vertical transformations for each SBT position. Figure 5 illustrates the LMCS architecture from the perspective of the decoder. Figure 6 illustrates the application of a second transformation. Figure 7 illustrates the Reduced Secondary Transform (RST). Figure 8 illustrates forward and reverse reduced transformations. Figure 9 illustrates an example of a forward RST8x8 process with a 16x48 matrix. FIGS. 10a and FIGS. 10b illustrate exemplary flowcharts according to at least an embodiment. Figure 11 illustrates the application of the forward and then inverse functions for luma reshaping. FIG. 12 illustrates an exemplary flowchart of an embodiment including parsing of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is inferred to be true for lossless coded blocks. FIG. 13 illustrates an exemplary flowchart of an embodiment including parsing of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when the transform skip is inferred to be false for lossless coded blocks. FIG. 14 illustrates an exemplary flowchart of an embodiment including the parsing of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when the transform skip is always parsed for lossless coded blocks. Figure 15 illustrates an example of a simplified block diagram of second-order lossless coding. Figure 16 illustrates an exemplary flowchart of the parsing process for region_transquant_bypass_flag and split_cu_flag when region-level signaling is used. Figure 17 illustrates different partitions for lumens and chromas in the case of a dual tree. Specific details for implementing the invention

[0012] Various embodiments relate to a post-processing method for the predicted values ​​of samples of a block of an image, wherein the values ​​are predicted according to an intra-prediction angle, and the values ​​of the samples are modified after prediction and determined based on the weighting of the difference between the value of a left-reference sample and the acquired predicted value for the sample, and the left-reference sample is determined based on the intra-prediction angle. An encoding method, a decoding method, an encoding device, and a decoding device based on such a post-processing method are proposed.

[0013] Furthermore, while these embodiments describe principles related to specific drafts of Versatile Video Coding (VVC) or High Efficiency Video Coding (HEVC) specifications, they are not limited to VVC or HEVC alone and may be applied to, for example, other standards and recommendations, whether existing or future-developed, and any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the embodiments described in this application may be used individually or in combination.

[0014] FIG. 1a illustrates a video encoder (100). Variations of this encoder (100) are considered, but the encoder (100) is described below for the sake of clarity without describing all expected variations. Before encoding, the video sequence undergoes pre-encoding processing (101), for example, applying color conversion to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input picture components to obtain a signal distribution more resilient to compression (e.g., by using histogram equalization of one of the color components). Metadata is associated with the preprocessing and can be attached to the bitstream.

[0015] In the encoder (100), the picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (102) and processed, for example, into units of CUs. Each unit is encoded, for example, using an intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder determines whether to use intra mode or inter mode to encode the unit (105), and indicates the intra / inter decision, for example, by a prediction mode flag. Prediction residuals are calculated, for example, by subtracting the predicted block from the original image block (110).

[0016] The predicted residuals are subsequently transformed (125) and quantized (130). The quantized transformation coefficients, as well as motion vectors and other syntax elements, are entropy-coded to output a bitstream (145). The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can bypass both transformation and quantization, that is, the residuals are coded directly without the application of transformation or quantization processes.

[0017] The encoder decodes the encoded block to provide a reference for additional predictions. To decode the prediction residuals, the quantized transform coefficients are dequantized (140) and inversely transformed (150). By combining the decoded prediction residuals and the predicted block (155), the image block is reconstructed. In-loop filters (165) are applied to the reconstructed picture to perform deblocking / SAO (Sample Adaptive Offset) and ALF (Adaptive Loop-Filter) filtering to reduce encoding artifacts, for example. The filtered image is stored in a reference picture buffer (180).

[0018] FIG. 1b illustrates a block diagram of a video decoder (200). In the decoder (200), the bitstream is decoded by decoder elements as described below. The video decoder (200) generally performs a decoding pass that is inversely to an encoding pass as described in FIG. 18. Additionally, the encoder (100) generally performs video decoding as part of encoding video data. In particular, the input of the decoder includes a video bitstream that can be generated by the video encoder (100). The bitstream is first entropy decoded to obtain transform coefficients, motion vectors, and other coded information (230). Picture partition information indicates how the picture is partitioned. Thus, the decoder can partition the picture according to the decoded picture partitioning information (235). To decode the prediction residuals, the transformation coefficients are dequantized (240) and inversely transformed (250). By combining the decoded prediction residuals and the predicted blocks (255), an image block is reconstructed. The predicted blocks can be obtained from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275) (270). In-loop filters (265) are applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0019] The decoded picture may further undergo post-decoding processing (285), for example, inverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the reverse of the remapping process performed in pre-encoding processing (101). The post-decoding processing may use metadata derived from pre-encoding processing and signaled in the bitstream.

[0020] FIG. 2 illustrates a block diagram of an example of a system in which various aspects and embodiments are implemented. The system (1000) may be implemented as a device comprising various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics, and servers. The elements of the system (1000) may be implemented, alone or in combination, as a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of the system (1000) are distributed across multiple ICs and / or discrete components. In various embodiments, the system (1000) is communically coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, the system (1000) is configured to implement one or more of the embodiments described in this document.

[0021] The system (1000) includes at least one processor (1010) configured to execute instructions loaded therein, for example, to implement the various embodiments described in this document. The processor (1010) may include embedded memory, an input / output interface, and various other circuits known in the art. The system (1000) includes at least one memory (1020) (e.g., a volatile memory device and / or a non-volatile memory device). The system (1000) includes a storage device (1040) that may include non-volatile memory and / or volatile memory, including but not limited to EEPROM (Electrically Erasable Programmable Read-Only Memory), ROM (Read-Only Memory), PROM (Programmable Read-Only Memory), RAM (Random Access Memory), DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), flash, a magnetic disk drive, and / or an optical disk drive. The storage device (1040) may include, as non-limiting examples, an internal storage device, an attached storage device (including detachable and non-detachable storage devices), and / or a network-accessible storage device.

[0022] The system (1000) includes, for example, an encoder / decoder module (1030) configured to process data to provide an encoded video or a decoded video, and the encoder / decoder module (1030) may include its own processor and memory. The encoder / decoder module (1030) represents a module(s) that may be included in a device to perform encoding and / or decoding functions. As is known, the device may include one or both of the encoding and decoding modules. Additionally, the encoder / decoder module (1030) may be implemented as a separate element of the system (1000) or may be integrated within the processor (1010) as a combination of hardware and software, as is known to those skilled in the art.

[0023] Program code to be loaded onto a processor (1010) or an encoder / decoder (1030) to perform the various embodiments described in this document may be stored in a storage device (1040) and subsequently loaded onto memory (1020) for execution by the processor (1010). According to various embodiments, one or more of the processor (1010), memory (1020), storage device (1040), and encoder / decoder module (1030) may store one or more of various items during the execution of the processes described in this document. These stored items may include, but are not limited to, input video, decoded video or parts of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of mathematical expressions, formulas, operations, and operation logic.

[0024] In some embodiments, memory located inside the processor (1010) and / or the encoder / decoder module (1030) is used to store instructions and to provide working memory for processing required during encoding or decoding. However, in other embodiments, memory outside the processing device (e.g., the processing device may be either the processor (1010) or the encoder / decoder module (1030)) is used for one or more of these functions. The external memory may be memory (1020) and / or storage device (1040), e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a high-speed external dynamic volatile memory, such as RAM, is used as a working memory for video coding and decoding operations such as MPEG-2 (MPEG refers to the Moving Picture Experts Group, MPEG-2 is also referred to as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by JVET, Joint Video Experts Team).

[0025] Inputs to the elements of the system (1000) may be provided through various input devices as shown in block (1130). These input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives, for example, an RF signal transmitted over the air by a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Other examples not shown in FIG. 18 include composite video.

[0026] In various embodiments, the input devices of block (1130) have their respective associated input processing elements as known in the art. For example, the RF portion may be associated with appropriate elements for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower band of frequencies to select a single frequency band (e.g.) which may be referred to as a channel in certain embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a stream of desired data packets. The RF portion of various embodiments includes one or more elements for performing these functions, e.g., frequency selectors, signal selectors, band-limiters, channel selectors, filters, down-converters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs various such functions, for example, including down-converting the received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to the baseband. In one set-top box embodiment, the RF section and its associated input processing element receive an RF signal transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0027] Additionally, USB and / or HDMI terminals may include their own interface processors for connecting the system (1000) to other electronic devices via USB and / or HDMI connections. It should be understood that various modes of input processing, e.g., Reed-Solomon error correction, may be implemented as needed, e.g., within a separate input processing IC or within the processor (1010). Similarly, modes of USB or HDMI interface processing may be implemented as needed within separate interface ICs or within the processor (1010). The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, e.g., a processor (1010) and an encoder / decoder (1030) operating in combination with memory and storage elements, to process the data stream as needed for presentation on an output device.

[0028] Various elements of the system (1000) may be provided within an integrated housing. Within the integrated housing, various elements may be interconnected and data may be transmitted between them using a suitable connection arrangement (1140) known in the relevant art, such as, for example, an I2C (Inter-IC) bus, wiring, and printed circuit board.

[0029] The system (1000) includes a communication interface (1050) that enables communication with other devices through a communication channel (1060). The communication interface (1050) may include a transceiver configured to transmit and receive data through the communication channel (1060), but is not limited thereto. The communication interface (1050) may include a modem or a network card, but is not limited thereto, and the communication channel (1060) may be implemented, for example, within a wired and / or wireless medium.

[0030] In various embodiments, data is streamed or otherwise provided to the system (1000) using a wireless network, such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE stands for Institute of Electrical and Electronics Engineers). Wi-Fi signals in these embodiments are received, for example, through a communication channel (1060) and a communication interface (1050) adapted for Wi-Fi communication. The communication channel (1060) in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top (OTT) communications. Other embodiments provide streamed data to the system (1000) using a set-top box that transmits data via the HDMI connection of the input block (1130). Yet another embodiment provides streamed data to the system (1000) using the RF connection of the input block (1130). As mentioned above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, for example, cellular networks or Bluetooth networks.

[0031] The system (1000) can provide output signals to various output devices, including a display (1100), speakers (1110), and other peripheral devices (1120). The display (1100) in various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display (1100) may be for a television, tablet, laptop, mobile phone, or other device. The display (1100) may be integrated with other components (e.g., as in a smartphone) or may be separate (e.g., an external monitor for a laptop). The other peripheral devices (1120) in various examples of the embodiments include one or more of a standalone digital video disc (or digital multifunction disc) (DVR, for both), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices (1120) that provide functions based on the output of the system (1000). For example, the disc player performs the function of playing the output of the system (1000).

[0032] In various embodiments, control signals are communicated between the system (1000) and the display (1100), speakers (1110), or other peripheral devices (1120) using signaling such as AV.Link, CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicably coupled to the system (1000) through dedicated connections via their respective interfaces (1070, 1080, and 1090). Alternatively, output devices may be connected to the system (1000) using a communication channel (1060) via a communication interface (1050). The display (1100) and speakers (1110) may be integrated with other components of the system (1000) in an electronic device, e.g., a television, as a single unit. In various embodiments, the display interface (1070) includes a display driver, e.g., a timing controller (T Con) chip.

[0033] Alternatively, the display (1100) and speaker (1110) may be separated from one or more of the other components, for example, if the RF portion of the input (1130) is part of a separate set-top box. In various embodiments where the display (1100) and speakers (1110) are external components, the output signal may be provided through dedicated output connections, for example, HDMI ports, USB ports, or COMP outputs.

[0034] The embodiments may be performed by computer software implemented by the processor (1010), by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory (1020) may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, e.g., as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor (1010) may be of any type suitable for the technical environment and may include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on a multi-core architecture as non-limiting examples.

[0035] Lossless mode is available in HEVC. In this mode, transform and quantization bypasses are indicated at the CU level by a flag at the beginning of the CU syntax structure. If the bypass is on, transform and quantizer scaling operations are skipped, and residual signals are coded directly without any degradation. This enables this mode to fully reconstruct the lossless representation of the coded block. Sample differences are encoded by reusing the transform block coding, along with transform sub-blocks, coefficient scans, and the final effective coefficient signaling, as if they were quantized transform coefficient levels. This mode can be useful, for example, for local coding of graphic content where quantization artifacts may be highly visible or completely unacceptable. The encoder may also switch to this mode if the rate distortion cost of normal transform coding exceeds the rate cost of bypass coding when using typically low quantization parameters. For CUs coded in lossless mode, post-filters are disabled.

[0036] When enabled in the Sequence Parameter Set (SPS), PCM coding can be applied at the CU level. When enabled for a CU under consideration, no prediction, no quantization, and no transformation are applied. Instead, the sample values ​​of the samples within the corresponding coding blocks are directly coded to become a bitstream at the PCM sample bit depth configured in the SPS. The granularity for applying PCM coding can be configured between the minimum of the 32x32 luminal coding tree block size and the minimum luminal coding block size at the low end. When a coding unit is coded in PCM mode, the size of other coding units within the same coding tree unit must not be smaller than the size of the PCM unit. By definition, since PCM coding enables lossless representation of the corresponding block, the bits consumed for PCM coding of the coding unit can be considered as an upper limit of the amount of bits required to encode the CU. Therefore, the encoder can switch to PCM coding, for example, when the application of transform-based residual coding exceeds its limits for exceptionally noisy content.

[0037] HEVC lossless mode is enabled using the transquant_bypass_enabled_flag flag coded in the PPS (Picture Parameter Set) (see Table 1 below). This flag enables the coding of cu_transquant_bypass_flag at the CU level (Table 2). cu_transquant_bypass_flag It specifies that the quantization, transformation process, and loop filter are bypassed.

[0038]

[0039]

[0040] If cu_transquant_bypass_flag is true for all CUs, lossless coding may be used for the entire frame, but it is also possible for only one region to be lossless coded. This is typically the case for mixed content with natural video containing overlaid text and graphics. The text and graphics regions may be lossless coded to maximize readability, while the natural content may be coded in a lossy manner.

[0041] Furthermore, since quantization is lossy by definition, it is disabled for lossless coding. The conversion process (DCT-2 and DST-4 are used in HEVC) is lossy in HEVC due to rounding operations. Deblocking filters and post-filters as Sample Adaptive Offsets are useless when the reconstructed signal is lossless.

[0042] FIG. 3 illustrates an example of a simplified block diagram of a decoding process for PCM, lossless, and transform skip modes. The input to the decoding process (300) is a bitstream. Data is decoded from the bitstream in step (310). Specifically, this provides quantized transformed coefficients and a transform type. In step (320), inverse quantization is applied to the quantized transformed coefficients. In step (330), inverse transform is applied to the resulting transformed coefficients. The resulting residual is added to the prediction signal coming from the intra or inter prediction of step (340) in step (350). The result is a reconstructed block. When the I_PCM mode is selected in step (305), the sample values ​​are decoded directly without entropy decoding. When the lossless coding mode is selected, steps (320 and 330) are skipped. When the transform skip mode is selected, step (330) is skipped.

[0043] In the more recent Versatile Video Coding (VVC) Test Model 4 (VTM4), large transform sizes of up to 64x64 are possible, which is primarily useful for high-resolution video, e.g., 1080p and 4K sequences. For transform blocks with a size equal to 64 (width or height, or both), high-frequency transform factors are zeroed out so that only lower-frequency factors are retained, and thus this information is lost. More precisely, this information is lost by not being encoded in the resulting bitstream and is therefore not available to the decoder. For example, for an MxN transform block where M is the block width and N is the block height, when M is equal to 64, only the left 32 columns of transform factors are retained. Similarly, when N is 64, only the top 32 rows of transform factors are retained. In both examples, the remaining 32 columns or rows of transform factors are lost. When the transform skip mode is used for a large block, the entire block is used without zeroing out any values. In practice, this consists of setting the maximum size for the transform to 5, as illustrated in the syntax excerpt from the current VVC specification (relevant lines are set in bold) shown in Table 3 below. Thus, the maximum actual transform size set to, for example, 32 (2^5) in the current VVC specification is used. Hereinafter, this parameter will be named "max_actual_transf_size".

[0044]

[0045] In addition to DCT-2 used in HEVC, VVC adds a Multiple Transform Selection (MTS) scheme used for residual coding for both inter- and intra-coded blocks. The newly introduced transformation matrices are DST-7 and DCT-8. To control the MTS scheme, individual enabling flags are specified at the SPS level for intra and inter, respectively. When MTS is enabled at the SPS level, a CU-level MTS index is signaled to indicate the pairs of separable transformations used for the CU among DCT-2, DST-7, and DCT-8. MTS is applied only to the luminance. The CU-level MTS index is signaled when the following conditions are met: both width and height are 32 or less, and the CBF flag is equal to 1.

[0046] To reduce the complexity of the large-sized DST-7 and DCT-8, the high-frequency conversion coefficients are zeroed out for DST-7 and DCT-8 blocks having the same size (width or height, or both width and height) as 32. Only the coefficients within the 16x16 low-frequency range are retained.

[0047] The block size limit for transform skips is the same as for MTS, which clearly states that transform skips are applicable to CU when both the block width and height are 32 or less.

[0048] For an inter-predicted CU with cu_cbf identical to 1, cu_sbt_flag may be signaled to indicate whether the entire residual block or a sub-part of the residual block is decoded. In the former case, inter-MTS information is further parsed to determine the transform type of the CU. In the latter case, one part of the residual block is coded with the inferred adaptive transform and another part of the residual block is zeroed out.

[0049] Figure 4 illustrates the horizontal and vertical transformations for each SBT position. The sub-block transformation is a position-dependent transformation applied to the Luma transformation blocks. Two positions, SBT-H (horizontal) and SBT-V (vertical), are associated with different core transformations. More specifically, the horizontal and vertical transformations for each SBT position are specified in Figure 4. For example, the horizontal and vertical transformations for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is greater than 32, the corresponding transformation is set as DCT-2. Thus, the sub-block transformation jointly specifies the TU tiling, cbf, and the horizontal and vertical transformations of the residual block, which can be considered as a syntax shortcut for cases where the main residual of the block is on one side of the block.

[0050] In VTM4, a coding tool called LMCS (luma mapping with chroma scaling) was added as a new processing block before loop filters. LMCS has two main components: 1) in-loop mapping of lumina components based on adaptive piecewise linear models; and 2) lumina-dependent chroma residual scaling applied to chroma components.

[0051] Figure 5 illustrates the LMCS architecture from the perspective of a decoder. The light gray shaded blocks in Figure 5 indicate where processing is applied in the mapped domain; these include inverse quantization, inverse transform, luminance intra prediction, and adding luminance prediction with luminance residuals. The unshaded blocks in Figure 5 indicate where processing is applied in the original (i.e., unmapped) domain; these include deblocking, loop filters such as ALF and SAO, motion compensation prediction, chroma intra prediction, adding chroma prediction with chroma residuals, and storing decoded pictures as reference pictures. The dark gray shaded blocks in Figure 5 are new LMCS function blocks that include forward and inverse mapping of the luminance signal and the luminance-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag.

[0052] As also known as NSST (Non-Separable Secondary Transform) or RST (Reduced Secondary Transform), a secondary transformation is applied between the forward first-order transformation and quantization (in the encoder) and between dequantization and the inverse first-order transformation (on the decoder side).

[0053] Figure 6 illustrates the application of a second transformation. In JEM, as shown in the figure, a 4x4 second transformation is applied to smaller blocks (i.e., min(width, height) < 8), and an 8x8 second transformation is applied to larger blocks (i.e., min(width, height) > 4) per 8x8 block.

[0054] The application of inseparable transformations is explained as follows, using input as an example. A 4x4 input block is represented by the following matrix:

[0055]

[0056] To apply an inseparable transformation, this 4x4 input block X is first a vector It is expressed as:

[0057]

[0058] Inseparable transformations It is calculated as, and here represents the transformation coefficient vector, and T is a 16x16 transformation matrix. 16x1 coefficient vector It is subsequently reorganized into a 4x4 block using the scanning order (horizontal, vertical, or diagonal) for that block. Coefficients with smaller indices will be placed in smaller scanning indices within the 4x4 coefficient block. There are a total of 35 transformation sets, and three inseparable transformation matrices (kernels) are used per transformation set. The mapping from the intra prediction mode to the transformation sets is predefined. For each transformation set, selected inseparable quadratic transformation candidates are further specified by an explicitly signaled quadratic transformation index. The index is signaled as a bitstream once per intra CU after the transformation coefficients.

[0059] Figure 7 illustrates the Reduced Secondary Transform (RST). In this technique, 16x48 and 16x16 matrices are used for 8x8 and 4x4 blocks, respectively. For convenience of notation, the 16x48 transformation is denoted as RST8x8, and the 16x16 transformation is denoted as RST4x4. The main idea of ​​the Reduced Transform (RT) is to map an N-dimensional vector to an R-dimensional vector in a different space, where R / N(R <N)은 감소 인자이다.

[0060] Figure 8 illustrates the forward and backward reduced transformations. The RT matrix is ​​the following RxN matrix:

[0061]

[0062] Here, the R row of the transformation is the R base of the N-dimensional space. The inverse transformation matrix for RT is the transpose of its forward transformation.

[0063] Figure 9 illustrates an example of a forward RST8x8 process with 16x48 matrices. In the adopted configuration, 16x48 matrices are applied instead of 16x64 matrices with the same transformation set configuration, each taking 48 input data from three 4x4 blocks within the top-left 8x8 block, excluding the right-bottom 4x4 block (Figure 4). With the help of reduced dimensions, memory usage for storing all RST matrices is reduced from 10KB to 8KB with reasonable performance degradation.

[0064] Furthermore, in VTM5, a coding tool called joint coding of chrominance residuals has been adopted. When this tool is activated, a single joint residual block is used to describe the residuals of both Cb and Cr blocks in the same transformation unit, as shown in Equation 1.

[0065]

[0066] After that, the Cb and Cr signals are reconstructed by subtracting the joint residual for Cb and adding it for Cr, as shown in Equation 2.

[0067]

[0068] The flag is coded at the TU level to enable joint coding of chroma residuals, and when the flag is disabled, individual coding of Cb and Cr residuals is used.

[0069] The embodiments described below are designed with the foregoing in mind. The encoder (100) of FIG. 1a, the decoder (200) of FIG. 1b, and the system (1000) of FIG. 2 are adapted to implement at least one of the embodiments described below.

[0070] In at least one embodiment, the present application relates to a lossless coding mode in a video coding system comprising a plurality of coding tools, some of which are lossy by design, and some of which can be adapted to be lossless or nearly lossless. For example, to enable a lossless mode in a video coding system such as VVC, the following strategy is proposed:

[0071] - Disable lossy tools by design and use only lossless tools.

[0072] - Adapt some tools to enable lossless coding.

[0073] - Some tools are adapted to enable near-lossless coding (where the difference from the original signal is limited and small), so that second-order lossless coding is applied after residual coding, and thus lossless coding can be provided.

[0074] Lossless coding can be handled at both the frame level and the region level.

[0075] FIGS. 10a and FIGS. 10b illustrate exemplary flowcharts according to at least an embodiment. The drawings illustrate an example of a general process for determining whether a given tool should be disabled at the frame level, block level, adapted to be lossless, or used to utilize this tool adapted to be nearly lossless for secondary lossless coding. Such a process can be implemented, for example, in the encoder device (100) of FIG. 1a.

[0076] In the first step (400 in FIG. 10a), whether the tool is lossy by design is evaluated.

[0077] ● If the tool is lossy, a second check (402 in FIG. 10b) tests whether the tool can be disabled at the block level (in this case, the tool can only be controlled at the frame level).

[0078] o If the tool cannot be disabled at the block level, the value of the flag transquant_bypass_enabled_flag (typically signaled at the PPS level) is checked at step (404) of FIG. 10b. If transquant_bypass_enabled_flag is true, the tool is disabled at step (410) of FIG. 10b (this applies to LMCS, for example). If transquant_bypass_enabled_flag is false, the tool is enabled at step (411) of FIG. 10b. Once step (410) or (411) is applied, the process is terminated.

[0079] o If the tool can be disabled at the block level, the value of the CU-level flag cu_transquant_bypass_flag is checked in step (405) of FIG. 10b. If cu_transquant_bypass_flag is true, the tool is disabled at the CU level in step (408) of FIG. 10b (this applies, for example, to SBT, MTS, LFNTS). If cu_transquant_bypass_flag is false, the tool can be enabled at the CU level in step (409) of FIG. 10b. Once step (408) or (409) is applied, the process is terminated.

[0080] ● If the tool is not lossy by design, a test is performed to check whether it is lossless in step (401) of FIG. 10a.

[0081] o If the tool is lossless, no special measures are applied, and the flow proceeds to the end of the process.

[0082] o If the tool is not lossless, in step (403) of FIG. 10a, it is checked whether the tool is nearly lossless.

[0083] ■ If the tool is not nearly lossless, the tool is also not lossy by design, which means that the tool can be adapted to be lossless (step 406 of FIG. 10a). This can be applied, for example, to an MTS that uses only lossless transformations within the MTS transformation set.

[0084] ■ When the tool is nearly lossless, an additional (secondary) lossless coding step is applied (step 407 in FIG. 10a). For example, a lossless transformation without quantization may be applied.

[0085] Once step (306) or (307) is applied, the process is terminated.

[0086] In the following disclosure, the application of this process to specific tools of the VVC specification is described.

[0087] Disabling tools incompatible with lossless coding

[0088] This first case corresponds to steps (410) (picture level) and (408) (CU level) of FIG. 10a. The first element relates to a Zero-Out Transform with large CUs. In the embodiment, the Zero-Out Transform cannot be used in the case of lossless-coded CUs or when a transform skip is used. This is exemplified in the syntax specification of Table 4, which highlights proposed changes compared to the current VVC syntax in italicized text. When a block is larger than max_actual_transf_size x max_actual_transf_size (actually in the current VVC version, 32x32) and is lossless coded, all coefficients are coded, in contrast to the case of lossy-coded blocks where only some of the coefficients are coded.

[0089] In fact, compared to the current VVC specification, the transform size limit is applied only when transform_skip_flag is false and cu_transquant_bypass_flag is false.

[0090]

[0091] In the variant, if the picture-level transquant_bypass_enabled_flag is enabled and the block is larger than max_actual_transf_size x max_actual_transf_size (e.g., 32x32 in the current VVC), quadtree splitting is inferred or enforced. In other words, when lossless coding is desired, blocks larger than 64x64 are systematically split so that the block size becomes 32x32 and thus avoids zeroing. Table 5 shows the modified syntax.

[0092]

[0093] In another variant, cu_transquant_bypass_flag is coded only when the current CU is less than or equal to max_actual_transf_size x max_actual_transf_size (e.g., 32x32 in the current VVC). This means that larger blocks cannot be losslessly coded. Table 6 shows the associated syntax.

[0094]

[0095] The second element relates to the Sub-block Transform (SBT). Since the SBT uses a transform tree subdivision with one of the transform units inferred to have no residuals, this tool cannot guarantee the reconstruction of a lossless CU. In the example, the SBT is disabled at the CU level when lossless mode is enabled (i.e., when transquant_bypass_enabled_flag is true). The corresponding syntax changes are shown in Table 7. Decoding of SBT-related syntax and activation of the SBT are possible only when transquant_bypass_enabled_flag is false.

[0096]

[0097] In an alternative implementation, SBT can be used only as TU tiling. In this embodiment, residuals can be coded for each sub-block, while some blocks are forced to have coefficients of value 0 in the initial design. Horizontal and vertical transformations are inferred as transformation skips. Corresponding syntax changes are shown in Table 8.

[0098]

[0099] Figure 11 illustrates the application of the forward and subsequent inverse function for luma reshaping. In practice, the third element is related to Luma Reshaping (LMCS). Reshaping is a lossy transformation because, due to rounding operations, the application of the forward and subsequent inverse function is not guaranteed to provide the original signal. In LMCS, the reference picture is stored in the original domain. In the intra case, the prediction process is achieved in the "reshaped" domain, and once the samples are reshaped, inverse reshaping is applied immediately before the loop-filtering steps. In the inter case, after motion compensation, the predicted signal is forward reshaped. Then, once the samples are reshaped, inverse reshaping is applied immediately before the loop-filtering steps.

[0100] In the examples, LMCS is disabled at the slice-level when lossless coding is allowed in PPS. Corresponding syntax changes are shown in Table 9.

[0101]

[0102] The fourth element relates to MTS (Multiple Transform Selection) and transform skips, and the embodiment relates to inferring the transform skips as true. DCT and DST transforms are lossy even if the quantization step is equal to 1, due to rounding errors that cause some loss. In the first embodiment, when the CU is lossless coded (checked by the value of the CU-level flag cu_transquant_bypass_flag), only transform skips may be used. transform_skip_flag is inferred as 1, and tu_mts_idx is not coded. The corresponding syntax changes are shown in Table 10. When transform_skip_flag is inferred as 1, residual coding for the transform skips used for lossless blocks is used.

[0103]

[0104] In VTM-5.0, two different residual coding processes can be used: the first residual coding process is used for non-transformed skip residual coding and is efficient for coding residuals of natural content blocks, and the second residual coding process is used for transformed skip residual coding and is efficient for coding screen content blocks. The selection of residual coding syntax is shown in Table 11.

[0105]

[0106] FIG. 12 illustrates an exemplary flowchart of an embodiment including the parsing of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is inferred to be true for lossless coded blocks. In the first step (800), cu_transquant_bypass_flag is parsed, and if the flag is true, transform_skip_flag is inferred to be true in step (801), otherwise transform_skip_flag is parsed in step (802). If transform_skip_flag is false, normalized residuals are parsed in step (803), otherwise, transform skip residuals are parsed in step (804).

[0107] FIG. 13 illustrates an exemplary flowchart of an embodiment including the parsing of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when transform skip is inferred as false for lossless coded blocks. In the first step (900), cu_transquant_bypass_flag is parsed, and if the flag is true, transform_skip_flag is inferred as false in step (901) and normalized residual coding is used in step (904), otherwise transform_skip_flag is parsed in step (902). If transform_skip_flag is false, normalized residuals are parsed in step (904), otherwise, transform skip residuals are parsed in step (903).

[0108] Since residual coding for transform skip is designed to code residuals from screen content-coded blocks, this may be less efficient for coding natural content for lossless blocks. Therefore, in another embodiment, when CU is lossless coded, transform_skip_flag is inferred to 0 and tu_mts_idx is not coded. Residual coding for normal transformation is used for lossless coded blocks. Thus, in this special mode (i.e., when cu_transquant_bypass_flag is true), normal coding will be used even when transform_skip_flag is true, whereas normal transform skip residual coding should be used when transform_skip_flag is true.

[0109] In other words, this embodiment described in FIG. 13 proposes determining the type of residual coding by acquiring a flag indicating a special mode when information indicates that transform skip residual coding is used, and selecting normal residual coding for residual coding instead of transform skip residual coding that should be used when this flag is true. The flag indicating a special mode may be derived from other information, such as an indication that the coding is lossless (e.g., cu_transquant_bypass_flag), or an indication that the quantization, transform process and loop filter are bypassed (not used), or an indication that the residual coding is forced to normal residual coding.

[0110] FIG. 14 illustrates an exemplary flowchart of an embodiment including the parsing of cu_transquant_bypass_flag, transform_skip_flag, and residual coding when the transform skip is always parsed for lossless coded blocks.

[0111] In this embodiment, to keep the design of lossless coded blocks close to that of lossy coded blocks, both normal residual coding and transform skip coding may be used. In this case, transform_skip_flag is coded for the lossless coded blocks. This allows competition between two different methods of coding residual coefficients and better adapts residual coding to the content.

[0112] In the first step (1400), cu_transquant_bypass_flag is parsed, and then in the next step (1401), transform_skip_flag is parsed. If transform_skip_flag is false, normalized residuals are parsed in the next step (1403), and if the flag is true, transform skip residuals are parsed in the next step (1402).

[0113] In practice, the transform-skip residual coding method is optimized for coding computer-generated content where the correlation between spatially adjacent coefficient values ​​is strong. Generally, in lossy coding, transform-skip coefficient coding is primarily used when content is computer-generated and is effective for this type of content.

[0114] However, in this embodiment, it is suggested to use a flag to select between the normal coefficient coding method and the transform-skip coefficient coding method. In practice, transform-skip coefficient coding can be used for computer-generated content, and the normal coefficient coding method can be used for normal content. This allows competition between two different ways of coding residual coefficients and adapts the residual coding method better to the content. This flag changes only the coefficient coding method used to code the residuals of the block.

[0115] In VVC, two methods are used to code residuals:

[0116] - The normal coefficient coding method is used when transformation and quantization are used. The method is inherited from HEVC with some improvements and is designed to code residuals after a transformation where energy is compacted in the low-frequency portion of the signal.

[0117] - Transform-skip coefficient coding methods are used when only quantization is used. The method is designed for computer-generated content where transformations are often skipped due to significant spatial correlations between coefficients. The key characteristics of transform-skip residual coding methods are that they do not signal the last non-zero coefficient, start from the top-left corner of the block, use a reduction template of previous coefficients to code the current one, and the signs of the coefficients are coded in Cabac context.

[0118] FIGS. 12, 13, and 14 show that, unlike in the current VVC specification, the residual coding method can be separated from the fact that a transformation was skipped or not. In fact, in the current VVC specification, the transform_skip_flag syntax element being true means that no transformation is applied to the residuals of the block and the transform-skip coefficient coding method is used. On the other hand, in the proposed methods, to use a better coefficient coding method when coding natural content, the transform_skip_flag is inferred to be false so that the normal coefficient coding method is used when the current block is losslessly coded (when transformation and quantization are skipped). In the transformation, even if cu_transquant_bypass_flag is true (in which case transformation and quantization are skipped), the flag (where transform_skip_flag is reused for this purpose) is coded to indicate only which residual coding method is used.

[0119] The embodiment relates to LFNTS (Low Frequency Non-Separable Transform). In this embodiment, LFNTS is disabled when the current CU is lossless coded. Corresponding syntax changes are shown in Table 12.

[0120]

[0121] The example relates to joint Cb-Cr coding. The joint coding process of Cb and Cr residuals cannot be reversed. By using Equations 1 and 2 where resCb=resCr, resjoint=0, recCb=predCb, and recCr=predCr are obtained. In the example, joint Cb-Cr coding is disabled at the CU level when cu_transquant_bypass_flag is true. The corresponding syntax changes are shown in Table 13.

[0122]

[0123] When mathematical formula 1 separates resjoint_Cb and res Joint_Cr as in mathematical formula 3.

[0124]

[0125] With this proposed modification, as shown in Equation 4, the process can be reversed, but it is lossy due to rounding errors.

[0126]

[0127] This variation can be used for lossless coding when the second lossless coding process is performed as described later.

[0128] Adaptation of VVC tools for lossless coding

[0129] Referring to FIG. 10a, this case corresponds to step (406).

[0130] In the examples, lossless transformations are added to the MTS transformation set. For lossless coding, these transformations will be selected. Examples of lossless transformations are the denormalized Walsh-Hadamard transformation and the denormalized Haar transformation. Examples of near-lossless transformations are the normalized Walsh-Hadamard transformation and the normalized Haar transformation.

[0131] Lossless transformations can be obtained by Walsh-Hadamard or Haar transformations. The denormalized transformations of Walsh-Hadamard and Haar consist of ±1s and 0s, as shown in Equations 5 and 6. For example, the denormalized 4x4 Walsh-Hadamard matrix is ​​as follows:

[0132]

[0133] Meanwhile, the denormalized 4x4 Haar matrix is ​​as follows:

[0134]

[0135] Lossless reconstruction can be achieved using Walsh-Hadamard or Haar transformations. To illustrate this, consider an example of a 2x2 transformation taking residual samples (r0 and r1). The transformation matrices for both Haar and Walsh-Hadamard are as follows:

[0136]

[0137] The transformation coefficients (c0 and c1) are obtained by the following:

[0138]

[0139] The inverse transformation is performed in this way:

[0140]

[0141] In another example of a 4x4 Walsh-Hadamard transformation, for the transformation of four residual samples (r0, r1, r2, r3), the transformation coefficients c0, c1, c2, and c3 are obtained by multiplying the matrix by the coefficient vector:

[0142]

[0143] The inverse transformation is performed in this way:

[0144]

[0145] Finally, to transform the four residual samples using the Haar transform, the following is used:

[0146]

[0147] The inverse transformation is performed in this way:

[0148]

[0149] Denormalized Haar or Walsh-Hadamard transforms can sometimes increase the bitrate because the dynamics of the coefficients increase. For example, when using a 4x4 denormalized Walsh-Hadamard matrix, the coefficient energy can be up to twice the energy of the residual samples. Therefore, a normalized matrix is ​​achieved by dividing the denormalized matrix by 2. However, doing so results in a loss of integer representation. Consider the same example to transform four residual samples using a "normalized" Walsh-Hadamard transform:

[0150]

[0151] Since integer representations are used for conversion factors, rounding causes an error of ±1 / 2. Therefore, an additional coding step is required to code the reminder after dividing by 2. This can be done in the following way:

[0152] First, calculate the division reminders for each coefficient:

[0153]

[0154] Here, the reminders (rem_c0, rem_c1, rem_c2 and rem_c3) can take values ​​of (-1, 0 or 1).

[0155] The reminders are subsequently encoded into a bitstream. First, a valid reminder bit indicating whether the reminder is non-zero is encoded, and then, if valid, the sign of the reminder (0 for negative and 1 for positive) is encoded. The syntax is shown in Table 15. Differential Pulse-Code Modulation (DPCM) can also be used to apply correlation between reminders. Once the reminders are decoded, the decoder can calculate the inverse transform in this manner:

[0156]

[0157] This technique can be used for other transformation sizes, but normalization cannot always be achieved. For example, a 2x2 Walsh-Hadamard transformation matrix requires division by the square root of 2 for normalization, which always results in loss because the reminder part is not simply ±0.5. However, dividing by 2, as in the 4x4 case, is beneficial because it reduces the increased dynamics of the transformation factors.

[0158] This technique is also applicable to the Haar transform, but it does not lead to a normalized transform due to the inhomogeneous norm of the Haar transform matrix. For example, the 4x4 Haar transform is described.

[0159]

[0160] Reminders are calculated in this way:

[0161]

[0162] Here, the reminders take values ​​of (0, -1, and 1). On the decoder side, the decoded reminders are used in the inverse transformation process in this way:

[0163]

[0164] By Haar, Walsh-Hadamard, and the absence of transformation tools, there can be multiple methods for transforming residual data in a lossless manner. In total, there are nine options for horizontal and vertical transformations of two-dimensional residual signals. The following table provides their details:

[0165]

[0166] A simple way to utilize these transformations is for the encoder to select the best transformation pair—the one that reduces the rate—and encode the index of the used pair, enabling the decoder to infer the used pair and perform the inverse transformation. In VTM, the current transformation pairs are the core DCT2 transformations for horizontal and vertical transformations, or a set of additional DST7 and DCT8 transformations known as MTS (Multiple Transform Selection). MTS can be switched on or off by a high-level flag. To be consistent with this design, the default lossless transformation is No-transform for both horizontal and vertical, and the "Multiple Transforms" are the Walsh-Hadamard and Haar transformations. These can also be controlled by the high-level flag "Lossless_MTS_Flag". Thus, the table of transformation selections can be modified as follows:

[0167]

[0168] Typically, two denormalized transformations fit residuals with small dimensions well. For example, they can be used up to a size of 8x8 or 4x4.

[0169] Block level with secondary lossless coding

[0170] Referring to FIG. 10a, this case corresponds to step (407).

[0171] If the CU is lossy coded but has an error limited below a given threshold, typically when the absolute value of the error is 1 or less, an additional coding stage may be introduced to allow lossless coding, as shown in Equation 20. The error is measured as the difference between the original pixel and the reconstructed sample.

[0172]

[0173] For example, if the DCT or DST transform is used with the same quantization step as 1, second-order lossless coding can be applied after lossy coding.

[0174] FIG. 15 illustrates an example of a simplified block diagram of second-order lossless coding. The input to the process is a bitstream. Data is decoded from the bitstream in step (700). Specifically, this provides quantized transformed coefficients, a transformation type, and newly introduced second-order coefficients. In step (701), inverse quantization is applied to the quantized transformed coefficients. In step (702), inverse transformation is applied to the resulting transformed coefficients. The resulting residual is added in step (703) to the prediction signal from the intra or inter prediction of step (704). The result is a reconstructed block. When the I_PCM mode is selected in step (710), the sample values ​​are decoded directly without entropy decoding. When the lossless coding mode is selected, steps (701 and 702) are skipped. When the transformation skip mode is selected, step (702) is skipped. When a block level having a second transformation mode is selected, step (701) is skipped, and in step (720), an inverse transformation is applied to obtain a first residual, and a second lossless residual is added to the first residual. Then, the sum of the resulting residuals is added to the predicted signal in step (703).

[0175] As the error is limited to below a given threshold, the syntax for coding these small residuals is very simple, and basically, when the threshold is 1, only a significant flag is needed, and a sign flag may be needed to code secondary residuals, and the syntax is shown in Table 16.

[0176]

[0177] This second-order lossless coding can also be applied by regions instead of by blocks.

[0178] Area level signaling

[0179] In use cases for lossless coding, the entire region is coded losslessly, and it is highly likely that some lossless coded blocks will not mix with lossy coded blocks. To handle this case, the actual syntax needs to code a cu_transquant_bypass_flag for each CU, which can be costly.

[0180] In the first embodiment, it is proposed to move the flag before split_cu_flag and code it so that all child CUs from the parent CU can share region_transquant_bypass_flag, and the associated syntax is shown in Table 17.

[0181]

[0182] FIG. 16 illustrates an exemplary flowchart of the parsing process for region_transquant_bypass_flag and split_cu_flag when region-level signaling is used. When parsing syntax for a Coding Tree Unit, the first step (500) is to initialize two variables to false: isRegionTransquantCoded, which indicates whether a flag indicating lossless coding has already been coded for the current region, and isRegionTransquant, which indicates whether the current region is lossless coded. Then, step (501) checks whether lossless coding is enabled at the picture level (Transquant_bypass_enabled_flag is true) and whether region_transquant_bypass_flag has not yet been coded for the current region (isRegionTransquantCoded is false). If such conditions are true, the flag region_transquant_bypass_flag is parsed in step (502). Otherwise, step (504) parses split_cu_flag. After step (502), step (503) checks whether region_transquant_bypass_flag is true. If region_transquant_bypass_flag is true, the variable isRegionTransquant is set to true in step (504). Then, split_cu_flag is parsed in step (504). If split_cu_flag is true, the process returns to step (500); otherwise, the tree is no longer split and the process terminates (step 505). The syntax of the current coding unit can be parsed.

[0183] Lossless coding in the case of a binary tree

[0184] Figure 17 illustrates different partitions for luminance and chroma in the case of a dual tree. In VVC, a coding structure called a dual tree allows trees to be coded separately for luminance and chroma for intra-slices.

[0185] In the example, cu_transquant_bypass_flag is parsed individually for each tree, allowing for more flexibility in signal lossless coding.

[0186] In a variant embodiment, cu_transquant_bypass_flag is not coded for the chroma tree but is inferred from the lumina tree. For a given chroma CU, if one of the juxtaposed lumina CUs is losslessly coded, the current chroma CU is also losslessly coded. The figure illustrates different partitions for the lumina and chroma trees, for example, if lumina CU L4 is losslessly coded, chroma CUs C3 and C4 are losslessly coded. In another example, if lumina CU L3 is losslessly coded, chroma CU C2 is also losslessly coded, even if the juxtaposed lumina CU L2 is not.

[0187] This application describes various aspects, including tools, features, embodiments, models, approaches, etc. Many of these aspects are described by specificity and, at least to demonstrate individual characteristics, are described in a manner that often sounds limiting. However, this is for clarity in the description and does not limit the application or scope of such aspects. In practice, all different aspects may be combined and exchanged to provide additional aspects. Furthermore, aspects may also be combined and exchanged with aspects described in earlier applications.

[0188] The embodiments described and considered in this application may be implemented in many different forms. FIGS. 1a, FIGS. 1b, and FIGS. 2 provide some embodiments, but other embodiments are considered, and the discussion of these figures does not limit the scope of the embodiments. At least one of the embodiments generally relates to video encoding and decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream. These and other embodiments may be implemented as a method, an apparatus, a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium storing a bitstream generated according to any of the described methods.

[0189] Various methods have been described herein, and each of the methods comprises one or more steps or actions to achieve the described method. If a specific order of steps or actions is not required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0190] The various methods and other embodiments described in this application may be used to modify modules of a video encoder (100) and a decoder (200), such as motion compensation and motion estimation modules (170, 175, 275), as illustrated in FIGS. 1a and 1b. Furthermore, the embodiments are not limited to VVC or HEVC and may be applied to other standards and recommendations, whether existing or future developments, and any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the embodiments described in this application may be used individually or in combination.

[0191] Various numerical values ​​are used in this application. Specific values ​​are for illustrative purposes only, and the described embodiments are not limited to these specific values.

[0192] Various implementations involve decoding. As used in this application, “decoding” may include all or part of the processes performed on a received encoded sequence, for example, to produce a final output suitable for display. In various embodiments, these processes include one or more of the processes typically performed by a decoder. In various embodiments, such processes also, or alternatively, include the processes performed by the decoder of the various implementations described in this application.

[0193] As additional examples, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to generally refer to a broader decoding process will be apparent based on the context of the specific descriptions and will be well understood by a person skilled in the art.

[0194] Various implementations involve encoding. In a manner similar to the above discussion regarding "decoding," "encoding" as used in this application may include all or part of the processes performed on, for example, an input video sequence to generate an encoded bitstream. In various embodiments, these processes include one or more of the processes typically performed by an encoder. In various embodiments, such processes also, or alternatively, include processes performed by the encoder of the various implementations described in this application.

[0195] As an additional example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in yet another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or to generally refer to a broader decoding process will be evident based on the context of the specific descriptions and will be well understood by a person skilled in the art.

[0196] It should be noted that the syntax elements used in this specification are descriptive terms. Accordingly, they do not exclude the use of other syntax element names.

[0197] It must be understood that when a drawing is presented as a flowchart, it also provides a block diagram of the corresponding device. Similarly, it must be understood that when a drawing is presented as a block diagram, it also provides a flowchart of the corresponding method / process.

[0198] Various embodiments refer to rate distortion optimization. In particular, during the encoding process, when computational complexity constraints are often given, a balance or trade-off between rate and distortion is usually considered. Rate distortion optimization is typically formulated as minimizing the rate distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving the rate distortion optimization problem. For example, these approaches may be based on extensive testing of all encoding options, including all considered modes or coding parameter values, along with a complete evaluation of the coding cost and associated distortion of the reconstructed signal after coding and decoding. To reduce encoding complexity, a faster approach may also be used, specifically by calculating approximated distortion based on predicted or predicted residual signals rather than the reconstructed ones. A combination of these two approaches may also be used, for example, by using approximated distortion for only some of the possible encoding options and full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches use any of the various techniques to perform optimization, but optimization does not necessarily have to be a complete evaluation of both coding costs and associated distortions.

[0199] This application describes various aspects, including tools, features, embodiments, models, approaches, etc. Many of these aspects are described by specificity and, at least to demonstrate individual characteristics, are described in a manner that often sounds limiting. However, this is for clarity in the description and does not limit the application or scope of such aspects. In practice, all different aspects may be combined and exchanged to provide additional aspects. Furthermore, aspects may also be combined and exchanged with aspects described in earlier applications.

[0200] The implementations and aspects described herein may be implemented, for example, as methods or processes, devices, software programs, data streams, or signals. Although discussed only in the context of a single form of implementation (e.g., discussed only as a method), implementations of the discussed features may also be implemented in other forms (e.g., devices or programs). Devices may be implemented, for example, as appropriate hardware, software, and firmware. Methods may be implemented as processors, for example, referring generally to processing devices, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include, for example, communication devices such as computers, tablets, smartphones, cell phones, PDAs (portable / personal digital assistants), and other devices that facilitate the communication of information between end users.

[0201] References to "one embodiment," "an embodiment," "an implementation," or "an implementation," as well as other variations thereof, imply that specific features, structures, characteristics, etc., described in relation to the embodiment are included in at least one embodiment. Accordingly, the appearance of phrases such as "in one embodiment," "in an embodiment," "in an implementation," or "in an implementation," as well as any other variations, at various locations throughout this application, do not necessarily all refer to the same embodiment.

[0202] Additionally, the present application may refer to "determining" various pieces of information. Determining information may include, for example, estimating information, calculating information, predicting information, or retrieving information from memory.

[0203] Additionally, the present application may refer to "accessing" various pieces of information. Accessing information may include, for example, receiving information, retrieving information (for example, from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0204] Additionally, the present application may refer to "receiving" various pieces of information. Receiving is intended in a broad sense, as in "accessing." Receiving information may include, for example, accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" is typically involved in one or another manner during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0205] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," "frame," "slice," and "tile" may be used interchangeably. Although not essential, the term "reconstructed" is typically used on the encoder side, while "decoded" is used on the decoder side.

[0206] For example, in the cases of "A / B", "A and / or B", and "at least one of A and B", it should be noted that the use of any of " / ", "and / or", and "at least one of ~" is intended to include the selection of only the first listed option (A), the selection of only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C", these phrases are intended to include the selection of only the first listed option (A), the selection of only the second listed option (B), the selection of only the third listed option (C), the selection of only the first and second listed options (A and B), the selection of only the first and third listed options (A and C), the selection of only the second and third listed options (B and C), or the selection of all three options (A, B, and C). This can be extended as many items are listed, as is obvious to a person skilled in the art in this field and related fields.

[0207] Additionally, as used herein, the word "signal" refers, among other things, to indicating something to a corresponding decoder. For example, in a specific embodiment, the encoder signals a specific one of the illumination compensation parameters. In this way, in one embodiment, the same parameter is used on both the encoder side and the decoder side. Thus, for example, the encoder may transmit the specific parameter to the decoder (explicit signaling) so that the decoder can use the same specific parameter. Conversely, if the decoder already has other things in addition to the specific parameter, signaling without transmission may be used simply to allow the decoder to know and select the specific parameter (implicit signaling). By avoiding the transmission of any actual functions, bit saving is realized in various embodiments. It will be understood that signaling can be achieved in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder. While the foregoing relates to the verb form of the word "signal," the word "signal" may also be used as a noun in this specification.

[0208] As is apparent to those skilled in the art, implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiment. Such a signal may be formatted as an electromagnetic wave or as a baseband signal, for example (e.g., using the radio frequency portion of the spectrum). Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted via various different wired or wireless links as known. The signal may be stored on a processor-readable medium.

Claims

Claim 1 A method for determining the type of residual coding for a block of an image or video, comprising: a step of obtaining first information for determining whether the block is a transform skip block; a step of obtaining second information for determining whether normalized residual coding is used for transform skip blocks; and a step of determining that normalized residual coding must be used instead of transform skip residual coding that should be used for the block when the first information indicates that the block is a transform skip block and the second information indicates that normalized residual coding is used for transform skip blocks. Claim 2 In claim 1, the coding is a method based on the Versatile Video Coding specification. Claim 3 A method for encoding coefficients for a block of pixels of an image or video, wherein the type of residual coding is determined by the method according to claim 1. Claim 4 A method for decoding coefficients for a block of pixels of an image or video, wherein the type of residual coding is determined by the method according to claim 1. Claim 5 A device for encoding coefficients for a block of pixels of an image or video, wherein the type of residual coding is determined by the method according to claim 1. Claim 6 A device for decoding coefficients for a block of pixels of an image or video, wherein the type of residual coding is determined by the method according to claim 1. Claim 7 A non-transient computer-readable medium comprising program code instructions for implementing the steps of the method according to claim 1 when executed by a processor. Claim 8 A computer program stored on a recording medium containing program code instructions for implementing the steps of the method according to claim 1 when executed by a processor. Claim 9 delete Claim 10 delete Claim 11 delete

Images