Updating the mapping function within the loop

Abstract

A method and apparatus for encoding or decoding video are provided, in which an intraloop luma mapping is used to encode or decode one or more blocks of video. In some embodiments, for at least one block of video, the update of at least one intraloop luma mapping function is determined based on adjacent reconstructed samples of at least one block, and at least one block is encoded or decoded using at least one updated mapping function.

Description

[Technical Field]

[0001] This application claims the benefit of European Application No. 23306414.6, filed on 24 August 2023, which is incorporated herein by reference in its entirety.

[0002] This embodiment relates, in general, to video compression. This embodiment relates to a method and apparatus for encoding or decoding images or videos. More specifically, this embodiment relates to updating a luma mapping function within a coding loop. [Background technology]

[0003] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transformation that leverage the spatial and temporal redundancy of video content. Generally, intra or inter-prediction is used to leverage intra or inter-picture correlation, and then the difference between the original block and the predicted block, often referred to as the prediction error or prediction residual, is transformed, quantized, and entropy-coded. In inter-prediction, the motion vectors used in motion compensation are often predicted from motion vector predictors. To reconstruct the video, the compressed data is decoded by the reverse process corresponding to entropy coding, quantization, transformation, and prediction. [Overview of the project]

[0004] According to one embodiment, a method for encoding or decoding video is provided. This method includes determining at least one mapping function for at least one block of video using at least one template sample of the block, and encoding or decoding at least one block using at least one mapping function.

[0005] In another embodiment, an apparatus for encoding or decoding video is provided. This apparatus comprises one or more processors capable of determining at least one mapping function for at least one block of video using at least one template sample of the block, and encoding or decoding at least one block using at least one mapping function.

[0006] In some embodiments, the mapping function is a forward mapping function, and encoding or decoding at least one block using the mapping function includes applying the mapping function to the predicted blocks of at least one block. In other embodiments, the mapping function is a reverse mapping function, and encoding or decoding at least one block using the mapping function includes applying the mapping function to the reconstruction of at least one block.

[0007] In some embodiments, determining the mapping function corresponds to updating a default mapping function, such as a mapping function defined by an LMCS tool, such as a mapping function calculated on the encoder side, signaled to the bitstream by the encoder, and then decoded and reconstructed by the decoder.

[0008] In other embodiments, the mapping function is determined from the update of the identity mapping function.

[0009] Further embodiments that may be used individually or in combination are described herein.

[0010] One or more embodiments also provide a computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform a method for encoding or decoding video in accordance with any of the embodiments described herein. One or more of the present embodiments also provide a non-transitory computer-readable medium and / or a computer-readable storage medium storing instructions for encoding or decoding video according to the methods described herein.

[0011] One or more embodiments also provide a computer-readable storage medium storing a bitstream generated according to the methods described herein. One or more embodiments also provide a method and apparatus for transmitting or receiving a bitstream generated according to the above-mentioned method. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] [Figure 1] Fig. 1 is a block diagram of a system in which aspects of the present embodiment may be implemented. [Figure 2] Fig. 2 is a block diagram of an embodiment of a video encoder in which aspects of the present embodiment may be implemented. [Figure 3] Fig. 3 is a block diagram of an embodiment of a video decoder in which aspects of the present embodiment may be implemented. [Figure 4] Fig. 4 is a block diagram of an embodiment of a video decoder in which another aspect of the present embodiment may be implemented. [Figure 5] Fig. 5 is a block diagram of an embodiment of a video decoder in which another aspect of the present embodiment may be implemented. [Figure 6] Fig. 6 is a block diagram of an embodiment of a video decoder in which another aspect of the present embodiment may be implemented. [Figure 7] Fig. 7 is a block diagram of an embodiment of a video decoder in which another aspect of the present embodiment may be implemented. [Figure 8] Fig. 8 shows an example of mapping update of a forward mapping function according to an embodiment. [Figure 9]An example of mapping update of an inverse mapping function according to one embodiment is shown. [Figure 10] An example of a reference region in a reference picture for updating a mapping function according to one embodiment is shown. [Figure 11] An example of a flowchart of a method for encoding at least one block of video according to one embodiment is shown. [Figure 12] An example of a flowchart of a method for decoding at least one block of video according to one embodiment is shown. [Figure 13] An example of a flowchart of a method for encoding at least one block of video according to another embodiment is shown. [Figure 14] An example of a flowchart of a method for decoding at least one block of video according to another embodiment is shown. [Figure 15] A block diagram of a system in which aspects of the present embodiment may be implemented according to another embodiment is shown. [Figure 16] Two remote devices communicating via a communication network according to an example of the present principles are shown. [Figure 17] A signal syntax according to an example of the present principles is shown. DETAILED DESCRIPTION OF EMBODIMENTS

[0013] The present application describes various aspects including tools, features, embodiments, models, approaches, etc. Many of these aspects are described specifically, and are often described in a way that may sound restrictive, at least for the purpose of illustrating individual features. However, this is for the sake of clarity of description, and does not limit the application or scope of these aspects. In fact, all different aspects may be combined and interchanged to provide further aspects. Furthermore, these aspects may also be combined with or interchanged with aspects described in earlier applications.

[0014] The embodiments described and intended in this application can be realized in many different forms. Figures 1, 2, and 3 below provide some embodiments, but other embodiments are intended, and the considerations in Figures 1, 2, and 3 are not intended to limit the scope of implementation forms. At least one of these embodiments generally relates to video encoding and decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream. These and other embodiments can be realized as a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the methods, apparatus, or described methods, and / or a computer-readable storage medium storing a bitstream generated according to any of the described methods.

[0015] In this application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, and the terms “image,” “picture,” and “frame” may be used interchangeably.

[0016] Various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a particular order of steps or actions is required for the normal operation of the method, the order and / or use of any particular steps and / or actions may be modified or combined. Furthermore, terms such as “first,” “second,” etc., may be used in various embodiments to modify elements, components, steps, actions, etc., such as “first decryption” and “second decryption.” The use of such terms does not imply a modified order of actions unless specifically required. Thus, in this example, the first decryption does not need to be performed before the second decryption, but may occur, for example, before, during, or overlapping with the second decryption.

[0017] These embodiments are not limited to VVC or HEVC and can be applied, for example, to other standards and recommendations, whether prior to exist or to be developed in the future, as well as to any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise specifically indicated or technically excluded, the embodiments described herein can be used individually or in combination.

[0018] Figure 1 shows a block diagram of an example of a system in which various embodiments and forms can be implemented. System 100 may be embodied as a device comprising various components described below and configured to perform one or more of the embodiments described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of System 100 may be embodied individually or in combination as a single integrated circuit, a plurality of ICs, and / or individual components. For example, in at least one embodiment, the processing and encoder / decoder elements of System 100 are distributed across a plurality of ICs and / or individual components. In various embodiments, System 100 is communicably coupled to other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, System 100 is configured to implement one or more of the embodiments described in this application.

[0019] System 100 includes at least one processor 110, which is configured to execute instructions loaded therein to realize, for example, various embodiments described in this application. The processor 110 may include embedded memory, input / output interfaces, and various other circuits known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes a storage device 140 which may include non-volatile memory and / or volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drives, and / or optical disk drives. The storage device 140 may, in non-limiting examples, include an internal storage device, a removable storage device, and / or a network-accessible storage device.

[0020] System 100 includes, for example, an encoder / decoder module 130 configured to process data to provide encoded or decoded video, the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents one or more modules that may be included in a device for performing encoding and / or decoding functions. As is known, the device may include one or both of the encoding module and the decoding module. Furthermore, the encoder / decoder module 130 may be implemented as a separate element of System 100, or it may be incorporated into the processor 110 as a combination of hardware and software, as is known to those skilled in the art.

[0021] To perform the various embodiments described in this application, program code to be loaded into the processor 110 or encoder / decoder 130 may be stored in the storage device 140 and subsequently loaded onto the memory 120 for execution by the processor 110. According to various embodiments, one or more of the processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more of various items during the execution of the processes described in this application. Such stored items may include, but are not limited to, input video, decoded video or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, expressions, operations, and arithmetic logic.

[0022] In some embodiments, internal memory of the processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory of the processing device (for example, the processing device can be either the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory may be memory 120 and / or storage device 140, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video coding and decoding operations such as MPEG-2, HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a standard developed by JVET, i.e., the Joint Video Experts Team, also known as H.266).

[0023] Inputs to the elements of system 100 can be provided through various input devices, as shown in block 105. Such input devices include, but are not limited to, (i) a radio frequency (RF) section for receiving RF signals wirelessly transmitted by, for example, a broadcasting station; (ii) a component (COMP) input terminal (or a set of COMP input terminals); (iii) a Universal Serial Bus (USB) input terminal; and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Another example not shown in Figure 1 is composite video.

[0024] In various embodiments, the input device of block 105 has associated input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also called selecting a signal or band-limiting a signal to a certain frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band so as to select a signal frequency band that may (for example) be called a channel in certain embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements that perform these functions, for example, a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to the baseband) or to the baseband. In one embodiment of a set-top box, the RF unit and its associated input processing elements perform frequency selection by receiving, filtering, down-converting, and filtering again to a desired frequency band of RF signals transmitted via a wired (e.g., cable) medium. In various embodiments, the order of these elements (and other elements) may be rearranged, some of these elements may be removed, and / or other elements performing similar or different functions may be added. Adding elements may include inserting elements between existing elements, for example, an amplifier and an analog-to-digital converter. In various embodiments, the RF unit includes an antenna.

[0025] Furthermore, the USB and / or HDMI terminals may include their respective interface processors for connecting the system 100 to other electronic devices over USB and / or HDMI connections. It should be understood that various forms of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, within a separate input processing IC or within the processor 110. Similarly, forms of USB or HDMI interface processing may be implemented as needed, within a separate interface IC or within the processor 110. For example, demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including a processor 110 and an encoder / decoder 130, which may, as needed, process the data stream for display on an output device.

[0026] Various elements of system 100 may be provided within an integrated housing, where the various elements are interconnected and data can be transmitted between them using an internal bus known in the art, such as a suitable connection arrangement 115, including an I2C bus, wiring, and a printed circuit board.

[0027] System 100 includes a communication interface 150 that enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 190. The communication interface 150 may also include, but is not limited to, a modem or a network card, and the communication channel 190 may be implemented, for example, in a wired and / or wireless medium.

[0028] In various embodiments, data is streamed to system 100 using a Wi-Fi network such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 190 and a communication interface 150 adapted for Wi-Fi communication. In these embodiments, the communication channel 190 is typically connected to an access point or router that provides access to an external network, including the Internet, to enable streaming of the application and other over-the-top communication. In other embodiments, the streamed data is provided to system 100 using a set-top box that distributes data via an HDMI connection of input block 105. Still other embodiments provide the streamed data to system 100 using an RF connection of input block 105. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0029] System 100 can provide output signals to various output devices, including a display 165, a speaker 175, and other peripheral devices 185. In various embodiments, the display 165 includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 165 may be for a television, tablet, laptop, mobile phone, or other device. The display 165 may also be integrated with other components (for example, in a smartphone) or separate (for example, an external monitor for a laptop). Other peripheral devices 185, in various examples of embodiments, include one or more of a standalone digital video disc (or digital multi-purpose disc) (both terms DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 185 that provide functions based on the output of System 100. For example, a disc player performs the function of playing back the output of System 100.

[0030] In various embodiments, control signals are communicated between the system 100 and the display 165, speaker 175, or other peripheral devices 185 using signaling such as AV Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicably coupled to the system 100 via dedicated connections through their respective interfaces 160, 170, and 180. Alternatively, output devices may be connected to the system 100 using a communication channel 190 via a communication interface 150. The display 165 and speaker 175 may be integrated into a single unit with other components of the system 100 in an electronic device such as a television. In various embodiments, the display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0031] The display 165 and speaker 175 may, alternatively, be separate from one or more of the other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments where the display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, for example, including an HDMI port, a USB port, or a COMP output.

[0032] The embodiments can be implemented by computer software implemented by the processor 110, by hardware, or by a combination of hardware and software. In a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 120 can be of any type appropriate for the technical environment and, in a non-limiting example, can be implemented using any appropriate data storage technology such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 110 can be of any type appropriate for the technical environment and, in a non-limiting example, can include one or more of microprocessors, general-purpose computers, dedicated computers, and processors based on multi-core architectures.

[0033] Figure 2 shows an example of a block-based hybrid video encoder 200. While variations of this encoder 200 are intended, the encoder 200 is described below for clarity without explaining all anticipated variations.

[0034] In some embodiments, Figure 2 also shows encoders that have been improved upon the HEVC or VVC standard (Multipurpose Video Coding, standard ITU-TH.266, ISO / IEC23090-3, 2020), or encoders that employ HEVC or VVC-like technologies, such as the ECM encoder currently under development by JVET (Joint Video Exploration Team).

[0035] Figure 2 shows an example of a block-based hybrid video encoder 200. Before encoding, the video sequence may undergo pre-encoding processing (201), for example, by applying a color conversion to the input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0), or by remapping the input picture components (e.g., using histogram equalization of one of the color components) to obtain a signal distribution that is more resilient to compression. Metadata can be associated with the pre-processing and attached to the bitstream.

[0036] In encoder 200, the picture is encoded by encoder elements as described below. The picture is first analyzed to determine when and what function luma mapping (LMCS) should be applied (295). Step 295 also applies forward luma mapping to the slice or picture, if applicable. The picture to be encoded is divided (202) and processed in units of coding units (CUs), for example. Each unit is encoded using either intra-mode or inter-mode, for example. If the unit is encoded in intra-mode, intra-prediction is performed (260). In inter-mode, motion estimation (275) and motion compensation (270) are performed. If LMCS is applied, forward luma mapping of motion compensation prediction is applied (285). The encoder decides whether to use intra-mode or inter-mode for encoding the unit (205), and indicates the intra / inter decision, for example, by a prediction mode flag. The predicted residual is calculated, for example, by subtracting the predicted blocks from the original image blocks (210).

[0037] Next, the predicted residuals are transformed (225) and quantized (230). In addition to the quantized transformation coefficients, other syntactic elements such as motion vectors and picture segmentation information are entropy coded to output a bitstream (245). The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transformation and quantization, i.e., the residuals are coded directly without applying any transformation or quantization process.

[0038] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantized transformation coefficients are inversely quantized (240), inversely transformed (250), and the predicted residuals are decoded. The decoded predicted residuals and the predicted blocks are combined (255) to reconstruct the image blocks. If LMCS is applied, inverse lmapping is performed on the reconstructed picture (290). Next, an in-loop filter (265) is applied to the reconstructed picture to reduce encoding artifacts, for example, by performing deblocking / SAO (sample adaptive offset) / ALF (adaptive loop filter) filtering. The filtered image is stored in the reference picture buffer 280.

[0039] Figure 3 shows a block diagram of an exemplary video decoder 300. In the decoder 300, the bitstream is decoded by the decoder elements as described below. The video decoder 300 generally performs a decoding path that is the reverse of the encoding path described in Figure 2. The encoder 200 also generally performs video decoding as part of encoding the video data.

[0040] Specifically, the decoder input includes a video bitstream, which may be generated by a video encoder 200. First, the bitstream is entropy-decoded to obtain transformation coefficients, prediction modes, motion vectors, and other coded information (330). Picture partitioning information indicates how the picture is partitioned. Thus, the decoder may partition the picture according to the decoded picture partitioning information (335). The transformation coefficients are inversely quantized (340) and inversely transformed (350) to decode the prediction residuals. The image blocks are reconstructed by combining the decoded prediction residuals with the predicted blocks (355). If LMCS is applied, inverse lumar mapping is performed on the reconstructed picture (390). Predicted blocks may be obtained from intra-prediction (360) or motion-compensated prediction (i.e., inter-prediction) (375) (370). If LMCS is applied, forward lumar mapping of motion-compensated prediction is applied (325). The loop filter (365) is applied to the reconstructed image. The filtered image is stored in the reference picture buffer (380). Note that for a given picture, the contents of the reference picture buffer 380 on the decoder 300 side are identical to the contents of the reference picture buffer 280 on the encoder 200 side for the same picture.

[0041] The decoded picture may undergo further post-decoded processing (385), such as reverse color transformation (e.g., conversion from YCbCr4:2:0 to RGB4:4:4) or reverse luma mapping, which is the reverse of the forward luma mapping process performed in pre-encoding processing (201). The post-decoded processing can use the metadata derived in pre-encoding processing and signaled within the bitstream.

[0042] Some embodiments described herein relate to in-loop luma mapping, and more specifically to updating the luma mapping function within the coding loop on both the encoder and decoder sides based on samples already reconstructed in the current slice or picture or other slices or pictures.

[0043] The objective is to improve compression efficiency, that is, to reduce the bitrate while maintaining quality, or equivalently, to improve quality while maintaining the bitrate.

[0044] Any one of the embodiments described herein can be implemented, for example, in the forward luma mapping module (285, 325) or reverse luma mapping module (290, 390) of a video encoder 200 or video decoder 300, or in an additional module positioned before these modules within the video encoder or decoder.

[0045] Luma mapping with chroma scaling (LMCS) In most SDR sequences for captured content, the data range of the lumens signal does not correspond to the 1024 levels expected for a 10-bit signal, but rather to a limited range of 64-940 (also known as the standard range). For HDR signals, when in PQ format, the actual code range is further reduced because the maximum value of 1023 corresponds to a peak luminance of 10000 cd / m2, which is never actually used. Instead, grading more frequently uses peak luminances up to 1000, 4000, or 5000 cd / m2, which result in codewords around 720, 850, and 875, respectively. In VTM and ECM, the LMCS function allows for modification of the content's lumens signal to better utilize the 10-bit capability. The mapping function is applied to the lumens signal using a pre-calculated function or LUT calculated on the encoder side. The value of the function or LUT is defined by the encoder based on specific characteristics of the content. For this purpose, the 10-bit representation (1024 levels) is divided into a given number of N segments (e.g., 64 if 16 intervals are used), resulting in 1024 / N levels. Thus, the Luma mapping function is defined as a piecewise linear function defined by N segments.

[0046] Next, for each input value x ([0, 1023] in the case of a 10-bit signal), a forward mapping LUT can be constructed from a piecewise linear function. In parallel, an inverse mapping LUT is also generated based on the same piecewise linear function. This is used whenever a conversion from a mapped region to an unmapped region is required. Forward and inverse mapping can be applied to any picture in the video.

[0047] LMCS parameters are signaled by an encoder, typically once per intra-period, in a high-level structure called APS. The table below shows an example of the corresponding syntax as implemented in the VVC / ITU-T H.266 standard.

[0048] [Table 1] Table 1. LMCS Syntax Table lmcs_data(){ Descriptor lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v) lmcs_delta_cw_prec_minus1 ue(v) for(i=lmcs_min_bin_idx;i<=LmcsMaxBinIdx;i++){ lmcs_delta_abs_cw[i] u(v) if(lmcs_delta_abs_cw[i]>0) lmcs_delta_sign_cw_flag[i] u(1) } if(aps_chroma_present_flag){ lmcs_delta_abs_crs u(3) if(lmcs_delta_abs_crs>0) lmcs_delta_sign_crs_flag u(1) } }

[0049] On the decoder side, the decoder decodes this information, reconstructs the forward and reverse mapping functions, and uses these functions during the decoding process (forward mapping is applied to interpredicted samples, and reverse mapping is applied after the reconstruction process and before intra-loop filtering).

[0050] When using the current implementations of forward and reverse mapping function tools in VVC and similar systems, the functions are pre-calculated and do not change or adapt within the slice or picture. Therefore, the mapping functions do not account for the distortions introduced by signal quantization. Furthermore, these methods do not account for local statistical variations in the signal. These drawbacks can affect coding efficiency.

[0051] Some embodiments provide a method and apparatus in which a mapping update step is added before applying a mapping to a coding block being processed. The mapping update step determines an update to the mapping function using, as input signals, predicted samples already processed within a slice or picture and reconstructed samples (typically samples located within the template region of the block). In one embodiment, the mapping update step is applied before the forward mapping step (285, 325) applied to the inter-prediction block. In another embodiment, the mapping update step is applied before the inverse mapping step (290, 390) applied to the reconstructed block. In yet another embodiment, the mapping update step is applied before both the forward mapping step and the inverse mapping step.

[0052] Figures 11 and 12 show, respectively, examples of flowcharts for a method (1100) of encoding (1100) and a method (1200) of decoding (1200) at least one block of video according to one embodiment.

[0053] In this embodiment, the default mapping function is updated on a block-by-block basis.

[0054] On the encoding side, at step 1110, the default mapping function is obtained. The default mapping function is, for example, the Luma mapping function determined for the LMCS tool and signaled in the bitstream. The current picture to be encoded refers to one of the parameter sets signaled in the bitstream to identify the default mapping function.

[0055] At 1120, the update of the default mapping function for the current block of the picture to be encoded is determined based on the template sample of the current block, and at 1130, the current block is encoded using the update of the default mapping function.

[0056] On the decoding side, at 1210, the default mapping function is obtained, for example, by decoding the mapping function parameters encoded in the bitstream referenced by the current picture to be decoded. At 1220, an update to the default mapping function for the current block is determined based on the template sample of the current block, and at 1230, the current block is decoded using the update to the default mapping function.

[0057] Figures 13 and 14 show example flowcharts of methods (1300) for encoding and (1400) for decoding at least one block of video according to a different embodiment, respectively. In this embodiment, the LMCS mapping function is no longer used as a starting point for determining updates to the mapping function on a block basis; rather, the identity mapping function is used.

[0058] To do this, on the encoding side, at 1310, a mapping function is determined for the current block of the picture to be encoded, based on the template sample of the current block, and at 1320, the current block is encoded using the determined mapping function. On the decoding side, at 1410, a mapping function is determined for the current block, based on the template sample of the current block, in the same way as on the encoder side, and at 1420, the current block is decoded using the determined mapping function.

[0059] In some modifications of the embodiments described herein, the mapping function is determined as a lookup table that maps sample values ​​within a range of signal bit depth values ​​(for example, for a 10-bit signal, the range considered may be [0, 1023]). The mapping function is an in-loop mapping function and can be a forward mapping function, a reverse mapping function, or both, as shown in Figures 2 to 7.

[0060] More detailed variations of these embodiments are provided below.

[0061] In the following, we assume that the current block of video is being processed. The following notation is used for this current block. • Use "prediction blocks" as prediction sample blocks obtained from intra-prediction (460, 560) or inter-prediction including motion compensation steps (470, 575). • Use the "reconstructed block" as the reconstructed sample block obtained from the residual and prediction addition step (455, 555). • Use the "filtered block" as the filtered sample block obtained from the in-loop filtering steps (465, 565).

[0062] In one embodiment, the mapping update stage of the forward mapping function is applied before the forward mapping of the interprediction block. The mapping update takes predicted and reconstructed samples from the currently processed region of the slice or picture as input. For example, adjacent predicted and reconstructed samples above and to the left of the interprediction block are used.

[0063] An example of the process is shown in Figure 4 for the encoder. A new mapping update step 495 is added between the motion compensation step 470 and the forward mapping step 485. The mapping update modifies the forward mapping function based on the adjacent predicted and reconstructed samples of the current block ("template predicted samples" and "template reconstructed samples" in Figure 4). In a modified example, adjacent filtered samples from the in-loop filtering step 465 ("template filtered samples" in Figure 4) are also used as input to the mapping update step. The forward mapping (485) is then applied to the predicted samples of the current block using the updated forward mapping function. Adjacent samples may be samples above and to the left of the current block, but may also be samples further away from the same picture or from the reference picture, from a region already processed by the encoder / decoder.

[0064] Figure 5 shows the decoder side. A new mapping update step 595 is added between the motion compensation step 575 and the forward mapping step 525. The mapping update modifies the forward mapping function based on the adjacent predicted and reconstructed samples of the current block ("template predicted samples" and "template reconstructed samples" in Figure 5). In the modified example, adjacent filtered samples from the in-loop filtering step 565 ("template filtered samples" in Figure 5) are also used as input to the mapping update step. Then, the forward mapping (525) is applied to the predicted samples of the current block using the updated forward mapping function.

[0065] This process is also shown in Figure 8. For the current block, the samples of the prediction block (800) have already been generated from the motion compensation step. The reconstructed and filtered samples are not yet available for the current block, but they may be available in the causal region surrounding the current block. The mapping update step (495 for the encoder, 595 for the decoder) takes as input the prediction samples in the causal neighborhood (801) of the prediction block (800) (the gray area around the prediction block, called the prediction template) and the reconstructed samples in the causal neighborhood (802) of the current block (the gray area around the current block, called the reconstructed template). In a modified example, the template prediction samples (801) used to update the mapping function are obtained by applying the intra-prediction or inter-prediction (460, 470) used to encode these samples. In this modified example, the predictions of the template samples can be buffered when these samples are encoded / decoded to avoid the need to recalculate the predictions of the template samples twice to update the mapping function. In another variation, the template prediction sample (801) used to update the mapping function is obtained by applying the same prediction mode (intra or inter-predict) with the same parameters as the prediction for the current block.

[0066] The template-reconstructed samples (802) are obtained at the output of the residual and prediction addition step (455, 555) when reconstructing these samples, before intra-loop filtering is applied to the template-reconstructed samples. Here, the predictions for the template samples are those generated to encode these samples.

[0067] In the modified example, filtered samples (803) within the causal neighborhood of the current block (the gray area around the current block, called the filtered template) can be used in place of, or as a complement to, the samples in the reconstructed template. The template filtered samples (803) are obtained as samples output by the in-loop filter module (465, 565) applied to the template reconstructed samples after the inverse mapping.

[0068] The output of the mapping update step is the updated forward mapping function UFm(.). This function is used in step 485 (encoder) or 525 (decoder) to map the predicted block sample (800), called Pred(x,y), to the output mapped predicted block sample (804), called MapPred(x,y). For any position (x,y) within a block, MapPred(x,y) = UFm(Pred(x,y)).

[0069] In another embodiment, the mapping update stage of the inverse mapping function is applied before the inverse mapping of the reconstructed block. The mapping update takes reconstructed and filtered samples from the already processed region of the current slice or picture as input. For example, adjacent reconstructed and filtered samples from above and to the left of the block are used.

[0070] This process is illustrated in Figure 6 for the encoder. A new mapping update step 496 is added between the residual and prediction addition step 455 and the inverse mapping step 490. The mapping update modifies the inverse mapping function based on the adjacent reconstructed and filtered samples of the current block ("template reconstructed samples" and "template filtered samples" in Figure 6). The updated inverse mapping function is then used to apply the inverse mapping (490) to the reconstructed samples (reconstructed samples) of the current block.

[0071] Figure 7 shows the decoder side. The new mapping update step 596 is added between the residual and prediction addition step 555 and the inverse mapping step 590. The mapping update modifies the inverse mapping function based on the adjacent reconstructed and filtered samples of the current block ("template reconstructed samples" and "template filtered samples" in Figure 7). The updated inverse mapping function is then used to apply the inverse mapping (596) to the reconstructed samples (reconstructed samples) of the current block.

[0072] This process is also shown in Figure 9. For the current block, the reconstructed block samples (900) have already been generated from the residual and predictive sum steps. Filtered samples are not yet available for the current block, but they may be available in the causal region surrounding the current block. The mapping update step (496 for the encoder, 596 for the decoder) takes as input the reconstructed samples (the gray area around the reconstructed block, called the reconstructed template) within the causal neighborhood (902) of the reconstructed block (900), and the filtered samples (the gray area around the current block, called the filtered template) within the causal neighborhood (903) of the current block. Adjacent samples may be above and to the left of the current block, but may be further away from the region already processed by the encoder / decoder, from the same picture, or from the reference picture.

[0073] The template-reconstructed samples (902) are obtained at the output of the residual and prediction addition step (455, 555) when reconstructing these samples, where the predictions for the template samples are those generated to encode these samples. The template-filtered samples (903) are obtained as samples output by the in-loop filter module (465, 565) applied to the template-reconstructed samples after inverse mapping.

[0074] The output of the mapping update step is the updated inverse mapping function UIm(.). This function is used in step 490 (encoder) or 596 (decoder) to reverse map a reconstructed block sample (900) called Rec(x,y) to the output inverse-mapped reconstructed block sample (904) called InvMapRec(x,y). For any position (x,y) within a block, InvMapRec(x,y) = UIm(Rec(x,y)).

[0075] In some embodiments, the use of an updated mapping function instead of the default (unupdated) mapping function is controlled by an indicator. In variations, the indicator is a flag signaled in the bitstream.

[0076] In some variations, signaling can be performed at high levels, such as sequence level (SPS signaling), picture level (PPS signaling), slice level (slice header signaling), or APS level (APS signaling). In other variations, signaling can be performed at local levels, such as CTU or CU level.

[0077] In a modified example, the choice between the default mapping function and the updated mapping function for a given block (CTU or CU) can be inferred on both the encoder and decoder sides based on an analysis of the health of the updated mapping function. This favorably avoids signaling of control flags. An example of checking the health of the updated mapping function is further explained below.

[0078] In a modified example, multimode updates are used, allowing multiple updated mapping functions to be derived, and an index is signaled or inferred at the local level (e.g., CTU or CU) to indicate which mapping function should be used. For example, multiple updated mapping functions may be derived using different sets of template samples. For instance, one first updated mapping function might be derived using upper and left-hand template samples, one second updated mapping function might be derived using upper template samples, and one third updated mapping function might be derived using left-hand template samples. The index might indicate, for example, which of the default mapping function, the first updated mapping function, the second updated mapping function, or the third updated mapping function should be used. When upper or left-hand samples in the template are unavailable, the coding length of the index may be reduced.

[0079] In one embodiment, the updated mapping function replaces the default mapping function for the mapping process used in the next coding block. In other words, the updated mapping function becomes the default mapping function for the next coding block.

[0080] In this embodiment, in order to ensure that applying the forward mapping function fwdLut and then the inverse mapping function invLut yields the identity function, the following inverse mapping function may also need to be updated (except in rare cases due to rounding errors). [0,2 BD For any x in -1], invLut[fwdLut[x]] = x, where BD is the bit depth of the sample.

[0081] Nevertheless, in principle, updating the inverse mapping function should not be necessary even if the forward mapping function is updated. In fact, updating the forward mapping function is done in the forward mapping step to apply a slight correction to the input samples, with the aim of limiting the corrections made in the following steps. For example, if the forward mapping function is updated in step 495, the goal is to bring the forward-mapped predicted signal closer to the forward-mapped original signal, and thus reduce the energy of the residual. For example, if the forward mapping function is updated in step 496, the goal is to bring the inverse-mapped reconstructed signal closer to the original signal, and thus reduce the energy of the correction signal brought about by the subsequent in-loop filter.

[0082] In another embodiment, the default mapping function remains the reference mapping function for the next coding block. Other updates to the mapping function may be determined for the next coding block.

[0083] An example of how to decide how to update the mapping function is described below.

[0084] Here, the default mapping function is assumed to correspond to LUT, defLut[.], where defLut[x] is the mapped version of x.

[0085] The input sample for the process is one of the following: - A set of template samples in in1(x,y) in the unmapped region (for example, in embodiments relating to forward mapping, these may be template prediction samples), and a set of template samples in in2(x,y) in the mapped region (for example, in embodiments relating to forward mapping, these may be template reconstructed samples), -or, a set of template samples in in1(x,y) in the mapped region (for example, in embodiments relating to reverse mapping, these may be template-reconstructed samples), and a set of template samples in in2(x,y) in the reverse-mapped region (for example, in embodiments relating to reverse mapping, these may be template-filtered samples).

[0086] The default mapping function defLut[.] that will be updated is one of the following: - Forward mapping function (e.g., Figures 4, 5, and 8), -or an inverse mapping function (e.g., Figures 6, 7, and 9), - Or both forward and reverse mapping functions.

[0087] The following embodiment provides a method for deriving an updated mapping function updLut[.] from defLut[.], in1(x,y), and in2(x,y) for (x,y) in the template region.

[0088] In one embodiment, the updated mapping function updLut[.] is derived by minimizing the distortion between the samples in in2(x,y) and the samples in in1(x,y) mapped by updLut across the template samples. Two examples of distortion are described below.

[0089]

number

[0090] A modified example to minimize this distortion is described below. Step 1 - Initialization -k=0 (2 BD -1) up to length 2 BDinitialize the three arrays (hereinafter referred to as occurrence, mean, and variance) (where BD is the bit depth of signals in1 and in2). ○occurrence[k] = 0 ○mean[k] = 0 ○variance[k] = 0 - for k = 0 ~ (2 BD - 1), initialize updLut to be equal to defLut. ○updLut[k] = defLut[k] Step 2 - Sample Occurrence Counting - Scan all template pixels (x,y) ○occurrence[in1(x,y)] = occurrence[in1(x,y)] + 1 ○mean[in1(x,y)] = mean[in1(x,y)] + in2(x,y) ○variance[in1(x,y)] = variance[in1(x,y)] + (in2(x,y) * in2(x,y)) Step 3 - Derivation of Sample Mean - for k = 0 ~ (2 BD - 1) ○when occurrence[k] > 0 ■mean[k] = mean[k] ÷ occurrence[k] ·variance[k] = (variance[k] ÷ occurrence[k]) - (mean[k] * mean[k]) ○The symbol "÷" represents floating-point division, and integer implementation can also be considered with some adaptations of these formulas. Step 4 - LUT Update - for k = 0 ~ (2 BD - 1) ○updLut[k] is updated based on mean[k], and optionally variance[k]. For example, updLut[k] = a * updLut[k] + (1 - a) * mean[k] (rounded to the nearest integer) Here, "a" is a parameter in [0,1] that is either predefined by default or depends on the variance [k]. For example, a=0.5 or a=0.75. In another exemplary case, a=0. The purpose of the parameter "a" is to control the deviation from the default LUT defLut. If "a" is close to 1, updLut is constrained to stay close to defLut.

[0091] Alternatively, a = variance[k] ÷ (b + variance[k]), or a = sqrt(variance[k]) ÷ (b + sqrt(variance[k])), where sqrt is the square root operator and b is a positive parameter, for example, equal to 1.

[0092] In the modified example, the occurrence [k] is at the threshold T. occ Below (for example, T occ If k = 4, or size_template / 4 (where size_template is the number of samples in the template), then mean[k] is set to 0.

[0093] In the modified version, a health check of the updated mapping function is applied. For example, if the variance [k] is too high for one or more k (e.g., T var =(2 BD / 16) * (2 BD A given threshold T that can be equal to / 16 var If the value exceeds [a certain threshold], the LUT will not be updated. Alternatively, if the maximum value of the variance [k] is too high, the LUT will not be updated.

[0094] In fact, a value of variance [k] that is too large indicates that the mapped samples are too scattered and that the updated mapping of value k is unreliable.

[0095] In another variation, the health check is based on the amplitude of the mean [k]. For one or more ks, the mean [k] is too high (e.g., T avg =(2 BDFor example, a given threshold T equal to ( / 16) avg The LUT is not updated if the maximum value of the mean[k] is greater than ( ). Alternatively, the LUT is not updated if the maximum value of the mean[k] is too high. In another variation, the LUT is updated, but all mean[k] for any k are equal to the maximum value, e.g., T avg. It will be clipped.

[0096] In another variation, the health check is based on occurrence[k]. The maximum occurrence[k] across all k values ​​is, for example, T occ A threshold T can be equal to =4 or size_template / 4. occ If the value falls below this level, the LUT will not be updated.

[0097] In a particular case of an embodiment of updating a forward mapping function, where sample in1 corresponds to a template-predicted sample (located in an unmapped region) and in2 corresponds to a template-filtered sample (also located in an unmapped region), the goal of the mapping update is to derive an updLut as follows:

[0098]

number

[0099] Another embodiment for deriving the updated mapping function updLut[.] is provided below. In this alternative approach, a single offset "off" is calculated and used for all values ​​identified within the template region. Step 1 - Initialization - Initialize the occurrence parameter "occurrence" and the offset parameter "off". ○ Occurrence = 0 ○ Off = 0 Step 2 - Sample Generation Count - Scan all template pixels (x,y). ○ Occurrence = Occurrence + 1 ○ Off = Off + (in2(x,y) - defLut[in1(x,y)]) Step 3 - Offset Derivation - If occurrence > 0. ○ Off = (Off + Occurrence / 2) / Occurrence Step 4 - LUT update based on offset - Scan all template pixels (x,y). ○updLut[in1(x,y)]=defLut[in1(x,y)]+off ○In some cases, the values ​​around in1(x,y) may also be modified. ■r=max(0,in1(x,y)-K)~min(2 BD -1, in the case of (x, y) + K) updLut[r]=defLut[r]+off Here, K is a predefined parameter, for example, equal to 2, 4, 8, 16, or 32.

[0100] In a modified example, several offsets are considered. For example, the mean of in1 is calculated across the template, and then one first offset is defined for samples below the mean, and one second offset is used for samples above the mean. In another example, the sample range is divided into N sub-intervals (e.g., N=4), and for each sub-interval i (i=0 to N-1) of the index, the offset "off" is estimated. Then, for each r in interval i, updLut[r] is (defLut[r] + off i It will be set to ).

[0101] In some embodiments, a history list of mapping functions is used to provide multiple mapping functions. For example, after processing each CTU, an updated mapping function is derived as described above (using a sample within the processed CTU instead of a template sample surrounding the CTU). The derived mapping function is then inserted into the history list of mapping functions, for example, using a first-in, first-out approach. This favorably allows for the dynamic adaptation of the mapping function used to adapt to local variations in the signal. For the current block, the updated mapping function can be selected from all the mapping functions stored in the history list of mapping functions.

[0102] At the block level (CU or CTU), an index may be signaled to indicate which mapping function in the list is used for the current block.

[0103] In one variation, the default LUT is always included in the list, and sometimes even in the first position of the list. In another variation, the first candidate in the list is the most recently derived and updated mapping function.

[0104] A variation demonstrates that using a single index, the updated mapping function must be computed based on the template sample and not selected from mapping functions previously stored in the list.

[0105] This approach can be applied to forward mapping functions as well as reverse mapping functions. The idea can be extended to other types of regions besides CTUs, such as groups, tiles, subpictures, and slices of CTUs. For example, a mapping function may be updated using a reference region, such as a group of preceding CTUs, tiles, subpictures, or slices. When applied to such a reference region, in one variation, the list may contain only one updated mapping function, which is the most recent updated mapping function from the reference region. In another variation, the list may also include the default mapping function.

[0106] In the embodiments described above, the template sample considered for updating the mapping function is taken from the causal spatial adjacency of the current block for encoding or decoding. In variations, the template sample may also belong to a reference region containing a time block or CTU within a reference frame. For example, a sequence of time blocks or CTUs in a reference picture used to predict the current block is used as the reference region for updating the mapping function of the current block. In another example, a time block or CTU in a reference picture pointed to by a motion vector MV(s) used to predict the current block is used as the reference region for updating the mapping function of the current block. A diagram of this example is provided at the top (a) of Figure 10.

[0107] When the current block is predicted from several reference pictures (an example of a bipredicted block), or from several subblocks within a reference picture (an example of subblock time-motion vector prediction or subblock affine motion prediction, as in the case of VVC), several reference regions may be used. This is shown in the lower part (b) of Figure 10, where the current block consists of two subblocks, each of which is predicted from a different reference picture using separate motion vectors (MV1, MV2).

[0108] In one embodiment, the update process from any of the embodiments described above may be applied even if the LMCS is not activated for the current block, slice, or picture. In that case, the default mapping function is considered to be the identity function of the default forward and reverse mapping functions. [0,(2 BD For any x in (-1), fwdLut[x] = InvLut[x] = x

[0109] In that case, even when it is determined that LMCS is disabled, the use of mapping functions can still be enabled by determining the updated mapping function from the identity function and applying the updated mapping function to the current block. Then, the remainder of the process described above can be applied in the same way as when LMCS is enabled. To determine the update of the mapping function updLut[k], the default mapping function defLut[k] is set to the identity function, or the updLut[k] function is initialized with the identity function.

[0110] In this embodiment, the mapping function update can be enabled or disabled independently of the LMCS tool activation.

[0111] The purpose of updating the mapping function is to enable local correction of block prediction or block reconstruction for each sample value (depending on where the mapping is performed). When LMCS is disabled, this embodiment provides a new block-based tool that enables correction of block prediction using a mapping function determined locally for the block. The parameters of the mapping function are determined for the current block based on a template sample, as described in any of the embodiments described above. Compared with LIC (local illumination compensation) tools that provide linear correction of prediction, the advantage of the embodiments provided herein, where the correction is performed by the mapping function, is that the correction is determined locally for each sample value. Advantageously, the mapping function is implemented using a LUT that enables providing corrections that are no longer linear (or based on a parametric model). The LUT representing the mapping function is determined based on a histogram or occurrence count of the template sample. Such embodiments described herein provide locally adapting the sample value mapping during its operation when encoding or decoding the current block.

[0112] In one embodiment, instead of considering the lookup table as a representation of the forward and reverse mapping functions, the mapping functions are expressed as piecewise linear functions. For example, let's use P elements. One part of index k (k=0 to P-1) is defined by two pivot points pivot(k)=(px(k),py(k)) and pivot(k+1)=(px(k+1),py(k+1)). The samples s in the part of index k are ss>=px(k) and s <px(k+1)となるようなものである。sは、map k This is mapped to (s). map k (s) = py(k) + (s - px(k)) * (py(k+1)-py(k))÷(px(k+1)-px(k)) k is ss>=px(k) and s <px(k+1)となるようなインデックスである。

[0113] This expression is expressed in floating-point terms, but a fixed-point formulation can be easily derived. Using lookup table notation, this is equivalent to the following: LUT[s]=map k (s) is such that k is ss>=px(k) and s <px(k+1)となるようなインデックスである。

[0114] (py(k+1)-py(k))÷(px(k+1)-px(k)) represents the slope at index k. The process of updating the mapping function may also involve updating the pivot point (or equivalently the slope) in the same way as described above. For example, the updated pivot point may be estimated to minimize the distortion.

[0115]

number

[0116] Figure 15 shows a block diagram of a system in which an aspect of this embodiment may be implemented according to another embodiment. Figure 15 shows one embodiment of a device 1500 for encoding or decoding video according to any one of the embodiments described herein. The device comprises a processor 1510 which can be interconnected to a memory 1520 via at least one port. Both the processor 1510 and the memory 1520 may also have one or more additional interconnections to external connections.

[0117] The processor 1510 is also configured using any one of the embodiments described herein. For example, the processor 1510 is configured using any one of the embodiments described herein to determine, for at least one block of video, to update at least one mapping function using at least one block template sample, wherein the mapping function is an in-loop mapping function, and to encode or decode at least one block using at least one updated mapping function. For example, the processor 1510 is configured using a computer program product that includes code instructions to implement any one of the embodiments described herein.

[0118] In the embodiment shown in Figure 16, in a transmission context between two remote devices A and B via a communication network NET, device A comprises a processor associated with memory RAM and ROM configured to perform a method for encoding video as described with reference to Figures 1 to 15, and device B comprises a processor associated with memory RAM and ROM configured to perform a method for decoding video as described with reference to Figures 1 to 15. For example, the network is a broadcast network and is adapted to broadcast / transmit coded video from device A to decoding devices including device B.

[0119] Figure 16 shows an example of the syntax of a signal transmitted via a packet-based transmission protocol. Each transmission packet P includes a header H and a payload PAYLOAD. In some embodiments, the payload PAYLOAD may include video data encoded by any one of the embodiments described above. The payload may also include arbitrary signaling as described above. For example, the payload may include one or more indicators. One or more indicators may include at least one of the following: an indicator indicating whether an updated mapping function or a default mapping function is used in at least one block, or an indicator indicating which updated mapping function should be used in at least one block when multiple updated mapping functions are available in at least one block.

[0120] Various implementations involve decoding. As used in this application, “decoding” can encompass all or part of the processes performed on a received encoded sequence to produce, for example, a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also, or alternatively, include processes performed by the decoder in the various implementations described in this application, such as entropy decoding a sequence of binary symbols to reconstruct image or video data.

[0121] As further examples, in one embodiment, “decoding” refers only to entropy decoding; in another embodiment, “decoding” refers only to differential decoding; in yet another embodiment, “decoding” refers to a combination of entropy decoding and differential decoding; and in yet another embodiment, “decoding” refers to the entire picture reconstruction process, including entropy decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or generally to a broader decoding process will become clear from the context of the particular description and will be well understood by those skilled in the art.

[0122] Various implementations involve encoding. As with the above considerations regarding "decoding," "encoding" as used in this application may encompass all or part of the processes performed on an input video sequence to generate an encoded bitstream. In various embodiments, such processes typically include one or more processes performed by the encoder, e.g., segmentation, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, such processes also, or alternatively, include processes performed by the encoder in the various implementations described in this application, e.g., determining resampling filter coefficients and resampling the decoded picture.

[0123] As further examples, in one embodiment, “encoding” refers only to entropy coding; in another embodiment, “encoding” refers only to differential coding; and in yet another embodiment, “encoding” refers to a combination of differential coding and entropy coding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or to a broader encoding process in general will become clear from the context of the particular description and will be well understood by those skilled in the art.

[0124] It should be noted that the syntactic elements used herein are descriptive terms; therefore, they do not preclude the use of other syntactic element names.

[0125] This disclosure describes various types of information that can be transmitted or stored, such as syntax. This information can be packaged or configured in various ways, including methods common in video standards, such as including the information in SPS, PPS, NAL units, headers (e.g., NAL unit headers, picture headers, or slice headers), or SEI messages. Other methods are also available, including methods common to system-level or application-level standards, such as including the information in one or more of the following: a. SDP (Session Description Protocol), a format for describing multimedia communication sessions for the purpose of session announcement and session invitation, such as being described in an RFC and used in conjunction with RTP (Real-time Transport Protocol) transmission. b. For example, a DASH MPD (Media Presentation Description) descriptor used in DASH and transmitted over HTTP, where the descriptor is associated with a representation or set of representations to provide additional characteristics to the content representation. c. RTP header extensions, such as those used during RTP streaming. d. An ISO-based media file format that uses boxes, which are object-oriented building blocks defined by a unique type identifier and length, as used in OMAF, for example, and also known as "atoms" in some specifications. e. An HLS (HTTP Live Streaming) manifest sent via HTTP. The manifest can, for example, be associated with a version of content or a set of versions of content, and can provide characteristics of the version or set of versions.

[0126] When a diagram is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0127] Several embodiments refer to rate-distortion optimization. In particular, during the coding process, a balance or trade-off between rate and distortion is usually considered, often given the constraints of computational complexity. Rate-distortion optimization is usually formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods for solving rate-distortion optimization problems. For example, these methods may be based on extensive testing of all coding options, including all considered mode or coding parameter values, but with a complete evaluation of their coding costs, as well as the associated distortions of the reconstructed signals after coding and decoding. In particular, faster approaches can be used to reduce coding complexity by calculating approximate distortion based on the predicted or predicted residual signal rather than the reconstructed signal. These two approaches can also be combined, for example, by using approximate distortion for only some of the possible coding options and full distortion for others. Other approaches evaluate only a subset of the possible coding options. More generally, many methods employ one of various techniques to perform optimization, but the optimization is not necessarily a complete evaluation of both coding costs and associated distortions.

[0128] The implementation forms and embodiments described herein can be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if considered only in the context of a single form of implementation (e.g., considered only as a method), the implementation forms of the considered features can also be implemented in other forms (e.g., apparatus or programs). Apparatus may be implemented, for example, in appropriate hardware, software, and firmware. These methods may be implemented, for example, in a processor, which refers to processing devices in general, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end users.

[0129] The terms "one embodiment" or "one implementation" or "one implementation," and any other variations thereof, mean that the specific features, structures, characteristics, etc., described in relation to the embodiments are included in at least one embodiment. Therefore, the appearance of the phrases "in one embodiment" or "in one embodiment" or "in one implementation," and any other variations, found in various places throughout this application, do not necessarily all refer to the same embodiment.

[0130] Additionally, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory.

[0131] Furthermore, this application may also refer to “accessing” various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0132] Additionally, this application may refer to “receiving” various types of information. Receiving is intended to be a broad term, similar to “accessing.” Receiving information may include, for example, accessing information or retrieving information (for example, from memory). Furthermore, “receiving” typically accompanies, in some way, operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0133] For example, in the cases of "A / B", "A and / or B", and "at least one of A and B", please understand that the use of any of the following " / ", "and / or", and "at least one of" is intended to cover the selection of only the first option (A), only the second option (B), or both options (A and B). As further examples, in the cases of "A, B, and / or C" and "at least one of A, B, and C", such phrasing is intended to cover the selection of only the first option (A), only the second option (B), only the third option (C), only the first and second options (A and B), only the first and third options (A and C), only the second and third options (B and C), or all three options (A, B, and C). This may be extended to the number of items listed, as is obvious to anyone with ordinary knowledge of this and related technologies.

[0134] Furthermore, as used herein, the word “signaling” refers, in particular, to pointing something to a corresponding decoder. In this way, in one embodiment, the same parameter is used on both the encoder and decoder sides. For example, the encoder can transmit a specific parameter to the decoder (explicit signaling), and as a result, the decoder can use the same specific parameter. Conversely, if the decoder already has a specific parameter as well as other parameters, signaling can be used without transmission (implicit signaling), allowing the decoder to easily recognize and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual function. It should be seen that signaling can be achieved in various ways. For example, one or more syntactic elements, flags, etc., can be used to signal information to the corresponding decoder in various embodiments. While this concerns the verb form of the word “signaling,” the word “signaling” can also be used as a noun in this specification.

[0135] As will be apparent to those skilled in the art, the implementation can generate various signals, for example, that are formatted to carry information that can be stored or transmitted. The information may include, for example, instructions for performing a method, or data generated by one of the implementations described. For example, a signal may be formatted to carry the bitstream of the embodiment described. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over various different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0136] Several embodiments have been described above. Features of these embodiments may be provided individually or in any combination across various claim categories and types.

Claims

1. It is a method, For at least one block of the video, determine at least one mapping function using the template sample of the said at least one block, A method comprising encoding or decoding the at least one block using the at least one mapping function.

2. A device comprising one or more processors, wherein the one or more processors For at least one block of the video, determine at least one mapping function using the template sample of the said at least one block. A device capable of encoding or decoding the at least one block using the at least one mapping function.

3. The method according to claim 1 or the apparatus according to claim 2, wherein the mapping function is a forward mapping function, and encoding or decoding the at least one block using the mapping function includes applying the mapping function to the predicted block of the at least one block.

4. The method according to claim 1 or 3, or the apparatus according to any one of claims 2 to 3, wherein the mapping function is an inverse mapping function, and encoding or decoding the at least one block using the mapping function includes applying the mapping function to the reconstruction of the at least one block.

5. The method according to any one of claims 1 or 3 to 4, or the apparatus according to any one of claims 2 to 4, wherein determining the mapping function determines a lookup table.

6. The method according to any one of claims 1 or 3 to 5, or the apparatus according to any one of claims 2 to 5, wherein the mapping function is an in-loop mapping function.

7. The method according to any one of claims 1 or 3 to 6, or the apparatus according to any one of claims 2 to 6, wherein determining the at least one mapping function is done using at least one of the template-reconstructed sample of the at least one block, the template-predicted sample of the prediction block of the at least one block, or the template-filtered sample of the at least one block.

8. The method according to any one of claims 1 or 3 to 7, or the apparatus according to any one of claims 2 to 7, wherein determining the mapping function updates the default mapping function.

9. The method or apparatus according to claim 8, wherein the default mapping function is constructed from mapping function parameters encoded in a bitstream for at least one picture of the video.

10. The method or apparatus according to claim 8, wherein the default mapping function is the identity mapping function.

11. The method or apparatus according to any one of claims 8 to 10, wherein an indicator indicates whether the updated mapping function or the default mapping function is used for the at least one block.

12. The method or apparatus according to claim 11, wherein the indicator is signaled in the bitstream at the sequence level, picture level, slice level, APS level, or block level.

13. The method or apparatus according to claim 11, wherein the indicator is inferred when determining the updated mapping function.

14. When multiple mapping functions are available for the at least one block, the indicator indicates which mapping function should be used for the at least one block, the method according to any one of claims 1 or 3 to 13, or the apparatus according to any one of claims 2 to 13.

15. The method or apparatus according to claim 14, wherein the indicator is signaled in a bitstream.

16. The method or apparatus according to claim 14 or 15, wherein multiple mapping functions are obtained using separate sets of template samples.

17. The method or apparatus according to claim 14 or 15, wherein the plurality of mapping functions are obtained from a history list of mapping functions.

18. The method according to any one of claims 1 or 3 to 17, or the apparatus according to any one of claims 2 to 17, wherein the mapping function used in the at least one block is also used in one or more blocks following the at least one block in the order of encoding or decoding.

19. The method or apparatus according to claim 18, wherein both the forward mapping function and its inverse mapping function are updated.

20. The method according to any one of claims 1 or 3 to 19, or the apparatus according to any one of claims 2 to 19, wherein determining the at least one mapping function includes determining an update to the forward mapping function, determining an update to the inverse mapping function, or determining an update to both the forward mapping function and the inverse mapping function.

21. The method or apparatus according to claim 20, wherein determining the at least one mapping function minimizes the distortion between a first version of the template sample and a second version of the template sample mapped by the updated mapping function across the template sample.

22. The method or apparatus according to claim 20, wherein determining the mapping function using the template sample of at least one block is based on a histogram or number of occurrences determined for the template sample.

23. The method or apparatus according to claim 20, wherein determining the at least one mapping function includes determining one or more offsets based on a template sample.

24. The method according to any one of claims 1 or 3 to 23, or the apparatus according to any one of claims 2 to 23, wherein the template sample is an adjacent sample of the at least one block and / or a reference time sample of the at least one block.

25. The method according to any one of claims 1 or 3 to 24, or the apparatus according to any one of claims 2 to 24, wherein determining the at least one mapping function is in response to a determination that the chroma mapping and chroma scaling tools are disabled, and the at least one mapping function is an identity mapping function.

26. A computer program product comprising instructions for causing one or more processors to perform the method according to any one of claims 1, 3 to 25.

27. A non-temporary computer-readable medium for storing executable program instructions, which cause a computer executing the program instructions to carry out the method according to any one of claims 1, 3 to 25.

28. A bitstream containing data representing a video, wherein the bitstream includes at least one indicator for at least one block of the video, which indicates whether an updated mapping function or a default mapping function is used for the at least one block, or which mapping function should be used for the at least one block when multiple mapping functions are available for the at least one block.

29. A non-temporary computer-readable medium for storing the bitstream described in claim 28.

30. It is a device, The apparatus according to claim 2, A device comprising: (i) an antenna configured to receive or transmit a signal including data representing the video; (ii) a band limiter configured to restrict the signal to a frequency band including the data representing the video; or (iii) a display configured to display the video.

31. The device according to claim 30, wherein the device includes at least one of a television, a mobile phone, a tablet, and a set-top box.

Images