Refining mode processing in video encoding and decoding

Abstract

A method includes decoding an indication at a block level of a picture portion, the indication providing a selection of a refinement parameter table from a plurality of refinement parameter tables for a block, the plurality of refinement parameter tables associated with a component refinement mode applied to a reconstructed version of a component of the picture portion; one or more refinement parameters for the block are obtained using the selected refinement parameter table, the picture portion is refined in the component refinement mode, and the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

Description

[0001] This application is a divisional application of the invention patent application with the application date of June 11, 2019, application number 201980054618.7, and invention name “Refined Mode Processing in Video Coding and Decoding”. Technical Field

[0002] The present disclosure relates to video encoding and decoding. Background Art

[0003] To achieve high compression efficiency, image and video coding schemes typically use prediction and transform to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original picture block and the predicted picture block (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy encoded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to prediction, transform, quantization, and entropy encoding. Summary of the invention

[0004] In general, an example of at least one embodiment of a method includes: obtaining a cost for: (i) encoding a picture portion based on applying a refinement mode to a reconstructed signal for each block, wherein the refinement mode is based on a refinement parameter, and (ii) encoding the picture portion using a mode other than the refinement mode; and encoding the picture portion based on the cost.

[0005] In general, an example of at least one embodiment of an apparatus includes one or more processors configured to: obtain a cost for: encoding a picture portion based on applying a refinement mode to a reconstructed signal for each block, wherein the refinement mode is based on a refinement parameter, and encoding the picture portion without using the refinement mode; and encoding the picture portion based on the cost.

[0006] In general, an example of at least one embodiment of a decoding method includes: obtaining an indication of applying a refinement mode to a reconstructed signal for each block during encoding of a picture portion, wherein the refinement mode is based on a refinement parameter; and decoding the encoded picture portion based on the indication.

[0007] In general, an example of at least one embodiment of an apparatus includes one or more processors configured to: obtain an indication of applying a refinement mode to a reconstructed signal for each block during encoding of an encoded picture portion, wherein the refinement mode is based on a refinement parameter; and decode the encoded picture portion based on the indication.

[0008] In general, an example of at least one embodiment includes providing a signal formatted to include data representing an encoded picture portion; and providing data indicating application of a refinement mode to a reconstructed signal for each block based on a refinement parameter during encoding of the encoded picture portion.

[0009] In general, an example of at least one embodiment includes providing a bitstream formatted to include data representing an encoded picture portion; and providing data indicating application of a refinement mode to a reconstructed signal for each block based on a refinement parameter during encoding of the encoded picture portion.

[0010] Generally, examples of at least one embodiment include a computer program product or a non-transitory computer-readable storage medium having computer-readable instructions stored thereon for causing one or more processors to perform any of the methods described herein.

[0011] Generally, one or more embodiments provide a computer-readable storage medium, such as a non-transitory computer-readable storage medium, on which instructions are stored for encoding or decoding video data according to the method or apparatus described herein, and / or on which a bitstream generated according to the method or apparatus described herein is stored. One or more of the present embodiments can also provide a method and apparatus for sending or receiving a bitstream generated according to the method or apparatus described herein.

[0012] In general, another example of at least one embodiment includes a device comprising one or more processors configured to encode or decode a picture portion according to the method or apparatus described herein and comprising at least one of: (i) an antenna configured to receive a signal comprising data representing an image, (ii) a band limiter configured to limit the received signal to a frequency band comprising data representing the image, or (iii) a display configured to display the image.

[0013] In general, an example of at least one embodiment of a method includes: decoding an indication at a block level of a picture portion, the indication providing for a block to select a refinement parameter table from a plurality of refinement parameter tables, the plurality of refinement parameter tables being associated with a component refinement mode applied to a reconstructed version of a component of the picture portion; obtaining one or more refinement parameters for the block using the selected refinement parameter table, refining the picture portion in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

[0014] Generally, one or more embodiments provide a non-transitory computer-readable medium storing executable program instructions to cause a computer to execute the instructions to perform a method according to the methods described herein.

[0015] In general, an example of at least one embodiment of an apparatus includes one or more processors configured to: decode an indication at a block level of a picture portion, the indication providing for a block to select a refinement parameter table from a plurality of refinement parameter tables, the plurality of refinement parameter tables being associated with a component refinement mode applied to a reconstructed version of a component of the picture portion; obtain one or more refinement parameters for the block using the selected refinement parameter table, refine the picture portion in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

[0016] In general, an example of at least one embodiment of a method includes: encoding an indication at a block level of a picture portion, the indication providing for a block to select a refinement parameter table from a plurality of refinement parameter tables, the plurality of refinement parameter tables being associated with component refinement modes applied to a reconstructed version of components of the picture portion; obtaining one or more refinement parameters for the block using the selected refinement parameter table, refining the picture portion in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

[0017] In general, an example of at least one embodiment of an apparatus includes one or more processors configured to: encode an indication at a block level of a picture portion, the indication providing for a block to select a refinement parameter table from a plurality of refinement parameter tables, the plurality of refinement parameter tables being associated with component refinement modes applied to a reconstructed version of components of the picture portion; obtain one or more refinement parameters for the block using the selected refinement parameter table, refine the picture portion in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

[0018] In general, one or more embodiments provide a non-transitory computer-readable medium comprising a bitstream formatted to include: data representing a picture portion; and at a block level of the picture portion, data representing an indication, the indication providing for a block to select a refinement parameter table from a plurality of refinement parameter tables, the plurality of refinement parameter tables being associated with component refinement modes applied to a reconstructed version of a component of the picture portion. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present disclosure may be better understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0020] Figure 1 A block diagram depicting an example of an embodiment of a video encoder is provided.

[0021] Figure 2 A block diagram depicting an example of an embodiment of a video decoder is provided.

[0022] Figure 3 A block diagram depicting another example of an embodiment of a video encoder is provided.

[0023] Figure 4 A block diagram depicting an example of another embodiment of a video decoder is provided.

[0024] Figure 5 A table depicting an example of an embodiment of a signal syntax is provided.

[0025] Figure 6 A diagram illustrating features of an embodiment of a refinement table is provided.

[0026] Figure 7 A table depicting another example of an embodiment of a signal syntax is provided.

[0027] Figure 8 A table is provided that depicts an example of features of another embodiment of a signal syntax.

[0028] Fig. 9 A flow chart depicting an example of an embodiment of a process for deriving refinement information is provided.

[0029] Fig.10 A table depicting another example of an embodiment of a signal syntax is provided.

[0030] Fig.11 A flow chart depicting another example of an embodiment of a process for deriving refinement information is provided.

[0031] Fig.12 Two flow charts are provided that depict examples of embodiments of processes for encoding and decoding refinement parameters.

[0032] Fig.13 A table depicting another example of an embodiment of a signal syntax is provided.

[0033] Fig.14 A flow chart is provided that depicts an example of an embodiment of a process for deriving information for use with one or more embodiments.

[0034] Fig.15 Block diagrams are provided that depict examples of embodiments of systems that include video encoding and / or decoding.

[0035] Fig.16 A block diagram is provided depicting an example of an embodiment of a system such as a transmitter including video encoding.

[0036] Fig.17 A block diagram depicting another example of an embodiment of a video encoder is provided.

[0037] Fig.18A block diagram depicting another example of an embodiment of a video encoder is provided.

[0038] Fig.19 A block diagram depicting another example of an embodiment of a video encoder is provided.

[0039] Fig. 20 A flow chart depicting an example of an embodiment of a process for video encoding is provided.

[0040] Fig.21 Provides a description of Fig. 20 A flowchart of an example of an embodiment of a process depicting features of at least one other embodiment.

[0041] Fig. 22 A diagram illustrating features of an embodiment of a refinement table is provided.

[0042] Fig.23 A diagram illustrating features of another embodiment of a refinement table is provided.

[0043] Fig.24 Provides a description Fig.21 Flowchart of an example of an embodiment of a variation of the process depicted.

[0044] Fig.25 A diagram is provided that features another embodiment of a refinement table.

[0045] Fig.26 A diagram is provided that features another embodiment of a refinement table.

[0046] Fig. 27 A block diagram is provided depicting an example of an embodiment of a system such as a receiver that includes video decoding.

[0047] Fig.28 A block diagram depicting another example of an embodiment of a video decoder is provided.

[0048] Fig.29 A block diagram depicting another example of an embodiment of a video decoder is provided.

[0049] Fig.30 A block diagram depicting another example of an embodiment of a video decoder is provided.

[0050] Fig.31 A flow chart depicting an example of an embodiment of a process for decoding a video signal is provided.

[0051] Fig.32 A block diagram depicting another example of an embodiment of a video encoder is provided.

[0052] Fig.33A block diagram depicting another example of an embodiment of a video decoder is provided.

[0053] It should be understood that the drawings are for the purpose of illustrating examples of various aspects and embodiments and are not necessarily the only possible configurations. Throughout the various drawings, the same reference numerals refer to the same or similar features. DETAILED DESCRIPTION

[0054] The video compression and reconstruction process may produce distortion. For example, distortion may be observed in the reconstructed video signal, particularly when the video signal is mapped before encoding, for example, to better utilize the sample codeword distribution of the video picture. In general, at least one embodiment may use, for example, video signal refinement based on chrominance components to provide flexibility and coding gain to provide reduced distortion and / or improved compression efficiency. At least one embodiment may involve a refinement step as an in-loop or out-of-loop filter for improving the reconstructed video signal after decoding. Refinement may be based on a refinement table encoded in a bitstream or signal during encoding, for example. The decoder then applies signal correction (e.g., filtering) based on the refinement table. Among various examples of possible modes, one example of a mode of at least one embodiment is cross-component refinement of chrominance components. Another example of a mode of at least one embodiment is intra-component refinement, for example, applied to a luminance component.

[0055] Applying certain processes, methods or techniques intended to provide refinement to an entire slice or given entire region may result in degradation of local portions of the slice or region to which the refinement is applied, even though the refinement is beneficial globally for the entire slice or region. In general, at least one embodiment addresses this issue and is also directed to improving the encoding of the data involved, such as refinement tables. Refinement for an entire slice or region may involve encoding one table per component and applying this one table to all samples of the entire slice or region.

[0056] Typically, in at least one embodiment, a table is calculated on the encoder side based on providing an improvement indicated by a metric, for example by reducing the rate-distortion cost (typically a weighted sum of distortion and coding cost) over the entire considered picture or region. For example, the distortion may be the mean square error between the refined reconstructed signal and the original signal. The resulting table is then encoded in the bitstream or signal. If the rate-distortion cost over the considered slice or region is globally reduced, this also means that there may be some portions in the considered slice or region where the rate-distortion cost is increased. Typically, at least one example of an embodiment may include a refinement mode that includes cross-component chroma refinement. Another example of an embodiment may include, for example, intra-component refinement applied to a luminance component.

[0057] As used herein, refinement of data may refer to a change, adjustment, modification, revision, or adaptation of a process that can result in an improvement (e.g., an improvement such as increased efficiency of encoding and / or decoding of information (such as video data) and / or other improvements such as reduced costs (such as rate-distortion costs described herein)). For ease of explanation, one or more embodiments described herein may refer to "costs" as "rate-distortion costs," which may be used to evaluate, determine, or obtain effects, impacts, or improvements associated with the use of refinement modes. For example, costs associated or unassociated with the use of refinement modes may be evaluated (e.g., compared) to determine whether to use a refinement mode as described herein, for example, based on improved compression efficiency and / or reduced distortion, such as indicated by a change in costs. Although one type of cost may include rate-distortion costs, references herein to rate-distortion costs are not limiting, as other types of costs and / or other methods for evaluating the effects of refinement modes are contemplated.

[0058] In addition, in the description of this document, the words "reconstructed" and "decoded" can be used interchangeably. Typically, but not necessarily, "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side. It should be noted that the term "decoded" or "reconstructed" may mean that the bitstream or signal is partially "decoded" or "reconstructed", for example, a signal obtained after deblocking filtering but before SAO filtering, and the reconstructed samples may be different from the final decoded output for display. Moreover, the terms "image", "picture" and "frame" can be used interchangeably.

[0059] In general, at least one embodiment may involve enabling activation of refinement for each block, and possibly enabling selection, for each block, of a set of possible refinement parameters previously encoded in a stream to be applied to the block. The term "block" may be replaced in variants by "coding unit" (CU), "coding tree unit" (CTU).

[0060] In general, one or more embodiments relate to encoders and decoders, and one or more embodiments may relate to decoder specifications and semantics.

[0061] Figure 1 An example of an embodiment of the encoder 100 is illustrated. Variations of the encoder 100 are contemplated, but for the sake of clarity, the encoder 100 is described below without describing all contemplated variations.

[0062] Prior to encoding, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream or signal.

[0063] In encoder 100, a picture is encoded by encoder elements as described below. The picture to be encoded is segmented (102) and processed in units such as CUs. Each unit is encoded using, for example, intra or inter mode. When the unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105D) which of intra mode or inter mode is used to encode the unit, and indicates the intra / inter decision by, for example, a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0064] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, along with motion vectors and other syntax elements, are entropy encoded (145) to output a bitstream or signal. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may bypass both the transform and quantization, i.e., the residual is directly encoded without applying a transform or quantization process.

[0065] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the predicted block are combined (155) and the image block is reconstructed. In-loop filtering (165) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0066] Figure 2 A block diagram of an example of an embodiment of a video decoder 200 is illustrated. In the decoder 200, a bitstream or signal is decoded by decoder elements as described below. The video decoder 200 generally performs the following operations: Figure 1 The encoding pass described in is relative to the decoding pass. The encoder 100 also typically performs video decoding as part of encoding the video data.

[0067] In particular, the input to the decoder includes a video bitstream or signal, which may be generated by the video encoder 100. The bitstream or signal is first entropy decoded (230) to obtain transform coefficients, motion vectors and other encoding information. The picture segmentation information indicates how the picture is segmented. Therefore, the decoder can segment (235) the picture based on the decoded picture segmentation information. The transform coefficients are dequantized (240) and inversely transformed (250) to decode the prediction residual. The decoded prediction residual and the predicted block are combined (255) and the image block is reconstructed. The predicted block can be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). In-loop filtering (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0068] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream or signal.

[0069] In general, according to various examples of features described herein, in at least one embodiment, one or more features as described herein can affect portions or segments or modules of an encoder / decoder (codec), such as in-loop filters (165 and 265) and entropy encoding (145) and decoding (230).

[0070] Figure 3 Pictured Figure 1 A variant 3000 of the video encoder 100. Figure 1 The modules are the same Figure 3 The modules are labeled with the same reference numerals and are not described further. Figure 3 In the embodiment shown, the thinning process is in a loop. The step of calculating the thinning parameters (step 3001) is completed after the in-loop filtering step (165). The thinning parameter calculation step takes as input the input video and the reconstructed video from the in-loop filtering step. It calculates the thinning parameters that are encoded into the bitstream or signal by the modified entropy encoder (3003). The thinning parameters are also used to apply the thinning step (3002) to the reconstructed video from the in-loop filter.

[0071] Figure 4 Pictured Figure 2 A variation 4000 of the video encoder 100. Figure 2 The modules are the same Figure 4 The modules are labeled with the same reference numerals and are not described further. Figure 4In the embodiment of the present invention, in step 4001, a bit stream or signal is decoded by an entropy decoder. The entropy decoder generates a reconstructed video and, in addition, generates decoded refinement parameters. In step 4002, refinement of the reconstructed video is applied using the refinement parameters to generate an output video.

[0072] As used herein, the following symbols represent or correspond to:

[0073] - B is the bit depth of the signal (e.g., 10 bits)

[0074] -MaxVal is the maximum value of the signal, calculated as (2 B -1)

[0075] - Sin_Y(p), Sin_cb(p), Sin_cr(p) correspond to the Y, cb and cr original (input) signals at relative position p in the picture

[0076] - Sout_cb(p), Sout_cr(p) correspond to the cb and cr (input) signals resulting from refinement at relative position p in the picture

[0077] -Srec_Y(p) corresponds to the reconstructed luminance signal at relative position p in the picture. It corresponds to the signal from the in-loop filter

[0078] - Srec_cb(p), Srec_cr(p) corresponds to the reconstructed chrominance signal to be refined at relative position p in the picture. It corresponds to the signal from the in-loop filter

[0079] -neutralVal corresponds to a predefined value. When the refinement value is equal to neutralVal, refinement does not modify the signal. A typical value of neutralVal is 64.

[0080] -Rcb, Rcr correspond to chroma refinement parameters

[0081] Rcb, Rcr can typically be defined as the pivot points of a piecewise linear (PWL) model. For example,

[0082] Rcb = {(Rcb_idx[pt], Rcb_val[pt]), for pt = 0 to Ncb-1}, Ncb is the number of pivot points. From the chroma refinement parameters Rcb, Rcr, the lookup tables LutRcb, LutRcr are derived. For example, LutRcb can be constructed by linearly interpolating between each pair of points in Rcb as follows.

[0083] For pt = 0 to Ncb-2

[0084] For idx=Rcb_idx[pt] to (Rcb_idx[pt+1]-1)

[0085] LutRcb[idx]=Rcb_val[pt]+(Rcb_val[pt+1]-Rcb_val[pt])*(idx-Rcb_idx[

[0086] pt]) / (Rcb_idx[pt+1]-Rcb_idx[pt])

[0087] An embodiment of cross-component chroma refinement is as follows (here is an explanation for component cb):

[0088] ·Sout_cb(p)=offset+LutRcb[Srec_Y(p)] / neutralVal*(Srec_cb(p)-offset)

[0089] Here, the offset is typically set to (MaxVal / 2), and p corresponds to the same relative position in the picture.

[0090] Or as follows:

[0091] ·Sout_cb(p)=(LutRcb[Srec_Y(p)]-neutralVal)+Srec_cb(p)

[0092] In another example of an embodiment, refinement may be applied as intra-component refinement of the luma component, which works as follows:

[0093] ·Sout_Y(p)=(LutRy[Srec_Y(p)] / neutralVal)*Srec_Y(p)

[0094] Or as follows:

[0095] ·Sout_Y(p)=LutRy[Srec_Y(p)]+Srec_Y(p)

[0096] Embodiments may include providing a method having one or more grammatical features such as Figure 5 An aspect may include that only the chroma components are refined. For example, when refinement_table_flag_cb is true, for all pts from 0 to (refinement_table_cb_size-1), the cb refinement table values ​​(Rcb_idx[pt], Rcb_val[pt]) are calculated as follows:

[0097] Rcb_idx[pt] is set equal to (Rcb_idx[pt-1]+refinement_cb_idx[pt])

[0098] Rcb_val[pt] is set equal to (neutralVal + refinement_cb_value[pt])

[0099] A similar process is applied for the cr component.

[0100] exist Figure 6 An example of a refinement table is shown in FIG. 1 , where refinement_table_cb_size = 6 (pt = 0 to 5). For symbol simplification, “Rcb” is Figure 6 Replaced by "R" in the text box.

[0101] In an embodiment, when block-based cross-component chroma refinement is activated, the K-to-cb, cr refinement table (Rcb) for k=1 to K is enabled. k ,Rcr k ). When K is equal to 0, no refinement is applied to the slice. One or more block-based indicators are encoded for each block to indicate which of these table pairs is used, or whether no refinement is activated or not for that block.

[0102] In a non-limiting example of an embodiment, only one pair of tables is used (K=1).

[0103] In another example of an embodiment, in a slice header, a set of refinement tables is first encoded. Figure 7 An example of the syntax for decoding a pair of cb, cr refinement tables is depicted in FIG. Figure 7 An example of slice header syntax using table pair identifiers is shown in FIG.

[0104] refinement_number_of_tables indicates the number of chroma refinement table pairs encoded in the stream. This number should be greater than or equal to 0. When equal to 0, this means that no chroma refinement is applied to the current slice. If refinement_number_of_tables is greater than 0, then for any idx value between 0 and (refinement_number_of_tables-1), at least refinement_table_flag_cb[idx] or refinement_table_flag_cr[idx] should be equal to 1.

[0105] NOTE: refinement_number_of_tables corresponds to the parameter K discussed above.

[0106] refinement_table_flag_cb[idx] indicates whether the refinement table is encoded for the cb component of the refinement table pair with identifier value idx. If refinement_table_flag_cb[idx] is equal to 0, refinement_table_flag_cr[idx] shall be equal to 1.

[0107] refinement_table_cb_size[idx] indicates the size of the cb refinement table of the refinement table pair with identifier value idx.

[0108] refinement_cb_idx[idx][i] indicates the index (first coordinate) of the i-th element of the cb refinement table of the refinement table pair with identifier value idx.

[0109] refinement_cb_value[idx][i] indicates the value (second coordinate) of the i-th element of the cb refinement table of the refinement table pair with identifier value idx.

[0110] refinement_table_flag_cr[idx] indicates whether the refinement table is encoded for the cr component of the refinement table pair with identifier value idx. If refinement_table_flag_cr[idx] is equal to 1, refinement_table_flag_cb[idx] shall be equal to 1.

[0111] refinement_table_cr_size[idx] indicates the size of the cr refinement table of the refinement table pair with identifier value idx.

[0112] refinement_cr_idx[idx][i] indicates the index (first coordinate) of the i-th element of the cr refinement table of the refinement table pair with identifier value idx.

[0113] refinement_cr_value[idx][i] indicates the value (second coordinate) of the i-th element of the cr refinement table of the refinement table pair with identifier value idx.

[0114] At the block (or CU, or CTU) level, a block-based identifier tables_id is signaled, corresponding to the identifier of the pair of cb, cr refinement tables to be applied to the current block. Figure 8 An example of a syntax for decoding block refinement information is depicted in Figure 8 shows an example of a block-level syntax using table pair identifiers. Figure 8 middle:

[0115] The syntax element refinement_activation_flag indicates whether refinement is used for the current block.

[0116] If the current block is using refinement (refinement_activation_flag is equal to 1), the syntax element tables_id is decoded and the current block is refined using the refinement table corresponding to the identifier tables_id:

[0117] For i = 0 to (refinement_table_cb_size[tables_id]-1)

[0118] οrefinement_cb_idx[tables_id][i]

[0119] οrefinement_cb_value[tables_id][i]

[0120] For i = 0 to (refinement_table_cr_size[tables_id]-1)

[0121] οrefinement_cr_idx[tables_id][i]

[0122] οrefinement_cr_value[tables_id][i]

[0123] exist Fig. 9 A corresponding block diagram illustrating an embodiment for deriving refinement information for a current block is shown in FIG. Fig. 9 In the embodiment of the present invention, at 601, the syntax element refinement_activation_flag is decoded. At 602, it is checked whether refinement_activation_flag is equal to 0. If refinement_activation_flag is equal to 0, no refinement is applied (603). If refinement_activation_flag is not equal to 0, tables_id is decoded at 604. At 605, refinement is applied using the table pair indicated by the identifier tables_id.

[0124] In an embodiment, the syntax element refinement_activation_flag is not coded, but a specific value of tables_id indicates that the block is not refined. For example, tables_id equal to 0 indicates that refinement is not used for the current block. The test "refinement_activation_flag == 0?" is replaced by "tables_id == 0?".

[0125] In another embodiment, an aspect relates to encoding refinement information relative to neighboring blocks. For example, a block-based indicator consists of a syntax element indicating whether to copy refinement information of a current block from a neighboring block. The refinement information consists of at least the following items:

[0126] - a parameter indicating whether the block uses refinement (here named refinedBlock, also corresponding to the syntax element named refinement_activation_flag),

[0127] - If refinedBlock indicates that the block is refined, the refinement table used to refine the samples of the block. Fig.10 An example of a syntax for decoding block refinement information is depicted in , where:

[0128] tables_from_left_flag indicates whether to copy refinement information from the left block of the current block.

[0129] tables_from_up_flag indicates whether to copy refinement information from the upper block of the current block.

[0130] refinement_activation_flag indicates whether refinement is activated for the current block.

[0131] exist Fig.11 A corresponding block diagram for deriving refinement information for the current block is provided in Fig.11, at (501) the syntax element tables_from_left_flag is decoded. At 502, it is checked that tables_from_left_flag is equal to 0. If tables_from_left_flag is not equal to 0, the refinement information of the current block is copied from the left block of the current block (504). Otherwise, at 503, the syntax element tables_from_up_flag is decoded. At 505, it is checked that tables_from_up_flag is equal to 0. If tables_from_up_flag is not equal to 0, the refinement information of the current block is copied from the upper block of the current block (507). Otherwise, at 506, the syntax element refinement_activation_flag is decoded. The refinedBlock of the current block is set to refinement_activation_flag. At 508, it is checked that refinement_activation_flag is equal to 0. If refinement_activation_flag is equal to 1, the refinement table is decoded (510). Otherwise, no specific process is applied (510). At 511, refinement is applied. In this step, if refinedBlock is equal to 0, the block is not processed by the refinement process. If refinedBlock is equal to 1, the block is refined using the refinement table derived for the current block.

[0132] In an embodiment, refinement_activation_flag is not explicitly encoded or decoded but implicitly contextually inferred. For example, the context is related to the SAO loop filtering process. In this embodiment, refinement_activation_flag is inferred from the SAO parameters. In one embodiment, if SAO is disabled for a block, refinement_activation_flag is set equal to 0. If SAO is enabled for a block, refinement_activation_flag is set equal to 1. In another embodiment, if SAO band offset is disabled for a block, refinement_activation_flag is set equal to 0. If SAO band offset is enabled for a block, refinement_activation_flag is set equal to 1. A similar rule may be applied to other SAO types, such as edge offset.

[0133] In an embodiment, the refinement process is applied jointly with another loop filtering process such as SAO.

[0134] In general, embodiments may include reducing table encoding costs by avoiding redundant neutral values. In embodiments, the chroma refinement tables may be considered to have the same size (Ncb=Ncr=N) and share the same index:

[0135] Rcb_idx[pt]=Rcr_idx[pt], for pt=0 to (N-1)

[0136] It can be observed that in most cases, when the refinement parameter for one component (eg cb) is equal to the neutral value neutralVal, the corresponding refinement parameters for the other chrominance components are often close to this neutral value.

[0137] This is illustrated in the example below, obtained from processing the first picture of the SDR content BasketballDrive_1920x1080, encoded with the VTM codec at a QP of 37. NeutralValue equals 64, and refinement values ​​close to 64 are indicated in bold font.

[0138] cb 64,32,44,63,65,65,64,64,65,64,63,56,54,54,56,37,64,

[0139] cr 64,35,60,64,65,64,64,64,65,64,61,57,51,45,49,46,64,Therefore, in an embodiment, the cost of the refinement parameters may be reduced by adjusting the encoding of the refinement parameters of the second chrominance component based on the corresponding refinement parameter values ​​of the refinement parameters of the first chrominance component. Fig.12 , which includes an upper block diagram corresponding to the encoder side and a lower block diagram corresponding to the decoder side.

[0140] At the encoder, the input values ​​are refinement parameters of a first chroma component Rp_c1 (e.g., cb component) and corresponding refinement parameters of a second chroma component Rp_c2 (i.e., specified for the same relative index in the refinement parameter table). A first step performs encoding of the refinement parameters of the first chroma component Rp_c1 (step 701). A second step checks a condition related to the value of Rp_c1 (step 702). If the condition is false, the value of the corresponding refinement parameter of the second chroma component Rp_c2 is encoded (step 704). If the condition is true, the value of Rp_c2 is inferred (step 703).

[0141] At the decoder, the first step performs the decoding of the refinement parameters of the first chroma component Rp_c1 (step 711). The second step involves checking a condition related to the value of Rp_c1 (step 712). If the condition is false, the value of the corresponding refinement parameter of the second chroma component Rp_c2 is decoded (step 714). If the condition is true, the value of Rp_c2 is inferred (step 713).

[0142] In an embodiment, the test condition may comprise checking whether the value of the refinement parameter of the first chroma component is equal to neutralValue:

[0143] Rp_c1==neutralValue?

[0144] In another embodiment, the test condition may include checking whether the value of the refinement parameter of the first chroma component is greater than or equal to (neutralValue-thresh1), or less than or equal to (neutralValue-thresh2), where thresh1 and thresh2 are predefined values ​​(typically equal to 1):

[0145] Rp_c1>=(neutralValue-thresh1)AND Rp_c1<=(neutralValue+thresh2)?

[0146] In an embodiment, the inferred value of Rp_c2 is neutralValue.

[0147] In an embodiment, the inferred value of Rp_c2 is Rp_c1.

[0148] In an embodiment, the inferred value of Rp_c2 is (neutralValue-Rp_c1).

[0149] exist Fig.13 An example of the modified syntax is shown in

[0150] Rp_c1(refinement_cb_value[i] in the table)==neutralValue?

[0151] When this condition is true, Rp_c2 (refinement_cr_value[i] in the table) is inferred to be neutralValue. Otherwise the element is explicitly signaled.

[0152] Generally, embodiments may include control of the size of the refinement table. For example, in an embodiment, in order to reduce the encoding cost of the refinement table, at least one of the following may be included:

[0153] The table size for cb and cr is the same and is fixed and defined by a default value named here N.

[0154] The indices Rcb_idx[pt] and Rcr_idx[pt] of the table correspond to equidistant points.

[0155] Each refinement table has a limited predefined number Nref of elements encoded. In this case, an indication of the first table point pt_init at which a table element is encoded is given. The encoded refinement value is thus:

[0156] οRcb_idx[pt_init] to Rcb_idx[pt_init+Nref-1]

[0157] οRcr_idx[pt_init] to Rcr_idx[pt_init+Nref-1]

[0158] Other (unencoded) values ​​are set to the neutral value (neutralVal)

[0159] o For pt=0 to (pt_init-1) and from (pt_init+Nref) to (N-1),

[0160] Rcr_idx[pt]=neutralVal and Rcb_idx[pt]=neutralVal

[0161] In an embodiment, the number of values ​​Nref of each refinement table is equal to four.

[0162] In an embodiment, the encoder may determine a block-based refinement activation and based on, for example, Fig.14 The actions described in the corresponding block diagrams shown in the figure are used to derive one or more refinement tables according to one or more embodiments described herein. In an implementation, the listed actions may occur iteratively.

[0163] 1. For blkld=0 to (Nblk-1), where Nblk is the number of blocks of the slice, initialize the map blkTablesId[blkld] to -1 (step 802).

[0164] 2. Initialize the parameter of the table identifier tablesId to 0 (step 803).

[0165] 3. Derive a refinement table of identifier tablesId from the samples in block blkId where blkTablesId[blkId]=-1 (step 804)

[0166] a. The results are cb and cr tables:

[0167] i.(refinement_cb_idx[tablesId][i],

[0168] refinement_cb_value[tablesId][i]), where i is from 0 to

[0169] (refinement_table_cb_size[tablesId]-1) and

[0170] (refinement_cr_idx[tablesId][i],

[0171] refinement_cr_value[tablesId][i]), where i is from 0 to

[0172] (refinement_table_cr_size[tablesId]-1)

[0173] b. See below for examples of performing the described calculations.

[0174] 4. For each block blkld for which blkTablesld[blkld] = -1, calculate the chroma rate distortion cost, both without refinement (RD_without[blkld]) and with refinement using the table calculated in step 3 (RD_with[blkld]) (step 805).

[0175] a. Calculate the chroma rate distortion cost as follows:

[0176] distortion(cb)+distortion(cr)+

[0177] L*cost(refinement_table_cb)+L*cost(refinement_table_cr)+

[0178] L*cost(activation map)

[0179] where L is the well-known “lambda” parameter in the derivation of rate-distortion cost.

[0180] 5. For all block blkIds with blkTablesId[blkId]=-1, update the block-based refinement activation as follows (step 806):

[0181] a.if (RD_with[blkId] <RD_without[blkId])blkTablesId[blkId]=tablesId

[0182] 6. If there is a block blkId, such as blkTablesId[blkId] = -1, and if tablesId <

[0183] MaxNbTables (step 807),

[0184] a. Increment tablesId by 1 and jump to step 3.

[0185] b. Otherwise stop (step 809).

[0186] MaxNbTables is a predefined parameter that specifies the maximum number of enabled pairwise refinement tables (step 808).

[0187] Regarding aspect 3b above, in an embodiment, the refined metadata can be derived as explained below. The refined metadata represents a correction function, denoted by R(), applied to each individual sample of a color component (e.g., a luminance component Y or a chrominance component Cb / Cr). One aspect of the refined metadata is to minimize the rate-distortion cost between the reconstructed image and the original image. The minimization can be performed on a given area A (e.g., an entire image, a slice, a tile, or a CTU). The refined metadata is denoted by R. In a preferred implementation, a piecewise linear model is used for R, defined by N pairs (R_idx[k], R_val[k]), k=0 to N-1.

[0188] The process begins by initializing the refinement metadata R by a neutral value (step 300). A neutral value is a value such that refinement does not change the signal. This will be explained further below. An initial rate-distortion cost initRD is calculated using R. The rate-distortion cost derivation using the function R will be described below. The parameter bestRD is initialized to initRD. Then, the steps for optimizing the refinement metadata R consist of the following steps. A loop is performed on the index pt of the continuous pivot point of the PWL model R. The parameters bestVal and initVal are initialized to R_val[pt]. A loop is performed on each value of R_val[pt]. A loop is implemented from the value (initVal-val0) to the value (initVal+Val1), where Val0 and Val1 are predefined parameters. Typical values ​​are given below. The rate-distortion cost curRD is calculated using the modified R (with the modified R_val[pt]). The comparison of curRD and bestRD is that if curRD is lower than bestRD, bestRD is set to curRD and bestValue is set to R_val[pt]. Next, a check is made for the end of the loop on the value of R_val[pt]. After the loop is complete, R_val[pt] is set to bestValue. Next, the loop is checked for the value of pt. After the loop is complete, R is the metadata obtained from the refinement process. The step of refining metadata optimization can be iterated several times.

[0189] Next, an embodiment of the refinement process is described using the following notation:

[0190] - B is the bit depth of the signal (e.g., 10 bits)

[0191] -MaxVal is the maximum value of the signal, calculated as (2 B -1)

[0192] - Sin(p) is the original (input) signal at position p in the picture

[0193] - Sout(p) is the signal resulting from the refinement at position p in the picture

[0194] Srec(p) is the signal to be refined. It corresponds to the signal resulting from the inverse mapping of the first variant, or to the signal resulting from the in-loop filter of the second variant.

[0195] - NeutralVal corresponds to a predefined value; when the refinement value is equal to NeutralVal, the refinement does not modify the signal; a typical value of NeutralVal is 128

[0196] The lookup table LutR may be constructed from the refined metadata R as follows.

[0197] For N=0 to N-2

[0198] For idx=R_idx[pt] to (R_idx[pt+1]-1)

[0199] LutR[idx]=R_val[pt]+(R_val[pt+1]-R_val[pt])*(idx-R_idx[pt])*(R_idx[

[0200] pt+1]-R_idx[pt])

[0201] Typically, at least two examples of refinement modes are defined. These examples will be referred to herein as "Mode 1" or "Intra-component refinement mode" and "Mode 2" or "Inter-component refinement mode":

[0202] In mode 1 or intra-component refinement mode, refinement is performed independently of other components. Preferably, this mode is applied to the luma component and is based on the following formula:

[0203] οSout(p)=LutR[Srec(p)] / NeutralVal*Srec(p)

[0204] In mode 2 or inter-component refinement mode, refinement is performed on one component in dependence on another. Preferably, this mode is applied to the chrominance component (denoted Srec_Y(p) in the following) in dependence on the luminance component. It is noted that in the first variant, Srec_Y(p) can be the signal obtained from the in-loop filter step or from the inverse mapping. It can also be a filtered version of the other (Y) component. The process is based on the following formula:

[0205] οSout(p)=offset+LutR[Srec_Y(p)] / NeutralVal*(Srec(p)-offset)

[0206] Here, offset is usually set to (MaxVal / 2), and p corresponds to the same relative position in the picture.

[0207] Finally, rounding and clipping between the minimum and maximum signal values ​​(typically 0 and 1023 for a 10-bit signal) are applied to the refined value Sout(p). Here, the refinement is applied as a multiplication operator mode. An additive refinement operator can also be used for refinement. In this case, mode 1 is applied as follows:

[0208] οSout(p)=LutR[Srec(p)] / NeutralVal+Srec(p)

[0209] And mode 2 is applied as follows:

[0210] οSout(p)=LutR[Srec_Y(p)] / NeutralVal+Srec(p)

[0211] Refinement mode 2 with multiplication operator mode is preferred for chroma components.

[0212] Next, in an embodiment, the rate-distortion cost of the refinement metadata R may be determined as follows, where the following notation is used:

[0213] - A is the picture area for which refinement is performed; the refinement may be performed, for example, on a whole picture, a slice, a tile, a CTU.

[0214] -Dist(x,y) is the distortion between sample value x and sample value y. Typically, the distortion is the squared difference (xy) 2

[0215] -Cost(R) is the encoding cost of encoding the refinement metadata R

[0216] -L is the lambda factor associated with the area. It is typically linearly dependent on 2 (QP / 6) , where QP represents the quantization parameter applied to the region.

[0217] The rate-distortion cost RDcost using the refined metadata R is defined as:

[0218]

[0219] R is constructed to minimize the total rate-distortion cost between the original signal Sin(p) and the refined signal Sout(p).

[0220] In general, embodiments provide coding gains, that is, improved quality for the same bit rate, or reduced bit rate for the same quality. The gains of refinement using block-based activation are shown in the following table. In the non-limiting example shown below, only one table is allowed to be used per component (K, or equivalently refinement_number_of_tables equals 1).

[0221] Performance is shown on 5 HDR HD 10-bit contents, 5 SDR HD 8-bit contents and 2 SDR HD 10-bit contents. The codec used is VTM (Versatile Video Coding (VVC) Test Model) and QTBT (Quadtree plus Binarytree Block Partitioning Structure) is activated. All sequences are encoded with 10-bit internal bit depth. The table below depicts the encoding results on a 17-frame sequence in random access configuration. neutraVal is set to 64 and table size is set to 17. Refinement metadata encoding is applied only to temporal level 0.

[0222] The following five tables show the BD rate gain as follows: Table 1 when refinement is applied for all blocks ("refine full slice"), Tables 2 to 4 when refinement activation is enabled for each block of size 128x128 ("refine blk128"), 64x64 ("refine blk64"), and 32x32 ("refine blk32"), and Table 5 when the best configuration is selected for each content. The results show the benefit or improvement of using block-based refinement activation.

[0223] refine full slice Y psnr U psnr V psnr YUV psnr cosmos 0.0％ -16.3％ -25.3％ -1.1％ hurdles 0.0％ -11.1％ -20.6％ -1.3％ market 0.1％ -1.9％ -0.9％ 0.0％ showgirl 0.0％ -15.5％ -9.6％ -1.0％ starting 0.1％ -2.7％ -10.8％ -0.6％ bbdrive 0.1％ -1.2％ -2.4％ -0.1％ bqterrace -0.1％ -3.4％ -6.1％ -0.4％ cactus 0.1％ -2.4％ -0.5％ -0.1％ kimono 0.2％ -4.1％ -0.4％ -0.1％ Parks Cene 0.0％ -5.2％ -0.7％ -0.4％ marketplace 0.1％ -4.7％ -4.9％ -0.2％ ritualdance 0.0％ -1.1％ -0.1％ 0.0％ average 0.1％ -5.8％ -6.9％ -0.4％

[0224] Table 1. Results when refinement is applied for all blocks

[0225] refine blk128 Y psnr U psnr V psnr YUV psnr cosmos 0.1％ -17.4％ -25.3％ -1.1％ hurdles 0.1％ -10.7％ -20.8％ -1.2％ market 0.0％ -2.6％ -1.5％ -0.1％ showgirl 0.1％ -18.3％ -9.8％ -1.1％ starting 0.1％ -4.5％ -12.5％ -0.8％ bbdrive 0.1％ -1.9％ -2.4％ -0.1％ baterrace 0.0％ -3.6％ -6.7％ -0.3％ cactus 0.1％ -3.0％ -0.9％ -0.1％ kimono 0.2％ -4.1％ -0.8％ 0.0％ Parks Cene 0.0％ -5.6％ -0.7％ -0.3％ marketplace 0.0％ -5.5％ -6.4％ -0.4％ ritualdance -0.2％ -1.5％ -0.7％ -0.3％ average 0.0％ -6.6％ -7.4％ -0.5％

[0226] Table 2. Results when enabling refinement activation for each block of size 128x128

[0227] refine blk64 Y psnr U psnr V psnr YUV psnr cosmos 0.1％ -17.4％ -25.3％ -1.1％ hurdles 0.1％ -11.0％ -20.9％ -1.2％ market 0.0％ -4.2％ -2.2％ -0.1％ showgirl 0.2％ -20.7％ -10.1％ -1.1％ starting 0.2％ -6.0％ -13.0％ -0.9％ bbdrive 0.0％ -2.5％ -2.0％ -0.2％ bqterrace 0.1％ -3.9％ -6.7％ -0.2％ cactus 0.0％ -2.7％ -0.9％ -0.2％ kimono 0.2％ -4.3％ -1.4％ -0.1％ parkscene 0.0％ -6.5％ -1.0％ -0.4％ marketplace 0.1％ -5.7％ -8.2％ -0.4％ ritualdance 0.1％ -1.3％ -0.4％ 0.0％ average 0.1％ -7.2％ -7.7％ -0.5％

[0228] Table 3. Results when enabling refinement activation for each block of size 64x64

[0229] refine blk32 Y psnr U psnr U psnr YUV psnr cosmos 0.0％ -18.5％ -27.0％ -1.3％ hurdles 0.1％ -12.4％ -21.2％ -1.3％ market 0.1％ -6.1％ -2.8％ 0.0％ showgirl 0.3％ -22.6％ -11.3％ -1.1％ starting 0.3％ -7.6％ -13.7％ -0.9％ bbdrive 0.1％ -3.3％ -3.3％ -0.2％ bqterrace 0.1％ -4.6％ -7.6％ -0.2％ cactus 0.2％ -3.1％ -0.9％ 0.0％ kimono 0.2％ -5.2％ -1.6％ -0.2％ Parks Cene 0.1％ -7.1％ -1.2％ -0.4％ marketplace 0.2％ -6.8％ -8.9％ -0.3％ ritualdance 0.2％ -1.2％ -1.0％ 0.1％ average 0.2％ -8.2％ -8.4％ -0.5％

[0230] Table 4. Results when enabling refinement activation for each block of size 32x32

[0231] best Y psnr U psnr V psnr YUV psnr cosmos 0.0％ -18.5％ -27.0％ -1.3％ hurdles 0.1％ -12.4％ -21.2％ -1.3％ market 0.0％ -2.6％ -1.5％ -0.1％ showgirl 0.2％ -20.7％ -10.1％ -1.1％ starting 0.3％ -7.6％ -13.7％ -0.9％ bbdrive 0.0％ -2.5％ -2.0％ -0.2％ bqterrace -0.1％ -3.4％ -6.1％ -0.4％ cactus 0.0％ -2.7％ -0.9％ -0.2％ kimono 0.2％ -5.2％ -1.6％ -0.2％ Parks Cene 0.0％ -6.5％ -1.0％ -0.4％ marketplace 0.1％ -5.7％ -8.2％ -0.4％ ritualdance -0.2％ -1.5％ -0.7％ -0.3％ average 0.0％ -7.4％ -7.8％ -0.6％

[0232] Table 5. Results when selecting the best result for each sequence

[0233] This document describes various examples of embodiments, features, models, methods, etc. Many of these examples are specifically described, and at least to illustrate individual characteristics, are often described in a manner that may appear limited. However, this is for the purpose of clarity of description and does not limit the application or scope. In fact, the various examples of embodiments, features, etc. described herein can be combined and interchanged in various ways to provide other examples of embodiments.

[0234] In general, examples of the embodiments described and contemplated in this document may be implemented in many different forms. Figure 1 and Figure 2 and described below Fig.15 Some embodiments are provided, but other embodiments are contemplated and are not intended to be construed in detail. Figure 1 , Figure 2 and Fig.15The discussion does not limit the scope of implementation. At least one embodiment generally provides examples related to video encoding and / or decoding, and at least one other embodiment generally relates to sending a generated or encoded bitstream or signal. These and other embodiments may be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream or signal generated according to any of the described methods.

[0235] In this application, the terms "reconstructed" and "decoded" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstructed" is used on the encoder side, and "decoded" is used on the decoder side.

[0236] The terms HDR (high dynamic range) and SDR (standard dynamic range) are used in this disclosure. These terms generally convey specific values ​​of dynamic range to those of ordinary skill in the art. However, additional embodiments are also intended in which references to HDR are understood to mean "higher dynamic range" and references to SDR are understood to mean "lower dynamic range". Such additional embodiments are not constrained by any specific value of dynamic range, which is often associated with the terms "high dynamic range" and "standard dynamic range".

[0237] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0238] Various methods and other aspects described in this document can be used to modify modules, such as Figure 1 and Figure 2 1 and 2. Intra prediction (160), entropy coding (145), and / or decoding modules (230) of the video encoder 100 and decoder 200 shown in FIG. 1 and 2. In addition, the present aspects are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether already existing or developed in the future, as well as extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise specified or technically prohibited, the various aspects described in this document can be used alone or in combination.

[0239] For example, various numerical values ​​are used in this document. The specific values ​​are for example purposes, and the various aspects described are not limited to these specific values.

[0240] Fig.15A block diagram of an example of a system in which various aspects and embodiments can be implemented is illustrated. System 1500 can be embodied as a device including various components described below, and is configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 1500 can be embodied in a single integrated circuit, multiple ICs and / or discrete components, either individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1500 are distributed over multiple ICs and / or discrete components. In various embodiments, system 1500 is coupled to other similar systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 1500 is configured to implement one or more aspects described in this document.

[0241] System 1500 includes at least one processor 1010, which is configured to execute instructions loaded therein to implement various aspects described in this document, for example. Processor 1010 may include embedded memory, input and output interfaces, and various other circuits known in the art. System 1500 includes at least one memory 1520 (e.g., volatile storage device and / or non-volatile storage device). System 1500 includes storage device 1540, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive and / or optical drive. As a non-limiting example, storage device 1540 may include internal storage device, attached storage device and / or network accessible storage device.

[0242] The system 1500 includes an encoder / decoder module 1530, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1530 may include its own processor and memory. The encoder / decoder module 1530 represents a module that can be included in a device to perform encoding and / or decoding functions. As is well known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1530 may be implemented as a separate element of the system 1500, or may be incorporated into the processor 1010 as a combination of hardware and software, which is well known to those skilled in the art.

[0243] Program code to be loaded onto the processor 1010 or the encoder / decoder 1530 to perform various aspects described in this document may be stored in the storage device 1540 and subsequently loaded onto the memory 1520 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1520, the storage device 1540, and the encoder / decoder module 1530 may store one or more of various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or a portion of decoded video, bitstreams or signals, matrices, variables, and intermediate or final results from equations, formulas, operations, and operational logic.

[0244] In several embodiments, memory internal to the processor 1010 and / or encoder / decoder module 1530 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1530) is used for one or more of these functions. The external memory may be memory 1520 and / or a storage device 1540, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations such as MPEG-2, HEVC, or VVC (Versatile Video Coding).

[0245] Input to the elements of system 1500 may be provided through various input devices as indicated in block 1130. Such input devices include, but are not limited to: (i) an RF section that receives an RF signal wirelessly transmitted, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0246] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower frequency band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF section of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal that sends on wired (for example cable) medium, and filter to the frequency band of expectation again by filtering, down-conversion and perform frequency selection.Each embodiment rearranges the order of above-mentioned (and other) element, removes some in these elements, and / or adds other elements that perform similar or different functions.Adding element can include and insert element between existing element, for example, inserts amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0247] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1500 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or in the processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented in a separate interface IC or in the processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 1010 and an encoder / decoder 1530 that operates in combination with memory and storage elements to process data streams for presentation on an output device.

[0248] The various elements of the system 1500 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data may be transmitted between them using suitable connection means 1140 (eg, an internal bus known in the art, including an I2C bus, wiring, and printed circuit boards).

[0249] The system 1500 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0250] In various embodiments, a Wi-Fi network such as IEEE 802.11 is used to stream data to the system 1500. The Wi-Fi signals of these embodiments are received on a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to an external network including the Internet to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to the system 1500, which delivers data through the HDMI connection of the input block 1130. Other embodiments also use the RF connection of the input block 1130 to provide streaming data to the system 1500.

[0251] The system 1500 can provide output signals to various output devices including a display 1100, a speaker 1110, and other peripherals 1120. In various examples of embodiments, the other peripherals 1120 include one or more of a stand-alone DVR, a hard disk player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 1500. In various embodiments, control signals are communicated between the system 1500 and the display 1100, the speaker 1110, or other peripherals 1120 using signaling such as AV.Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 1500 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be connected to the system 1500 using a communication channel 1060 via a communication interface 1050. The display 1100 and the speaker 1110 can be integrated into a single unit with other components of the system 1500 in an electronic device (e.g., a television). In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0252] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0253] The embodiments may be implemented by computer software implemented by the processor 1010 or by hardware or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1520 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as optical storage devices, magnetic storage devices, semiconductor-based storage devices, fixed memory, and removable memory as non-limiting examples. As a non-limiting example, the processor 1010 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0254] The following describes Figures 16 to 30 , which generally depicts various other examples of embodiments suitable for implementing one or more aspects described herein.

[0255] Fig.16 16 shows an example of the architecture of a transmitter 1600 configured to encode a picture in a bitstream according to a specific and non-limiting embodiment.

[0256] The transmitter 1600 includes one or more processors 1005, which may include, for example, a CPU, a GPU and / or a DSP (English abbreviation for a digital signal processor), and an internal memory 1630 (e.g., RAM, ROM and / or EPROM). The transmitter 1600 includes one or more communication interfaces 1610 (e.g., a keyboard, a mouse, a touchpad, a webcam), each suitable for displaying output information and / or allowing a user to input commands and / or data; and a power supply 1620 that may be located outside the transmitter 1600. The transmitter 1600 may also include one or more network interfaces (not shown). The encoder module 1640 represents a module that may be included in the device to perform an encoding function. In addition, the encoder module 1640 may be implemented as a separate element of the transmitter 1600, or may be incorporated into the processor 1005 as a combination of hardware and software known to those skilled in the art.

[0257] The image can be obtained from a source. According to different embodiments, the source can be but is not limited to:

[0258] - local storage, such as video memory, RAM, flash memory, hard disk;

[0259] - storage interfaces, such as interfaces to mass storage, ROM, optical disks or magnetic supports;

[0260] - a communication interface, for example, a wired interface (eg, a bus interface, a wide area network interface, a local area network interface) or a wireless interface (such as an IEEE 802.11 interface or a Bluetooth interface); and

[0261] - picture capture circuitry (e.g., such as, for example, a CCD (or charge coupled device) or a CMOS

[0262] (or complementary metal oxide semiconductor) sensors).

[0263] According to different embodiments, the bitstream can be sent to the destination. As an example, the bitstream is stored in a remote or local memory, such as a video memory or RAM, a hard disk. In a variant, the bitstream is sent to a storage interface, such as an interface with a mass storage, ROM, flash memory, optical disk or magnetic support, and / or sent through a communication interface (e.g., an interface to a point-to-point link, a communication bus, a point-to-multipoint link or a wide area network).

[0264] According to a non-limiting example of embodiment, the transmitter 1600 further comprises a computer program stored in the memory 1630. The computer program comprises instructions which, when executed by the transmitter 1600, in particular by the processor 1005, enable the transmitter 1600 to perform a reference Fig. 20 The encoding method described. According to a variant, the computer program is stored outside the transmitter 1600 on a non-transitory digital data support, for example on an external storage medium such as a HDD, a CD-ROM, a DVD, a read-only and / or DVD drive and / or a DVD read / write drive, all of which are well known in the art. The transmitter 1600 therefore includes a mechanism for reading the computer program. In addition, the transmitter 1600 can access one or more storage devices of the Universal Serial Bus (USB) type (e.g., a "memory stick") via corresponding USB ports (not shown).

[0265] According to one or more non-limiting examples of embodiments, the transmitter 1600 may be, but is not limited to:

[0266] -mobile device;

[0267] -communication equipment;

[0268] -Gaming equipment;

[0269] - Tablet (or tablet computer);

[0270] - Laptop;

[0271] -Still picture camera;

[0272] - Video camera;

[0273] - coding chip or coding device / means;

[0274] -Still image server; and

[0275] - Video server (eg, broadcast server, video-on-demand server, or web server).

[0276] Fig.17 The diagram is suitable for execution Fig. 20 The encoder 100 is an example of a video encoder 100 (eg, an encoder of the HEVC type) for encoding a method of encoding a video signal of the present invention. The encoder 100 is an example of a transmitter 1600 or a part of such a transmitter 1600.

[0277] For encoding, pictures are typically divided into basic coding units, such as coding tree units (CTUs) in HEVC or macroblock units in H.264. A set of possible consecutive basic coding units is grouped into slices. A basic coding unit contains basic coding blocks for all color components. In HEVC, the smallest coding tree block (CTB) size of 16x16 corresponds to the macroblock size used in previous video coding standards. It will be understood that although the terms CTU and CTB are used herein to describe encoding / decoding methods and encoding / decoding devices, these methods and devices should not be limited to these specific terms that may be differently worded (e.g., macroblocks) in other standards (such as H.264).

[0278] In HEVC encoding, pictures are partitioned into square CTUs of configurable size (typically 64x64, 128x128, or 256x256). A CTU is the root of a quadtree of four square coding units (CUs) of equal size (i.e., width and height are half the size of the parent block). A quadtree is a tree in which a parent node can be split into four child nodes, each of which can become a parent node for another split into four child nodes. In HEVC, a coding block (CB) is partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree, which is partitioned into transform blocks (TBs). Corresponding to the coding blocks, prediction blocks, and transform blocks, a coding unit (CU) includes a prediction unit (PU) and a tree-structured transform unit (TU) set, the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The sizes of the CB, PB, and TB of the luminance component apply to the corresponding CU, PU, ​​and TU.

[0279] In recent coding systems, CTU is the root of the coding tree that is divided into coding units (CUs). A coding tree is a tree in which a parent node (usually corresponding to a CU) can be split into child nodes (e.g., split into 2, 3, or 4 child nodes), each of which can become a parent node for further splitting into child nodes. In addition to the quadtree splitting mode, new splitting modes (binary tree symmetric splitting mode, binary tree asymmetric splitting mode, and ternary tree splitting mode) are defined, which increase the total number of possible splitting modes. The coding tree has a unique root node, such as CTU. The leaves of the coding tree are the terminal nodes of the tree. Each node of the coding tree represents a CU, which can be further split into smaller CUs, also called sub-CUs or more generally sub-blocks. Once it is determined that the CTU is split into CUs, the CU corresponding to the leaves of the coding tree is encoded. The splitting of CTU to CU and the coding parameters for encoding each CU (corresponding to the leaves of the coding tree) can be determined on the encoder side by a rate-distortion optimization process. The CB is not split into PB and TB, that is, the CU consists of a single PU and a single TU.

[0280] Hereinafter, the term "block" or "picture block" may be used to refer to any one of CTU, CU, PU, ​​TU, CB, PB, and TB. In addition, the term "block" or "picture block" may be used to refer to macroblocks, partitions, and sub-blocks specified in H.264 / AVC or other video coding standards, and more generally to sample arrays of various sizes.

[0281] Back to Fig.17 In an example of an embodiment of the encoder 100, a picture is encoded by the encoder elements described below. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra or inter mode. When a CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105D) which of intra mode or inter mode is used to encode the CU, and indicates the intra / inter decision by a prediction mode flag. The residual is calculated by subtracting (110) a block of prediction samples (also called a predictor) from the original picture block.

[0282] A CU in intra mode is predicted from reconstructed neighboring samples (e.g., in the same slice). A set of 35 intra prediction modes are available in HEVC, including DC, planar, and 33 angular prediction modes. A CU in inter mode is predicted from reconstructed samples of a reference picture stored in a reference picture buffer (180).

[0283] The residual is transformed (125) and quantized (130). The quantized transform coefficients, along with motion vectors and other syntax elements, are entropy encoded (145) to output a bitstream. The encoder can also skip the transform, or bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process.

[0284] Entropy coding can be, for example, context adaptive binary arithmetic coding (CABAC), context adaptive variable length coding (CAVLC), Huffman, arithmetic, exp-Golomb, etc. CABAC is an entropy coding method first introduced in H.264 and also used in HEVC. CABAC involves binarization, context modeling, and binary arithmetic coding. Binarization maps syntax elements to binary symbols (bins). Context modeling determines the probability of each regularly coded bin (i.e., not bypassed) based on a particular context. Finally, binary arithmetic coding compresses the bins into bits according to the determined probabilities.

[0285] The encoder comprises a decoding loop and thus decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the residual. The picture block is reconstructed by combining (155) the decoded residual and the predicted sample block. Optionally, an in-loop filter (165) is applied to the reconstructed picture, for example, to perform DBF (deblocking filtering) / SAO (sample adaptive offset) / ALF (adaptive loop filtering) to reduce coding artifacts. The filtered picture can be stored in a reference picture buffer (180) and used as a reference for other pictures. In this embodiment, refinement data is determined (190) from the reconstructed picture after filtering (i.e., the output of the in-loop filter) and its original version. In a first variant, the refinement data is determined (190) from the reconstructed picture before in-loop filtering and its original version. In this first variant, the refinement is applied before in-loop filtering. In a second variant, refinement data is determined (190) from a partially filtered reconstructed picture (e.g. after deblocking filtering but before SAO) and its original version. In this second variant, refinement is applied, e.g. immediately after partial filtering, e.g. after deblocking filtering but before SAO. The refinement data represents a correction function, denoted R(), which is applied to individual samples of a color component (e.g. a luminance component Y or a chrominance component Cb / Cr or a color component R, G or B). The refinement data is then entropy encoded in the bitstream. In this embodiment, the refinement process is outside the decoding loop. Therefore, the refinement process is only applied as post-processing in the decoder.

[0286] Fig.18 Picture shows Fig.17 A variation 1800 of the video encoder 100 is provided. Fig.17 The modules are the same Fig.18 The modules of are labeled with the same reference numerals and are not described further. Before being encoded by the video encoder 1800, the picture can be mapped (105). Such mapping can be used to better utilize the sample codeword value distribution of the picture. Fig.18 The mapping is applied to the original (input) samples before the core encoding shown in . Usually a static mapping function is used, i.e. the same function for all content, in order to limit complexity. The mapping function fmap(), which may be modeled by a one-dimensional lookup table LUTmap[x] (x is a value), is directly applied to the input signal x as follows:

[0287] y = fmap(x) or

[0288] y=LUTmap[x]

[0289] Where x is the input signal (eg, from 0 to 1023 for a 10-bit signal) and y is the mapped signal.

[0290] In a variant, the mapping function of one component depends on another component (cross-component mapping function). For example, the chrominance component c is mapped according to the luma component y located at the same relative position in the picture. The chrominance component c is mapped as follows:

[0291] c = offset + fmap (y) * (c - offset) or

[0292] c=offset+LUTmap[y]*(c-offset)

[0293] The offset is usually the center value of the chrominance signal (eg, 512 for a 10-bit chrominance signal). This parameter can also be a dynamic parameter encoded in the stream, which can ultimately improve the compression gain.

[0294] The mapping function can be defined by default or signaled in the bitstream, for example using a piecewise linear model, a scaling table, or a delta QP (dQP) table.

[0295] The filtered reconstructed picture (i.e., the output of the in-loop filter) is inverse mapped (185). Inverse mapping (185) is an implementation of the inverse process of mapping (105). Refinement data is determined (190) from the filtered reconstructed picture after inverse mapping and its original version. The refinement data is then entropy encoded in the bitstream. In this embodiment, the refinement process is outside the decoding loop. Therefore, the refinement process is only applied as post-processing in the decoder.

[0296] Fig.19 Picture shows Fig.17 A variation 1900 of the video encoder 100 is provided. Fig.17 The modules are the same Fig.19 The modules are labeled with the same reference numerals and are not described further. Mapping module 1905 is optional. Refinement data is determined (190) from the filtered reconstructed picture (i.e., the output of the in-loop filter) and its original version when mapping is not applied, or from its mapped original version when mapping is applied. The refinement data is then entropy encoded (145) in the bitstream. The refined data is also used to refine (182) the filtered reconstructed picture. The refined picture, rather than the filtered reconstructed picture, is stored in the reference picture buffer (180). In this embodiment, the refinement process is an in-loop process, i.e., it is part of the decoding loop. Therefore, the refinement process is applied to both the decoding loops of the encoder and the decoder. Modules 182 and 190 can be inserted at different locations. Refinement module 182 can be inserted before the in-loop filter or between the in-loop filters (e.g., after DBF and before SAO) in the case of at least two in-loop filters. Module 190 is placed so that it takes as input the same picture as refinement module 182, ie the reconstructed picture in case module 182 is before the in-loop filter, or the partially filtered reconstructed picture in case module 182 is between in-loop filters.

[0297] Fig. 20 Flowchart showing a method for encoding a picture portion in a bitstream according to a specific and non-limiting embodiment. The method starts at step S100. At step S110, a transmitter 1600 (such as, for example, encoder 100, 1800 or 1900) accesses a picture portion. Before being encoded, the accessed picture portion may optionally be encoded as Fig.18 and Fig.19 In step S120, the transmitter encodes and reconstructs the accessed picture portion to obtain a reconstructed picture portion. For this purpose, the picture portion can be split into blocks. Encoding the picture portion includes encoding the blocks of the picture portion. Encoding the block typically but not necessarily includes subtracting the predictor from the block to obtain a residual block, converting the residual block into a transform coefficient block, quantizing the coefficient block with a quantization step size to obtain a quantized transform coefficient block, and entropy encoding the quantized transform coefficient block in the bitstream. Reconstructing the block on the encoder side typically but not necessarily includes dequantizing and inverse transforming the quantized transform coefficient block to obtain a residual block, and adding the predictor to the residual block to obtain a decoded block. The reconstructed picture portion can then be filtered by an in-loop filter (e.g. Figure 17-Figure 19 deblocking / SAO / ALF filter in , and also as Fig.18 The inverse mapping is performed as in .

[0298] In step S130, refinement data is determined, for example, by module 190, such that a rate-distortion cost is reduced or minimized, the rate-distortion cost being calculated as the data encoding cost (i.e., the encoding cost of the refinement data and the refined picture portion) and the original version of the picture portion (i.e., the one that is accessed, possibly as Fig.19 ) and possibly Fig.17 and Fig.19 Filter as in or as Fig.18 More precisely, the refinement data is determined such that:

[0299] - If there is no mapping Fig.17 and Fig.19 a rate-distortion cost calculated as a weighted sum of a data coding cost (i.e., a coding cost of the refined picture portion and a refined picture portion) and a distortion between an original version of the picture portion and the reconstructed picture portion after refinement by the refinement data is reduced compared to a rate-distortion cost calculated as a weighted sum of a data coding cost (i.e., a coding cost of the picture portion without refinement) and a distortion between the original version of the picture portion and the reconstructed picture portion without refinement, when no mapping is applied before encoding as in

[0300] - In Fig.18 wherein, in case mapping is applied before encoding and the refinement is outside a decoding loop as in (with mapping and outside loop refinement), a rate-distortion cost calculated as a weighted sum of a data encoding cost (refinement data and encoding cost of the refined picture portion) and a distortion between the original version of the picture portion and the reconstructed picture portion after inverse mapping and refinement by the refinement data is reduced compared to a rate-distortion cost calculated as a weighted sum of a data encoding cost and a distortion between the original version of the picture portion and the reconstructed picture portion after inverse mapping and without refinement;

[0301] - In such as with mapping Fig.19 In a case where mapping is applied before encoding as in , and the refinement is within a decoding loop, a rate-distortion cost calculated as a weighted sum of a data encoding cost and a distortion between the mapped original version of the picture portion and the reconstructed picture portion after refinement by the refinement data is reduced compared to a rate-distortion cost calculated as a weighted sum of a data encoding cost and a distortion between the mapped original version of the picture portion and the reconstructed picture portion without refinement.

[0302] The reconstructed picture portion used for determining refinement data may be an in-loop filtered version or an in-loop locally filtered version of the reconstructed picture portion.

[0303] In a specific and non-limiting embodiment, the refined data, labeled R, is advantageously modeled by a piecewise linear model (PWL) defined by N pairs (R_idx[k], R_val[k]), k=0 to N-1. Each pair defines a pivot point of the PWL model. Figure 21-26 Step S130 is described in detail above. R_idx[k] is typically a value within the range of the signal under consideration, for example, for a 10-bit signal, the range is 0 to 1023. Advantageously, R_idx[k] is greater than R_idx[k-1].

[0304] At step S140, refinement data is encoded in the bitstream or signal. A non-limiting example of an embodiment of the syntax is provided below. It can be considered that refinement is performed on these three components. In a variant, syntax only for some components (e.g., only for two chrominance components) can be encoded and applied. The refinement data is encoded in the form of a refinement table.

[0305] refinement_data(){ refinement_table_new_flag if(refinement_table_new_flag){ refinement_table_flag_luma if(refinement_table_flag_luma){ refinement_table_luma_size for(i=0;i<refinement_table_luma_size;i++){ refinement_luma_idx[i] refinement_luma_value[i] } } refinement_table_flag_cb if (refinement_table_flag_cb) { refinement_table_cb_size for(i=0;i<refinement_table_cb_size;i++){ refinement_cb_idx[i] refinement_cb_value[i] } } refinement_table_flag_cr if (refinement_table_flag_cr) { refinement_table_cr_size for(i=0;i<refinement_table_cr_size;i++){ refinement_cr_idx[i] refinement_cr_value[i] } } } }

[0306] If refinement_table_new_flag is equal to 0, or refinement_table_flag_luma is equal to 0, no refinement is applied for luma.

[0307] Otherwise, for all pt from 0 to (refinement_table_luma_size-1), the luma refinement values ​​(R_idx[pt], R_val[pt]) are calculated as follows:

[0308] R_idx[pt] is set equal to (default_idx[pt] + refinement_luma_idx[pt])

[0309] R_val[pt] is set equal to (NeutralVal + refinement_luma_value[pt])

[0310] A similar process is applied for the cb or cr component.

[0311] For example, NeutralVal = 128. It can be used for Fig.21 and Fig.24 The value R_val is initialized in step S130 as described in detail in . NeutralVal and default_idx[pt] may be default values ​​known to both the encoder and decoder sides, in which case they do not need to be sent. In a variant, NeutralVal and default_idx[pt] may be values ​​encoded in the bitstream or signal.

[0312] Preferably, for pt from 0 to (refinement_table_luma_size-1), default_idx[pt] is defined as:

[0313] default_idx[pt]=(MaxVal / (refinement_table_luma_size-1))*pt

[0314] or

[0315] default_idx[pt]=((MaxVal+1) / (refinement_table_luma_size-1))*pt

[0316] It corresponds to equidistant indices from 0 to (MaxVal) or (Max+1), where MaxVal is the maximum value of the signal (eg, 1023 when the signal is represented by 10 bits).

[0317] In a variant, when mapping is applied and based on the PWL mapping table defined by the (map_idx[k],map_val[k]) pair, R_idx[k] is initialized by map_idx[k] for k=0 to N-1. In other words, default_idx[k] is equal to map_idx[k]. Similarly, for k=0 to N-1, R_val[k] can be another variant initialized by map_val[k] for k=0 to N-1.

[0318] The syntax elements refinement_table_luma_size, refinement_table_cb_size, refinement_table_cr_size may be defined by default values, in which case they do not need to be coded in the stream.

[0319] In the case of equidistant points of the luma PWL model, refinement_luma_idx[pt] does not need to be coded. refinement_luma_idx[pt] is set to 0, so that for pt from 0 to (refinement_table_luma_size-1), R_idx[pt]=default_idx[pt]. The same applies for the cb or cr table.

[0320] In a variation, a syntax element may be added for each table to indicate which refinement mode is used (between intra-component (mode 1) and / or inter-component (mode 2)). The syntax elements may be signaled at SPS, PPS, slice, tile or CTU level.

[0321] In a variation, a syntax element may be added for each table to indicate whether the table is applied as a multiplication operator or an addition operator. The syntax element may be signaled at SPS, PPS, slice, tile or CTU level.

[0322] In an embodiment, the refinement table is not encoded in the bitstream or signal. Instead, a default inverse mapping table (corresponding to Fig.18 and Fig.19 The inverse of the mapping table used by the mapping 105 on (i) is modified by the refinement table, and the modified inverse mapping table is encoded in the bitstream or signal.

[0323] In an embodiment, the refinement table is encoded only for pictures at low temporal levels. For example, the table is encoded only for pictures at temporal level 0 (the lowest level in the temporal coding hierarchy).

[0324] In an embodiment, refinement tables are encoded only for random access pictures such as intra pictures.

[0325] In an embodiment, refinement tables are encoded only for high quality pictures, which correspond to the average QP over pictures below a given value.

[0326] In an embodiment, a refinement table is encoded only when the refinement table encoding cost relative to the full picture encoding cost is below a given value.

[0327] In an embodiment, the refinement table is encoded only when the rate-distortion gain compared to not encoding the refinement table is above a given value. For example, the following rule may be applied:

[0328] If the rate-distortion cost gain is greater than (0.01*width*height), where width and height are the dimensions of the considered component, then the table is encoded.

[0329] If the rate-distortion cost gain is greater than (0.0025*initRD), where initRD is the rate-distortion cost when component refinement is not applied, the table is encoded. This corresponds approximately to a minimum PSNR gain of 0.01 dB when the rate-distortion cost is based on squared difference distortion. If the given value is set to (0.01*initRD), this corresponds approximately to a minimum PSNR gain of 0.05 dB when the rate-distortion cost is based on squared difference distortion. The values ​​described are examples only and can be modified.

[0330] Back to Fig. 20 In optional step S150, the refined data is applied to possible Fig.19 On the part of the reconstructed image that is filtered as in .

[0331] For this purpose, a lookup table LutR is determined from pairs of points of the PWL (R_idx[pt], R_val[pt]) for pt=0 to N-1.

[0332] For example, LutR is determined by linear interpolation between each pair of points in PWL (R_idx[pt], R_val[pt]) and (R_idx[pt+1], R_val[pt+1]) as follows:

[0333] For pt = 0 to N-2

[0334] For idx=R_idx[pt] to (R_idx[pt+1]-1)

[0335] LutR[idx]=R_val[pt]+(R_val[pt+1]-R_val[pt])*(idx-R_idx[pt]) / (R_idx[

[0336] pt+1]-R_idx[pt])

[0337] In a variation, LutR is determined as follows:

[0338] For pt = 0 to N-2

[0339] For idx=R_idx[pt] to (R_idx[pt+1]-1)

[0340] LutR[idx]=(R_val[pt]+R_val[pt+1]) / 2

[0341] Two examples of refinement mode are as follows:

[0342] Mode 1 - Intra-component refinement. In mode 1, refinement is performed independently of other components. The signal Srec(p) is refined as follows:

[0343] Sout(p)=LutR[Srec(p)] / NeutralVal*Srec(p)

[0344] Where Srec(p) is the reconstructed signal to be refined at position p in the picture portion and corresponds to the signal from the in-loop filter or possibly from the inverse mapping, and Sout(p) is the refined signal. It is assumed here that the signals Srec and Sout use the same bit depth. When different bit depths are used for the two signals (i.e. Bout for Sout and Brec for Srec), a scaling factor related to the bit depth difference can be applied. For example, when the bit depth Bout of Sout is greater than the bit depth Brec of Srec, the following formula is used:

[0345] Sout(p)=2 (Bout-Brec) *LutR[Srec(p)] / NeutralVal*Srec(p)

[0346] The scaling factor can be integrated directly into the LutR value. For example, when the bit depth Bout of Sout is lower than the bit depth Brec of Srec, the following formula is used:

[0347] Sout(p)=LutR[Srec(p)] / NeutralVal*Srec(p) / 2 (Brec-Bout)

[0348] The scaling factor can be integrated directly in the LutR value.

[0349] Advantageously, mode 1 is used for the luminance component.

[0350] Mode 2 - Inter-component refinement. In mode 2, refinement is performed on one component C0 and depends on another component C1. The signal Srec_C0(p) is refined as follows:

[0351] Sout(p)=offset+LutR[Srec_C1(p)] / NeutralVal*(Srec_C0(p)-offset)

[0352] where offset is set to (MaxVal / 2), Srec_C0(p) is the reconstructed signal of the component C0 to be refined, and p is the sample position in the picture portion. MaxVal is the maximum value of the signal Srec_C0 and is calculated as (2 B -1), where B is the bit depth of the signal. Srec_C1(p) is the reconstructed signal of component C1. Srec_C1(p) can be a signal from an in-loop filter or from an inverse mapping. After filtering by the in-loop filter, Srec_C1(p) can also be further filtered, for example with a low-pass filter. It is assumed here that the signals Srec_C0, Srec_C1 and Sout use the same bit depth. Advantageously, mode 2 can be applied to chrominance components and depends on the luminance component. When the bit depth Bout of Sout is higher than the bit depth Brec of Srec_C0 and Srec_C1, the formula is adopted as follows:

[0353] Sout(p)=2 (Bout-Brec) *(offset+LutR[Srec_C1(p)] / NeutralVal*(Srec_C0(p)-offset))

[0354] When the bit depth Bout of Sout is lower than the bit depth Brec of Srec_C0 and Srec_C1, the formula is used as follows:

[0355] Sout(p)=(offset+LutR[Srec_C1(p)] / NeutralVal*(Srec_C0(p)-offset)) / 2 (Bout-Brec)

[0356] Rounding and clipping between minimum and maximum signal values ​​(typically 0 and 1023 for a 10-bit signal) are finally applied to the refined value Sout(p).

[0357] In the above described embodiment, the refinement is used as a multiplication operator.

[0358] In a variant, the refinement is used as an addition operator. In this case, in mode 1, the signal Srec(p) is refined as follows:

[0359] οSout(p)=LutR[Srec(p)] / NeutralVal+Srec(p)

[0360] In this case, in mode 2, the signal Srec_C0(p) is refined as follows:

[0361] οSout(p)=LutR[Srec_C1(p)] / NeutralVal+Srec_C0(p)

[0362] For NeutralVal=64 and N=17, an example of cross-component refinement of table values ​​for chrominance components Cb and Cr is as follows:

[0363] Cb

[0364] k 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 R_idx[k] 0 64 128 192 256 320 384 448 512 576 640 702 766 830 896 960 1024 R_val[k] 64 64 64 52 58 63 66 65 62 66 66 64 72 49 64 64 64

[0365] Cr

[0366] k 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 R_idx[k] 0 64 128 192 256 320 384 448 512 576 640 702 766 830 896 960 1024 R_val[k] 64 64 62 61 63 66 65 66 64 60 70 64 64 41 64 64 64

[0367] return Fig. 20 , the method ends at step S160.

[0368] Fig.21 A flowchart illustrating an example of further details of step S130 is shown. The refinement data may be determined over a given picture region A (e.g., a slice, tile, or CTU) or over the entire picture. The refinement data R may be advantageously modeled by a piecewise linear model (PWL) defined by N pairs (R_idx[k], R_val[k]), k=0 to N-1. Fig. 22An example of a PWL model is illustrated in , where N = 6 (k∈{0,1,2,3,5}). Each pair defines a pivot point of the PWL model.

[0369] Initialize the values ​​R_idx and R_val (step S1300). Typically, for k=0 to N-1, R_idx[k] is initialized so that there is equidistant space between consecutive indices, i.e., (R_idx[k+1]-R_idx[k])=D, where D=Range / (N-1), Range is the range of the signal to be refined (e.g., 1024 for a signal represented in 10 bits). In the example, N=17 or 33. Initialize the value R_val with the value NeutralVal, which is defined so that refinement does not change the signal, e.g., 128. In a variant, when mapping is applied and based on the PWL mapping table defined by the (map_idx[k],map_val[k]) pair, for k=0 to N-1, (R_idx[k], R_val[k]) is initialized by (map_idx[k], map_val[k]).

[0370] exist Fig.21 In the embodiment of , only the value R_val[k] is determined. The value R_idx[k] is fixed to its initial value. The initial rate distortion cost initRD is calculated (step 1301) using R initialized as in S1300. The calculation of the initial rate distortion cost initRD is as follows:

[0371] initRD=L*Cost(R)+∑ pinA dist(Sin(p),Sout(p)) (Equation 1)

[0372] in:

[0373] -R is the refinement data initialized in S1300,

[0374] - A is the image region for which refinement is performed;

[0375] -Sin(p) is defined as follows:

[0376] When no mapping is applied, Sin(p) is the sample value of pixel p in the original picture region;

[0377] When mapping is applied and refinement is outside of a loop ( Fig.18 ), Sin(p) is the sample value of pixel p in the original image area;

[0378] When mapping is applied and refinement is in a loop ( Fig.19 ), Sin(p) is the sample value of pixel p in the mapped original image area;

[0379] - Sout(p) is the sample value of pixel p in the refined image area;

[0380] -dist(x,y) is the distortion between sample value x and sample value y, for example, the distortion is the squared difference (xy) 2 ; Other possible distortion functions are the absolute difference |xy|, or distortion based on subjectively oriented metrics such as SSIM (Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity", IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612, April 2004), or variations of SSIM may also be used;

[0381] Cost(R) is the coding cost of coding the refinement data R and the refinement picture area; and

[0382] - L is a value associated with the picture area A. Advantageously, it depends linearly on 2 (QP / 6) , where QP is a quantization parameter when a single quantization parameter value is used within the picture region A, or QP represents a quantization parameter applied to the picture region A when different quantization parameter values ​​are used within the picture region A. For example, QP is an average value of QPs used within the blocks of region A.

[0383] When the value R_val is initialized using the value NeutralVal, the initial rate-distortion cost initRD may be calculated using the encoding cost of the picture region without the encoding cost of the refinement data.

[0384] In step S1302, the parameter bestRD is initialized to initRD. Then, in step S1303, the refinement data R is determined. In step S1304, a loop is performed on the index pt of the consecutive pivot points of the PWL model R. In step S1305, the parameters bestVal and initVal are initialized to R_val[pt]. In step S1306, a loop is performed on the various values ​​of R_val[pt], that is, from the value (initVal-Val0) to the value (initVal+Val1), where Val0 and Val1 are predefined parameters. Typical values ​​Val0=Val1=NeutralVal / 4. In step S1307, the rate-distortion cost curRD is calculated using equation 1 with the current R (with the current R_val[pt]). In step S1308, curRD and bestRD are compared. If curRD is lower than bestRD, bestRD is set to curRD and bestValue is set to R_val[pt]. Otherwise, the method continues at step S1310. In step S1310, it is checked whether the loop on the value of R_val[pt] is finished. In the case of the loop being finished, R_val[pt] is set to bestValue in step S1311. In step S1312, it is checked whether the loop on the pt value is finished. In the case of the loop being finished, the current R is the output refinement data.

[0385] Step S1303 may be iterated n times, where n is a fixed integer, for example, n=3. Fig.23 The determination of R_val is illustrated. The dashed line represents the PWL after updating R_val.

[0386] Fig.24 express Fig.21 A variation of the process. Fig.24 In the embodiment of FIG. 1 , only the value R_idx[k] is determined. The value R_val[k] is fixed to its initial value. The process uses the initial R_idx and R_val data (e.g., from Fig.21method) as input. In step S1401, the initial rate-distortion cost initRD is calculated using (Equation 1). In step S1402, the parameter bestRD is initialized to initRD. Then in step S1403, the refinement data R is determined. In step S1404, a loop is performed on the index pt of the consecutive pivot points of the PWL model R. In step S1405, the parameters bestIdx and initIdx are initialized to R_idx[pt]. In step S1406, a loop is performed on each value of R_idx[pt] (from initIdx-idxVal0 to initIdx+idxVal1), where idxVal0 and idxVal1 are defined values, for example idxVal0=idxVal1=D / 4, where D=Range / (N-1), and Range is the signal range to be refined (for example, 1024 for a signal represented in 10 bits). In step S1407, the rate-distortion cost curRD is calculated using equation 1 with the current R (with the current R_idx[pt]). In step S1408, curRD and bestRD are compared. If curRD is lower than bestRD, bestRD is set to curRD and bestIdx is set to R_idx[pt]. Otherwise, the method continues at step S1310. In step S1410, check whether the loop on the value of R_idx[pt] is over. In the case where the loop is over, R_idx[pt] is set to bestIdx in step S1411. In step S1412, check whether the loop on the pt value is over. In the case where the loop is over, the current R is the output refined data. Step S1403 can be iterated n times, where n is an integer whose value is fixed, for example n=3.

[0387] Fig.25 The determination of R_idx is illustrated. The dashed line represents the PWL after the R_idx update.

[0388] Fig.26 The determination of R_idx and R_val is illustrated. The dashed line represents the PWL after the update of R_val and R_idx.

[0389] Fig. 272000 is an example of an architecture of a receiver 2000 configured to decode a picture from a bitstream or signal to obtain a decoded picture according to a specific and non-limiting embodiment. The receiver 2000 includes one or more processors 2005, which may include, for example, a CPU, a GPU and / or a DSP (English abbreviation for digital signal processor), and an internal memory 2030 (e.g., RAM, ROM and / or EPROM). The receiver 2000 includes one or more communication interfaces 2010 (e.g., keyboard, mouse, touchpad, webcam), each suitable for displaying output information and / or allowing a user to input commands and / or data (e.g., decoded pictures); and a power supply 2020 that may be external to the receiver 2000. The receiver 2000 may also include one or more network interfaces (not shown). The decoder module 2040 represents a module that may be included in the device to perform a decoding function. In addition, the decoder module 2040 may be implemented as a separate element of the receiver 2000, or may be incorporated into the processor 2005 as a combination of hardware and software known to those skilled in the art.

[0390] The bitstream or signal may be obtained from a source. According to different embodiments, the source may be, but is not limited to:

[0391] - local storage, such as video memory, RAM, flash memory, hard disk;

[0392] - storage interfaces, such as interfaces to mass storage, ROM, optical disks or magnetic supports;

[0393] - a communication interface, for example, a wired interface (eg, a bus interface, a wide area network interface, a local area network interface) or a wireless interface (such as an IEEE 802.11 interface or a Bluetooth interface); and

[0394] - Image capture circuitry (e.g., such as, for example, a CCD (or charge coupled device) or a CMOS

[0395] (or complementary metal oxide semiconductor) sensors).

[0396] According to different embodiments, the decoded picture can be sent to a destination, such as a display device. As an example, the decoded picture is stored in a remote or local memory, such as a video memory or RAM, a hard disk. In a variant, the decoded picture is sent to a storage interface, such as an interface with a mass storage, ROM, flash memory, optical disk or magnetic support, and / or sent through a communication interface (e.g., an interface to a point-to-point link, a communication bus, a point-to-multipoint link or a wide area network).

[0397] According to a non-limiting example of embodiment, the receiver 2000 further comprises a computer program stored in the memory 2030. The computer program comprises instructions which, when executed by the receiver 2000, in particular by the processor 2005, enable the receiver to perform a reference Fig.31 The decoding method described. According to a variant, the computer program is stored external to the receiver 2000 on a non-transitory digital data support, for example on an external storage medium such as a HDD, a CD-ROM, a DVD, a read-only and / or DVD drive and / or a DVD read / write drive, which are all well known in the art. The receiver 2000 therefore includes a mechanism for reading the computer program. In addition, the receiver 2000 can access one or more storage devices of the Universal Serial Bus (USB) type (e.g., a "memory stick") via corresponding USB ports (not shown).

[0398] According to one or more non-limiting examples of embodiments, the receiver 2000 may be, but is not limited to:

[0399] -mobile device;

[0400] -communication equipment;

[0401] -Gaming equipment;

[0402] - Set-top box;

[0403] -TV set;

[0404] - Tablet (or tablet computer);

[0405] - Laptop;

[0406] - Video players, such as Blu-ray players, DVD players;

[0407] - a display; and

[0408] -Decoding chip or decoding device / apparatus.

[0409] Fig.28 Illustration of the Fig.31 2 is a block diagram of an example of an embodiment of a video decoder 200 (e.g., of the HEVC type) for decoding methods of the present invention. The video decoder 200 is an example of a receiver 2000 or a portion of such a receiver 2000. In the illustrated example of an embodiment of the decoder 200, a bitstream or signal is decoded by a decoder element as described below. The video decoder 200 generally performs the same Fig.17 The encoding pass described in is opposite to the decoding pass, which performs video decoding as part of encoding the video data.

[0410] In particular, the input to the decoder includes a video bitstream or signal, which may be generated by the video encoder 100. The bitstream or signal is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other encoding information, such as refinement data. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the residual. The decoded residual is then combined (255) with a predicted block (also referred to as a predictor) to obtain a decoded / reconstructed picture block. The predicted block may be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). As described above, AMVP and merge mode techniques may be used during motion compensation, which may use interpolation filters to calculate interpolations of sub-integer samples of a reference block. An in-loop filter (265) is applied to the reconstructed picture. The in-loop filter may include a deblocking filter and a SAO filter. The filtered picture is stored in a reference picture buffer (280). The reconstructed picture, which may be filtered, is refined (290). Thinning is done outside the decoding loop and used as a post-processing pass.

[0411] Fig.29 Picture shows Fig.28 A variation 201 of the video decoder 200 is provided. Fig.28 The modules are the same Fig.29 The modules are labeled with the same reference numerals and are not described further. The filtered reconstructed picture (i.e., the output of the in-loop filter) is inverse mapped (285). The inverse mapping (285) is the inverse process of the mapping (105) applied on the encoder side. The inverse mapping can use an inverse mapping table decoded from the bitstream or signal or a default inverse mapping table. The inverse mapped picture is refined (290) using refinement data decoded from the bitstream or signal (230).

[0412] In a variant, the inverse mapping and refinement are combined in a single module which applies the inverse mapping using an inverse mapping table decoded from a bitstream or signal, wherein the inverse mapping table is modified in the encoder to take into account the refinement data. In a variant, for a given component being processed, a lookup table LutComb is applied to perform the inverse mapping and refinement process, and the lookup table is constructed as a concatenation of a lookup table LutInvMap derived from the mapping table and a lookup table derived from the refinement table LutR:

[0413] For x = 0 to MaxVal, LutComb[x] = LutR[LutInvMap[x]]

[0414] In this embodiment, the refinement process is outside the decoding loop. Therefore, the refinement process is only used as post-processing in the decoder.

[0415] Fig.30 Picture shows Fig.28A variation 202 of the video decoder 200 is provided. Fig.28 The modules are the same Fig.30 The modules are labeled with the same reference numerals and are not described further. Refinement data (230) is decoded from a bitstream or signal. The filtered reconstructed picture is refined (290) using the decoded refinement data. The refined picture is stored in a reference picture buffer (280) instead of the filtered reconstructed picture. Module 290 can be inserted at different locations. Refinement module 290 can be inserted before the in-loop filter or between the in-loop filters (e.g., after DBF and before SAO) in the case of at least two in-loop filters. The refined picture can optionally be inverse mapped (285). In this embodiment, the refinement process is in a decoding loop.

[0416] Fig.31 A flow chart showing a method for decoding a picture from a bitstream or signal according to a specific and non-limiting embodiment. The method starts at step S200. At step S210, a receiver 2000, such as decoder 200, accesses a bitstream or signal. At step S220, the receiver decodes a picture portion from the bitstream or signal to obtain a decoded picture portion. For this purpose, a block of the picture portion is decoded. Decoding the block typically, but not necessarily, comprises entropy decoding a portion of the bitstream or signal representing the block to obtain a block of transform coefficients, dequantizing and inverse transforming the block of transform coefficients to obtain a residual block, and adding a predictor to the residual block to obtain a decoded block. The decoded picture portion can then be decoded as shown in FIG. Figure 28-Figure 30 is filtered by the in-loop filter and also as Fig.29 In step S230, the refinement data is decoded from the bitstream or signal. This step is the inverse step of the encoding step S140. All variants and embodiments described with respect to step S140 are applicable to step S230. In step S240, the decoded picture is refined. This step is the same as the refinement step S150 on the encoder side.

[0417] Fig.32 Another example of an embodiment is shown in , which is a method for encoding video data, the method comprising providing refinement mode capability. Fig.32In the embodiment of the present invention, video data such as data included in a digital video signal or bitstream is processed at 3210 to identify costs associated with encoding the video data (such as a picture portion) using a refinement mode for each block as described herein and using a mode other than the refinement mode (e.g., no refinement). Then, at 3220, the video data is encoded based on the costs. For example, costs (such as rate-distortion costs) for encoding both using and not using the refinement mode can be obtained. If the use of the refinement mode results in a rate-distortion cost indicating an improvement in encoding, the refinement mode can be used for encoding. The rate-distortion cost indicating an improvement without limited improvement may result in not using the refinement mode or using a different mode that does not justify increasing the complexity of the refinement mode. Processing of the video data for obtaining costs and for encoding the video data may occur, such as described above with respect to one or more embodiments described herein. The encoded video data is output from 3220.

[0418] Fig.33 An example of an embodiment of a method for decoding video data is shown. According to one or more embodiments described herein, encoded video data (e.g., a digital data signal or a bitstream) is processed at 3310 during encoding of the video data to obtain an indication of a refinement mode to be used (e.g., a refinement mode for each block). Then, at 3320, the encoded video data, such as an encoded picture portion, is decoded based on the indication, for example, as described herein based on the refinement mode that has been used during encoding. Processing of the video data for obtaining costs and for encoding the video data may occur, for example, as described above with respect to one or more embodiments described herein. The decoded video data is output from 3320.

[0419] Throughout this disclosure, various implementations involve decoding. As used in this application, "decoding" may encompass, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also includes or alternatively includes a process performed by a decoder of the various implementations described in this application, such as extracting a picture from a tiled (packed) picture, determining the upsampling filter to be used, then upsampling the picture, and flipping the picture back to its intended orientation.

[0420] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or broadly to a broader decoding process will be apparent based on the context of the particular description and is believed to be well understood by those skilled in the art.

[0421] Likewise, various implementations involve encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application may encompass, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream or signal. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as segmentation, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such a process also includes or alternatively includes a process performed by an encoder of the various implementations described in this application.

[0422] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or broadly to a broader encoding process will be apparent based on the context of the particular description and is believed to be well understood by those skilled in the art.

[0423] Note that the syntax elements used herein are descriptive terms. Therefore, they do not exclude the use of their syntax element names.

[0424] When the figures are presented as flow charts, it should be understood that they also provide block diagrams of the corresponding apparatus. Similarly, when the figures are presented as block diagrams, it should be understood that they also provide flow charts of the corresponding methods / processes.

[0425] Various embodiments relate to rate-distortion optimization. In particular, during the encoding process, given the constraints of computational complexity, it is often necessary to consider the balance or trade-off between rate and distortion. Rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are a variety of methods for solving the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options, including all considered modes or coding parameter values, and a complete evaluation of their coding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save coding complexity, especially calculations based on approximate distortions of predicted or predicted residual signals rather than reconstructed signals. A mixture of these two methods can also be used, such as using approximate distortions only for some possible coding options and using complete distortions for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many methods use any of a variety of different techniques to perform optimization, but optimization does not necessarily have to be a complete evaluation of coding costs and associated distortions.

[0426] The implementations and aspects described herein can be implemented, for example, in methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0427] References to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations thereof mean that the particular features, structures, characteristics, etc. described in connection with the embodiment are included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations appearing in various places herein do not necessarily all refer to the same embodiment.

[0428] Additionally, this document may refer to “obtaining” various information. Obtaining information may include, for example, one or more of determining information, estimating information, calculating information, predicting information, or retrieving information from a memory.

[0429] Additionally, this document may refer to “accessing” various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0430] Additionally, this document may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, during operations such as, for example, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is typically involved in one way or another.

[0431] It should be understood that the use of any of the following “ / ”, “and / or”, and “at least one of…”, for example, in the context of “A / B”, “A and / or B”, and “at least one of A and B”, is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the context of “A, B, and / or C” and “at least one of A, B, and C”, such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). As will be apparent to one of ordinary skill in this and related arts, this can be extended to as many items as listed.

[0432] Moreover, as used herein, the word "signaling" especially refers to indicating something to the corresponding decoder. For example, in some embodiments, the encoder signals a specific one of the multiple parameters for refinement. In this way, in an embodiment, the same parameters are used on both the encoder side and the decoder side. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be understood that signaling can be completed in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signaling (signal)" is related to the above, the word "signal (signal)" can also be used as a noun in this article.

[0433] As will be apparent to one of ordinary skill in the art, implementations may generate various signals formatted to carry information, which may be stored or transmitted, for example. The information may include, for example, instructions for executing the method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bit stream or signal of the described embodiment. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, the signal may be sent via a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0434] Various embodiments have been described. Embodiments may include any of the following features or entities, alone or in any combination, across a variety of claim categories and types:

[0435] • Apply refinements that do not depend on neighboring reconstructed samples.

[0436] • Refinement is applied based on a global function with one or more refinement parameters sent in the bitstream.

[0437] • Providing a mode in the encoder and / or decoder that includes cross-component refinement of chroma components.

[0438] • Providing a mode in the encoder and / or decoder that includes intra-component refinement of the luma component.

[0439] Enable per-block refinement activation in the decoder and / or encoder.

[0440] • Apply refinement using multiple functions for a particular component and selecting the refinement function block by block.

[0441] • Enabling in a decoder and / or encoder the selection for each block of a refinement parameter to be applied to the block from a set of possible parameters encoded in the bitstream or signal.

[0442] • Apply refinement to the reconstructed signal on a per-block basis.

[0443] • Include a refinement step as an in-loop or out-of-loop filter to improve the reconstructed signal after decoding.

[0444] • Base refinements in the decoder and / or encoder on refinement tables encoded in the bitstream or signal. • Reduce table encoding costs by avoiding redundant neutral values.

[0445] • Inserting in the signaling syntax elements that enable the decoder to provide refinement as described herein.

[0446] • Including in the syntax element an identifier associated with a refinement table, such as a table pair identifier.

[0447] Provides size limits for refinement tables to reduce coding costs.

[0448] • Include one or more syntax elements in the syntax elements that indicate whether refinement information for the current block is copied from a neighboring block.

[0449] • Based on these syntax elements, refinements to be applied at the decoder are selected.

[0450] • A bitstream or signal comprising one or more described syntax elements or variations thereof.

[0451] • Insertion in the signaling enables the decoder to provide refined syntax elements in a manner corresponding to that used by the encoder.

[0452] • Creating and / or sending and / or receiving and / or decoding a bitstream or signal comprising one or more described syntax elements or variations thereof.

[0453] • A television, set-top box, cell phone, tablet or other electronic device implementing refinements according to any of the described embodiments.

[0454] • A television, set-top box, cell phone, tablet or other electronic device that performs refinement according to any of the described embodiments, and displays (eg, using a monitor, screen or other type of display) the resulting image.

[0455] A television, set-top box, cellular phone, tablet or other electronic device that tunes (eg, using a tuner) to a channel to receive a signal including an encoded image and performs refinement according to any of the described embodiments.

[0456] • A television, set-top box, cellular phone, tablet, or other electronic device that receives over-the-air (eg, using an antenna) a signal comprising an encoded image and performs refinement according to any of the described embodiments.

[0457] A computer program product storing a program code which, when executed by a computer, implements the refinements according to any one of the described embodiments.

[0458] A non-transitory computer-readable medium comprising executable program instructions causing a computer executing the instructions to implement the refinements according to any one of the described embodiments.

[0459] Various other general and specific embodiments are also supported and contemplated throughout the present disclosure. For example, at least one example of an embodiment includes a method of encoding, the method comprising: obtaining a cost for: (i) encoding a picture portion based on applying a refinement mode to a reconstructed signal for each block, wherein the refinement mode is based on a refinement parameter, and (ii) encoding the picture portion using a mode other than the refinement mode; and encoding the picture portion based on the cost.

[0460] Another example of at least one embodiment includes an apparatus comprising one or more processors configured to: obtain a cost for: encoding a picture portion based on applying a refinement mode to a reconstructed signal for each block, wherein the refinement mode is based on a refinement parameter, and encoding the picture portion without using the refinement mode, and encoding the picture portion based on the cost.

[0461] Another example of at least one embodiment includes a decoding method comprising: obtaining an indication of applying a refinement mode to a reconstructed signal for each block during encoding of a picture portion, wherein the refinement mode is based on a refinement parameter; and decoding the encoded picture portion based on the indication.

[0462] Another example of at least one embodiment includes an apparatus comprising one or more processors configured to: obtain an indication to apply a refinement mode to a reconstructed signal for each block during encoding of an encoded picture portion, wherein the refinement mode is based on a refinement parameter, and decode the encoded picture portion based on the indication.

[0463] Another example of at least one embodiment includes a signal formatted to include data representing an encoded picture portion; and data providing an indication of applying a refinement mode to a reconstructed signal for each block based on a refinement parameter during encoding of the encoded picture portion.

[0464] Another example of at least one embodiment includes a bitstream formatted to include data representing an encoded picture portion; and data providing an indication of applying a refinement mode to a reconstructed signal for each block based on a refinement parameter during encoding of the encoded picture portion.

[0465] In at least one embodiment described herein involving a refinement mode, the refinement mode may include component refinement.

[0466] In at least one embodiment described herein involving a refinement mode, the refinement mode may include at least one of cross-component refinement or intra-component refinement.

[0467] In at least one embodiment described herein involving cross-component refinement or intra-component refinement, the cross-component refinement may include cross-component chroma refinement, and the intra-component refinement may include intra-component luma refinement.

[0468] In at least one embodiment described herein involving a refinement mode, the refinement mode may include enabling selection of refinement parameters for each block.

[0469] In at least one embodiment described herein involving refinement parameters, the refinement parameters may include one or more refinement parameters contained in a refinement table.

[0470] In at least one embodiment described herein involving encoding a picture portion based on applying a refinement mode and a refinement parameter, the refinement parameter may be selected on a block basis from a plurality of refinement parameters based on an improvement in cost when encoding the picture portion using the refinement mode.

[0471] According to another embodiment, a method for encoding video data is proposed, the method comprising: obtaining a cost for: (i) encoding a picture portion using a refinement mode for each block, wherein the refinement mode is based on a refinement parameter, and (ii) encoding the picture portion using a mode other than the refinement mode; and encoding the picture portion based on the cost.

[0472] According to another embodiment, a method for decoding video data is proposed, the decoding method comprising: obtaining an indication of a refinement mode per block used during encoding of an encoded picture portion based on a refinement parameter; and decoding the encoded picture portion based on the indication.

[0473] According to another embodiment, an apparatus for decoding video data is proposed, the apparatus comprising one or more processors configured to: obtain an indication of a refinement mode used for each block based on a refinement parameter during encoding of an encoded picture portion; and decode the encoded picture portion based on the indication.

[0474] According to another embodiment, a signal format is proposed, the signal format comprising data representing an encoded picture portion; and data providing an indication of a refinement mode for each block used during encoding of the encoded picture portion based on a refinement parameter.

[0475] According to another embodiment, a bitstream is provided, the bitstream being formatted to include encoded video data, wherein the encoded video data is encoded in the following manner: obtaining a cost for: (i) encoding a picture portion using a refinement mode for each block, the refinement mode being based on a refinement parameter; and (ii) encoding the picture portion using a mode other than the refinement mode; and encoding the picture portion based on the cost.

[0476] According to another embodiment, the refinement scheme described herein may include component refinement.

[0477] According to another embodiment, the refinement mode described herein may include component refinement, wherein the component refinement includes at least one of cross-component refinement and inter-component refinement and intra-component refinement.

[0478] According to another embodiment, the refinement modes described herein may include inter-component chroma refinement.

[0479] According to another embodiment, the refinement modes described herein may include intra-component luminance refinement.

[0480] According to another embodiment, the refinement mode described herein may include enabling selection of refinement parameters for each block.

[0481] According to another embodiment, the refinement modes described herein may be based on refinement parameters including one or more chroma refinement parameters contained in a chroma refinement table.

[0482] According to another embodiment, the method, device or signal according to the present disclosure may include encoding and / or decoding video data using a refinement mode of a base refinement parameter, wherein the encoding and / or decoding includes encoding and / or decoding a picture portion and the refinement parameter based on an improvement in the cost when encoding and / or decoding the picture portion using the refinement mode.

[0483] According to another embodiment, a method, apparatus or signal according to the present disclosure may include encoding and / or decoding video data based on using a refinement mode during encoding and / or decoding and determining a cost associated with using the refinement mode, wherein the cost may include a rate-distortion cost, and determining the cost may include determining the rate-distortion cost when encoding a picture portion using the refinement mode.

[0484] Generally, at least one embodiment of a device for encoding and / or decoding video data may include the apparatus described herein and may include at least one of: (i) an antenna configured to receive a signal comprising data representing an image, (ii) a band limiter configured to limit the received signal to a frequency band comprising data representing the image, or (iii) a display configured to display the image.

[0485] In general, at least one embodiment of an apparatus for encoding and / or decoding video data may include at least one of a television, a cellular phone, a tablet, a set-top box, and a gateway device.

[0486] One or more of the embodiments further provides a computer-readable storage medium or a computer program product on which instructions for encoding or decoding video data according to the method or apparatus described herein are stored. The embodiments further provide a computer-readable storage medium or a computer program product on which a bitstream generated according to the method or apparatus described herein is stored. The embodiments further provide a method and apparatus for sending or receiving a bitstream generated according to the method or apparatus described herein.

Claims

1. A method comprising: decoding, at a block level for a picture portion, an indication providing for selection of a refinement parameter table for the block from a plurality of refinement parameter tables associated with component refinement modes applied to a reconstructed version of a component of the picture portion; obtaining one or more refinement parameters for the block using the selected refinement parameter table, The picture portion is refined in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

2. The method according to claim 1, wherein: The one or more refinement parameters are pivot points of the piecewise linear model. The method of claim 2 , further comprising constructing a lookup table using the one or more refinement parameters.

4. The method according to claim 3, wherein: The lookup table is used to refine a reconstructed version of the components of the picture portion.

5. The method according to claim 1, wherein: The plurality of refinement parameter tables include a plurality of pairs of refinement tables. 6 . The method of claim 1 , further comprising decoding the plurality of refinement parameter tables from a slice header of the picture portion.

7. The method according to claim 6, wherein: The plurality of refinement parameter tables include one or more parameters for refining the one or more refinement parameters.

8. The method according to claim 7, wherein: The one or more parameters include an index for updating an index of the one or more refinement parameters and a refinement value for updating a value of the one or more refinement parameters.

9. The method according to claim 1, wherein: A given value of the indication provides for enabling the component refinement mode for the block.

10. The method according to claim 1, wherein: The enabling or disabling of the component refinement mode for the block is inferred from the SAO parameters for the block.

11. The method of claim 6, further comprising decoding a first flag and a second flag for the picture portion, the first flag indicating whether at least one first refinement parameter table is encoded for a first component of the picture portion, the second flag indicating whether at least one second refinement parameter table is encoded for a second component of the picture portion, and wherein the plurality of refinement parameter tables are decoded further based on at least one of the first flag or the second flag.

12. The method according to claim 1, wherein: The component refinement mode is an intra-component refinement mode.

13. The method according to claim 1, wherein: The component refinement mode is an inter-component refinement mode.

14. A non-transitory computer readable medium storing executable program instructions to cause a computer to execute the instructions to perform the method according to claim 1.

15. An apparatus comprising one or more processors configured to: decoding, at a block level for a picture portion, an indication providing for selection of a refinement parameter table for the block from a plurality of refinement parameter tables associated with component refinement modes applied to a reconstructed version of a component of the picture portion; obtaining one or more refinement parameters for the block using the selected refinement parameter table, The picture portion is refined in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

16. A method comprising: encoding, at a block level for a picture portion, an indication providing for selection of a refinement parameter table for the block from a plurality of refinement parameter tables associated with component refinement modes applied to a reconstructed version of a component of the picture portion; obtaining one or more refinement parameters for the block using the selected refinement parameter table, The picture portion is refined in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

17. The method according to claim 16, wherein: The refinement parameter table is selected from the plurality of refinement parameter tables on a block basis based on a refinement of a cost associated with encoding the picture portion using the component refinement mode.

18. A non-transitory computer readable medium storing executable program instructions to cause a computer to execute the instructions to perform the method according to claim 15.

19. An apparatus comprising one or more processors configured to: encoding, at a block level for a picture portion, an indication providing for selection of a refinement parameter table for the block from a plurality of refinement parameter tables associated with component refinement modes applied to a reconstructed version of a component of the picture portion; obtaining one or more refinement parameters for the block using the selected refinement parameter table, The picture portion is refined in the component refinement mode, wherein the component refinement mode is applied to the block based on the obtained one or more refinement parameters.

20. A non-transitory computer-readable medium comprising a bitstream, the bitstream being formatted to include: Data representing a portion of an image; as well as At a block level of the picture portion, data representing an indication providing for selection of a refinement parameter table for the block from a plurality of refinement parameter tables associated with component refinement modes applied to a reconstructed version of a component of the picture portion.

Images