Method and device for picture encoding and decoding

By using bidirectional prediction and illuminance compensation technology with weighting and error minimization in image encoding and decoding, the problem of insufficient illuminance compensation and bidirectional prediction efficiency in the prior art is solved, and the efficiency and accuracy of encoding and decoding are improved.

CN111989924BActive Publication Date: 2024-11-29INTERDIGITAL VC HOLDINGS INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN201980025576.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-03-19
Publication Date
2024-11-29
Estimated Expiration
2039-03-19

AI Technical Summary

Technical Problem

When existing image and video encoding technologies use intra- or inter-frame prediction, it is difficult to effectively perform illuminance compensation and bidirectional prediction, resulting in insufficient encoding efficiency and decoding accuracy.

Method used

By determining the predicted value as a weighted sum of the first motion compensation reference block sample value in the first reference image and the second motion compensation reference block sample value in the second reference image, the weight of the weighted sum is determined using error minimization, Bidirectional prediction and illuminance compensation are performed during the encoding and decoding process to form a reconstructed sample in the L shape to correct the predicted value.

Benefits of technology

Improves the efficiency and accuracy of image encoding and decoding, enhances the performance of encoder and decoder, reduces encoding artifacts, and optimizes the transmission and storage of bitstreams.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111989924B_ABST
    Figure CN111989924B_ABST
Patent Text Reader

Abstract

A method for encoding a picture block is disclosed. A prediction value is determined as a weighted sum of sample values ​​of a first motion compensated reference block in a first reference image and sample values ​​of a second motion compensated reference block in a second reference image. A residual block determined from the picture block and the prediction value is then encoded in a bitstream. Advantageously, the weights of the weighted sum are determined by minimizing the error between reconstructed samples adjacent to the picture block and samples adjacent to the first and second reference blocks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present embodiments generally relate to methods and apparatus for picture encoding and decoding, and more particularly, to methods and apparatus for picture encoding and decoding with illumination compensation and bidirectional prediction. Background Art

[0002] In order to achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to exploit spatial and temporal redundancy in video content. Generally, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image block and the predicted image block (usually expressed as prediction error, prediction residual or prediction value (predictor)) is transformed, quantized and entropy encoded. During encoding, quadtree partitioning may be used, and the original image block is usually partitioned / split into sub-blocks. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to prediction, transformation, quantization and entropy coding. Summary of the invention

[0003] A method for encoding a picture block is disclosed. The method comprises:

[0004] - determining a prediction value as a weighted sum of sample values ​​of a first motion compensated reference block in the first reference picture and sample values ​​of a second motion compensated reference block in the second reference picture;

[0005] - encoding the residual block determined from the picture block and the prediction value,

[0006] The weight of the weighted sum is determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0007] A machine-readable medium is disclosed. The machine-readable medium stores machine-executable instructions, which, when executed, implement a method for encoding a picture block, the method comprising:

[0008] - determining a prediction value as a weighted sum of sample values ​​of a first motion compensated reference block in the first reference picture and sample values ​​of a second motion compensated reference block in the second reference picture;

[0009] - encoding the residual block determined from the picture block and the prediction value,

[0010] The weight of the weighted sum is determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0011] A coding device for coding a picture block is disclosed. The coding device comprises:

[0012] - A component for determining a predicted value as a weighted sum of sample values of a first motion-compensated reference block in a first reference image and sample values of a second motion-compensated reference block in a second reference image;

[0013] - A component for encoding a residual block determined from a picture block and the predicted value,

[0014] wherein the weights of the weighted sum are determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0015] An encoding device is disclosed. The encoding device includes: a communication interface configured to access at least one picture block; and at least one processor configured to:

[0016] - Determine a predicted value as a weighted sum of sample values of a first motion-compensated reference block in a first reference image and sample values of a second motion-compensated reference block in a second reference image;

[0017] - Encode a residual block determined from the accessed picture block and the predicted value,

[0018] wherein the weights of the weighted sum are determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0019] A bitstream is also disclosed. The bitstream is formatted to include encoded data representing a picture block, and the encoded data is encoded according to an encoding method according to one of the above embodiments. The bitstream is stored on a non-transitory processor-readable medium.

[0020] Advantageously, the reconstructed samples adjacent to the picture block and the reconstructed samples adjacent to the first and second reference blocks form an L shape.

[0021] In an embodiment, wherein the weighted sum is also right-shifted, and wherein the shift S is determined such that the weight is an integer value with a precision close to 1 / (1<<S).

[0022] In an embodiment, determining the weights and / or the shift includes selecting the weights and the shift from a set of defined values.

[0023] In an embodiment, determining the weights and / or the shift includes least-squares minimization of the error and selecting the weights and the shift whose values are closest to the values obtained by least-squares minimization from a set of defined values.

[0024] In an embodiment, determining the weights and / or the shift includes calculating the error obtained with a set of defined weight and shift values and selecting the weights and the shift corresponding to the minimum error.

[0025] A method for decoding a picture block, the method comprising:

[0026] - determining a prediction value as a weighted sum of sample values ​​of a first motion compensated reference block in the first reference picture and sample values ​​of a second motion compensated reference block in the second reference picture;

[0027] -decode the picture block from the prediction value and residual block,

[0028] The weight of the weighted sum is determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0029] A machine-readable medium is disclosed. The machine-readable medium stores machine-executable instructions, which, when executed, implement a method for decoding a picture block, the method comprising:

[0030] - determining a prediction value as a weighted sum of sample values ​​of a first motion compensated reference block in the first reference picture and sample values ​​of a second motion compensated reference block in the second reference picture;

[0031] -decode the picture block from the prediction value and residual block,

[0032] The weight of the weighted sum is determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0033] A decoding device for decoding a picture block is disclosed. The decoding device comprises:

[0034] - means for determining a prediction value as a weighted sum of sample values ​​of a first motion compensated reference block in a first reference image and sample values ​​of a second motion compensated reference block in a second reference image;

[0035] - means for decoding a picture block from a prediction value and a residual block,

[0036] The weight of the weighted sum is determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0037] A decoding device is also disclosed. The decoding device includes: a communication interface configured to access at least one stream; and at least one processor configured to:

[0038] - determining a prediction value as a weighted sum of sample values ​​of a first motion compensated reference block in the first reference picture and sample values ​​of a second motion compensated reference block in the second reference picture;

[0039] - Decode a picture block from the predicted value and the residual block, where the residual is decoded from the accessed stream;

[0040] Wherein, the weights of the weighted sum are determined by minimizing the error between the reconstructed samples adjacent to the picture block and the samples adjacent to the first and second reference blocks.

[0041] Advantageously, the reconstructed samples adjacent to the picture block and the reconstructed samples adjacent to the first and second reference blocks form an L shape.

[0042] In an embodiment, wherein the weighted sum is also right-shifted, and wherein the shift S is determined such that the weights are integer values with a precision close to 1 / (1<<S).

[0043] In an embodiment, determining the weights and / or the shift includes selecting the weights and the shift from a set of defined values.

[0044] In an embodiment, determining the weights and / or the shift includes least-squares minimization of the error and selecting the weights and the shift whose values are closest to the values obtained by least-squares minimization from a set of defined values.

[0045] In an embodiment, determining the weights and / or the shift includes calculating the error obtained with a set of defined weights and shift values and selecting the weights and the shift corresponding to the minimum error. Description of the Drawings

[0046] Figure 1 Represents an exemplary architecture of a transmitter configured to encode a picture in a bitstream according to a specific and non-limiting embodiment;

[0047] Figure 2 Illustrates an exemplary video encoder suitable for performing the encoding method referred to Figure 5 and Figure 6 described;

[0048] Figure 3 represents a flowchart of a method for deriving a predicted value using bidirectional prediction and illumination compensation according to the prior art;

[0049] Figure 4 represents an example of an L shape for deriving illumination compensation parameters according to the prior art;

[0050] Figure 5 Represents a flowchart of a method for encoding a picture block in a bitstream according to a specific and non-limiting embodiment;

[0051] Figure 6 Represents a flowchart of a method for deriving a predicted value using bidirectional prediction and illumination compensation according to a specific and non-limiting embodiment;

[0052] Figure 7represents an exemplary architecture of a receiver configured to decode a picture from a bitstream to obtain a decoded picture according to a specific and non-limiting embodiment;

[0053] Figure 8 Illustration of a method suitable for executing a reference Figure 6 and Fig. 9 A block diagram of an exemplary video decoder for the decoding method described in the method; and

[0054] Fig. 9 A flowchart representing a method for decoding a picture from a bitstream according to a specific and non-limiting embodiment. DETAILED DESCRIPTION

[0055] It should be understood that the drawings and descriptions have been simplified to illustrate the elements relevant to a clear understanding of the present embodiment, while many other elements found in typical encoding and / or decoding devices have been eliminated for clarity. It should be understood that although the terms first and second may be used to describe various elements here, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0056] Various methods are described below, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0057] Reference to "one embodiment" or "an embodiment" and other variations thereof of the present principles means that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places throughout the specification are not necessarily all referring to the same embodiment.

[0058] It should be understood that the use of any of the following " / ", "and / or", and "at least one of", for example, in the context of "A / B", "A and / or B", and "at least one of A and B", is intended to include: selecting only the first listed option (A), or selecting only the second listed option (B), or selecting both options (A and B). As a further example, in the context of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to include: selecting only the first listed option (A), or selecting only the second listed option (B), or selecting only the third listed option (C), or selecting only the first and second listed options (A and B), or selecting only the first and third listed options (A and C), or selecting only the second and third listed options (B and C), or selecting all three options (A and B and C). As will be apparent to one of ordinary skill in this and related arts, this may be extended for many listed options.

[0059] A picture is an array of luma samples in monochrome format, or an array of luma samples and two corresponding chroma sample arrays (or three tricolor color sample arrays, such as RGB) in 4:2:0, 4:2:2, and 4:4:4 color formats. Generally, a "block" is for a specific area in a sample array (e.g., luma Y), while a "unit" includes a concatenated block of all color components (luma Y and possibly chroma Cb and chroma Cr). A slice is an integer number of basic coding units such as HEVC coding tree units or H.264 macroblock units. A slice can consist of a complete picture or a part of it. Each slice can include one or more slice segments. A tile is an integer number of basic coding units grouped in square or rectangular areas within a slice or picture.

[0060] In the following, the words "reconstruct" and "decode" may be used interchangeably. Usually, but not necessarily, "reconstruct" is used on the encoder side and "decode" is used on the decoder side. It should be noted that the term "decode" or "reconstruct" may mean that the bitstream is partially "decoded" or "reconstructed", for example, the signal obtained after deblocking filtering but before SAO filtering, and the reconstructed samples may be different from the final decoded output for display. We may also use the terms "image", "picture" and "frame" interchangeably.

[0061] Various embodiments are described with reference to the HEVC standard. However, the present embodiments are not limited to HEVC and may be applied to other standards, recommendations and extensions thereof, including, for example, HEVC or HEVC extensions such as format range (RExt), scalability (SHVC), multi-view (MV-HEVC) extensions, and future video coding standards such as those developed by the Joint Video Experts Team (JVET). Various embodiments are described with reference to encoding / decoding of picture blocks. They may be used to encode / decode an entire picture or an entire sequence of pictures.

[0062] Figure 1 1 shows an exemplary architecture of a transmitter 1000 configured to encode a picture in a bitstream according to a specific and non-limiting embodiment.

[0063] The transmitter 1000 includes one or more processors 1005, which may include, for example, a CPU, a GPU and / or a DSP (English abbreviation for Digital Signal Processor), and an internal memory 1030 (e.g., RAM, ROM and / or EPROM). The transmitter 1000 includes one or more communication interfaces 1010 (e.g., keyboard, mouse, touchpad, webcam) that are suitable for displaying output information and / or allowing a user to input commands and / or data; and a power supply 1020 that may be external to the transmitter 1000. The transmitter 1000 may also include one or more network interfaces (not shown). The encoder module 1040 represents a module that may be included in the device to perform encoding functions. In addition, the encoder module 1040 may be implemented as an independent element of the transmitter 1000, or may be incorporated in the (multiple) processors 1005 as a combination of hardware and software known to those skilled in the art.

[0064] The image can be obtained from a source. According to different embodiments, the source can be but is not limited to:

[0065] - local storage, such as video memory, RAM, flash memory, hard disk;

[0066] - Storage interfaces, such as interfaces to mass storage, ROM, optical disks or magnetic supports;

[0067] - a communication interface, such as a wired interface (eg a bus interface, a wide area network interface, a local area network interface) or a wireless interface (such as an IEEE 802.11 interface or a Bluetooth interface); and

[0068] - Picture capture circuit (eg, a sensor such as, for example, a CCD (or Charge-Coupled Device) or a CMOS (or Complementary Metal-Oxide-Semiconductor)).

[0069] According to different embodiments, the bitstream can be sent to a destination. As an example, the bitstream is stored in a remote or local memory, such as a video memory or RAM, a hard disk. In a variant, the bitstream is sent to a storage interface (e.g., an interface with a mass storage, ROM, flash memory, optical disk, or magnetic support), and / or the bitstream is transmitted via a communication interface (e.g., an interface to a point-to-point link, a communication bus, a point-to-multipoint link, or a broadcast network).

[0070] According to an exemplary and non-limiting embodiment, the transmitter 1000 further comprises a computer program stored in the memory 1030. The computer program comprises instructions which, when executed by the transmitter 1000, in particular by the processor 1005, enable the transmitter 1000 to perform the reference Figure 5 and Figure 6 The encoding method described. According to a variant, the computer program is stored externally to the transmitter 1000 on a non-transitory digital data support, for example on an external storage medium, such as an HDD, a CD-ROM, a DVD, a read-only and / or DVD drive and / or a DVD read / write drive, all of which are known in the art. Therefore, the transmitter 1000 includes a mechanism for reading the computer program. In addition, the transmitter 1000 can access one or more storage devices of the Universal Serial Bus (USB) type (e.g., a “memory stick”) via corresponding USB ports (not shown).

[0071] According to an exemplary and non-limiting embodiment, the transmitter 1000 may be, but is not limited to:

[0072] -mobile device;

[0073] -communication equipment;

[0074] -Gaming equipment;

[0075] - Tablet (or tablet computer);

[0076] - Laptop;

[0077] -Still picture camera;

[0078] - Camera;

[0079] - coding chip or coding device / means;

[0080] -Still picture server; and

[0081] - Video server (eg, broadcast server, video-on-demand server, or web server).

[0082] Figure 2 Illustration of a method suitable for executing a reference Figure 5 and Figure 6 An exemplary video encoder 100 of the described encoding method is, for example, an encoder of the HEVC type. The encoder 100 is an example of a transmitter 1000 or a part of such a transmitter 1000.

[0083] For encoding, pictures are typically partitioned into basic coding units, for example, into coding tree units (CTUs) in HEVC, or into macroblock units in H.264. A set of possibly consecutive basic coding units is grouped into slices. A basic coding unit contains basic coding blocks for all color components. In HEVC, the minimum coding tree block (CTB) size of 16×16 corresponds to the macroblock size used in previous video coding standards. It will be understood that although the terms CTU and CTB are used here to describe encoding / decoding methods and encoding / decoding devices, these methods and devices should not be limited to these specific terms, which may be worded differently (e.g., macroblocks) in other standards such as H.264.

[0084] In HEVC coding, pictures are partitioned into square-shaped CTUs with configurable sizes, typically 64×64, 128×128, or 256×256. A CTU is the root of a quadtree partitioned into 4 square coding units (CUs) of equal size (i.e., width and height are half the size of the parent block). A quadtree is a tree in which a parent node can be partitioned into four child nodes, each of which can become a parent node and thus be partitioned into four child nodes. In HEVC, a coding block (CB) is partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree partitioned into transform blocks (TBs). Corresponding to coding blocks, prediction blocks, and transform blocks, a coding unit (CU) includes a tree-structured set of prediction units (PUs) and transform units (TUs), the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The sizes of the CB, PB, and TB of the luminance component apply to the corresponding CU, PU, ​​and TU.

[0085] In recent coding systems, CTU is the root of a coding tree partitioned into coding units (CUs). A coding tree is a tree in which a parent node (usually corresponding to a CU) can be partitioned into child nodes (e.g., partitioned into 2, 3, or 4 child nodes), each of which can become a parent node and thus be partitioned into child nodes. In addition to the quadtree partitioning mode, new partitioning modes (binary tree symmetric partitioning mode, binary tree asymmetric partitioning mode, and ternary tree partitioning mode) are defined, which increases the total number of possible partitioning modes. The coding tree has a unique root node, such as CTU. The leaves of the coding tree are the terminal nodes of the tree. Each node of the coding tree represents a CU that can also be partitioned into smaller CUs (also called sub-CUs or more generally sub-blocks). Once the partitioning of CTU to CU is determined, the CU corresponding to the leaf of the coding tree is encoded. The partitioning of CTU to CU and the coding parameters for encoding each CU (corresponding to the leaf of the coding tree) can be determined on the encoder side through a rate-distortion optimization process. There is no partitioning of CB to PB and TB, that is, the CU consists of a single PU and a single TU.

[0086] Hereinafter, the term "block" or "picture block" may be used to refer to any of CTU, CU, PU, ​​TU, CB, PB, and TB. In addition, the term "block" or "picture block" may be used to refer to macroblocks, partitions, and sub-blocks specified in H.264 / AVC or other video coding standards, and more generally to sample arrays of various sizes.

[0087] Back to Figure 2 In the exemplary encoder 100, as described below, a picture is encoded by an encoder element. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra or inter mode. When a CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. Motion information (i.e., motion vector and reference index) can be signaled in two methods, i.e., "Advanced Motion Vector Prediction (AMVP)" and "Merge Mode". In AMVP, a video encoder or decoder assembles a candidate list based on a motion vector determined from an encoded block. The video encoder then signals an index to the candidate list to identify a motion vector predictor (MVP) and signals a motion vector difference (MVD). On the decoder side, the motion vector (MV) is reconstructed as MVP+MVD.

[0088] In merge mode, the video encoder or decoder assembles a candidate list based on the coded blocks, and the video encoder signals the index of one of the candidates in the candidate list. At the decoder side, motion vectors and reference picture indexes are reconstructed based on the signaled candidates.

[0089] The encoder decides (105) which of intra mode or inter mode to use to encode the CU and indicates the intra / inter decision via a prediction mode flag. The residual is calculated by subtracting (110) the predicted block (also called prediction value) from the original picture block.

[0090] For example, within the same slice, a CU in intra mode is predicted from reconstructed neighboring samples. A CU in inter mode is predicted from reconstructed samples of a reference picture stored in a reference picture buffer (180). In inter mode, bidirectional prediction combines two predictions pred0 and pred1 obtained via motion compensation (170) of two reconstructed reference blocks:

[0091] Bi-pred=w 0 *pred 0 +w 1 *pred 1 (1)

[0092] In addition, bidirectional prediction can be combined with block-based illumination compensation (IC). IC allows to correct block prediction samples obtained using motion compensation (MC) by taking into account spatial or temporal local illumination changes. In this case, a flag (IC flag) is encoded for the current block to indicate whether IC correction should be applied. For example, if the current block inherits parameters from an adjacent reconstructed block (merge mode), the IC flag may not be encoded, but derived from the adjacent reconstructed block.

[0093] The residual is transformed (125) and quantized (130). The quantized transform coefficients, along with motion vectors and other syntax elements, are entropy encoded (145) to output a bitstream. The encoder can also skip the transform, or bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process. In direct PCM encoding, prediction is not applied, and the coding unit samples are encoded directly into the bitstream.

[0094] The encoder comprises a decoding loop and thus decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the residual. The picture block is reconstructed by combining (155) the decoded residual and the predicted block. An in-loop filter (165) is applied to reconstruct the picture, for example to perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered picture can be stored in a reference picture buffer (180) and used as a reference for other pictures.

[0095] FIG. 3 shows a flow chart of a method for deriving a prediction value using bidirectional prediction and illumination compensation according to the prior art.

[0096] Illumination compensation

[0097] In intra mode, IC allows to correct the block prediction samples obtained using MC by taking into account spatial or temporal local illumination variations. Typically, the set S of reconstructed neighboring samples (L-shape-cur) of the current block is cur The corresponding set S of neighboring samples (L-shape-ref-i) of the reference-i block (i=0 or 1) ref-i Compare to determine IC parameters (S306, S308), as shown in Figure 4. Figure 4 depicts an example of a set of reconstructed adjacent samples (also referred to as L-shape in this document). In this example, the L-shape consists of a row of samples above the current or reference block and a column of samples on the left. In another example, the L-shape may consist of only a row of samples above, or only a column of samples on the left, or 2 columns of samples on the left, and so on.

[0098] At steps S306 and S308, the IC parameters (a) are calculated by minimizing the difference between the samples in L-shape-cur corrected with the IC parameters and the samples in L-shape-ref-i (eg, using the least squares method). i , b i ). Typically, the IC model is linear: IC(x) = a.x + b, where x is the value of the sample to be compensated.

[0099] The parameters (ai, bi) are determined by solving the least squares minimization of the L shape:

[0100]

[0101] Finally, a i Transformed into integer weights (a i ) and shift (sh i ), and correct pred by IC parameters i (S314 and S316), to obtain predIc as follows i :

[0102] PredIc i =(a i *x i >>sh i )+b i (3)

[0103] Among them, x i is the motion compensation block (pred i ) samples, ">>" is the right shift operator.

[0104] The IC parameters (a 0 , b 0 ) and (a 1 , b 1 ). (a0, b0) is derived from L-shape-ref0 (S306), and (a1, b1) is derived from L-shape-ref1 (S308).

[0105] Bidirectional prediction

[0106] The predicted values ​​pred0 (predIc0 if IC is enabled with ref0) and pred1 (or predIc1 if IC is enabled with ref1) are then combined into a single predicted value Bi-pred (S318). Conventionally, the combination includes averaging the two predictions with weights w0=w1=1 / 2. More generally, the two predicted values ​​pred0 and pred1 are combined into a single predicted value as follows:

[0107] Bi-pred=(w 0 *pred 0 +w 1 *pred 1 +off)>>shift (4)

[0108] In a variation, weights w0 and w1 are different. As an example, w0 may be equal to w, and w1 may be equal to (1-w), with predefined weight values ​​w listed in Table 1. An index corresponding to the weight value may be encoded in the bitstream for each block.

[0109]

[0110]

[0111] Table 1

[0112] The weights can also be derived from global illumination compensation, also known as the "weight prediction" (WP) tool. In this case, the weights are determined by the encoder and are constant for the entire slice. They can be transmitted in the bitstream. However, the WP tool is redundant with the IC tool, so activating both WP and IC is usually counterproductive.

[0113] In the bitstream, when IC is enabled for the current slice, picture or sequence, an IC flag can be coded per block to indicate whether IC is enabled for that block. It can also be derived from reconstructed neighboring blocks, such as in the case of merge mode.

[0114] When IC is enabled with bidirectional prediction, two equations (3) are calculated independently for each reference 0 and 1 at S314 and S316, respectively. Their results are combined using equation (4) at step S318: Bi-Pred = (w0*(a0*x0>>sh0+b0)+w1*(a1*x1>>sh1+b1)+off)>>shift (5)

[0115] Cascading several shifts may result in insufficient precision and increase the amount of computation and memory bandwidth. In the case of generalized bidirectional prediction, the encoder must check all weights and select the best weights using, for example, the RD cost (rate distortion cost) criterion. This can severely slow down the encoder.

[0116] Figure 5 A flowchart representing a method for encoding a picture block in a bitstream according to a specific and non-limiting embodiment.

[0117] The method begins at step S500. At step S510, a transmitter 1000 (e.g., such as encoder 100) accesses a block of a picture slice. At step S520, a prediction value is determined for the accessed block. The prediction value is determined as a weighted sum of sample values ​​of a first motion compensated reference block (pred0) in a first reference image and sample values ​​of a second motion compensated reference block (pred1) in a second reference image, the weight of the weighted sum being determined by minimizing the error (i.e., difference) between reconstructed samples adjacent to the picture block and samples adjacent to the first reference block and the second reference block. Obtaining the weighted sum may include adding an offset. Figure 6 The specific implementation of step S520 is described in detail.

[0118] At step S530, the transmitter encodes the accessed block. Encoding the block typically, but not necessarily, includes: subtracting the prediction value from the block to obtain a residual block, transforming the residual block into a block of transform coefficients, quantizing the coefficient block with a quantization step size QP to obtain a quantized block of transform coefficients, and entropy encoding the quantized block of transform coefficients in the bitstream. Reconstructing the block at the encoder side typically, but not necessarily, includes: dequantizing and inverse transforming the quantized block of transform coefficients to obtain a residual block, and adding the prediction value to the residual block to obtain a decoded block.

[0119] The method ends at step S540.

[0120] Figure 6 Flowchart showing the derivation of prediction values ​​using bi-directional prediction and illumination compensation according to a specific and non-limiting embodiment.

[0121] In step S600, a motion compensation reference block ref0 is obtained, which is denoted as pred0.

[0122] In step S602, a motion compensation reference block ref1 is obtained. It is labeled as pred1.

[0123] In step S604, the combined IC and the bi-prediction weights A0, A1, B, and S are determined.

[0124] The values A′0, A′1, B′, and S are directly determined by the encoder (and decoder) using the L shape of the current block. As an example, the least squares method is used to minimize the quadratic error (7) to determine A′0, A′1, B′, and S:

[0125]

[0126] Adjust the value of the shift S such that A 0 、A 1 、B are integers with the required precision P. For example, with P≈1 / (1<<S) (equivalent to fixed-point arithmetic with S bits as the fractional part):

[0127] A 0 =E(A′ 0 *(1<<S))

[0128] A 1 =E(A′ 1 *(1<<S))

[0129] B=E(B′*(1<<S))

[0130] E(x) represents the integer part of x.

[0131] When the following conditions are true, S can be iteratively decreased:

[0132] |A′ 0 –(A 0 >>S)|≤P

[0133] |A′ 1 –(A 1 >>S)|≤P

[0134] |B′–(B>>S)|≤P

[0135] In a variant, only the predefined values A 0 、A 1 、B, and S (e.g., the weight values in Table 1) are checked, and the parameters (A′ 0 、A′ 1 、B′, and S) corresponding to the minimum quadratic error (7) are retained. In the latter case, the least squares method is not used. The weight values in Table 1 can be transmitted in the bitstream (e.g., in the SPS), or are known a priori. SPS stands for Sequence Parameter Set.

[0136] In another variant, the value A is calculated using the least squares method 0 、A 1 、B, and S, and then adjustments are made so that they match a predefined value (e.g., a weight value in Table 1).

[0137] In an exemplary embodiment, a flag (a generalized bidirectional prediction flag) is encoded (or decoded on the decoder side), which indicates whether the current block is encoded using the Figure 6 method or whether the current block is encoded using a conventional method. In a variant, the flag and the IC flag are combined into a single flag. In a variant, the flag and the generalized bidirectional prediction flag are combined into a single flag.

[0138] In step S606, the final prediction value Bi-pred is determined by combining pred0 and pred1 using a weighted sum. In an exemplary embodiment, the combination weights A0, A1, B, and S obtained in step S604 are used to combine pred0 and pred1. In a specific embodiment, B can be equal to zero. In a single step (the steps S314, S316, and S318 in Figure 3 are combined into Figure 6 a single step S604) the IC parameters and the prediction weights are applied together. Equation (5) is modified to the following equation (6):

[0139] Bi-Pred = (A0 * x0 + A1 * x1) + B) >> S (6)

[0140] If sh0 = sh1, then: A0 = w0 * a0

[0141] A1 = w1 * a1

[0142] B = (w0 * b0 + w1 * b1 + off) << sh0

[0143] S = shift + sh0

[0144] If sh0 < sh1, then: A0 = (w0 * a0) << (sh1 - sh0)

[0145] A1 = w1 * a1

[0146] B = (w0 * b0 + w1 * b1 + off) << sh1

[0147] S = shift + sh1

[0148] If sh0 > sh1, then: A0 = w0 * a0

[0149] A1 = (w1 * a1) << (sh0 – sh1)

[0150] B=(w0*b0+w1*b1+off)< <sh0

[0151] S=shift+sh0

[0152] Figure 7 2000 illustrates an exemplary architecture of a receiver 2000 configured to decode a picture from a bitstream to obtain a decoded picture according to a specific and non-limiting embodiment.

[0153] The receiver 2000 includes one or more processors 2005, which may include, for example, a CPU, a GPU and / or a DSP (English abbreviation for a digital signal processor), and an internal memory 2030 (e.g., RAM, ROM and / or EPROM). The receiver 2000 includes one or more communication interfaces 2010 (e.g., keyboard, mouse, touchpad, webcam) that are suitable for displaying output information and / or allowing a user to input commands and / or data (e.g., decoded pictures); and a power supply 2020 that may be external to the receiver 2000. The receiver 2000 may also include one or more network interfaces (not shown). The decoder module 2040 represents a module that may be included in the device to perform a decoding function. In addition, the decoder module 2040 may be implemented as an independent element of the receiver 2000, or may be incorporated in the (multiple) processors 2005 as a combination of hardware and software known to those skilled in the art.

[0154] The bitstream may be obtained from a source. According to different embodiments, the source may be, but is not limited to:

[0155] - local storage, such as video memory, RAM, flash memory, hard disk;

[0156] - Storage interfaces, such as interfaces to mass storage, ROM, optical disks or magnetic supports;

[0157] - a communication interface, such as a wired interface (eg a bus interface, a wide area network interface, a local area network interface) or a wireless interface (such as an IEEE 802.11 interface or a Bluetooth interface); and

[0158] - Image capture circuitry (eg, a sensor such as, for example, a CCD (or Charge Coupled Device) or a CMOS (or Complementary Metal Oxide Semiconductor)).

[0159] According to different embodiments, the decoded picture can be sent to a destination such as a display device. As an example, the decoded picture is stored in a remote or local memory, such as a video memory or RAM, a hard disk. In a variant, the decoded picture is sent to a storage interface (e.g., an interface with a mass storage, ROM, flash memory, optical disk, or magnetic support), and / or the decoded picture is transmitted via a communication interface (e.g., an interface to a point-to-point link, a communication bus, a point-to-multipoint link, or a broadcast network).

[0160] According to a particular and non-limiting embodiment, the receiver 2000 further comprises a computer program stored in the memory 2030. The computer program comprises instructions which, when executed by the receiver 2000, in particular by the processor 2005, enable the receiver to perform a reference Figure 6 and Fig. 9 The decoding method described. According to a variant, the computer program is stored externally to the receiver 2000 on a non-transitory digital data support, for example on an external storage medium, such as an HDD, a CD-ROM, a DVD, a read-only and / or DVD drive and / or a DVD read / write drive, all of which are known in the art. Therefore, the receiver 2000 includes a mechanism for reading the computer program. In addition, the receiver 2000 can access one or more storage devices of the Universal Serial Bus (USB) type (e.g., a "memory stick") via corresponding USB ports (not shown).

[0161] According to an exemplary and non-limiting embodiment, the receiver 2000 may be, but is not limited to:

[0162] -mobile device;

[0163] -communication equipment;

[0164] -Gaming equipment;

[0165] - Set-top box;

[0166] -TV set;

[0167] - Tablet (or tablet computer);

[0168] - Laptop;

[0169] - Video players, such as Blu-ray players, DVD players;

[0170] - a display; and

[0171] -Decoding chip or decoding device / apparatus.

[0172] Figure 8 Illustration of a method suitable for executing a reference Figure 6 and Fig. 91 is a block diagram of an exemplary video decoder 200 (e.g., a HEVC-type video decoder) for describing the decoding method. The video decoder 200 is an example of a receiver 2000 or a portion of such a receiver 2000. In the exemplary decoder 200, a bitstream is decoded by a decoder element as described below. The video decoder 200 generally performs the same Figure 2 The decoding pass is the opposite of the encoding pass described in , which performs video decoding as part of encoding the video data.

[0173] Specifically, the input of the decoder includes a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other encoded information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the residual. The decoded residual is then combined (255) with the predicted block (also referred to as the prediction value) to obtain a decoded / reconstructed picture block. The predicted block may be obtained (270) according to intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). As described above, AMVP and merge mode techniques may be used during motion compensation, which may use an interpolation filter to calculate interpolation of sub-integer samples of a reference block. An in-loop filter (265) is applied to reconstruct the picture. The in-loop filter may include a deblocking filter and a SAO filter. The filtered picture is stored in a reference picture buffer (280).

[0174] Fig. 9 A flowchart representing a method for decoding a picture from a bitstream according to a specific and non-limiting embodiment.

[0175] The method begins at step S900. At step S910, a receiver 2000, such as decoder 200, accesses a bitstream. At step S920, the receiver determines a prediction value as a weighted sum of sample values ​​of a first motion compensated reference block (pred0) in a first reference image and sample values ​​of a second motion compensated reference block (pred1) in a second reference image, the weights of the weighted sum being determined by minimizing the error (i.e., difference) between reconstructed samples adjacent to the picture block and samples adjacent to the first reference block and the second reference block. Obtaining the weighted sum may include adding an offset. A specific embodiment of step S920 is Figure 6 is described in detail above, and is the same as S520 of the encoding method.

[0176] At step S930, the receiver decodes a block of the picture from the bitstream. Decoding a block typically, but not necessarily, includes entropy decoding a portion of the bitstream representing the block to obtain a transform coefficient block, dequantizing and inverse transforming the transform coefficient block to obtain a residual block, and adding a prediction value to the residual block to obtain a decoded block.

[0177] The method ends at step S840.

[0178] For example, the implementations described herein may be implemented in methods or processes, devices, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method or device), the implementations of the features discussed may also be implemented in other forms (e.g., programs). For example, a device may be implemented in appropriate hardware, software, and firmware. For example, a method may be implemented in a device, such as, for example, a processor, which generally refers to a processing device, such as a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants ("PDAs"), and other devices that assist in information communication between end users.

[0179] The implementation of the various processes and features described herein can be embodied in various different devices or applications, particularly, for example, devices or applications. Examples of such devices include encoders, decoders, postprocessors for processing outputs from decoders, preprocessors for providing inputs to encoders, video encoders, video decoders, video codecs, network servers, set-top boxes, laptop computers, personal computers, cellular phones, PDAs, and other communication devices. It should be clear that the device can be mobile and can even be installed in a mobile vehicle.

[0180] Furthermore, the methods may be implemented by instructions executed by a processor, and such instructions (and / or data values ​​produced by the implementation) may be stored on a processor-readable medium, such as, for example, an integrated circuit, a software carrier, or other storage device, such as, for example, a hard disk, a compact disk ("CD"), an optical disk (such as, for example, a DVD, commonly referred to as a digital versatile disk or digital video disk), a random access memory ("RAM"), or a read-only memory ("ROM"). The instructions may form an application program tangibly embodied on a processor-readable medium. For example, the instructions may be in hardware, firmware, software, or a combination. For example, the instructions may be found in an operating system, a separate application, or a combination of both. Thus, for example, a processor may be characterized as a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Furthermore, in addition to or in lieu of instructions, the processor-readable medium may store data values ​​produced by the implementation.

[0181] It will be apparent to those skilled in the art that implementations may generate various formatted signals to carry information that may be stored or transmitted, for example. For example, the information may include instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the grammar of the described embodiments, or to carry as data the actual grammar values ​​written by the described embodiments. For example, such a signal may be formatted as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. For example, formatting may include encoding a data stream and modulating a carrier with the encoded data stream. For example, the information carried by the signal may be analog or digital information. As is well known, a signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0182] Many implementations have been described. However, it should be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. In addition, it will be understood by those skilled in the art that other structures and processes may be substituted for those disclosed, and that the resulting implementations will perform at least substantially the same functions as the disclosed implementations in at least substantially the same manner to achieve at least substantially the same results. Therefore, the present application contemplates these and other implementations.

Claims

1. A method for encoding, the method comprising: - obtaining a picture block to which an illumination compensation prediction mode and a weighted prediction mode are jointly applied, the illumination compensation prediction mode comprising obtaining a predictor of a current block by applying an illumination compensation model defined by illumination compensation parameters to a motion compensated reference block, the illumination compensation parameters comprising two illumination compensation triplets, the illumination compensation triplets comprising weights, offsets and shift values, the weighted prediction mode comprising obtaining a predictor of the current block as a weighted sum of two motion compensated reference blocks using the weighted prediction parameters, the weighted prediction parameters comprising two weights, an offset value and a shift value, each shift value being used to obtain an integer weight and an offset; - determining a first motion compensated reference block in a first reference image for the picture block and a second motion compensated reference block in a second reference image for the picture block; - based on the shift values ​​of the two illumination compensation triplets, determining a first weight to be applied to the reference block for the first motion compensation, a second weight to be applied to the reference block for the second motion compensation, an offset value and a shift value allowing a weighted sum of the reference block for the first motion compensation and the reference block for the second motion compensation to be obtained, wherein the determined first weight, the second weight, the offset value and the shift value jointly represent the illumination compensation parameter and the weighted prediction parameter of the illumination compensation prediction mode and are calculated once for the picture block using the illumination compensation parameter and the weighted prediction parameter; and - jointly applying the illumination compensation prediction mode and the weighted prediction mode by obtaining the weighted sum using the determined first weight, the second weight, the offset value and the shifted value, and encoding the picture block using the weighted sum as a predictor of the picture block.

2. The method according to claim 1, wherein: The determined first weight, the second weight, the offset value, and the shift value are determined by minimizing a difference between reconstructed samples adjacent to the picture block and reconstructed samples adjacent to the first motion compensated reference block and the second motion compensated reference block.

3. The method according to claim 2, wherein: Reconstructed samples adjacent to the picture block form an L shape, reconstructed samples adjacent to the reference block for the first motion compensation form an L shape, and reconstructed samples adjacent to the reference block for the second motion compensation form an L shape.

4. A method for decoding, the method comprising: - obtaining a picture block to which an illumination compensation prediction mode and a weighted prediction mode are jointly applied, the illumination compensation prediction mode comprising obtaining a predictor of a current block by applying an illumination compensation model defined by illumination compensation parameters to a motion compensated reference block, the illumination compensation parameters comprising two illumination compensation triplets, the illumination compensation triplets comprising weights, offsets and shift values, the weighted prediction mode comprising obtaining a predictor of the current block as a weighted sum of two motion compensated reference blocks using the weighted prediction parameters, the weighted prediction parameters comprising two weights, an offset value and a shift value, each shift value being used to obtain an integer weight and an offset; - determining from the video data a first motion compensated reference block in a first reference picture of the picture block and a second motion compensated reference block in a second reference picture of the picture block; - based on the shift values ​​of the two illumination compensation triplets, obtaining from the video data a first weight to be applied to the reference block for the first motion compensation, a second weight to be applied to the reference block for the second motion compensation, an offset value and a shift value allowing a weighted sum of the reference block for the first motion compensation and the reference block for the second motion compensation, the determined first weight, the second weight, the offset value and the shift value jointly representing the illumination compensation parameter and the weighted prediction parameter of the illumination compensation prediction mode and being calculated once for the picture block using the illumination compensation parameter and the weighted prediction parameter; and - jointly applying the illumination compensation prediction mode and the weighted prediction mode by obtaining the weighted sum using the determined first weight, the second weight, the offset value, and the shifted value, and decoding the picture block using the weighted sum as a predictor of the picture block.

5. The method according to claim 4, wherein: The determined first weight, the second weight, the offset value, and the shift value are determined by minimizing a difference between reconstructed samples adjacent to the picture block and reconstructed samples adjacent to the first motion compensated reference block and the second motion compensated reference block.

6. The method according to claim 5, wherein: Reconstructed samples adjacent to the picture block form an L shape, reconstructed samples adjacent to the reference block for the first motion compensation form an L shape, and reconstructed samples adjacent to the reference block for the second motion compensation form an L shape.

7. A device for encoding, the device comprising an electronic circuit adapted to: - obtaining a picture block to which an illumination compensation prediction mode and a weighted prediction mode are jointly applied, the illumination compensation prediction mode comprising obtaining a predictor of a current block by applying an illumination compensation model defined by illumination compensation parameters to a motion compensated reference block, the illumination compensation parameters comprising two illumination compensation triplets, the illumination compensation triplets comprising weights, offsets and shift values, the weighted prediction mode comprising obtaining a predictor of the current block as a weighted sum of two motion compensated reference blocks using the weighted prediction parameters, the weighted prediction parameters comprising two weights, an offset value and a shift value, each shift value being used to obtain an integer weight and an offset; - determining a first motion compensated reference block in a first reference image for the picture block and a second motion compensated reference block in a second reference image for the picture block; - based on the shift values ​​of the two illumination compensation triplets, determining a first weight to be applied to the reference block for the first motion compensation, a second weight to be applied to the reference block for the second motion compensation, an offset value and a shift value allowing a weighted sum of the reference block for the first motion compensation and the reference block for the second motion compensation to be obtained, wherein the determined first weight, the second weight, the offset value and the shift value jointly represent the illumination compensation parameter and the weighted prediction parameter of the illumination compensation prediction mode and are calculated once for the picture block using the illumination compensation parameter and the weighted prediction parameter; and - jointly applying the illumination compensation prediction mode and the weighted prediction mode by obtaining the weighted sum using the determined first weight, the second weight, the offset value and the shifted value, and encoding the picture block using the weighted sum as a predictor of the picture block.

8. The device according to claim 7, wherein: The determined first weight, the second weight, the offset value, and the shift value are determined by minimizing a difference between reconstructed samples adjacent to the picture block and reconstructed samples adjacent to the first motion compensated reference block and the second motion compensated reference block.

9. The device according to claim 8, wherein: Reconstructed samples adjacent to the picture block form an L shape, reconstructed samples adjacent to the reference block for the first motion compensation form an L shape, and reconstructed samples adjacent to the reference block for the second motion compensation form an L shape.

10. A device for decoding, the device comprising an electronic circuit adapted to: - obtaining a picture block to which an illumination compensation prediction mode and a weighted prediction mode are jointly applied, the illumination compensation prediction mode comprising obtaining a predictor of a current block by applying an illumination compensation model defined by illumination compensation parameters to a motion compensated reference block, the illumination compensation parameters comprising two illumination compensation triplets, the illumination compensation triplets comprising weights, offsets and shift values, the weighted prediction mode comprising obtaining a predictor of the current block as a weighted sum of two motion compensated reference blocks using the weighted prediction parameters, the weighted prediction parameters comprising two weights, an offset value and a shift value, each shift value being used to obtain an integer weight and an offset; - determining from the video data a first motion compensated reference block in a first reference picture of the picture block and a second motion compensated reference block in a second reference picture of the picture block; - based on the shift values ​​of the two illumination compensation triplets, obtaining from the video data a first weight to be applied to the reference block for the first motion compensation, a second weight to be applied to the reference block for the second motion compensation, an offset value and a shift value allowing a weighted sum of the reference block for the first motion compensation and the reference block for the second motion compensation, the determined first weight, the second weight, the offset value and the shift value jointly representing the illumination compensation parameter and the weighted prediction parameter of the illumination compensation prediction mode and being calculated once for the picture block using the illumination compensation parameter and the weighted prediction parameter; and - jointly applying the illumination compensation prediction mode and the weighted prediction mode by obtaining the weighted sum using the determined first weight, the second weight, the offset value, and the shifted value, and decoding the picture block using the weighted sum as a predictor of the picture block.

11. The device according to claim 10, wherein: The determined first weight, the second weight, the offset value, and the shift value are determined by minimizing a difference between reconstructed samples adjacent to the picture block and reconstructed samples adjacent to the first motion compensated reference block and the second motion compensated reference block.

12. The device according to claim 11, wherein Reconstructed samples adjacent to the picture block form an L shape, reconstructed samples adjacent to the reference block for the first motion compensation form an L shape, and reconstructed samples adjacent to the reference block for the second motion compensation form an L shape.

13. An information storage medium having program code instructions stored thereon, which, when executed by a processor, cause the processor to perform the method of claim 1.

14. An information storage medium having program code instructions stored thereon, which, when executed by a processor, cause the processor to perform the method of claim 4.

Citation Information

Patent Citations

  • Systems and methods for adaptive selection of weights for video coding

    US20180098070A1

  • Systems and methods for generalized multi-hypothesis prediction for video coding

    WO2017197146A1