Local illumination compensation method, video encoding and decoding method, device and system

Abstract

The invention discloses a local illumination compensation method, a video encoding and decoding method, a video encoding and decoding device and a video encoding and decoding system, and a decoding end determines an LIC mode used by a current block according to the availability of a reconstructed sample adjacent to the current block under the condition of determining that the current block uses an LIC according to an LIC use flag bit. And under the condition that the LIC is allowed to be used when the inter-frame prediction is performed on the current block, the coding end determines the LIC mode used when the coding attempt is performed on the current block according to the availability of the adjacent reconstructed sample of the current block. The embodiment of the invention further provides a corresponding video encoding and decoding method, device and system. According to the embodiment of the invention, coding attempts of one or more LIC modes can be skipped according to the availability of the reconstructed samples adjacent to the current block, the coding time is shortened, data redundancy is eliminated, bits needing to be transmitted are reduced, and the coding performance is improved.

Description

A local illumination compensation method, video encoding and decoding method, device and system Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to a local illumination compensation method, a video encoding and decoding method, a device, and a system. Background Art

[0002] Currently common video codec standards, such as H.266 / Versatile Video Coding (VVC), all utilize a block-based hybrid coding framework. Each video frame is divided into square largest coding units (LCUs) of the same size (e.g., 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) according to a set of rules. Coding units may also be divided into prediction units (PUs) and transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and inter-frame prediction. Inter-frame prediction includes motion estimation and motion compensation. Because adjacent pixels within a video frame are strongly correlated, intra-frame prediction is used in video codecs to eliminate spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent frames in a video, inter-frame prediction methods are used in video coding and decoding technology to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency. However, the current inter-frame prediction methods need to be improved to achieve higher coding efficiency.

[0003] SUMMARY OF THE INVENTION

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] An embodiment of the present disclosure provides a local illumination compensation method, which is applied to a decoding end and includes:

[0006] Analyze the local illumination compensation LIC usage flag of the current block;

[0007] When it is determined according to the LIC use flag bit that the current block uses LIC, the LIC mode used by the current block is determined according to the availability of reconstructed samples adjacent to the current block;

[0008] When inter prediction is performed on the current block, LIC is used according to the determined LIC mode.

[0009] An embodiment of the present disclosure further provides a video decoding method, including:

[0010] When determining that the current block uses the first inter-frame mode, obtaining sequence-level and frame-level syntax elements related to LIC use;

[0011] When it is determined according to the sequence-level and frame-level syntax elements that LIC is allowed to be used when performing inter-frame prediction on the current block, LIC is used when performing inter-frame prediction on the current block according to the local illumination compensation method applied to the decoding end as described in any embodiment of the present disclosure.

[0012] An embodiment of the present disclosure further provides a local illumination compensation method, which is applied to an encoding end and includes:

[0013] When LIC is allowed to be used when performing inter-frame prediction on the current block, determining the LIC mode to be used when attempting to encode the current block based on the availability of reconstructed samples adjacent to the current block;

[0014] An encoding attempt is performed on the current block using the determined LIC mode and a rate-distortion cost is calculated. Whether to use the LIC when performing inter-frame prediction on the current block is determined according to the rate-distortion cost.

[0015] An embodiment of the present disclosure further provides a video encoding method, including:

[0016] When the current block uses the first inter-frame mode, obtaining sequence-level and frame-level syntax elements related to local illumination compensation (LIC);

[0017] When it is determined according to the sequence-level and frame-level syntax elements that LIC is allowed to be used when performing inter-frame prediction on the current block, LIC is used when performing inter-frame prediction on the current block according to the local illumination compensation method applied to the encoding end as described in any embodiment of the present disclosure.

[0018] An embodiment of the present disclosure further provides a local illumination compensation device, comprising a processor and a memory storing a computer program, wherein the processor can implement the local illumination compensation method as described in any embodiment of the present disclosure when executing the computer program.

[0019] An embodiment of the present disclosure further provides a video decoding device, comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0020] An embodiment of the present disclosure further provides a video encoding device, comprising a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0021] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0022] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the local illumination compensation method as described in any embodiment of the present disclosure, or implements the video decoding method as described in any embodiment of the present disclosure, or implements the video encoding method as described in any embodiment of the present disclosure.

[0023] An embodiment of the present disclosure further provides a code stream, wherein the code stream is generated by the video encoding method as described in any embodiment of the present disclosure.

[0024] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0025] Summary of the Figures

[0026] The accompanying drawings are used to provide an understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation to the technical solutions of the present disclosure.

[0027] FIG1A is a schematic diagram of a coding and decoding system according to an embodiment of the present disclosure;

[0028] FIG1B is a framework diagram of an encoding end according to an embodiment of the present disclosure;

[0029] FIG1C is a framework diagram of a decoding end according to an embodiment of the present disclosure;

[0030] 2A and 2B are schematic diagrams of two images with different illumination;

[0031] 3A and 3B are schematic diagrams of reconstructed samples adjacent to a reference block and a current block, respectively, according to an embodiment of the present disclosure;

[0032] FIG4 is a schematic diagram of performing linear transformation on a prediction block of a current block when local illumination compensation is used according to an embodiment of the present disclosure;

[0033] FIG5A is a schematic diagram of an embodiment of the present disclosure in which only reconstructed samples adjacent to the upper sides of the current block and the reference block are selected to perform linear transformation on the prediction block of the current block;

[0034] FIG5B is a schematic diagram of an embodiment of the present disclosure in which only reconstructed samples adjacent to the left of the current block and the reference block are selected to perform linear transformation on the prediction block of the current block;

[0035] FIG6 is a flowchart of a local illumination compensation method for an encoding end according to an embodiment of the present disclosure;

[0036] FIG7 is a flowchart of a video encoding method according to an embodiment of the present disclosure;

[0037] FIG8 is a flowchart of a local illumination compensation method for a decoding end according to an embodiment of the present disclosure;

[0038] FIG9 is a flowchart of a video decoding method according to an embodiment of the present disclosure;

[0039] FIG10 is a schematic diagram of a local illumination compensation device according to an embodiment of the present disclosure.

[0040] Details

[0041] The present disclosure describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure.

[0042] In the description of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment described as "exemplary" or "for example" in the present disclosure should not be interpreted as being more preferred or advantageous than other embodiments. "And / or" in this article is a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "Multiple" refers to two or more than two. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present disclosure, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0043] When describing representative exemplary embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art can readily understand that these sequences can vary and still remain within the spirit and scope of the disclosed embodiments.

[0044] The local illumination compensation method and video coding and decoding method proposed in the embodiments of the present disclosure can be applied to various video coding and decoding standards, such as: H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), AVS (Audio Video Coding Standard), and other standards developed by MPEG (Moving Picture Experts Group), AOM (Alliance for Open Media), JVET (Joint Video Experts Team) and extensions of these standards, or any other customized standards.

[0045] FIG1A is a block diagram of a video encoding and decoding system that can be used in an embodiment of the present disclosure. As shown in the figure, the system is divided into an encoding end device 1 and a decoding end device 2. The encoding end device 1 generates a code stream. The decoding end device 2 can decode the code stream. The decoding end device 2 can receive the code stream from the encoding end device 1 via a link 3. The link 3 includes one or more media or devices capable of moving the code stream from the encoding end device 1 to the decoding end device 2. In one example, the link 3 includes one or more communication media that enable the encoding end device 1 to send the code stream directly to the decoding end device 2. The encoding end device 1 modulates the code stream according to a communication standard (e.g., a wireless communication protocol) and sends the modulated code stream to the decoding end device 2. The one or more communication media may include wireless and / or wired communication media and may form part of a packet network. In another example, the code stream can also be output from the output interface 15 to a storage device, and the decoding end device 2 can read the stored data from the storage device via streaming or downloading.

[0046] As shown in the figure, the encoding end device 1 includes a data source 11, a video encoding device 13 and an output interface 15. The data source 11 includes a video capture device (such as a camera), an archive containing previously captured data, an input interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. The video encoding device 13 encodes the data from the data source 11 and outputs it to the output interface 15. The output interface 15 may include at least one of a regulator, a modem and a transmitter. The decoding end device 2 includes an input interface 21, a video decoding device 23 and a display device 25. The input interface 21 includes at least one of a receiver and a modem. The input interface 21 can receive a code stream via the link 3 or from a storage device. The video decoding device 23 decodes the received code stream. The display device 25 is used to display the decoded data. The display device 25 can be integrated with other devices of the decoding end device 2 or set separately. The display device 25 is optional for the decoding end. In other examples, the decoding end may include other devices or equipment that apply the decoded data.

[0047] Based on the video coding and decoding system shown in FIG. 1A , various video coding and decoding methods can be used to implement video compression and decompression.

[0048] FIG1B is a block diagram of an exemplary video encoding device that can be used in embodiments of the present disclosure. As shown, the video encoding device 1000 includes a prediction unit 1100, a partitioning unit 1101, a residual generation unit 1102 (indicated by the circle with a plus sign after the partitioning unit 1101 in the figure), a transform processing unit 1104, a quantization unit 1106, an inverse quantization unit 1108, an inverse transform processing unit 1110, a reconstruction unit 1112 (indicated by the circle with a plus sign after the inverse transform processing unit 1110 in the figure), a filter unit 1113, a decoded image buffer 1114, and an entropy coding unit 1115. The prediction unit 1100 includes an inter-frame prediction unit 1121 and an intra-frame prediction unit 1126. The decoded image buffer 1114 may also be referred to as a decoded image buffer, a decoded picture buffer, or a decoded picture buffer. The video encoder 20 may also include more, fewer, or different functional components than shown in this example. For example, the transform processing unit 1104 and the inverse transform processing unit 1110 may be eliminated in some cases.

[0049] The division unit 1101 cooperates with the prediction unit 1100 to divide the received video data into slices, coding tree units (CTUs) or other larger units. The video data received by the division unit 1101 can be a video sequence including video frames such as I frames, P frames or B frames.

[0050] The prediction unit 1100 can divide a CTU into coding units (CUs) and perform intra-frame prediction coding or inter-frame prediction coding on the CU. When performing intra-frame prediction and inter-frame prediction on the CU, the CU can be divided into one or more prediction units (PUs).

[0051] The inter-frame prediction unit 1121 can perform inter-frame prediction on the PU and generate prediction data for the PU, which includes the prediction block of the PU, the motion information of the PU, and various syntax elements. The inter-frame prediction unit 1121 can include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be used to estimate motion to generate a motion vector, and the motion compensation unit can be used to obtain or generate a prediction block based on the motion vector.

[0052] The intra-prediction unit 1126 may perform intra prediction on the PU and generate prediction data for the PU. The prediction data for the PU may include a prediction block and various syntax elements for the PU.

[0053] The residual generation unit 1102 may subtract the prediction blocks of the PUs into which the CU is partitioned from the original blocks of the CU to generate the residual blocks of the CU.

[0054] The transform processing unit 1104 may divide a CU into one or more transform units (TUs). The division of a prediction unit (PU) and a transform unit (TU) may be different. A TU-associated residual block is a sub-block obtained by dividing the residual block of the CU. One or more transforms are applied to the TU-associated residual block to generate a TU-associated coefficient block.

[0055] The quantization unit 1106 may quantize the coefficients in the coefficient block based on the selected quantization parameter. The degree of quantization of the coefficient block may be adjusted by adjusting the quantization parameter (QP: Quantizer Parameter).

[0056] The inverse quantization unit 1108 and the inverse transform unit 1110 may apply inverse quantization and inverse transform to the coefficient block, respectively, to obtain a reconstructed residual block associated with the TU.

[0057] The reconstruction unit 1112 may add the reconstructed residual block and the prediction block generated by the prediction unit 1100 to generate a reconstructed image.

[0058] Filter unit 1113 performs loop filtering on the reconstructed image and stores the filtered reconstructed image in decoded image buffer 1114 as a reference image. Intra-frame prediction unit 1126 extracts reference images of blocks adjacent to the PU from decoded image buffer 1114 to perform intra-frame prediction. Inter-frame prediction unit 1121 can use the reference image of the previous frame cached in decoded image buffer 1114 to perform inter-frame prediction on the PU of the current frame image.

[0059] The entropy coding unit 1115 may perform entropy coding operations on received data (eg, syntax elements, quantized coefficient blocks, motion information, etc.).

[0060] FIG1C is a block diagram of an exemplary video decoding device that can be used in embodiments of the present disclosure. As shown, the video decoding device 101 includes an entropy decoding unit 150, a prediction unit 152, an inverse quantization unit 154, an inverse transform unit 156, a reconstruction unit 158 ​​(represented by the circle with a plus sign after the inverse transform unit 155), a filter unit 159, and a decoded image buffer 160. In other embodiments, the video decoder 30 may include more, fewer, or different functional components. For example, the inverse transform unit 155 may be eliminated in some cases.

[0061] Entropy decoding unit 150 may perform entropy decoding on the received bitstream to extract syntax elements, quantized coefficient blocks, motion information for the PU, etc. Prediction unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 159 may each perform corresponding operations based on the syntax elements extracted from the bitstream.

[0062] The inverse quantization unit 154 may perform inverse quantization on the coefficient block associated with the quantized TU.

[0063] Inverse transform processing unit 156 may apply one or more inverse transforms to the inverse quantized coefficient block to generate a reconstructed residual block for the TU.

[0064] Prediction unit 152 includes inter-prediction unit 162 and intra-prediction unit 164. If a PU is encoded using intra prediction, intra prediction unit 164 may determine an intra prediction mode for the PU based on syntax elements parsed from the codestream, perform intra prediction based on the determined intra prediction mode and reconstructed reference information of the PU's neighbors obtained from decoded picture buffer 160, and generate a prediction block for the PU. If a PU is encoded using inter prediction, inter prediction unit 162 may determine one or more reference blocks for the PU based on the PU's motion information and corresponding syntax elements, and generate a prediction block for the PU based on the reference blocks obtained from decoded picture buffer 160.

[0065] The reconstruction unit 158 ​​may obtain a reconstructed image based on the reconstructed residual block associated with the TU and the prediction block of the PU generated by the prediction unit 152 .

[0066] The filter unit 159 may perform loop filtering on the reconstructed image, and the filtered reconstructed image is stored in the decoded image buffer 160. The decoded image buffer 160 may provide a reference image for subsequent motion compensation, intra-frame prediction, inter-frame prediction, etc., and may also output the filtered reconstructed image as decoded video data for presentation on a display device.

[0067] Based on the above-described video encoding and decoding devices, the following basic encoding and decoding process can be performed. On the encoding side, a frame of an image is divided into blocks. Intra-frame prediction, inter-frame prediction, or other algorithms are performed on the current block to generate a predicted block for the current block. The predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain quantized coefficients, and the quantized coefficients are entropy encoded to generate a bitstream. On the decoding side, intra-frame prediction or inter-frame prediction is performed on the current block to generate a predicted block for the current block. The quantized coefficients obtained by parsing the bitstream are then inversely quantized and inversely transformed to obtain a residual block. The predicted block and residual block are added together to obtain a reconstructed block. The reconstructed block forms a reconstructed image. The reconstructed image is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoding side also performs similar operations as the decoding side to obtain a decoded image, which is often referred to as a reconstructed image. The decoded image obtained by the encoding side can be used as a reference frame for inter-frame prediction of subsequent frames. The block division information determined by the encoding side, as well as information and parameters for prediction, transform, quantization, entropy coding, and loop filtering modes, can be written into the bitstream if necessary. The decoding end determines the same block division information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information and parameter information as the encoding end by parsing the bit stream or analyzing based on existing information, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end.

[0068] Although the above is an example of a block-based hybrid coding framework, the embodiments of the present disclosure are not limited thereto. With the development of technology, one or more modules in the framework and one or more steps in the process can be replaced or optimized.

[0069] Local Illuminance Compensation (LIC) is an inter-frame prediction coding technique, also referred to as LIC in this article. LIC affects the inter-frame prediction portion of a hybrid video coding framework and is used simultaneously on both the encoder and decoder sides.

[0070] During inter-frame prediction coding, motion vector information (MV) is used to determine the reference block used for inter-frame prediction of the current block. The reference block typically comes from a different image (e.g., a different frame). Video content often experiences variations in illumination intensity, such as increases or decreases over time, or variations in illumination intensity due to cloud cover or camera flash. The difference between these frames lies primarily in the strength of the image's DC component, which manifests as changes in brightness. Texture information remains largely unchanged. Motion estimation and motion compensation used in inter-frame prediction techniques cannot effectively predict these variations. Due to the large DC component, a large amount of residual information is often encoded during coding. Local illumination compensation methods can effectively remove this redundant DC information, accurately predict brightness changes, and compensate accordingly, minimizing residual information and thus improving coding efficiency.

[0071] Taking Figure 2A and Figure 2B as an example, the texture information of the two images is basically the same, and the difference lies in the brightness change between the two. Since the image in Figure 2B appears very bright due to the illumination of the camera flash, while the image in Figure 2A was taken under normal natural light, there is a difference between the two. The burden brought by this difference in video encoding is enormous. Assuming that the current block in the image of Figure 2A uses the corresponding reconstructed block in the image of Figure 2B as a reference block, the texture information of the two is the same, but the overall residual is large. Because the brightness value of the pixels in the image of Figure 2B is offset as a whole due to the effect of the flash, this offset is included in the calculated residual. If the residual is directly transformed and quantized and written into the bitstream, it will incur a lot of overhead.

[0072] In a first embodiment, a local illumination compensation method is provided. During the digital video encoding and decoding process, this method uses an illumination compensation model to correct illumination differences and obtain an illumination-compensated prediction block. This eliminates the effects of flash or illumination variations, resulting in better overall prediction results.

[0073] This embodiment uses the correlation between adjacent reconstructed samples of the current block and the reference block to fit the changing correlation between the prediction samples and the reference samples within the coding block. The reconstructed samples adjacent to the top and left of the current block in the current image, if available, can be obtained. The reconstructed samples adjacent to the top and left of the reference block in the reference image can also be obtained. Modeling based on the adjacent reconstructed samples of the current block and the adjacent reconstructed samples of the reference block yields a corresponding fitting model, referred to as a local illumination compensation model, also referred to as a linear model or simply a model.

[0074] In this article, "adjacent" means spatially adjacent, and a reconstructed sample may also be referred to as a reference sample, a reconstructed reference sample, a template sample, a template reference sample, etc.

[0075] In this embodiment, a linear fitting method is used in the modeling process of local illumination compensation. The illumination change between the current block and the reference block is fitted by the scaling parameter a and the bias parameter b. The predicted value of the pixel in the current block is updated by the calculated model parameters. The formula is as follows:

[0076] Pred′(x,y)=a·Pred(x,y)+b (1)

[0077] Among them, Pred(x,y) is the predicted value before illumination compensation (that is, the predicted value without illumination compensation), such as the pixel value in the reference image pointed to by MV at (x,y); Pred′(x,y) is the predicted value after illumination compensation, a is the scaling parameter in the model, and b is the offset parameter in the model.

[0078] The above-mentioned update of the prediction value is applicable to both the luminance component and the chrominance component.

[0079] The calculation of the scaling parameter a and the offset parameter b requires the help of the reconstructed samples adjacent to the current block and the reference block. The reconstructed samples shown in FIG3A include the reconstructed samples adjacent to the upper side of the reference block and the reconstructed samples adjacent to the left side. The reconstructed samples in the reference image can be obtained by interpolation of reconstructed pixels. The reconstructed samples shown in FIG3B include the reconstructed samples adjacent to the upper side of the current block and the reconstructed samples adjacent to the left side. The reconstructed samples of the current image can be the reconstructed pixels in the current image. The reconstructed samples adjacent to the reference block and the current block in the figure are all in a single row or a single column, but the present disclosure is not limited to this.

[0080] By modeling and solving the linear relationship between the adjacent reconstructed samples of the reference block and the adjacent reconstructed samples of the current block, the scaling parameter a and the offset parameter b can be obtained. The derivation formula is as follows:

[0081] Curr_Rec neigh =a·Ref_Rec neigh +b (2)

[0082] Where Curr_Rec neigh is the reconstructed sample adjacent to the current block, Ref_Rec neigh are the reconstructed samples adjacent to the reference block.

[0083] In this article, the current block can be the current coding unit (CU) or current prediction unit (PU) in the current image. The reference block is the reconstructed CU or PU in the reference image used for inter-frame prediction of the current block. The position of the reference block is determined by the current block and the motion vector. The current block and the reference block are of equal size.

[0084] Before calculating the LIC model parameters for the current block, the number and positions of reconstructed samples used to calculate the model parameters are determined based on the width and height of the current block. In one example, if at least one of the width and height of the current block is equal to 4, four samples are selected from the reconstructed samples adjacent to the top and left of the current block for calculating the LIC model parameters. For example, if the current block has a width of 16 and a height of 4, all four reconstructed samples adjacent to the left of the current block are selected, and then four samples are selected from the 16 reconstructed samples adjacent to the top of the current block in steps of 3 to calculate the LIC model parameters. In another example, if both the width and height of the current block are not equal to 4, the number of reconstructed samples selected from the top and left sides of the current block is determined based on the smaller side length of the current block. This number can be the base-2 logarithm of the smaller side length of the current block, expressed as sample_num = log2(min(width, height)), where sample_num is the number of reconstructed samples selected, and width and height represent the width and height of the current block, respectively. For example, if the smaller side length of the current block is 8, 3 reconstructed samples can be selected from the top and left sides of the current block, respectively. The number of reconstructed samples adjacent to the reference block used to calculate the LIC model parameters and the position of the reconstructed samples adjacent to the current block used to calculate the LIC model parameters are the same and correspond in position.

[0085] In one example of this embodiment, after obtaining the reconstructed samples used to calculate the model parameters, the scaling factor a and the offset parameter b are obtained by the least square error method. The reconstructed samples obtained from the reference image (i.e., the reconstructed samples adjacent to the reference block) are denoted as x, and the reconstructed samples obtained from the current image (i.e., the reconstructed samples adjacent to the current block) are denoted as y; the sum of the reconstructed samples x obtained from the reference image is denoted as sumX, and the sum of the reconstructed samples y obtained from the current image is denoted as sumY; the sum of the squares of the reconstructed samples x obtained from the reference image is denoted as sumXX, and the sum of the products of the reconstructed samples x obtained from the reference image and the reconstructed samples y obtained from the current image (summed after bitwise multiplication) is denoted as sumXY. Parameters a and b can be calculated using the following formula:

[0086] a=(sumXY-sumXsumY) / (sumXX–sumXsumX)

[0087] b=sumY–a·sumX

[0088] In the above formula, sumXsumY represents sumX multiplied by sumY. In the specific calculation process, some shift operations can be used to simplify the calculation process, which will not be elaborated here. It should be noted that the calculation of model parameters is not limited to the specific algorithm in the above example.

[0089] After obtaining the LIC model parameters (also called linear model parameters), a linear transformation is performed on the prediction block obtained through motion compensation to obtain the prediction block after local illumination compensation, as shown in Figure 4. In this figure, the square in the current image represents the current block, the square in the reference image represents the reference block of the current block, PredCU represents the prediction block before local illumination compensation, and PredCU′ represents the prediction block after local illumination compensation. The change in the prediction value after local illumination compensation is shown in Equation (1).

[0090] This embodiment calculates LIC model parameters based on the reconstructed samples adjacent to the top and left of the current block and the reference block, and is referred to as the LIC_TL mode, where T represents top and L represents left. The LIC_TL mode can also be referred to as the original illumination compensation mode.

[0091] In LIC_TL mode, LIC model parameters are calculated based on the reconstructed samples adjacent to the upper and left sides of the current block and the reference block. When both the upper and left sides of the current block are available, the LIC model parameters are calculated using the upper and left sides of the current block and the reference block (including the upper and left sides of the current block, and the upper and left sides of the reference block). However, LIC_TL mode does not require both the upper and left sides of the current block to be available. When only the upper side of the current block is available, in LIC_TL mode, only the upper side of the current block and the reference block (including the upper side of the current block and the upper side of the reference block) are selected to calculate the LIC model parameters. When only the left side of the current block is available, in LIC_TL mode, only the left side of the current block and the reference block (including the left side of the current block and the left side of the reference block) are selected to calculate the LIC model parameters.

[0092] The local illumination compensation method of the LIC_TL mode of this embodiment can be used in ordinary inter-frame prediction, namely, inter mode, merge prediction mode, namely, merge mode (also called merge mode), and sub-block mode.

[0093] The local illumination compensation method in the LIC_TL mode of this embodiment is only used in the single-frame prediction mode and is prohibited in the multi-frame bidirectional reference mode. Furthermore, the illumination compensation method of this embodiment is coupled with other adopted inter-frame prediction methods. For the current block, the illumination compensation method cannot be used simultaneously with bidirectional optical flow (BDOF) technology or symmetric motion vector difference (SMVD) mode.

[0094] In the second embodiment, two different LIC modes are added: LIC_T and LIC_L. These two modes are extensions of the LIC_TL mode and are applied to Advanced Motion Vector Prediction (AMVP). Under common test conditions, the method of this embodiment can provide an average encoding performance improvement of 0.07% without any decoding complexity.

[0095] In LIC_T mode, only the reconstructed samples adjacent to the upper side of the current block and the reference block are selected to calculate the LIC model parameters. Although in LIC_TL mode, when only the reconstructed samples adjacent to the upper side of the current block are available, only the reconstructed samples adjacent to the upper side of the current block and the reference block are selected to calculate the LIC model parameters, unlike in LIC_TL mode, in LIC_T mode, even if both the reconstructed samples adjacent to the upper side of the current block and the reconstructed samples adjacent to the left of the current block are available, only the reconstructed samples adjacent to the upper side of the current block and the reference block are selected to calculate the LIC model parameters (see Figure 5A);

[0096] In LIC_L mode, only the reconstructed samples adjacent to the left of the current block and the reference block are selected to calculate the LIC model parameters. Although in LIC_TL mode, when only the reconstructed samples adjacent to the left of the current block are available, only the reconstructed samples adjacent to the left of the current block and the reference block are selected to calculate the LIC model parameters, unlike in LIC_TL mode, in LIC_L mode, even if both the reconstructed samples adjacent to the top and the left of the current block are available, only the reconstructed samples adjacent to the left of the current block and the reference block are selected to calculate the LIC model parameters (see Figure 5B).

[0097] However, when the second embodiment determines that LIC is permitted for inter-frame prediction of the current block, encoding attempts are made using each of the three LIC modes, and the rate-distortion cost is calculated to select the optimal LIC mode. This approach is relatively simple. For example, when parsing the bitstream, the decoder does not check the reconstructed samples when the current block is at the boundary, which may result in data redundancy.

[0098] For example, if the current block uses LIC and is located at the top edge of the image, only the reconstructed samples adjacent to the left are available. When attempting to encode using LIC_TL mode, only the reconstructed samples adjacent to the left of the current block and the reference block are selected for LIC model parameter calculation based on the availability of reconstructed samples. When attempting to encode using LIC_L mode, only the reconstructed samples adjacent to the left of the current block and the reference block are selected for LIC model parameter calculation. In this case, the LIC_TL mode is identical to the LIC_L mode, resulting in duplicate computations and an extra bit in the bitstream representing the LIC_L mode. This undoubtedly results in extra bits transmitted and increased overhead for coding units using illumination compensation techniques at the edge. A similar situation occurs when the current block uses LIC and is located at the left edge of the image. In this case, the LIC_TL mode is identical to the LIC_T mode.

[0099] Furthermore, in LIC_TL mode, the number of reconstructed samples selected for the current block at the edge and outside the edge is the same, determined by the smaller of width or height. If the current block is at the upper edge of the image and LIC is used, and the block size is 4x32, 4 of the 32 reconstructed samples adjacent to the left of the current block are selected to calculate the LIC model parameters, with a step size of 8. This selection method results in a larger step size, which may lead to significant deviation and inaccuracy in the calculation of model parameters, resulting in reduced encoding performance.

[0100] An embodiment of the present disclosure provides a local illumination compensation method, which is applied to an encoding end, as shown in FIG6 , and includes:

[0101] Step 110 , when LIC is allowed to be used when performing inter-frame prediction on the current block, determining the LIC mode to be used when attempting to encode the current block based on the availability of reconstructed samples adjacent to the current block;

[0102] In this step, the encoder can traverse the prediction modes and, if the current block uses the first inter mode, obtain LIC-related syntax elements to determine whether LIC is allowed for inter prediction of the current block. If LIC is allowed, a coding attempt is required to determine whether to use LIC based on the rate-distortion cost.

[0103] Step 120: Use the determined LIC mode to attempt to encode the current block and calculate a rate-distortion cost, and determine whether to use LIC when performing inter-frame prediction on the current block based on the rate-distortion cost.

[0104] In this step, after encoding the current block using the determined LIC mode and calculating the rate-distortion cost, the LIC mode with the lowest rate-distortion cost is the optimal LIC mode. The encoder can then continue to traverse other inter-frame prediction techniques (also known as inter-frame prediction modes), such as BDOF and SMVD, calculate the rate-distortion cost corresponding to each mode, and compare it with the rate-distortion cost of the optimal LIC mode. If the optimal LIC mode has the lowest rate-distortion cost, LIC is used for inter-frame prediction of the current block; otherwise, LIC is not used. The "optimal LIC mode" here is not limited to using a single LIC mode but can also refer to a combination of an LIC mode and other inter-frame modes that are allowed to be used simultaneously with the LIC mode.

[0105] In the embodiments of the present disclosure, when LIC is allowed to be used during inter-frame prediction of the current block, boundary detection of the current block is added to determine the availability of reconstructed samples adjacent to the current block. The LIC mode to be used when attempting to encode the current block is determined based on the availability of the reconstructed samples adjacent to the current block. If only the reconstructed samples adjacent to the upper side or the reconstructed samples adjacent to the left side of the current block are available, only one or two of the LIC_TL mode, LIC_T mode, and LIC_L mode are used for the encoding attempt, rather than using the three LIC modes one by one for the encoding attempt. This eliminates data redundancy, reduces the bits required for transmission, and reduces overhead.

[0106] In an exemplary embodiment of the present disclosure, determining the LIC mode to be used when attempting to encode the current block based on the availability of reconstructed samples adjacent to the current block includes: when both the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current block are available, using the LIC_TL mode, the LIC_T mode, and the LIC_L mode to attempt to encode.

[0107] In an exemplary embodiment of the present disclosure, when only one-side adjacent reconstructed samples of the current block are available, an encoding attempt using one LIC mode is skipped, and no encoding attempt is performed using the LIC_T mode or the LIC_L mode.

[0108] In this embodiment, determining the LIC mode to be used when attempting to encode the current block based on the availability of reconstructed samples adjacent to the current block includes:

[0109] When only the reconstructed samples of the upper adjacent block are available for the current block, use the LIC_TL mode and the LIC_L mode for coding attempts;

[0110] When only the left adjacent reconstructed samples of the current block are available, encoding attempts are made using the LIC_TL mode and the LIC_T mode. In the LIC_T mode, only the upper adjacent reconstructed samples of the current block and the reference block are selected to calculate the LIC model parameters.

[0111] If only the reconstructed samples adjacent to the upper side of the current block are available, when the LIC_L mode is used for encoding, padding can be used to obtain reconstructed samples adjacent to the left of the current block for use in calculating the LIC model parameters. During padding, the value of the leftmost reconstructed sample among the adjacent samples adjacent to the upper side of the current block can be assigned to all adjacent reconstructed samples to the left of the current block, or a specified value (such as the middle value of a sample value range) can be assigned to all adjacent reconstructed samples to the left of the current block. If only the reconstructed samples adjacent to the left side of the current block are available, when the LIC_T mode is used for encoding, padding can be used to obtain reconstructed samples adjacent to the upper side of the current block for use in calculating the LIC model parameters. The padding method is similar and will not be repeated here.

[0112] Performing an encoding attempt and calculating a rate-distortion cost in the manner described above, and determining, based on the rate-distortion cost, to use LIC when performing inter-frame prediction on the current block, the method further includes:

[0113] Set the LIC usage flag of the current block (the flag at the coding unit level) to true and write it into the bitstream;

[0114] When the LIC mode with the lowest rate-distortion cost is LIC_TL mode during the encoding attempt, the LIC extension flag of the current block is set to false and written into the bitstream;

[0115] When the LIC mode with the lowest rate-distortion cost during the encoding attempt is LIC_T mode or LIC_L mode, the LIC extension flag of the current block is set to true and written into the bitstream;

[0116] In one example of this embodiment, when the LIC mode with the lowest rate-distortion cost is the LIC_T mode or the LIC_L mode, and if both the reconstructed samples adjacent to the upper and left sides of the current block are available, the mode index of the LIC mode with the lowest rate-distortion cost is also encoded and written into the bitstream. However, when only the reconstructed samples adjacent to the upper or left sides of the current block are available, encoding of the mode index is unnecessary because encoding attempts are not performed simultaneously using the LIC_T and LIC_L modes. The decoder determines the optimal LIC mode (i.e., the LIC mode with the lowest rate-distortion cost) based on the availability of the adjacent reconstructed samples of the current block. Specifically, when only the reconstructed samples adjacent to the upper side of the current block are available, the optimal LIC mode is determined to be the LIC_T mode; when only the reconstructed samples adjacent to the left side of the current block are available, the optimal LIC mode is determined to be the LIC_T mode.

[0117] In another exemplary embodiment of the present disclosure, when only one-side adjacent reconstructed samples of the current block are available, encoding attempts in two LIC modes are skipped, and encoding attempts in the LIC_T mode and the LIC_L mode are no longer used.

[0118] In this embodiment, determining the LIC mode to be used when attempting to encode the current block based on the availability of reconstructed samples adjacent to the current block includes: determining to use the LIC_TL mode for the encoding attempt on the current block when only the reconstructed samples adjacent to the upper side or only the reconstructed samples adjacent to the left side of the current block are available.

[0119] Performing an encoding attempt and calculating a rate-distortion cost in the manner described above, and determining, based on the rate-distortion cost, to use LIC when performing inter-frame prediction on the current block, the method further includes:

[0120] Set the LIC usage flag of the current block to true and write it into the code stream;

[0121] When only the reconstructed samples adjacent to the upper side or the reconstructed samples adjacent to the left side of the current block are available, there is no need to encode the LIC extension flag of the current block and the mode index of the LIC mode. In other words, when only the reconstructed samples adjacent to one side of the current block are available, this embodiment only needs to encode the LIC usage flag of the current block, reducing the codewords that need to be transmitted and correspondingly simplifying the decoding process at the decoding end.

[0122] When both the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current block are available, if the LIC mode with the lowest rate-distortion cost during the encoding attempt is the LIC_TL mode, the LIC extension flag of the current block is set to false and written into the bitstream; if the LIC mode with the lowest rate-distortion cost is the LIC_T mode or the LIC_L mode, the LIC extension flag of the current block is set to true and written into the bitstream, and the mode index of the LIC mode with the lowest rate-distortion cost is encoded and written into the bitstream.

[0123] The local illumination compensation method in the above-described embodiment of the present disclosure adds boundary detection to the current block when LIC is permitted. When only reconstructed samples adjacent to one side of the current block are available, the method skips attempting one or more LIC modes, simplifying the computational process. Furthermore, the encoding of LIC-related syntax elements for the current block can be modified, such as by no longer encoding the LIC mode index or the mode index and LIC extension flag. This can save encoding time, reduce overhead, and improve encoding efficiency.

[0124] An embodiment of the present disclosure further provides a video encoding method, as shown in FIG7 , including:

[0125] Step 210: When the current block uses the first inter-frame mode, obtain sequence-level and frame-level syntax elements related to LIC;

[0126] In this step, the encoder can traverse the prediction modes and, when the current block uses the first inter-frame mode, obtain the sequence-level LIC usage permission flag and the frame-level LIC usage flag. The sequence-level LIC usage permission flag indicates whether the current encoder allows the use of LIC, i.e., local illumination compensation, and the frame-level LIC usage flag indicates whether the current frame allows the use of LIC.

[0127] Step 220: When it is determined based on the sequence-level and frame-level syntax elements that the LIC is allowed to be used when performing inter-frame prediction on the current block, use the LIC when performing inter-frame prediction on the current block according to the local illumination compensation method applied to the encoder as described in any embodiment of the present disclosure.

[0128] In one example, if the LIC usage permission flag at the sequence level is true, indicating that the LIC is permitted to be used, and the LIC usage flag at the frame level is true, indicating that the LIC is used, it can be determined that the LIC is permitted to be used when performing inter-frame prediction on the current block. If at least one of the two flags is false (i.e., false), it is determined that the LIC is not permitted to be used when performing inter-frame prediction on the current block.

[0129] In this paper, the bit indicating that LIC is allowed to be used at the frame level is called the LIC use identification bit, and the mark indicating that LIC is allowed to be used at the coding unit level is called the LIC use flag bit, to facilitate the distinction between the two.

[0130] The video encoding method of this embodiment uses the local illumination compensation method for the encoding end of any embodiment of the present disclosure to perform a boundary check on the current block to determine the availability of reconstructed samples adjacent to the current block. When only reconstructed samples adjacent to one side of the current block are available, the encoding attempt of at least one of the LIC_T mode and the LIC_L mode is skipped, and the encoding of LIC-related syntax elements (such as flag bits and indexes) is reduced. This can save encoding time and bit overhead, thereby improving encoding efficiency.

[0131] In some embodiments, the prediction mode that requires parsing the flag bit when using LIC is a normal inter-frame prediction mode, and does not include the merge mode. In an exemplary embodiment of the present disclosure, the application of the LIC_T mode and the LIC_L mode is extended to the merge mode, and a syntax element is added in the merge mode to indicate the LIC_T mode or the LIC_L mode. In one example, the local illumination compensation method of any embodiment of the present disclosure is applied to the merge mode in the unidirectional reference mode, but is not used simultaneously with the combined inter and intra prediction (CIIP) mode, the geometry partition mode (GPM) and the affine mode (Affine_merge) in the merge mode in the unidirectional reference mode, and can be used simultaneously with the merge mode with motion vector difference (MMVD_Merge). In another example, the local illumination compensation method of any embodiment of the present disclosure is applied to the merge mode in the bidirectional reference mode (only used for the merge mode in the bidirectional reference mode), but is not used simultaneously with the bidirectional optical flow mode, the decoding-side motion vector derivation mode and the bidirectional prediction weighted mode in the merge mode in the bidirectional reference mode. Can be used together with regular_merge mode.

[0132] In an exemplary embodiment of the present disclosure, when constructing a motion vector prediction (MVP) list in merge mode, the current block's LIC-related flags (such as the LIC usage flag) and the LIC mode index are inherited from neighboring blocks or corresponding prediction blocks (neighboring blocks for inter-frame prediction represent surrounding spatially adjacent blocks, while prediction blocks obtained from reference frames represent temporal blocks). Alternatively, the current block's LIC usage flag is inherited from a neighboring block or corresponding prediction block. However, only the LIC usage flag is inherited, not the LIC mode index. The LIC_TL mode index is used as the default mode index. Inheriting the LIC usage flag eliminates the need for an additional rate-distortion optimization (RDO) process at the encoder.

[0133] An embodiment of the present disclosure further provides a local illumination compensation method, which is applied to a decoding end, as shown in FIG8 , and includes:

[0134] Step 310, parsing the LIC usage flag of the current block;

[0135] In this step, when the decoder determines that the current block uses inter-frame mode, it obtains the sequence-level LIC use permission flag and the frame-level LIC use flag. If the LIC use permission flag is true, it means that the current decoder allows the use of LIC, and the LIC use flag is true, it means that LIC is used. The LIC use flag of the current block is parsed to determine whether the current block uses LIC.

[0136] Step 320: When it is determined that the current block uses LIC according to the LIC use flag, the LIC mode used by the current block is determined according to the availability of reconstructed samples adjacent to the current block;

[0137] In this step, because the encoder skips the encoding attempt for the LIC_T mode and / or LIC_L mode when only reconstructed samples adjacent to the current block are available on one side, and may no longer encode the LIC mode index or LIC extension flag, the decoder needs to parse LIC-related syntax elements based on the availability of reconstructed samples adjacent to the current block to determine the LIC mode used for the current block.

[0138] Step 330: When performing inter-frame prediction on the current block, use LIC according to the determined LIC mode.

[0139] In this step, when performing inter-frame prediction on the current block, adjacent reconstructed samples of the current block and the reference block may be selected based on the determined LIC mode to calculate LIC model parameters. A linear transformation is then performed on the prediction block obtained after motion compensation of the current block based on the LIC model parameters to obtain a prediction block after local illumination compensation. The reconstructed block is then obtained by adding the prediction block after local illumination compensation to the decoded residual.

[0140] The disclosed embodiments perform boundary detection on the current block during decoding and parse LIC-related syntax elements based on the availability of adjacent reconstructed samples of the current block. When only adjacent reconstructed samples on one side of the current block are available, the LIC mode used by the current block can be determined using fewer syntax elements and a simpler parsing process, thereby saving bit overhead and improving coding efficiency.

[0141] In an exemplary embodiment of the present disclosure, when only reconstructed samples adjacent to a current block on one side are available, the encoder skips the encoding attempt using the LIC_T mode or the LIC_L mode and no longer encodes the mode index. The parsing process and semantics of this embodiment also change accordingly. Determining the LIC mode to use for the current block based on the availability of reconstructed samples adjacent to the current block includes:

[0142] Continue parsing the LIC extension flag of the current block;

[0143] When the LIC extension flag is true and only the upper adjacent reconstructed samples of the current block are available, determining that the current block uses the LIC_L mode;

[0144] When the LIC extension flag is true and only the left adjacent reconstructed samples of the current block are available, it is determined that the current block uses the LIC_T mode.

[0145] Since the encoder has only one side of adjacent reconstructed samples available for the current block and determines to use the LIC_T mode or the LIC_L mode, the available side is associated with the LIC mode used. Therefore, the LIC mode used by the current block can be determined without using a mode index.

[0146] In this embodiment, determining the LIC mode used by the current block according to the availability of reconstructed samples adjacent to the current block further includes:

[0147] When the LIC extension flag is negative, it is determined that the current block uses the LIC_TL mode.

[0148] When the LIC extension flag is true and the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current block are both available, the LIC mode index of the current block is further parsed, and it is determined whether the current block uses the LIC_T mode or the LIC_L mode according to the LIC mode index.

[0149] In another exemplary embodiment of the present disclosure, when only reconstructed samples adjacent to one side of the current block are available, the encoder skips encoding attempts using the LIC_T mode and the LIC_L mode, and no longer encodes the LIC extension flag (used to indicate whether the LIC_T mode or the LIC_L mode is used) and the mode index. The parsing process and semantics of this embodiment also change accordingly.

[0150] In this embodiment, determining the LIC mode used by the current block according to the availability of reconstructed samples adjacent to the current block further includes:

[0151] When both the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current block are available, continue parsing the LIC extension flag of the current block;

[0152] When the LIC extension flag is negative, determining that the current block uses the LIC_TL mode;

[0153] When the LIC extension flag is true, the LIC mode index of the current block is continuously parsed, and it is determined whether the current block uses the LIC_T mode or the LIC_L mode according to the LIC mode index.

[0154] In this embodiment, determining the LIC mode to use for the current block based on the availability of adjacent reconstructed samples of the current block further includes: when both the upper and left adjacent reconstructed samples of the current block are not available, there is no need to parse the LIC extension flag, and the LIC extension flag defaults to false, and determining that the current block uses the LIC_TL mode. When only adjacent reconstructed samples on one side of the current block are available, the encoder does not allow use of the LIC_T mode or the LIC_L mode. Therefore, when the decoder determines that the current block uses LIC and only adjacent reconstructed samples on the one side are available, it can directly determine that the current block should use the LIC_TL mode without parsing other syntax elements.

[0155] In an exemplary embodiment of the present disclosure, using the LIC according to the determined LIC mode includes:

[0156] When the current block uses the LIC_T mode, only the reconstructed samples adjacent to the upper side of the current block and the reference block are selected to calculate the LIC model parameters, and the prediction block of the current block is linearly transformed according to the LIC model parameters to obtain the prediction block after LIC;

[0157] When the current block uses the LIC_L mode, only the reconstructed samples adjacent to the left of the current block and the reference block are selected to calculate the LIC model parameters, and the prediction block of the current block is linearly transformed according to the LIC model parameters to obtain the prediction block after LIC;

[0158] When the current block uses the LIC_TL mode, LIC model parameters are calculated based on the reconstructed samples adjacent to the upper side and the left side of the current block and the reference block, and the prediction block of the current block is linearly transformed according to the LIC model parameters to obtain the prediction block after LIC.

[0159] An embodiment of the present disclosure further provides a video decoding method, as shown in FIG9 , including:

[0160] Step 410: When it is determined that the current block uses the first inter-frame mode, obtain sequence-level and frame-level syntax elements related to the use of LIC;

[0161] In this step, the parsed sequence-level and frame-level syntax elements may include a sequence-level LIC usage permission flag bit and a frame-level LIC usage identification bit.

[0162] Step 420: When determining, based on the sequence-level and frame-level syntax elements, that LIC is allowed to be used when performing inter-frame prediction on the current block, use LIC when performing inter-frame prediction on the current block according to the local illumination compensation method for the decoding side as described in any embodiment of the present disclosure.

[0163] In this step, if both the LIC usage permission flag and the LIC usage flag are true, the decoding-side local illumination compensation method described in any embodiment of the present disclosure may be used to continue parsing the local illumination compensation LIC usage flag for the current block. If the current block is determined to use LIC based on the LIC usage flag, the LIC mode to use for the current block is determined based on the availability of adjacent reconstructed samples of the current block. Furthermore, when performing inter-frame prediction on the current block, the LIC is used based on the determined LIC mode. This embodiment performs boundary detection on the current block during decoding and parses LIC-related syntax elements based on the availability of adjacent reconstructed samples of the current block, thereby saving bit overhead and improving coding efficiency.

[0164] In an exemplary embodiment of the present disclosure, the application of the LIC_T mode and the LIC_L mode is extended from AMVP to the merge mode, and a syntax element is added in the merge mode to indicate the LIC_T mode or the LIC_L mode. In one example, the local illumination compensation method of any embodiment of the present disclosure is applied to the merge mode in the unidirectional reference mode, but is not used simultaneously with the combined inter and intra prediction (CIIP) mode, GPM, and affine mode (Affine_merge) in the merge mode in the unidirectional reference mode. It can be used simultaneously with the merge mode with motion vector difference (MMVD_Merge). In another example, the local illumination compensation method of any embodiment of the present disclosure is applied to the merge mode in the bidirectional reference mode (only used for the merge mode in the bidirectional reference mode), but is not used simultaneously with the bidirectional optical flow mode, the decoding-side motion vector derivation mode, and the bidirectional prediction weighted mode in the merge mode in the bidirectional reference mode. It can be used simultaneously with the regular merge mode (regular_Merge).

[0165] When constructing the motion vector prediction (MVP) list in merge mode, the current block's LIC-related flags (such as the LIC usage flag) and the LIC mode index are inherited from neighboring blocks or the corresponding prediction block. Alternatively, the current block's LIC usage flag is inherited from a neighboring block or the corresponding prediction block. However, only the LIC usage flag is inherited, not the LIC mode index. The LIC_TL mode index is used as the default mode index. Inheriting the LIC usage flag eliminates the need for an additional rate-distortion optimization (RDO) process at the encoder.

[0166] In an embodiment of the present disclosure, for the LIC_TL mode, a method for calculating the number of selected reconstructed samples is improved, and the number of reconstructed samples is determined in combination with the boundary detection of the current block.

[0167] The algorithm before improvement can be expressed as:

[0168] SampleNum=min(cuWidth, cuHeight).

[0169] The improved algorithm of this embodiment can be expressed as:

[0170] SampleNum=cuAbove&&cuLeft? min(cuWidth,cuHeight):cuAbove? cuWidth:cuHeight;

[0171] In the above expression, SampleNum is the cardinality for calculating the number of selected reconstruction samples, cuWidth and cuHeight are the width and height of the current block, respectively, cuAbove indicates that the adjacent reconstruction samples above the current block are available, and cuLeft indicates that the adjacent reconstruction samples to the left of the current block are available.

[0172] The algorithm before the improvement calculated the number of selected reconstruction samples based on the smaller value of the current block width and high and low, regardless of whether the current block was at the boundary. The improved algorithm, however, incorporates boundary detection. When the conditions are met that the adjacent reconstruction samples above and adjacent reconstruction samples to the left of the current block are both available, the number of selected reconstruction samples is calculated based on the smaller value of the current block width and high and low. If this condition is not met, the algorithm then determines whether the adjacent reconstruction samples above the current block are available. If so, the number of selected reconstruction samples is calculated based on the width of the current block. If the adjacent reconstruction samples above the current block are available, the number of selected reconstruction samples is calculated based on the height of the current block.

[0173] The improved algorithm of this embodiment can be incorporated into the local illumination compensation method of any embodiment of the present disclosure, and can also be applied to other local illumination compensation methods, such as a local illumination compensation method that only adopts the LIC_TL mode, and so on.

[0174] When the improved algorithm of this embodiment is combined with the local illumination compensation method for an encoder in the embodiment of the present disclosure, when attempting to encode the current block using a determined LIC mode, the algorithm includes: when attempting to encode the current block using the LIC_TL mode, if only the reconstructed samples adjacent to the upper side of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined based on the width of the current block; if only the reconstructed samples adjacent to the left side of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined based on the height of the current block. In one example, when determining the number of reconstructed samples selected for calculating the LIC model parameters based on the width of the current block, the number is the base-2 logarithm of the width. For example, if the width is 8, 3 reconstructed samples may be selected, but the present disclosure is not limited to this. The number of reconstructed samples selected for calculating the LIC model parameters based on the height of the current block is similar and will not be further described.

[0175] When the improved algorithm of this embodiment is combined with the local illumination compensation method for a decoding end of the embodiment of the present disclosure, the LIC model parameters are calculated based on the reconstructed samples adjacent to the upper side and the left side of the current block and the reference block, including:

[0176] When only the upper adjacent reconstructed samples of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined according to the width of the current block;

[0177] In the case that only the reconstructed samples adjacent to the left of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined according to the height of the current block.

[0178] This embodiment optimizes the number of reconstructed samples selected for calculating LIC model parameters when the LIC_TL mode is used for encoding and the current block is at the image boundary. This avoids excessively large step sizes in selecting reconstructed samples, increases the accuracy of model parameter calculation, and improves encoding performance.

[0179] An embodiment of the present disclosure provides a video encoding method and a video decoding method, which add illumination compensation mode boundary detection at the coding unit (CU) level at both the encoding and decoding ends. The encoding end skips trying the LIC_L mode when the current block is at the upper boundary; and skips trying the LIC_T mode when the current block is at the left boundary. Accordingly, when parsing the syntax elements of the current block, the decoding end determines the boundary of the current block. If the block is at the boundary, the last syntax element (mode index) does not need to be parsed. Otherwise, all syntax elements related to the LIC in the current block must be parsed.

[0180] In this embodiment, the current block takes the current coding unit as an example. The video coding method of this embodiment includes:

[0181] Step 1: The encoder traverses the prediction mode. If the current coding unit is an inter-frame mode that can use LIC, the encoder obtains the LIC usage permission flag and the frame-level LIC usage flag. The LIC usage permission flag is a sequence-level flag, indicating that the current encoder allows the use of LIC.

[0182] If the LIC usage permission flag is true and the frame-level LIC usage flag is true, the encoder attempts to use the LIC prediction method and executes step 2;

[0183] If the LIC allowed use flag bit or the LIC use flag bit is false, the encoder does not attempt to use the LIC prediction method, skips step 2 and directly executes step 3;

[0184] Step 2: The encoder obtains the adjacent reconstructed sample information of the current coding unit. If the upper adjacent reconstructed samples and the left adjacent reconstructed reference samples of the current coding unit are both available, the first, second, and third rounds of encoding attempts are performed; if only the left adjacent reconstructed samples of the current coding unit are available, the first and second rounds of encoding attempts are performed; if only the upper adjacent reconstructed samples of the current coding unit are available, the first and third rounds of encoding attempts are performed;

[0185] in,

[0186] The first round of encoding attempts to use the LIC_TL mode to select reconstructed samples for calculating the LIC model parameters. The process includes: the encoder selects the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left of the current coding unit. The number of selected samples is determined based on the smaller value of the width and height of the current coding unit. The reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left of the corresponding coding unit in the reference frame (i.e., the reference coding unit) are also selected. When the current coding unit is not at the boundary, the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left of the current coding unit and the reference coding unit can be selected. When the current coding unit is at the boundary, only the reconstructed samples adjacent to the upper side or the reconstructed samples adjacent to the left of the current coding unit and the reference coding unit can be selected.

[0187] After selecting the reconstructed samples, the obtained reconstructed samples are modeled using the linear model calculation method described above to calculate the scaling factor a and offset parameter b. The scaling factor a and offset parameter b are then used to perform a linear transformation on the motion-compensated prediction block to obtain the prediction block of the current coding unit after local illumination compensation. The prediction block is subtracted from the original sample corresponding to the current coding unit to obtain the residual of the current coding unit. The rate-distortion cost is calculated through operations such as transformation and quantization, which is recorded as cost1.

[0188] The second round of encoding attempts to use the LIC_T mode to select reconstructed samples for calculating the LIC model parameters. The process includes: the encoder only selects the reconstructed samples adjacent to the upper side of the current coding unit, and also only selects the reconstructed samples adjacent to the upper side of the corresponding coding unit in the reference frame. The processing after selecting the reconstructed samples is the same as the first round, and a rate-distortion cost is calculated, denoted as cost2.

[0189] The third round of encoding attempts to use the LIC_L mode to select reconstructed samples for calculating the LIC model parameters. The encoder selects only the reconstructed samples adjacent to the left of the current coding unit, and also selects only the reconstructed samples adjacent to the left of the corresponding coding unit in the reference frame. The processing after selecting the reconstructed samples is the same as in the first round, and a rate-distortion cost is calculated, denoted as cost3.

[0190] Compare cost1, cost2, and cost3, record the minimum rate-distortion cost as costLic, and save the information of the current local illumination compensation mode, including the mode index of the LIC mode, where the mode index of the LIC_TL mode corresponding to cost1 is 0, the mode index of the LIC_T mode corresponding to cost2 is 1, and the mode index of the LIC_L mode corresponding to cost3 is 2;

[0191] Step 3: The encoder continues to traverse the prediction modes of other frames, calculates the rate-distortion cost corresponding to each prediction mode, and selects the prediction mode corresponding to the minimum rate-distortion cost as the optimal prediction mode for the current coding unit.

[0192] It is divided into the following situations:

[0193] The first method is to determine that the current coding unit uses local illumination compensation if local illumination compensation is allowed and costLic is the smallest compared to the rate-distortion cost of other prediction modes. The coding unit is then determined to use local illumination compensation, and the coding unit-level LIC usage flag of the current coding unit is set to true and written into the bitstream.

[0194] If the current coding unit is at a non-boundary position, and the mode index of the LIC mode corresponding to costLic is greater than 0 (the mode index of LIC_T mode and LIC_L mode is greater than 0), the LIC extension flag position is written into the bitstream, and the mode index is coded into the bitstream, and equal probability coding can be used; if the mode index of the LIC mode corresponding to costLic is equal to 0 (the mode index of LIC_TL mode is equal to 0), the LIC extension flag position is written into the bitstream;

[0195] If the current coding unit is at the boundary, if the mode index of the LIC mode corresponding to costLic is greater than 0, the LIC extension flag position is written into the bitstream. If the mode index of the LIC mode corresponding to costLic is equal to 0, the LIC extension flag position is written into the bitstream. In this case, the mode index does not need to be encoded.

[0196] In the second case, if the current coding unit allows the use of local illumination compensation and costLic is not the minimum compared to the rate-distortion cost of other prediction modes, the current coding unit does not use local illumination compensation, and the LIC usage flag of the coding unit level of the current coding unit is set to false and written into the bitstream;

[0197] In the third case, if the current coding unit does not allow the use of illumination compensation technology, the optimal prediction mode information of other non-illumination compensation technologies is written into the bitstream, which is not described in detail here.

[0198] In this step, if the mode index of the LIC mode corresponding to costLic is greater than 0, costLic may correspond to the LIC_T mode with a mode index of 1, or it may correspond to the LIC_L mode with a mode index of 2. When encoding the mode index into the bitstream, the mode index may be decremented by 1. In this way, when the mode index is 1, the encoded mode index is 0, and when the mode index is 2, the encoded mode index is 1. When the decoder parses the encoded mode index as 0, it determines to use the LIC_T mode. When the decoder parses the encoded mode index as 1, it determines to use the LIC_L mode.

[0199] In the fourth step, after the encoder traverses all coding units, it outputs the bit stream after processing such as entropy coding.

[0200] Herein, the LIC_TL mode may also be referred to as the upper and left local illumination compensation mode, the LIC_T mode may also be referred to as the upper-only local illumination compensation mode, and the LIC_T mode may also be referred to as the left-only local illumination compensation mode.

[0201] The video decoding method of this embodiment includes:

[0202] In step 1, the decoder parses the coding unit type flag. If the mode is inter, it retrieves the LIC usage flag, a sequence-level flag that indicates the current decoder allows the use of LIC. If the LIC usage flag is true, the frame-level LIC usage flag must be parsed when decoding each frame.

[0203] If both the sequence-level LIC usage flag and the frame-level LIC usage flag are true, continue parsing the LIC usage flag of the current coding unit (coding unit-level flag):

[0204] If the LIC usage flag lic_flag of the current coding unit is true, continue to parse the LIC extension flag lic_ext:

[0205] If lic_ext is true and the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current coding unit are available, continue to parse the mode index lic_idx (encoded mode index) of the LIC mode. If lic_idx is 1, it is determined that the current coding unit uses the LIC_L mode. If lic_idx is 0, it is determined that the current coding unit uses the LIC_T mode.

[0206] If lic_ext is true and only the reconstructed samples adjacent to the upper side or only the reconstructed samples adjacent to the left side are available for the current coding unit, there is no need to parse the mode index. When only the reconstructed samples adjacent to the upper side of the current coding unit are available, it is determined that the current coding unit uses the LIC_L mode; when only the reconstructed samples adjacent to the left side of the current coding unit are available, it is determined that the current coding unit uses the LIC_T mode.

[0207] If lic_ext is no, it is determined that the current coding unit uses the LIC_TL mode.

[0208] Then proceed to step 2;

[0209] If the LIC usage flag lic_flag of the current coding unit is no, go to step 3:

[0210] If one of the sequence-level LIC usage flag and the frame-level LIC usage flag is false, the current decoding process does not need to parse the LIC usage flag of the current coding unit. The coding unit-level LIC usage flag defaults to false, and step three is executed.

[0211] In step 2, the encoder determines the LIC mode to use based on the index parsed in the previous step. Based on the determined LIC mode, it selects reconstructed samples adjacent to the current coding unit. The number of selected reconstructed samples depends on the width or height of the current coding unit. It also selects reconstructed samples adjacent to the corresponding coding unit in the reference frame. The selected reconstructed samples are modeled using a linear model calculation method, and the scaling factor a and offset parameter b are calculated. A linear transformation is then performed on the motion-compensated prediction block, scaling it by a factor of a and compensating it by b, to obtain the prediction block for the current coding unit after local illumination compensation.

[0212] The third step is to continue parsing the information such as the usage flags or indexes of other inter-frame prediction modes, and obtain the final prediction block of the current coding unit based on the parsed information;

[0213] The fourth step is to parse the bitstream and obtain the residual information. The time domain residual information (also called decoding residual or reconstruction residual) is obtained by inverse quantization and inverse transformation. The final prediction block is superimposed with the time domain residual information to obtain the reconstructed sample block of the current block, which is also called the reconstructed block.

[0214] In the fifth step, all reconstructed sample blocks are processed through loop filtering and other technologies to obtain a reconstructed image. The reconstructed image can be used as a video output or as a decoding reference for subsequent images.

[0215] In this embodiment or other embodiments, when the flag bit or identification bit related to the LIC is 1, it indicates that the flag bit is true, and when it is 0, it indicates that the flag bit or identification bit is false, but the present disclosure is not limited thereto.

[0216] In this embodiment, LIC can set usage restrictions at the coding unit level. When the product of the width and height of the current coding unit is less than 32, local illumination compensation is not allowed.

[0217] The syntax table at the coding unit level of this embodiment is as follows:

[0218]

[0219] Among them, sps_lic_enable_flag is the flag bit for enabling the use of LIC at the sequence level, ph_lic_enable_flag is the flag bit for using LIC at the frame level, lic_flag is the flag bit for using LIC at the current coding unit, lic_ext is the flag bit for extending the LIC of the current coding unit, and lic_index is the mode index after encoding the current coding unit. lic_ext+lic_index can restore the mode index of the LIC mode used before encoding the current coding unit.

[0220] JVET has established a group to research beyond the H.266 / VVC coding model and named this model, the platform testing software, the Enhanced Compression Model (ECM). Based on VTM10.0, ECM incorporates newer and more efficient compression algorithms and integrates numerous intra-frame and inter-frame prediction technologies. After integrating this embodiment into the latest ECM4.0, tests were conducted under random access (RA) conditions. Taking a class D sequence as an example, the test results are as follows:

[0221]

[0222]

[0223] Based on test data for class D sequences, this embodiment demonstrates improvements in chrominance components compared to the second embodiment. From the encoder's perspective, skipping encoding attempts at boundaries effectively reduces encoding time by an average of 2% to 3%. Furthermore, this embodiment removes redundant information from the bitstream.

[0224] Another embodiment of the present disclosure provides a video encoding method and a video decoding method, which also add illumination compensation mode boundary detection at the coding unit level at both the encoding and decoding ends. When the current block is at any boundary, the encoding end only allows the original illumination compensation mode, namely the LIC_TL mode, and does not allow attempts at the LIC_L mode and the LIC_T mode. When the decoding end parses the syntax elements related to local illumination compensation of the current block, it determines the boundary of the current block. If the current block is at any boundary, it only needs to parse the first syntax element, namely the LIC use flag. If the LIC use flag is true, it can be determined that the LIC_TL mode is used. If the LIC use flag is false, it indicates that LIC is not used when performing inter-frame prediction on the current block. The current block in this embodiment also takes the current coding unit as an example.

[0225] The video encoding method of this embodiment includes:

[0226] In step 1, the encoder traverses the prediction mode. If the current coding unit is an inter-frame mode that can use LIC, the LIC usage permission flag and the frame-level LIC usage identification bit are obtained. Among them, the LIC usage permission flag is a sequence-level flag, indicating that the current encoder allows the use of local illumination compensation.

[0227] If the LIC usage flag at the sequence level is true and the LIC usage flag at the frame level is true, the encoder attempts to use the LIC prediction method and executes step 2;

[0228] If at least one of the sequence-level LIC usage flag and the frame-level LIC usage flag is false, the encoder does not attempt to use the LIC prediction method, skips step 2, and proceeds directly to step 3.

[0229] In step 2, the encoder obtains the reconstructed sample information adjacent to the current coding unit. If the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current coding unit are both available, the first, second, and third rounds of encoding attempts are performed; if only the reconstructed samples adjacent to the left side or only the reconstructed samples adjacent to the upper side of the current coding unit are available, only the first round of encoding attempts is performed.

[0230] The first round of encoding attempts to use the LIC_TL mode to select reconstructed samples for calculating the LIC model parameters. The process includes: the encoder selects the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left of the current coding unit. The number of selected samples is determined based on the smaller value of the width and height of the current coding unit. The reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left of the corresponding coding unit in the reference frame (i.e., the reference coding unit) are also selected. When the current coding unit is not at the boundary, the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left of the current coding unit and the reference coding unit can be selected. When the current coding unit is at the boundary, only the reconstructed samples adjacent to the upper side or the reconstructed samples adjacent to the left of the current coding unit and the reference coding unit can be selected.

[0231] After selecting the reconstructed samples, the obtained reconstructed samples are modeled using the linear model calculation method described above to calculate the scaling factor a and offset parameter b. The scaling factor a and offset parameter b are then used to perform a linear transformation on the motion-compensated prediction block to obtain the prediction block of the current coding unit after local illumination compensation. The prediction block is subtracted from the original sample corresponding to the current coding unit to obtain the residual of the current coding unit. The rate-distortion cost is calculated through operations such as transformation and quantization, which is recorded as cost1.

[0232] The second round of encoding attempts to use the LIC_T mode to select reconstructed samples for calculating the LIC model parameters. The process includes: the encoder only selects the reconstructed samples adjacent to the upper side of the current coding unit, and also only selects the reconstructed samples adjacent to the upper side of the corresponding coding unit in the reference frame. The processing after selecting the reconstructed samples is the same as the first round, and a rate-distortion cost is calculated, denoted as cost2.

[0233] The third round of encoding attempts to use the LIC_L mode to select reconstructed samples for calculating the LIC model parameters. The encoder selects only the reconstructed samples adjacent to the left of the current coding unit, and also selects only the reconstructed samples adjacent to the left of the corresponding coding unit in the reference frame. The processing after selecting the reconstructed samples is the same as in the first round, and a rate-distortion cost is calculated, denoted as cost3.

[0234] Compare cost1, cost2, and cost3, record the minimum rate-distortion cost as costLic, and save the information of the current local illumination compensation mode, including the mode index of the LIC mode, where the mode index of the LIC_TL mode corresponding to cost1 is 0, the mode index of the LIC_T mode corresponding to cost2 is 1, and the mode index of the LIC_L mode corresponding to cost3 is 2;

[0235] Step 3: The encoder continues to traverse the prediction modes of other frames, calculates the rate-distortion cost corresponding to each prediction mode, and selects the prediction mode corresponding to the minimum rate-distortion cost as the optimal prediction mode for the current coding unit.

[0236] It is divided into the following situations:

[0237] The first method is to use local illumination compensation for the current coding unit if it is allowed and the costLic is the smallest compared to the rate-distortion cost of other prediction modes. Then the current coding unit is determined to use local illumination compensation and the coding unit-level LIC usage flag of the current coding unit is set to true and written into the bitstream.

[0238] If the current coding unit is not at a boundary, if the mode index of the LIC mode corresponding to costLic is greater than 0, the LIC extension flag position is written into the bitstream, and the mode index is encoded (such as equal probability coding) into the bitstream; if the mode index of the LIC mode corresponding to costLic is equal to 0, the LIC extension flag position is written into the bitstream.

[0239] If the current coding unit is at the boundary, the LIC_TL mode is used by default and no other syntax elements need to be written into the bitstream.

[0240] The second type is that if the current coding unit allows the use of illumination compensation technology and costLic is not the minimum compared with the rate-distortion cost of other prediction modes, the current coding unit does not use local illumination compensation, and the coding unit level usage flag position of the current coding unit is written into the bitstream;

[0241] The third method is to write the optimal prediction mode information without using local illumination compensation into the bitstream if the current coding unit does not allow local illumination compensation. This will not be described in detail here.

[0242] Step 4: After the encoder traverses all coding units, it outputs the bit stream after using technologies such as entropy coding.

[0243] The video encoding method of this embodiment includes:

[0244] Step 1: The decoder parses the type flag at the coding unit level. If it is inter-frame mode, it obtains the LIC usage permission flag, which is a sequence-level flag and indicates that the current decoder allows the use of LIC. If the LIC usage permission flag is true, the frame-level LIC usage flag needs to be parsed when decoding each frame:

[0245] If both the sequence-level LIC usage flag and the frame-level LIC usage flag are true, continue parsing the LIC usage flag of the current coding unit:

[0246] If the LIC usage flag lic_flag of the current coding unit is true and the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current coding unit are available, continue to parse the LIC extension flag lic_ext of the current coding unit:

[0247] If lic_ext is true, continue to parse the mode index lic_idx (encoded mode index) of the LIC mode. If lic_idx is 1, it is determined that the current coding unit uses the LIC_L mode. If lic_idx is 0, it is determined that the current coding unit uses the LIC_T mode.

[0248] If lic_ext is no, it is determined that the current coding unit uses the LIC_TL mode.

[0249] If the lic_flag of the current coding unit is true and only the reconstructed samples of the upper adjacent side or only the reconstructed samples of the left adjacent side of the current coding unit are available, it is determined that the current coding unit uses the LIC_TL mode;

[0250] Then proceed to step 2;

[0251] If the LIC usage flag of the current coding unit is no, go to step 3:

[0252] If at least one of the sequence-level LIC usage flag and the frame-level LIC usage flag is false, the current decoding process does not need to parse the coding unit-level LIC usage flag. The coding unit-level LIC usage flag defaults to false, and step 3 is executed.

[0253] In step 2, the encoder determines the LIC mode to use based on the index parsed in the previous step. Based on the determined LIC mode, it selects reconstructed reference samples adjacent to the current coding unit. The number of selected reconstructed samples depends on the width or height of the current coding unit. Reconstructed samples adjacent to the corresponding coding unit in the reference frame are also selected. A linear model is used to model the selected reconstructed samples, and the scaling factor a and offset parameter b are calculated. A linear transformation is then performed on the motion-compensated prediction block, scaling it by a factor of a and offsetting it by b to obtain the prediction block for the current coding unit after local illumination compensation.

[0254] Step 3: Continue parsing information such as usage identification bits or indexes of other inter-frame prediction technologies, and obtain the final prediction block of the current coding unit based on the parsed information;

[0255] Step 4: parse the bitstream and obtain the residual information, and obtain the time domain residual information through inverse quantization and inverse transformation. The final prediction block is superimposed with the time domain residual information to obtain the reconstructed sample block of the current coding unit;

[0256] In step five, all reconstructed sample blocks are subjected to loop filtering and other techniques to obtain a reconstructed image. The reconstructed image can be output as a video or used as a reference for subsequent decoding.

[0257] In this embodiment, LIC can set usage restrictions at the coding unit level. When the product of the width and height of the current coding unit is less than 32, local illumination compensation is not allowed.

[0258] The syntax table at the coding unit level of this embodiment is as follows:

[0259]

[0260]

[0261] Among them, sps_lic_enable_flag is the flag bit for enabling the use of LIC at the sequence level, ph_lic_enable_flag is the flag bit for using LIC at the frame level, lic_flag is the flag bit for using LIC at the current coding unit, lic_ext is the flag bit for extending the LIC of the current coding unit, and lic_index is the mode index after encoding the current coding unit. lic_ext+lic_index can restore the mode index of the LIC mode used before encoding the current coding unit.

[0262] An embodiment of the present disclosure further provides a local illumination compensation device, as shown in FIG10 , comprising a processor 71 and a memory 73 storing a computer program, wherein the processor 71 can implement the local illumination compensation method as described in any embodiment of the present disclosure when executing the computer program.

[0263] An embodiment of the present disclosure further provides a video decoding device, see FIG10 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0264] An embodiment of the present disclosure further provides a video encoding device, see FIG10 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0265] The processors of the above embodiments of the present disclosure may be general-purpose processors, including central processing units (CPUs), network processors (NPs), microprocessors, etc., or other conventional processors; the processors may also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components; or combinations of the above devices. That is, the processors of the above embodiments may be any processing device or combination of devices that implement the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. If the embodiments of the present disclosure are partially implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and one or more processors may be used to execute the instructions in hardware to implement the methods of the embodiments of the present disclosure.

[0266] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0267] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the local illumination compensation method as described in any embodiment of the present disclosure, or implements the video decoding method as described in any embodiment of the present disclosure, or implements the video encoding method as described in any embodiment of the present disclosure.

[0268] An embodiment of the present disclosure further provides a code stream, wherein the code stream is generated by the video encoding method as described in any embodiment of the present disclosure.

[0269] In one or more exemplary embodiments above, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that facilitates the transfer of a computer program from one place to another, such as according to a communication protocol. In this way, a computer-readable medium may generally correspond to a non-transitory tangible computer-readable storage medium or a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this disclosure. A computer program product may include a computer-readable medium.

[0270] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may also be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient (transient) media, but rather refer to non-transient tangible storage media. As used herein, disk and optical disk include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, or Blu-ray disc, among others, where disks typically reproduce data magnetically, while optical discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0271] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0272] The technical solutions of the embodiments of the present disclosure can be implemented in a wide variety of devices or equipment, including wireless mobile phones, integrated circuits (ICs), or a group of ICs (e.g., chipsets). Various components, modules, or units are described in the embodiments of the present disclosure to emphasize the functional aspects of the devices configured to perform the described techniques, but they do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in a codec hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

Claims

1. A local illumination compensation method, applied to a decoding end, comprising: Analyze the local illumination compensation LIC usage flag of the current block; When it is determined according to the LIC use flag bit that the current block uses LIC, the LIC mode used by the current block is determined according to the availability of reconstructed samples adjacent to the current block; When inter prediction is performed on the current block, LIC is used according to the determined LIC mode.

2. The method of claim 1, wherein: The determining, based on the availability of adjacent reconstructed samples of the current block, the LIC mode used by the current block includes: Continue parsing the LIC extension flag of the current block; When the LIC extension flag is true and only the reconstructed samples adjacent to the upper side of the current block are available, determining that the current block uses the LIC_L mode, in which only the reconstructed samples adjacent to the left side of the current block and the reference block are selected to calculate the LIC model parameters; When the LIC extension flag is true and only the left adjacent reconstructed samples of the current block are available, the current block is determined to use the LIC_T mode. In the LIC_T mode, only the upper adjacent reconstructed samples of the current block and the reference block are selected to calculate the LIC model parameters.

3. The method of claim 2, wherein: The determining, based on the availability of adjacent reconstructed samples of the current block, the LIC mode used by the current block further includes: When the LIC extension flag is false, determining that the current block uses the LIC_TL mode, and calculating the LIC model parameters according to the reconstructed samples adjacent to the upper side and the left side of the current block and the reference block in the LIC_TL mode; When the LIC extension flag is true and the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current block are both available, the LIC mode index of the current block is further parsed, and it is determined whether the current block uses the LIC_T mode or the LIC_L mode according to the LIC mode index.

4. The method of claim 1, wherein: The determining, based on the availability of adjacent reconstructed samples of the current block, the LIC mode used by the current block includes: When both the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current block are available, continue parsing the LIC extension flag of the current block; When the LIC extension flag is negative, determining that the current block uses the LIC_TL mode; When the LIC extension flag is true, the LIC mode index of the current block is continuously parsed, and it is determined whether the current block uses the LIC_T mode or the LIC_L mode according to the LIC mode index.

5. The method of claim 4, wherein: The determining, based on the availability of adjacent reconstructed samples of the current block, the LIC mode used by the current block further includes: When both the upper adjacent reconstructed samples and the left adjacent reconstructed samples of the current block are available, the LIC extension flag does not need to be parsed and the LIC extension flag defaults to no, and the current block is determined to use the LIC_TL mode.

6. The method of claim 1, wherein: The using the LIC according to the determined LIC mode includes: When the current block uses the LIC_T mode, only the reconstructed samples adjacent to the upper side of the current block and the reference block are selected to calculate the LIC model parameters, and the prediction block of the current block is linearly transformed according to the LIC model parameters to obtain the prediction block after LIC; When the current block uses the LIC_L mode, only the reconstructed samples adjacent to the left of the current block and the reference block are selected to calculate the LIC model parameters, and the prediction block of the current block is linearly transformed according to the LIC model parameters to obtain the prediction block after LIC; When the current block uses the LIC_TL mode, LIC model parameters are calculated based on the reconstructed samples adjacent to the upper side and the left side of the current block and the reference block, and the prediction block of the current block is linearly transformed according to the LIC model parameters to obtain the prediction block after LIC.

7. The method of claim 6, wherein: The calculating of LIC model parameters according to the reconstructed samples adjacent to the upper side and the left side of the current block and the reference block includes: When only the upper adjacent reconstructed samples of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined according to the width of the current block; In the case that only the reconstructed samples adjacent to the left of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined according to the height of the current block.

8. A video decoding method, comprising: When determining that the current block uses the first inter-frame mode, obtaining sequence-level and frame-level syntax elements related to the use of local illumination compensation (LIC); When it is determined according to the sequence-level and frame-level syntax elements that LIC is allowed to be used when performing inter-frame prediction on the current block, the LIC is used when performing inter-frame prediction on the current block according to the method according to any one of claims 1 to 7.

9. The method of claim 8, wherein: The first inter-frame mode includes at least one of the following: Merge mode in unidirectional reference mode, but not used simultaneously with intra-frame and inter-frame joint prediction modes, geometric partitioning mode, and affine mode in merge mode in unidirectional reference mode; Merge mode in bidirectional reference mode, but not used simultaneously with bidirectional optical flow mode, decoding-side motion vector derivation mode, and bidirectional prediction weighting mode in merge mode in bidirectional reference mode.

10. The method of claim 9, wherein: The method further includes: in a process of constructing a motion vector prediction list in merge mode, the LIC usage flag of the current block and the mode index of the LIC mode are inherited from a neighboring block or a corresponding prediction block; or, the LIC usage flag of the current block is inherited from the neighboring block or the corresponding prediction block, and the mode index of the LIC_TL mode is used as a default mode index.

11. A local illumination compensation method, applied to an encoding end, comprising: When LIC is allowed to be used when performing inter-frame prediction on the current block, determining the LIC mode to be used when attempting to encode the current block based on the availability of reconstructed samples adjacent to the current block; An encoding attempt is performed on the current block using the determined LIC mode and a rate-distortion cost is calculated. Whether to use the LIC when performing inter-frame prediction on the current block is determined according to the rate-distortion cost.

12. The method of claim 11, wherein: The determining, based on availability of reconstructed samples adjacent to the current block, a LIC mode to be used when attempting to encode the current block includes: When both the upper and left adjacent reconstructed samples of the current block are available, encoding attempts are made using the LIC_TL mode, the LIC_T mode, and the LIC_L mode.

13. The method according to claim 11 or 12, wherein: The determining, based on availability of reconstructed samples adjacent to the current block, a LIC mode to be used when attempting to encode the current block includes: When only the reconstructed samples adjacent to the upper side of the current block are available, the encoding attempt is made using the LIC_TL mode and the LIC_L mode. In the LIC_TL mode, the LIC model parameters are calculated based on the reconstructed samples adjacent to the upper side and the left side of the current block and the reference block. In the LIC_L mode, only the reconstructed samples adjacent to the left side of the current block and the reference block are selected to calculate the LIC model parameters. When only the left adjacent reconstructed samples of the current block are available, encoding attempts are made using the LIC_TL mode and the LIC_T mode. In the LIC_T mode, only the upper adjacent reconstructed samples of the current block and the reference block are selected to calculate the LIC model parameters.

14. The method of claim 13, wherein: After determining whether to use LIC when performing inter-frame prediction on the current block according to the rate-distortion cost, the method further includes: If LIC is used, the LIC use flag of the current block is set to true and written into the code stream; When the LIC mode with the lowest rate-distortion cost is LIC_TL mode during the encoding attempt, the LIC extension flag of the current block is set to false and written into the bitstream; When the LIC mode with the lowest rate-distortion cost during the coding attempt is the LIC_T mode or the LIC_L mode, the LIC extension flag of the current block is set to true and written into the bitstream.

15. The method of claim 14, wherein: When the LIC mode with the lowest rate-distortion penalty during the encoding attempt is the LIC_T mode or the LIC_L mode, the method further includes: In the case where only the reconstructed samples of the upper adjacent block or only the reconstructed samples of the left adjacent block are available for the current block, the mode index does not need to be encoded; When both the upper adjacent reconstructed samples and the left adjacent reconstructed samples of the current block are available, the mode index of the LIC mode with the minimum rate-distortion cost is encoded and written into the bitstream.

16. The method of claim 11 or 12, wherein: The determining, based on availability of reconstructed samples adjacent to the current block, a LIC mode to be used when attempting to encode the current block includes: In a case where only the upper adjacent reconstructed samples or only the left adjacent reconstructed samples of the current block are available, it is determined that the current block uses the LIC_TL mode for encoding attempt.

17. The method of claim 16, wherein: After determining whether to use LIC when performing inter-frame prediction on the current block according to the rate-distortion cost, the method further includes: If LIC is used, the LIC use flag of the current block is set to true and written into the code stream; In the case that only the upper adjacent reconstructed samples or only the left adjacent reconstructed samples of the current block are available, there is no need to encode the LIC extension flag of the current block and the mode index of the LIC mode.

18. The method of claim 17, wherein: In the case of determining to use LIC, the method further includes: When both the reconstructed samples adjacent to the upper side and the reconstructed samples adjacent to the left side of the current block are available, if the LIC mode with the lowest rate-distortion cost during the encoding attempt is the LIC_TL mode, the LIC extension flag of the current block is set to false and written into the bitstream; if the LIC mode with the lowest rate-distortion cost is the LIC_T mode or the LIC_L mode, the LIC extension flag of the current block is set to true and written into the bitstream, and the mode index of the LIC mode with the lowest rate-distortion cost is encoded and written into the bitstream.

19. The method of claim 11, wherein: The using the determined LIC mode to attempt encoding of the current block includes: When attempting to encode the current block using the LIC_TL mode, if only the reconstructed samples adjacent to the upper side of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined according to the width of the current block; if only the reconstructed samples adjacent to the left side of the current block are available, the number of reconstructed samples selected for calculating the LIC model parameters is determined according to the height of the current block.

20. A video encoding method, comprising: When the current block uses the first inter-frame mode, obtaining sequence-level and frame-level syntax elements related to local illumination compensation (LIC); When it is determined according to the sequence-level and frame-level syntax elements that LIC is allowed to be used when performing inter-frame prediction on the current block, LIC is used when performing inter-frame prediction on the current block according to the method according to any one of claims 11 to 19.

21. The method of claim 20, wherein: The first inter-frame mode includes at least one of the following: Merge mode in unidirectional reference mode, but not used simultaneously with intra-frame and inter-frame joint prediction modes, geometric partitioning mode, and affine mode in merge mode in unidirectional reference mode; Merge mode in bidirectional reference mode, but not used simultaneously with bidirectional optical flow mode, decoding-side motion vector derivation mode, and bidirectional prediction weighting mode in merge mode in bidirectional reference mode.

22. The method of claim 21, wherein: The method further includes: in a process of constructing a motion vector prediction list in merge mode, the LIC usage flag of the current block and the mode index of the LIC mode are inherited from a neighboring block or a corresponding prediction block; or, the LIC usage flag of the current block is inherited from the neighboring block or the corresponding prediction block, and the mode index of the LIC_TL mode is used as a default mode index.

23. A local illumination compensation device, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it can implement the local illumination compensation method as described in any one of claims 1 to 7 and 11 to 19.

24. A video decoding device comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it is capable of implementing the video decoding method according to any one of claims 8 to 10.

25. A video encoding device comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it can implement the video encoding method according to any one of claims 20 to 22.

26. A video encoding and decoding system, wherein: The method comprises the video encoding device according to claim 25 and the video decoding device according to claim 24.

27. A non-transitory computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, it can implement the local illumination compensation method as described in any one of claims 1 to 7, 11 to 19, or implement the video decoding method as described in any one of claims 8 to 10, or implement the video encoding method as described in any one of claims 20 to 22.

28. A code stream, wherein The code stream is generated by the video encoding method according to any one of claims 20 to 22.

Images