LIGHT COMPENSATION METHODS, ENCODERS, DECODERS, AND METHODS FOR BIT STREAM TRANSMISSION

Abstract

The invention relates to a method for illumination compensation, an encoder, and a decoder. A bitstream is received, and the bitstream is parsed to receive a flag enabling illumination compensation, current motion information, and current forecast mode. The flag enabling the illumination compensation frame level is in the received bitstream, when the flag enabling illumination compensation is valid. The flag using illumination compensation is in the received bitstream, when the flag enabling the illumination compensation frame level is valid. The index information of the target illumination compensation mode is in the received bitstream, when the flag using illumination compensation is valid. Illumination compensation is performed on the current block based on the index information of the target illumination compensation mode.

Description

Illumination compensation method, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to video coding technology, and are related to but not limited to an illumination compensation method, an encoder, a decoder, and a storage medium. Background Art

[0002] In video encoding and decoding, in addition to intra-frame prediction, inter-frame prediction can also be used to encode and decode the current block. Inter-frame prediction can perform an inter-frame prediction process on each coding unit to calculate the predicted block.

[0003] In related technologies, illumination compensation techniques are used in inter-frame prediction to eliminate variations in illumination, changes in illumination angle caused by object motion, shadow changes caused by relative motion of objects, and brightness changes introduced by post-production. This can cause similar textures but different brightness between the current block (encoded or decoded) and the reference block. In this case, illumination compensation is applied to the current block, using certain pixels from the reconstructed pixels in the column to the left and row above the current block, and pixels at corresponding positions in the reference block's column to the left and row above, to determine an illumination compensation factor for the current block and perform illumination compensation.

[0004] However, for the current block in ordinary natural videos, the use of illumination compensation technology will increase the bit rate of the code stream due to the identification bit overhead at the coding unit level, resulting in a large encoding and decoding bit overhead for sequences that do not require illumination compensation for specific illumination videos or images, affecting the final encoding and decoding efficiency.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide an illumination compensation, an encoder, a decoder, and a storage medium, which can add a frame-level illumination compensation frame-level permission flag when selecting illumination compensation for frame-level images based on different situations to indicate whether illumination compensation technology is used, so as to save the bit overhead of encoding and decoding and improve the performance of encoding and decoding.

[0007] In a first aspect, an embodiment of the present application provides a method for illumination compensation, applied to a decoder, the method comprising:

[0008] Obtaining a code stream, parsing the code stream, and obtaining a light compensation permission flag;

[0009] If the illumination compensation permission flag is valid, obtaining the illumination compensation frame-level permission flag in the code stream;

[0010] If the illumination compensation frame level permission flag is valid, obtaining an illumination compensation use flag in the bitstream;

[0011] If the illumination compensation usage flag is valid, obtaining index information of the target illumination compensation mode in the bitstream;

[0012] Perform illumination compensation on the current block based on the index information of the target illumination compensation mode.

[0013] In a second aspect, an embodiment of the present application further provides a method for illumination compensation, which is applied to an encoder, and the method includes:

[0014] Determine the illumination compensation frame level permission flag based on the brightness information of the current frame;

[0015] If the illumination compensation frame level permission flag is valid, the illumination compensation function is enabled for the current frame, illumination compensation is allowed for the current block of the current frame, and an illumination prediction value is obtained;

[0016] The illumination compensation frame level permission flag is written into the bitstream.

[0017] In a third aspect, an embodiment of the present application provides a decoder, including:

[0018] The parsing part is configured to obtain a code stream, parse the code stream, and obtain a light compensation permission flag;

[0019] a first acquisition section configured to, if the illumination compensation enable flag is valid, acquire an illumination compensation frame-level enable flag in the codestream; if the illumination compensation frame-level enable flag is valid, acquire an illumination compensation usage flag in the codestream; and if the illumination compensation usage flag is valid, acquire index information of a target illumination compensation mode in the codestream;

[0020] The first illumination compensation part is configured to perform illumination compensation on the current block based on the index information of the target illumination compensation mode.

[0021] In a fourth aspect, an embodiment of the present application provides an encoder, including:

[0022] The second determining part is configured to determine the illumination compensation frame level permission flag based on the brightness information of the current frame;

[0023] The second prediction part is configured to enable the illumination compensation function of the current frame if the illumination compensation frame level permission flag is valid, perform illumination compensation on the current block of the current frame, and obtain an illumination prediction value;

[0024] The writing part is configured to write the illumination compensation frame level permission flag into the code stream.

[0025] In a fifth aspect, an embodiment of the present application further provides a decoder, including:

[0026] a first memory and a first processor;

[0027] The first memory stores a computer program that can be run on the first processor, and the first processor implements the illumination compensation method of the decoder when executing the program.

[0028] In a sixth aspect, an embodiment of the present application further provides an encoder, including:

[0029] a second memory and a second processor;

[0030] The second memory stores a computer program that can be run on the second processor, and the illumination compensation method of the encoder is performed when the second processor executes the program.

[0031] An embodiment of the present application provides a storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed by a first processor, it implements the illumination compensation method of the claim decoder; or when the computer program is executed by a second processor, it implements the illumination compensation method of the claim encoder.

[0032] The present invention provides an illumination compensation method, encoder, decoder, and storage medium. The method comprises obtaining a bitstream and parsing the bitstream to obtain an illumination compensation permission flag, current motion information, and current prediction mode. If the illumination compensation permission flag is valid, an illumination compensation frame-level permission flag is obtained from the bitstream. If the illumination compensation frame-level permission flag is valid, an illumination compensation usage flag is obtained from the bitstream. If the illumination compensation usage flag is valid, an index of a target illumination compensation mode is obtained from the bitstream. Based on the index of the target illumination compensation mode, illumination compensation is performed on the current block. The above-mentioned implementation method allows the decoder to directly obtain the frame-level illumination compensation frame-level permission flag from the bitstream when illumination compensation technology is used during decoding of the current block. This allows the encoder to first determine at the frame level whether to continue parsing the block-level illumination compensation usage flag. Only when the illumination compensation frame-level permission flag is valid can the encoder obtain the next-level flag in the bitstream and then perform subsequent decoding to perform prediction (e.g., inter-frame prediction) on the current block. Therefore, when the illumination compensation frame-level permission flag is invalid, the transmission bits of the code stream will be greatly reduced. In this way, when selecting illumination compensation for frame-level images based on different situations, adding the frame-level illumination compensation frame-level permission flag to indicate whether illumination compensation technology is used can save the bit overhead of encoding and decoding and improve the performance of encoding and decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] 1A-1C are exemplary component distribution diagrams in different color formats provided by an embodiment of the present application;

[0034] FIG2 is a schematic diagram of an exemplary division of coding units provided in an embodiment of the present application;

[0035] FIG3 is a schematic diagram of the composition structure of an exemplary video encoding and decoding network architecture provided in an embodiment of the present application;

[0036] FIG4 is a structural diagram of an exemplary video encoding system provided in an embodiment of the present application;

[0037] FIG5 is a structural diagram of an exemplary video decoding system provided in an embodiment of the present application;

[0038] FIG6 is a schematic diagram of exemplary illumination compensation for a current block and a reference block provided by an embodiment of the present application;

[0039] FIG7A is a first schematic diagram of an exemplary arrangement of a current block and a reference block provided in an embodiment of the present application;

[0040] FIG7B is a second schematic diagram of an exemplary arrangement of a current block and a reference block provided in an embodiment of the present application;

[0041] FIG8 is a flow chart of a light compensation method provided in an embodiment of the present application;

[0042] FIG9 is a third schematic diagram of an exemplary arrangement of a current block and a reference block provided in an embodiment of the present application;

[0043] FIG10 is a fourth schematic diagram of an exemplary arrangement of a current block and a reference block provided in an embodiment of the present application;

[0044] FIG11 is a fifth exemplary arrangement diagram of a current block and a reference block provided in an embodiment of the present application;

[0045] FIG12 is a flow chart of a light compensation method provided in an embodiment of the present application;

[0046] FIG13 is a structural diagram of a decoder provided in an embodiment of the present application;

[0047] FIG14 is a second structural diagram of a decoder provided in an embodiment of the present application;

[0048] FIG15 is a structural diagram 1 of an encoder provided in an embodiment of the present application;

[0049] FIG16 is a second structural diagram of an encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the related applications and are not intended to limit the applications. It should also be noted that for ease of description, only the parts relevant to the related applications are shown in the drawings.

[0051] In a video image, a first image component, a second image component, and a third image component are generally used to represent a current block (Coding Block, CB); wherein the three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. Specifically, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V; thus, the video image can be represented in either the YCbCr format or the YUV format.

[0052] Typically, digital video compression technology operates on image data encoded in the YCbCr (YUV) color encoding format. The YUV ratio is typically 4:2:0, 4:2:2, or 4:4:4. Y represents luminance (Luma), Cb (U) represents blue chrominance, and Cr (V) represents red chrominance. U and V represent chroma, used to describe color and saturation. Figures 1A through 1C illustrate the distribution of components in different color formats, with white representing the Y component and dark gray representing the UV components. As shown in Figure 1A, a 4:2:0 color format indicates four luminance components and two chrominance components for every four pixels (YYYYCbCr). As shown in Figure 1B, a 4:2:2 format indicates four luminance components and four chrominance components for every four pixels (YYYYCbCrCbCr). As shown in Figure 1C, a 4:4:4 format indicates a full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0053] Currently, common video codec standards all use a block-based hybrid coding framework. Each frame in a video image is divided into square Largest Coding Units (LCUs) of the same size (e.g., 128×128, 64×64, etc.). Each LCU can be further divided into rectangular Coding Units (CUs) according to rules; and Coding Units may be further divided into smaller Prediction Units (PUs). Specifically, the hybrid coding framework may include modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module may include intra-frame prediction and inter-frame prediction, and inter-frame prediction may include motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels within a frame of a video image, the use of intra-frame prediction in video coding and decoding technology can eliminate spatial redundancy between adjacent pixels. Inter-frame prediction can refer to image information from different frames and use motion estimation to search for the motion vector information that best matches the current partition block to eliminate temporal redundancy; the transformation converts the predicted image block to the frequency domain, redistributes the energy, and combines it with quantization to remove information that the human eye is not sensitive to, thereby eliminating visual redundancy; entropy coding can eliminate character redundancy based on the current context model and the probability information of the binary code stream.

[0054] It should be noted that during the video encoding process, the encoder first reads the image information and divides the image into several coding tree units (CTUs). A coding tree unit can be further divided into several coding units (CUs). These coding units can be rectangular blocks or square blocks. The specific relationship can be shown in Figure 2.

[0055] In the inter-frame prediction process, the current coding unit is predicted with reference to the information of the reference blocks of different frame images, that is, according to the current majority of left-to-right and top-to-bottom coding orders, the current coding unit can use the upper left coding unit, the upper coding unit and the left coding unit of the reference block as reference information to predict that the digital video input by the current coding unit is in color format, that is, the current mainstream digital video encoder input source is YUV 4:2:0 format, that is, every 4 pixels of the image are composed of 4 Y components and 2 UV components. The encoder will encode the Y component and UV component separately, and the encoding tools and technologies used are slightly different. At the same time, the decoding end will also decode according to different formats.

[0056] For inter-frame prediction in digital video encoding and decoding, the coding block of the current frame is predicted by referring to the reconstructed image information of the decoded frame. Motion estimation searches for image blocks in the reference frame, and based on MSE (mean square error) or other algorithms, finds the matching block with the minimum error, obtaining the motion information (motion vector, MV) corresponding to the current coding block. Motion compensation is performed on the reference block pointed to by the MV to obtain the predicted block. The residual information is calculated by comparing the predicted block with the original image block, and then transmitted to the decoder through processes such as transformation and quantization. After receiving and parsing the bitstream, the decoder performs steps such as inverse transformation and inverse quantization to obtain the residual information. The predicted image block predicted by the decoder is superimposed with the residual information to obtain the reconstructed image block.

[0057] Based on the above concepts, for example, illumination compensation technology can be applied to various prediction technologies, such as inter-frame prediction, inter-frame non-rectangular prediction, affine transformation prediction, etc., which is not limited in the embodiments of the present application.

[0058] Taking inter-frame prediction as an example, an embodiment of the present application provides a network architecture of a video encoding and decoding system including an illumination compensation method. FIG3 is a schematic diagram of the composition structure of the network architecture of the video encoding and decoding in the embodiment of the present application. As shown in FIG3 , the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During the implementation process, the electronic device can be various types of devices with video encoding and decoding functions. For example, the electronic device can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensor device, a server, etc., and the embodiment of the present application does not limit this. Among them, the illumination compensation device in the embodiment of the present application can be the above-mentioned electronic device.

[0059] Among them, the electronic device in the embodiment of the present application has a video encoding and decoding function, generally including a video encoder (ie, encoder) and a video decoder (ie, decoder).

[0060] The present application provides a video encoding system, as shown in FIG4 , the video encoding system 11 includes:

[0061] Transformation unit 111, quantization unit 112, mode selection and coding control logic unit 113, intra-frame prediction unit 114, inter-frame prediction unit 115 (including motion compensation and motion estimation), inverse quantization unit 116, inverse transformation unit 117, loop filtering unit 118, coding unit 119 and decoded image buffer unit 110; for the input original video signal, through the coding tree block (Coding Tree Unit, A video reconstructed block can be obtained by dividing the video reconstructed block into a CTU. The coding mode is determined by the mode selection and coding control logic unit 113. Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transform unit 111 and the quantization unit 112 to transform the video reconstructed block, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame prediction unit 114 is used to perform intra-frame prediction on the video reconstructed block; wherein, the intra-frame prediction unit 114 is used to determine the optimal intra-frame prediction mode (i.e., target prediction mode) of the video reconstructed block; the inter-frame prediction unit 115 is used to perform inter-frame prediction coding of the received video reconstructed block relative to one or more blocks in one or more reference frames to provide temporal prediction information; wherein, motion estimation is to generate motion direction The motion vector can estimate the motion of the video reconstructed block. Motion compensation is then performed based on the motion vector determined by motion estimation. After determining the inter-frame prediction mode, the inter-frame prediction unit 115 is further configured to provide the selected inter-frame prediction data to the encoding unit 119, and also to send the calculated motion vector data to the encoding unit 119. In addition, the inverse quantization unit 116 and the inverse transform unit 117 are configured to reconstruct the video reconstructed block, reconstructing a residual block in the pixel domain. This reconstructed residual block is passed through the loop filter unit 118 to remove blocking artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video reconstructed block. The encoding unit 119 is configured to encode various coding parameters and quantized transform coefficients. The decoded image buffer unit 110 is configured to store the reconstructed video reconstructed block for prediction reference. As video encoding progresses, new reconstructed video reconstructed blocks are continuously generated and stored in the decoded image buffer unit 110.

[0062] The embodiment of the present application provides a video decoding system. FIG5 is a schematic diagram of the composition structure of the video encoding system of the embodiment of the present application. As shown in FIG5 , the video encoding system 12 includes:

[0063] The decoding unit 121, the inverse transform unit 127, the inverse quantization unit 122, the intra-frame prediction unit 123, the motion compensation unit 124, the loop filter unit 125 and the decoded image cache unit 126 are configured; after the input video signal is encoded by the video encoding system 11, a code stream of the video signal is output; the code stream is input to the video decoding system 12, and first passes through the decoding unit 121 to obtain a decoded transform coefficient; the transform coefficient is processed by the inverse transform unit 127 and the inverse quantization unit 122 to generate a residual block in the pixel domain; the intra-frame prediction unit 123 can be used to generate prediction data of the current video decoding block based on the determined intra-frame prediction direction and data from a previously decoded block of the current frame or picture; the motion compensation unit 124 determines prediction information for a video decoding block by analyzing motion vectors and other associated syntax elements, and uses the prediction information to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform unit 127 and the inverse quantization unit 122 with the corresponding predictive block generated by the intra-frame prediction unit 123 or the motion compensation unit 124; the decoded video signal passes through the loop filter unit 125 to remove block effect artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image cache unit 126, which stores reference images for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal to obtain the restored original video signal.

[0064] In practical applications, real-world natural videos often contain changes in illumination intensity, such as a decrease in illumination over time, cloud cover, or changes in camera flash intensity. These video contents differ from the preceding and following frames primarily in the strength of the image's DC component, with little change to the texture information within the content. However, due to the large DC component, inter-frame prediction techniques like motion search and motion compensation cannot effectively predict these changes, leading to the inclusion of a significant amount of residual information. Illuminance compensation (IC) technology can effectively remove this redundant DC information, accurately predicting brightness changes and making corresponding compensation, resulting in smaller residual information and improved coding efficiency.

[0065] Related technologies include illumination compensation technologies in JEM of H.266 (Versatile Video Coding, VVC) and illumination compensation technologies in the current Audio Video Coding Standard (AVS) 3 standard.

[0066] In inter-frame prediction mode, each CU block that can use illumination compensation technology will calculate and compare the rate-distortion cost (Rdcost) with and without illumination compensation technology. The implementation of illumination compensation technology is mainly achieved by linearly compensating the prediction block through an illumination compensation linear model. The illumination compensation model mainly consists of a scaling factor a and an offset factor b. The specific formula is shown in formula (1):

[0067] Pred′(x,y)=a·Pred(x,y)+b (1)

[0068] Where Pred(x,y) is the predicted block before illumination compensation, Pred'(x,y) is the predicted block after illumination compensation, a is the scaling factor in the illumination compensation model, and b is the offset factor in the illumination compensation model. During the digital video encoding and decoding process, as shown in Figure 6, the illumination compensation model is used to correct illumination differences in the coding block of the current frame (i.e., the current block) based on the reconstructed reference block of the reference frame to obtain the compensated predicted block.

[0069] It should be noted that both a and b need to be calculated using the image information of the current frame and the image information of the reference frame. They are obtained by modeling the adjacent reconstructed pixels of the current block space and the adjacent reference pixels of the reconstructed block in the corresponding reference frame. The derivation formula is shown in formula (2):

[0070] Curr_Rec neigh =a·Ref_Rec neigh +b (2)

[0071] Among them, Curr_Rec neigh is the reconstructed pixel of the reconstructed image of the current frame, Ref_Rec neigh is the reconstructed reference pixel of the reconstructed image of the reference frame.

[0072] In the embodiment of the present application, the calculation of the scaling factor a and the offset factor b requires the use of the adjacent reconstructed reference pixels of the corresponding reconstructed reference block in the reference frame and the adjacent reconstructed pixels of the coding block in the current frame. Based on the correlation between the adjacent reconstructed pixels of the coding block of the current frame and the reconstructed reference pixels at the same corresponding pixel position in the reference frame, the block in formula (2) is modeled and solved to obtain the illumination-compensated prediction block corresponding to the current block.

[0073] For example, as shown in Figures 7A and 7B, the Reconstructed pixels are the most adjacent reconstructed pixels of the CU, the Reference picture CU is the corresponding reconstructed CU (reference block) in the reference frame, and the Current picture CU is the CU to be encoded in the current frame (current block). By modeling the corresponding Reconstructed pixels in the two frames and solving the linear relationship, the scaling factor a and the offset factor b are obtained, and then the linear relationship is applied to the Reference picture CU to obtain the predicted block of the Current picture CU. In detail, the adjacent Reconstructed pixels of the Reference picture CU are sorted, and in order to prevent some noise points from affecting the accuracy of the linear model, the maximum and minimum values ​​after sorting are removed. The average of the two maximum values ​​of the reconstructed pixels after the current sort is taken as Max1, and the average of the two minimum values ​​of the reconstructed pixels after the current sort is taken as Min1. In addition, the average of the two maximum values ​​of the reconstructed pixels corresponding to the reconstructed pixels after the current picture CU is taken as Max2, and the average of the two minimum values ​​of the reconstructed pixels corresponding to the reconstructed pixels after the current picture CU is taken as Min2. Solving the linear model, we can derive formulas (3) and (4):

[0074]

[0075] b=a×Min2-Min1 (4)

[0076] In the embodiment of the present application, Max1, Max2, Min1 and Min2 are substituted into formulas (3) and (4) to calculate a and b.

[0077] The illumination compensation technology in the embodiment of the present application acts on the inter-frame prediction part in the video coding hybrid framework, and is specifically applied to all prediction modes of inter-frame prediction, and acts on both the encoding end and the decoding end.

[0078] The implementation of the encoder using inter-frame prediction illumination compensation technology (IC) for inter-frame prediction coding mode is as follows:

[0079] The input digital video information is divided into several coding tree units at the encoding end. Each coding tree unit is divided into several rectangular or square coding units (i.e., current blocks). Each coding unit performs an inter-frame prediction process to calculate a prediction block.

[0080] In the current coding unit, if the illumination compensation enable flag is '1', all the following steps are performed; if the illumination compensation enable flag is '0', only steps a), b), and f) are performed:

[0081] a) Inter-frame prediction first traverses all candidate MVs to perform motion compensation, calculates the predicted pixels after motion compensation under each MV, and calculates the rate-distortion cost based on the original pixels and the predicted pixels;

[0082] b) According to the principle of minimizing the rate-distortion cost of all MVs mentioned above, the optimal MV and prediction mode of the current coding unit, such as (SKIP, MERGE / DIRECT or INTER), are selected, and the optimal information and the corresponding rate-distortion cost information are recorded;

[0083] c) Traverse all candidate MVs again. This process starts the illumination compensation technology. First, match the reference block (reconstructed pixels of the reference frame) according to the prediction mode and MV. Extract the left and top reconstructed pixels of the reference block and the left and top reconstructed pixels of the block to be encoded in the current frame. Sort and average the reconstructed pixels (the specific operation is as described above) and bring them into the above formula to obtain the linear model parameters a and b.

[0084] d) Perform motion compensation on the reference block to obtain a prediction block after ordinary motion compensation, and then perform illumination compensation on it. That is, linearly offset the pixels in each prediction block according to a linear model to obtain the final prediction block of the current coding unit;

[0085] e) Calculate the rate-distortion cost information of each MV based on the final predicted pixels and the original pixels after the illumination compensation technology, record the MV index of the minimum rate-distortion cost information and its corresponding prediction mode (such as skip mode SKIP, merge mode / direct export mode MERGE / DIRECT, or ordinary inter-frame mode INTER) and its corresponding rate-distortion cost;

[0086] f) If the illumination compensation technology permission flag is '0', the MV index and prediction mode recorded in b) are transmitted to the decoding end via the code stream; if the illumination compensation technology permission flag is '1', the minimum cost value recorded in b) is compared with the minimum cost value recorded in e). If the rate-distortion cost in b) is smaller, the MV index and prediction mode recorded in b) are encoded as the optimal information of the current coding unit and transmitted to the decoding end via the code stream, and the illumination compensation technology current coding unit flag position is set to invalid, indicating that the illumination compensation technology is not used, and is also transmitted to the decoding end via the code stream; if the rate-distortion cost in e) is smaller, the MV index and prediction mode recorded in e) are encoded as the optimal information of the current coding unit and transmitted to the decoding end via the code stream, and the illumination compensation technology current coding unit flag position is set to valid, indicating that the illumination compensation technology is used, and is also transmitted to the decoding end via the code stream.

[0087] The implementation of the decoder using IC for inter-frame prediction coding mode is as follows:

[0088] The decoding end obtains the code stream and parses it to obtain the digital video sequence information, and parses it to obtain the illumination compensation permission flag of the current video sequence, the encoding mode of the current decoding unit (ie, the current block) as the inter-frame prediction mode, and the illumination compensation usage flag of the current decoding unit.

[0089] In the current decoding unit, if the illumination compensation permission flag is '1', all the following steps are performed; if the illumination compensation permission flag is '0', only steps a), b), d), and f) are performed:

[0090] a) Obtaining code stream information, parsing the residual information of the current decoding unit, and obtaining the time domain residual information through inverse transformation and inverse quantization;

[0091] b) Parse the code stream and obtain the prediction mode information and MV index of the current decoding unit;

[0092] c) parse and obtain the usage identification bit of the illumination compensation technology;

[0093] d) Obtain the reference block in the reference frame according to the prediction mode and MV index of the current decoding unit, and obtain the prediction block of the current decoding unit after motion compensation;

[0094] e) If the illumination compensation technology usage flag of the current decoding unit is '1', obtain the left and top reconstructed pixels of the reference block in the reference frame, obtain the left and top reconstructed pixels of the current decoding unit, sort and average the obtained reconstructed pixels of the reference frame and the current frame (the specific operation is as described above), and obtain the scaling factor a and offset factor b of the linear model. Perform a linear offset on the prediction block of the current decoding unit to obtain the final prediction block of the current decoding unit;

[0095] f) The predicted block is superimposed on the restored residual information to obtain the reconstructed block of the current decoding unit, which is output after post-processing.

[0096] It should be noted that the illumination compensation permission flag in the embodiment of the present application is a numerical expression of the illumination compensation permission flag, and the illumination compensation usage flag is a numerical expression of the illumination compensation usage flag.

[0097] In the embodiment of the present application, when calculating the linear model, the illumination compensation technology only has a good prediction compensation effect on specific video sequences, such as images of the flash effect taken by a mobile phone or camera, images of the changing headlights of a motor vehicle passing by at night, etc. The illumination compensation technology can minimize the gap between the predicted value and the actual sample value. However, for non-illumination compensation video sequences, such as ordinary natural videos, the illumination compensation technology cannot play a good prediction compensation role. On the contrary, the code stream bit rate will increase due to the coding unit level flag overhead. Therefore, for specific illumination videos or images, the codec needs to reflect the adaptive opening or disabling of illumination compensation technology from the frame level permission flag to achieve the purpose of saving bit overhead and improving compression efficiency.

[0098] Based on this, the present application is implemented on the current block when performing inter-frame prediction. An illumination compensation method provided in an embodiment of the present application mainly acts on the inter-frame prediction unit 115 of the video encoding system 11 and the inter-frame prediction unit of the video decoding system 12, that is, the motion compensation unit 124; that is, if the video encoding system 11 can obtain a better prediction effect through the illumination compensation method provided in the embodiment of the present application, then, correspondingly, at the decoding end, the video decoding recovery quality can also be improved.

[0099] Based on this, the technical solution of this application is further described in detail below with reference to the accompanying drawings and embodiments. Before proceeding with the detailed description, it should be noted that the terms "first", "second", "third", etc. mentioned throughout the specification are only used to distinguish different features and do not have the function of defining priority, sequence, size relationship, etc.

[0100] The present embodiment provides an illumination compensation method, which is applied to a video decoding device, i.e., a decoder. The functions implemented by this method can be implemented by a first processor in the video decoding device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the video decoding device includes at least a first processor and a first storage medium. Hereinafter, the current decoding unit and the current encoding unit are both referred to as the current block.

[0101] FIG8 is a schematic diagram of a flow chart of an implementation method of an illumination compensation method according to an embodiment of the present application. As shown in FIG8 , the method includes:

[0102] S101: Acquire a code stream, parse the code stream, and obtain a light compensation permission flag.

[0103] S102: If the illumination compensation permission flag is valid, obtain the illumination compensation frame-level permission flag in the code stream.

[0104] S103: If the illumination compensation frame-level permission flag is valid, obtain an illumination compensation use flag in the code stream.

[0105] S104: If the illumination compensation usage flag is valid, obtain index information of the target illumination compensation mode in the bitstream.

[0106] S105 : Perform illumination compensation on the current block based on the index information of the target illumination compensation mode.

[0107] The illumination compensation method provided in the embodiment of the present application can be used in various application scenarios of prediction technology, such as inter-frame prediction, inter-frame non-rectangular prediction, affine transformation prediction, etc., and the embodiment of the present application does not limit it.

[0108] In an embodiment of the present application, after obtaining the bitstream, the decoder can first parse out from the header information of the bitstream whether the video sequence to be parsed is allowed to perform illumination compensation processing, that is, it can parse out the illumination compensation permission flag, which indicates whether illumination compensation is allowed in the video sequence.

[0109] It should be noted that in the embodiment of the present application, if the illumination compensation permission flag parsed by the decoder is valid, indicating that the illumination compensation function is allowed to be turned on in the video sequence, then the code stream obtained by the decoder must also have a frame-level illumination compensation frame-level permission flag that further indicates whether illumination compensation is allowed for each frame, that is, the decoder can also parse the illumination compensation frame-level permission flag in the code stream. Among them, when the illumination compensation frame-level permission flag corresponding to a certain frame in the video sequence (such as the current frame) is valid, indicating that the illumination compensation function is allowed to be turned on in the frame, the decoder needs to further parse and confirm which blocks in the frame use illumination compensation technology. Then, when decoding the current block, the block-level illumination compensation use flag can be used to determine whether the current block uses illumination compensation technology, so as to perform the subsequent decoding process. If the illumination compensation usage flag corresponding to the current block is valid, it means that the current block uses illumination compensation technology during encoding. Therefore, the decoder can obtain the index information of the target illumination compensation mode used when encoding the current block transmitted in the bitstream during parsing. In this way, the decoder can use the index information of the target illumination compensation mode to perform illumination compensation processing on the current block.

[0110] In an embodiment of the present application, when the decoder parses out that IC (illumination compensation) technology is adopted, the illumination compensation permission flag is valid (wherein, the value of the illumination compensation permission flag of the illumination compensation permission flag reflects whether it is valid). At this time, the decoder knows that the video sequence to which the current block belongs can adopt IC technology, then it is necessary to further parse out from the bitstream whether the illumination compensation frame-level permission flag corresponding to the current frame to which the current block belongs is valid. If it is valid, it indicates that the current block is allowed to adopt illumination compensation, and then it is necessary to further parse out from the bitstream whether the illumination compensation usage flag corresponding to the current block is valid. If it is valid, it indicates that the current block is processed by illumination compensation, and if it is invalid, it indicates that the current block is allowed to use illumination compensation technology, but it is not used. In an embodiment of the present application, if illumination compensation is used during encoding, then the decoder can obtain the index information of the target illumination compensation mode used from the bitstream.

[0111] For example, the decoder can also parse the current prediction mode and current motion information used by the encoder. The decoder simply performs inter-frame prediction on the current block based on the target illumination compensation mode index, the current prediction mode, and the current motion information to obtain a predicted value, and then proceeds with the subsequent decoding process based on the predicted value.

[0112] It should be noted that in the embodiment of the present application, the identification bit can be represented by a numerical value, and the embodiment of the present application does not limit this.

[0113] For example, the illumination compensation permission flag is 1, indicating that the illumination compensation permission flag is valid; the illumination compensation permission flag is 0, indicating that the illumination compensation permission flag is invalid; the illumination compensation frame level permission flag is 1, indicating that the illumination compensation frame level permission flag is valid; the illumination compensation frame level permission flag is 0, indicating that the illumination compensation frame level permission flag is invalid; the illumination compensation use flag is 1, indicating that the illumination compensation use flag is valid; the illumination compensation use flag is 0, indicating that the illumination compensation use flag is invalid. Or,

[0114] The illumination compensation permission flag is 0, indicating that the illumination compensation permission flag is valid; the illumination compensation permission flag is 1, indicating that the illumination compensation permission flag is invalid; the illumination compensation frame level permission flag is 0, indicating that the illumination compensation frame level permission flag is valid; the illumination compensation frame level permission flag is 1, indicating that the illumination compensation frame level permission flag is invalid; the illumination compensation use flag is 0, indicating that the illumination compensation use flag is valid; the illumination compensation use flag is 1, indicating that the illumination compensation use flag is invalid, which is not limited in the embodiments of the present application.

[0115] In some embodiments of the present application, the illumination compensation frame-level permission flag includes at least one illumination compensation frame-level permission flag. That is, in embodiments of the present application, a single frame-level illumination compensation permission flag may be used to indicate that illumination compensation technology is enabled for the current image frame to be encoded (i.e., the current frame); or multiple frame-level illumination compensation permission flags may be used to indicate that illumination compensation technology is enabled for the current image frame to be encoded.

[0116] In an embodiment of the present application, at least one illumination compensation frame-level permission flag may correspond one-to-one with different prediction modes of the current frame; or at least one illumination compensation frame-level permission flag may correspond one-to-one with different regions of the current frame. In other words, different illumination compensation frame-level permission flags may be used for different prediction modes or for different image regions of a frame.

[0117] In some embodiments of the present application, the illumination compensation frame level permission flag includes: N-level illumination compensation frame level permission flags corresponding to the prediction mode; in this case, if the illumination compensation frame level permission flag is valid, the process for the decoder to obtain the illumination compensation usage flag in the code stream may be: if the i-th level illumination compensation frame level permission flag in the N-level illumination compensation frame level permission flags is valid, then determining the i+1-th level illumination compensation frame level permission flag in the code stream, until the N-th level illumination compensation frame level permission flag in the code stream is determined; i is a positive integer greater than or equal to 1 and less than or equal to N, and N is a positive integer greater than or equal to 1; if the i+1-th level illumination compensation frame level permission flag is invalid, then obtaining the illumination compensation usage flag in the code stream.

[0118] It should be noted that when the decoder parses the bitstream, it can parse it in sequence according to the order of the N-level illumination compensation frame-level permission identifiers. There is a priority relationship among the N-level illumination compensation frame-level permission identifiers. The priorities of the 1st-level illumination compensation frame-level permission identifier to the Nth-level illumination compensation frame-level permission identifier are successively reduced, and the 1st-level illumination compensation frame-level permission identifier is the highest. In this case, when the decoder decodes a high-priority illumination compensation frame-level permission identifier, it can continue to parse subsequent illumination compensation frame-level permission identifiers according to priority only when the high-priority illumination compensation frame-level permission identifier is valid. In other words, when the high-priority illumination compensation frame-level permission identifier is parsed and found to be invalid, the subsequent levels of illumination compensation frame-level permission identifiers will no longer be parsed.

[0119] In an embodiment of the present application, if certain prediction modes corresponding to the current frame can use illumination compensation technology, the first-level illumination compensation frame-level enable flag indicates that the current frame can use illumination compensation technology, but not all prediction modes have illumination compensation technology enabled. It is necessary to parse the subsequent illumination compensation frame-level enable flag and enable illumination compensation technology for the corresponding prediction mode at the codec according to the bitstream syntax. If illumination compensation technology is not enabled for the current frame, the first-level illumination compensation frame-level enable flag is false, and there is no need to parse the subsequent flags, and no codewords for the subsequent flags are present in the bitstream.

[0120] It should be noted that if the i-th level illumination compensation frame level permission flag in the N-level illumination compensation frame level permission flag is valid, then the i+1-th level illumination compensation frame level permission flag in the code stream is determined. If the i+1-th level illumination compensation frame level permission flag is still valid, then the next level will be parsed. However, if the N-level illumination compensation frame level permission flag is still valid when parsed, then the next level will not be parsed because there is no next level. In this process, if the i+1-th level illumination compensation frame level permission flag is parsed to be invalid, then the next level will not be parsed, but the illumination compensation technology will be enabled for the prediction mode corresponding to the i-th level illumination compensation frame level permission flag, and further determination will be made as to which current block will use illumination compensation, thereby obtaining the illumination compensation usage flag in the code stream. In some embodiments of the present application, if the 1-th level illumination compensation frame level permission flag is invalid, then all prediction modes of the current frame do not enable illumination compensation technology, and therefore illumination compensation is not required.

[0121] In some embodiments of the present application, the decoder can also directly parse N levels of illumination compensation frame-level enable flags from the bitstream, without any priority relationship. All of them are directly parsed. If the illumination compensation frame-level enable flag of a level is valid, it indicates that the prediction mode corresponding to the illumination compensation frame-level enable flag of that level has illumination compensation mode enabled, and the decoding of the illumination compensation use flag can continue. If it is invalid, it indicates that the prediction mode corresponding to the illumination compensation frame-level enable flag of that level has not illumination compensation mode enabled, and normal decoding can be performed.

[0122] In some embodiments of the present application, the prediction mode may include: merge mode / direct derivation mode, skip mode and normal inter-frame mode, etc., which is not limited in the embodiments of the present application.

[0123] Exemplarily, when N=3, the first level illumination compensation frame level permission flag corresponds to the merge mode / direct export mode; the second level illumination compensation frame level permission flag corresponds to the skip mode; and the third level illumination compensation frame level permission flag corresponds to the normal inter-frame mode.

[0124] It is understandable that the decoder can determine which prediction modes have the illumination compensation technology enabled and which have not enabled based on different prediction modes, and then filter out part of the codewords of the current frame that has not enabled the illumination compensation technology at the frame level, thereby improving the decompression efficiency.

[0125] In some embodiments of the present application, when parsing the bitstream, the decoder can also obtain an image area division identifier; if the image area division identifier is valid, M illumination compensation frame-level permission identifiers are obtained; if the image area division identifier is invalid, illumination compensation is not enabled, and M is a positive integer greater than or equal to 1.

[0126] It should be noted that the decoder can set different illumination compensation frame level permission flags for different image regions divided into the current frame. In the embodiment of the present application, the current frame is divided into M image regions.

[0127] In some embodiments of the present application, the illumination compensation frame level enable flag includes: M illumination compensation frame level enable flags corresponding to the image region. In this case, if the illumination compensation frame level enable flag is valid, the decoder may further obtain the illumination compensation usage flag in the bitstream by: if the jth illumination compensation frame level enable flag among the M illumination compensation frame level enable flags is valid, then determining the j+1th illumination compensation frame level enable flag in the bitstream, and so on until the Mth illumination compensation frame level enable flag in the bitstream is determined; j is a positive integer greater than or equal to 1 and less than or equal to M; if the j+1th illumination compensation frame level enable flag is invalid, then obtaining the illumination compensation usage flag in the bitstream.

[0128] It should be noted that when the decoder parses the bitstream, it can parse them in sequence according to the order of the M illumination compensation frame-level permission identifiers. There is a priority relationship among the M illumination compensation frame-level permission identifiers. The priorities of the first illumination compensation frame-level permission identifier to the Mth illumination compensation frame-level permission identifier decrease in sequence, with the first illumination compensation frame-level permission identifier being the highest. In this case, when the decoder decodes a high-priority illumination compensation frame-level permission identifier, it can continue to parse subsequent illumination compensation frame-level permission identifiers according to priority only when the high-priority illumination compensation frame-level permission identifier is valid. In other words, when the high-priority illumination compensation frame-level permission identifier is parsed and found to be invalid, the subsequent levels of illumination compensation frame-level permission identifiers will no longer be parsed.

[0129] In an embodiment of the present application, if illumination compensation technology can be used in certain image areas corresponding to the current frame, the first illumination compensation frame-level permission flag indicates that illumination compensation technology can be used in the first image area of ​​the current frame, but illumination compensation technology is not enabled in all image areas, and subsequent illumination compensation frame-level permission flags need to be parsed, and illumination compensation technology is enabled for the corresponding image areas at the codec end according to the code stream syntax. If illumination compensation technology is not enabled in the first image area of ​​the current frame, the first illumination compensation frame-level permission flag is "no", and subsequent flags do not need to be parsed, and no codewords for subsequent flags exist in the code stream, which means that illumination compensation technology is not enabled in all image areas of the current frame.

[0130] It should be noted that if the i-th illumination compensation frame level permission flag among the M illumination compensation frame level permission flags is valid, then the i+1-th illumination compensation frame level permission flag in the code stream is determined. If the i+1-th illumination compensation frame level permission flag is still valid, then the next one is parsed. However, if the M-th illumination compensation frame level permission flag is still valid when parsed, then the next one is no longer parsed because there is no next one. In this process, if the i+1-th illumination compensation frame level permission flag is parsed to be invalid, then the next one is no longer parsed, but the illumination compensation technology is enabled for the image area corresponding to the i-th illumination compensation frame level permission flag, and the illumination compensation usage flag in the code stream is further determined to be which current block needs to use illumination compensation, thereby obtaining the illumination compensation usage flag in the code stream. In some embodiments of the present application, if the first illumination compensation frame level permission flag is invalid, then the illumination compensation technology is not enabled for all image areas of the current frame, and therefore illumination compensation is not required.

[0131] In some embodiments of the present application, the illumination compensation frame-level enable flag includes M illumination compensation frame-level enable flags corresponding to image regions. In this case, if the illumination compensation frame-level enable flag is valid, the decoder may obtain the illumination compensation usage flag in the bitstream by: if any one of the M illumination compensation frame-level enable flags is valid, then obtaining the illumination compensation usage flag in the bitstream.

[0132] In some embodiments of the present application, the decoder can also directly parse M illumination compensation frame-level enable flags from the bitstream, without any priority relationship. All of them are directly parsed. If the illumination compensation frame-level enable flag is valid, it indicates that the image area corresponding to the illumination compensation frame-level enable flag at that level has illumination compensation mode enabled, and the decoder can continue to parse the illumination compensation use flag. If it is invalid, it indicates that the image area corresponding to the illumination compensation frame-level enable flag at that level has not illumination compensation mode enabled, and normal decoding can be performed.

[0133] In some embodiments of the present application, if the first illumination compensation frame level permission flag is invalid, illumination compensation is not performed.

[0134] It can be understood that the decoder can determine which image areas have the illumination compensation technology turned on and which have not turned on the illumination compensation mode based on the different image areas divided in the current frame, and then can filter out part of the codewords in the image areas where the illumination compensation technology is not turned on at the frame level, thereby improving the decompression efficiency.

[0135] In some embodiments of the present application, under the premise that the illumination compensation function or illumination compensation technology is turned on in the current frame, when decoding the current block, the decoder can directly decode the current block, or divide the current block to obtain at least two sub-blocks corresponding to at least two areas, and then decode the at least two sub-blocks in sequence. The embodiments of the present application do not impose any restrictions.

[0136] In some embodiments of the present application, when the decoder performs illumination compensation on the current block based on the index information of the target illumination compensation mode, for at least two sub-blocks, the decoder can determine the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame based on the index information of the target illumination compensation mode; determine the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame; h is a positive integer greater than or equal to 1 and less than the number of at least two sub-blocks; based on the h-th model parameter, perform illumination compensation on the h-th region to obtain the h-th region prediction block; continue to perform illumination compensation processing on the h+1-th region until the illumination compensation processing of at least two sub-blocks is completed.

[0137] It should be noted that in the embodiment of the present application, there can be three illumination compensation modes: upper left illumination compensation mode (IC_TL), upper illumination compensation mode (IC_T), and left illumination compensation mode (IC_L). The index information of the target illumination compensation mode corresponds to one of the upper left illumination compensation mode (IC_TL), upper illumination compensation mode (IC_T), and left illumination compensation mode (IC_L). The upper left illumination compensation mode, upper illumination compensation mode, and left illumination compensation mode are determined by pixels on adjacent edges at different positions.

[0138] The decoder can determine which illumination compensation mode is used based on the index information of the target illumination compensation mode, and thus can determine which adjacent edge is used by the final illumination compensation mode, thereby determining the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame; and perform illumination compensation processing on each sub-block in turn based on at least the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame.

[0139] In an embodiment of the present application, at least two sub-blocks may be processed in a certain order, for example, the sub-block in the upper left region is processed first, the sub-block in the upper right region is processed second, the sub-block in the lower right region is processed third, and so on, which is not limited in the embodiment of the present application. The decoder may determine the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame, use the h-th model parameter to construct the h-th target illumination compensation mode corresponding to the h-th region, and use the h-th target illumination compensation mode to perform illumination compensation on the h-th region, thereby obtaining an illumination prediction value of the h-th region prediction block of the h-th region; and continue illumination compensation processing for the next region until illumination compensation processing for at least two sub-blocks is completed.

[0140] It can be understood that, in the process of decoding the current block, the decoder can divide the current block into block areas and obtain at least two sub-blocks, on the premise that the illumination compensation function is performed on the current block (that is, when the illumination compensation usage flag of the current block is valid), and calculate the corresponding target illumination compensation mode for each sub-block, and then perform illumination compensation based on the respective target illumination compensation modes. In this way, since the target illumination mode is determined in a targeted manner for the sub-block, different sub-blocks can be effectively targeted and processed more accurately, thereby improving the decoding precision and decoding performance.

[0141] It should be noted that when the decoder calculates the linear model parameters (ie, model parameters), it may divide the current block into several regions of specified sizes, calculate the linear model parameters separately, and act on the regions of corresponding sizes.

[0142] In some embodiments of the present application, the decoder determines the hth model parameter corresponding to the hth region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame in at least one of the following ways:

[0143] Method 1: If there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the h-1-th region prediction block and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0144] It should be noted that the reference samples used by the decoder to calculate the linear model can be the adjacent reconstructed samples corresponding to the respective regions (adjacent reconstructed reference pixels). If the adjacent region reconstructed samples do not exist, the predicted samples (region prediction blocks) after illumination compensation of the adjacent regions within the block can be used to participate in the calculation of the linear model.

[0145] For example, if the current decoding unit size is 32x32, it is divided into four sample areas of 16x16, and respective linear models are calculated for each of the four areas, and linear compensation is performed on the respective sample areas of 16x16.

[0146] Method 2:

[0147] If h is not 1, and there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel, the adjacent pixels in the 1st regional prediction block to the h-1th regional prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the 1st regional prediction block to the h-1th regional prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0148] It should be noted that the above-mentioned illumination compensation technology can divide the current block into several areas of specified sizes when calculating the linear model parameters, and allows the existence of pixel inheritance relationships. When determining the adjacent reconstruction reference pixels of the latter area, the adjacent reconstruction reference pixels of the current block can be used, or the adjacent reference pixels of the adjacent sub-blocks of the current sub-block in the current block can be used (that is, the pixels in the regional prediction block of the previous area). If the current sub-block has adjacent reconstruction reference pixels, the adjacent reconstruction reference pixels are used to calculate the model parameters corresponding to the current sub-block, but if the current sub-block does not have adjacent reconstruction reference pixels, the pixels in the regional prediction block of the previous area are used to calculate the model parameters corresponding to the current sub-block.

[0149] For example, if the current block size is 32x32, it is divided into four 16x16 sample regions. Assuming that the upper left 16x16 sample region 1 is first illuminated, followed by the upper right 16x16 sample region 2, the upper right 16x16 sample region can use all the predicted samples and their adjacent reconstructed samples in the upper left and upper right 16x16 sample regions to calculate the linear model for illumination compensation.

[0150] In some embodiments of the present application, if the current sub-block has adjacent reconstructed reference pixels, the model parameters corresponding to the current sub-block are calculated using the adjacent reconstructed reference pixels and the pixels of at least one regional prediction block in the previous region. However, if the current sub-block does not have adjacent reconstructed reference pixels, the model parameters corresponding to the current sub-block are calculated using the pixels of at least one regional prediction block in the previous region. The number of the at least one regional prediction block is not limited and can be full or partial.

[0151] It should be noted that when h is 1, the pixel relationship of the current block is not inherited, and the first model parameter is calculated based on the reconstructed pixels of the first area among the first adjacent reconstructed reference pixels and the reconstructed pixels of the current frame.

[0152] Method 3:

[0153] After the decoder determines the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame, if h is not 1, it performs weighted processing on the h-th model parameter based on at least one of the obtained 1st model parameter to the h-1th new model parameter to obtain the h-th new model parameter; based on the h-th new model parameter, it performs illumination compensation on the h-th region to obtain the h-th region prediction block.

[0154] It should be noted that when calculating linear model parameters, the above-mentioned illumination compensation technology can divide the current block into several regions of specified sizes and allow for inheritance of model parameters. When calculating model parameters for the latter region, the model parameters can be calculated using Method 1 or Method 2, and then combined with at least one of the model parameters calculated for the previous region (which can be the model parameters calculated directly using Method 1 or Method 2, or the new model parameters obtained by weighting the previous region) to obtain the final model parameters, i.e., the new model parameters.

[0155] It should be noted that if h is 1, the first model parameter is also the first new model parameter.

[0156] In the embodiment of the present application, the weighted weight information may be such that the weight of the model parameters calculated by the current sub-block itself is the largest, and the weight of the model parameters calculated by the sub-blocks adjacent to the current sub-block is the second largest. In other words, the weights may be distributed according to the degree of relevance to the current sub-block, with the greater the relevance, the greater the weight. The embodiment of the present application does not limit the weight distribution.

[0157] For example, if the current block size is 32x32, it is divided into four sample areas of 16x16 size. When performing illumination compensation on the upper right 16x16 sample area 2, the illumination compensated information can be referred to. For example, after the upper right 16x16 sample area 2 calculates the linear parameters using only the corresponding prediction samples and adjacent samples, it can refer to the linear parameters of the upper left 16x16 sample area 1. Simple operations such as weighting can be performed to obtain new linear model parameters to be applied to the upper right 16x16 sample area 2 for illumination compensation. When calculating the new model parameters for sample area 2, the model parameters a1 and b1 calculated for sample area 1 are inherited with the a2 and b2 parameters calculated for sample area 2, a1 and a2 are weighted to obtain the final a of sample area 2, and b1 and b2 are weighted to obtain the final b of sample area 2.

[0158] It should be noted that the model parameters include a scaling factor a and an offset factor b.

[0159] In some embodiments of the present application, when the decoder parses the code stream, it obtains a block region division identifier; if the block region division identifier is valid, the current block is divided to obtain at least two sub-blocks; if the block region division identifier is invalid, the block model parameters of the current block are calculated.

[0160] It should be noted that the block area division identifier may be present, or the block division may be performed directly by default without transmission to reduce codewords, which is not limited in the embodiments of the present application.

[0161] In an embodiment of the present application, the decoder does not divide the current block into regions to calculate linear model parameters, but calculates model parameters as a whole, and calculates block model parameters of the current block (when performing illumination compensation).

[0162] It should be noted that the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode are all derived from the model principle shown in formula (1). The difference is that when obtaining the model parameters of the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode, namely the scaling factor and the offset factor, the adjacent edges of the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame are different. Among them, the upper left illumination compensation mode mainly uses the upper adjacent and left adjacent, the upper illumination compensation mode uses the upper adjacent, and the left illumination compensation mode uses the left adjacent.

[0163] In some embodiments of the present application, the decoder uses the index information of the target illumination compensation mode, and the detailed implementation process of determining the target illumination compensation mode can be: the decoder determines the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame according to the index information of the target illumination compensation mode; based on the reconstructed pixels of the current block of the current frame and the reconstructed reference pixels of the reference frame, determines the model parameters of the target illumination compensation mode, and then determines the target illumination compensation mode.

[0164] In some embodiments of the present application, the decoder may determine the target illumination compensation mode based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame by:

[0165] Determining the target illumination compensation mode as the upper-left illumination compensation mode based on the upper reconstructed pixels and the left reconstructed pixels of the current block of the current frame and the upper reconstructed reference pixels and the left reconstructed reference pixels of the reference frame with the same corresponding pixel positions; or

[0166] Determining a target illumination compensation mode as an upper illumination compensation mode based on upper reconstructed pixels of a current block of a current frame and upper reconstructed reference pixels of a reference frame having the same corresponding pixel positions;

[0167] Based on the left reconstructed pixels of the current block of the current frame and the left reconstructed reference pixels of the reference frame with the same corresponding pixel positions, the target illumination compensation mode is determined to be the left illumination compensation mode.

[0168] In some embodiments of the present application, the upper reconstructed pixel of the current frame includes: at least one first pixel of an upper row of adjacent reconstructed blocks of the current block in the current frame;

[0169] The left reconstructed pixel of the current frame includes: at least one second pixel of a reconstructed block adjacent to the left column of the current block in the current frame;

[0170] The upper reconstructed reference pixel of the reference frame includes: at least one third pixel of an adjacent reconstructed reference block in an upper row of the initial prediction block in the reference frame, or at least one fourth pixel of the Nth row in the initial prediction block in the reference frame; N is a positive integer greater than or equal to 1 and less than the number of rows of the current block;

[0171] The left reconstructed reference pixel of the reference frame includes: at least one fifth pixel of a reconstructed reference block in a column adjacent to the left of the initial prediction block in the reference frame, or at least one sixth pixel in the Mth column of the initial prediction block in the reference frame, where M is a positive integer greater than or equal to 1 and less than the number of columns of the current block.

[0172] Exemplarily, as shown in FIG9 , in the illumination compensation mode IC_T, the fifth pixel of the adjacent reconstructed block in the previous row is used in the current frame, and the fifth pixel of the adjacent reconstructed reference block in a certain row in the reference frame is used.

[0173] Exemplarily, as shown in FIG10 , in illumination compensation mode IC_L, the first pixel of the adjacent reconstructed block in the left column is used in the current frame, and the first pixel of the adjacent reconstructed reference block in the left column is used in the reference frame.

[0174] For example, as shown in FIG11 , after motion information compensation is completed and a motion-compensated prediction block is obtained, when illumination compensation is performed, in illumination compensation mode IC_T, the fifth pixel of the adjacent reconstructed block in the previous row in the current frame is used as the fifth predicted pixel in a row of the prediction block.

[0175] In an embodiment of the present application, the reference frame can also use pixels of a certain row or column within the reference block, that is, in some embodiments of the present application, the pixel position and number of the reconstructed reference pixels in the reference frame obtained by the illumination compensation technology can be any position within the reference block in the reference frame and an integer number greater than 0, such as the first row and / or first column in the reference block pointed to by the motion information. The embodiment of the present application does not impose any restrictions. In addition, the pixel position and number of the reconstructed pixels obtained in the current frame should be consistent with the position and number of the reconstructed pixels obtained in the above-mentioned reference frame.

[0176] It should be noted that, when the current block is divided, the reconstructed pixels of the current block refer to the reconstructed pixels of each sub-block when the model parameters are calculated.

[0177] It can be understood that, since the decoder uses the illumination compensation technology in the process of encoding the current block, it determines which illumination compensation mode to use for illumination compensation from the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode through the index information of the target illumination compensation mode parsed by the code stream. In this process, since the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode are determined by the pixels of adjacent edges at different positions, different illumination compensation modes can be used for illumination compensation based on the consideration of different adjacent edges. Therefore, in the case where the pixels of different adjacent edges are relatively different, it is possible to improve the decoding accuracy by selecting different illumination compensation models corresponding to different pixel positions to perform the prediction process of the current block.

[0178] In some embodiments of the present application, the decoder can also obtain current motion information and current prediction mode when parsing the bitstream; predict the current block based on the index information of the target illumination compensation mode, the current prediction mode and the current motion information to obtain a predicted value.

[0179] In some embodiments of the present application, the decoder can determine the initial prediction block corresponding to the current block based on the current motion information; perform motion compensation on the initial prediction block to obtain a first prediction block; use the index information of the target illumination compensation mode to determine the model parameters; use the model parameters to determine the target illumination compensation mode, and then use the target illumination compensation mode to perform illumination compensation processing on the first prediction block to obtain a second prediction block; use the current prediction mode to perform inter-frame prediction on the second prediction block to obtain a prediction value.

[0180] In some embodiments of the present application, the decoder performs inter-frame prediction on the current block based on the index information of the target illumination compensation mode, the current prediction mode, and the current motion information. The implementation of obtaining the predicted value may include the following three methods:

[0181] Method 1:

[0182] The decoder uses the index information of the target illumination compensation mode to determine the target illumination compensation mode; based on the current motion information, determines the initial prediction block corresponding to the current block; performs motion compensation on the initial prediction block to obtain a first prediction block; uses the target illumination compensation mode to perform illumination compensation on the first prediction block to obtain a second prediction block; uses the current prediction mode to perform inter-frame prediction on the second prediction block to obtain a predicted value.

[0183] Method 2:

[0184] The decoder uses the index information of the target illumination compensation mode to determine the target illumination compensation mode; based on the current motion information, determines the initial prediction block corresponding to the current block; uses the target illumination compensation mode to perform illumination compensation on the initial prediction block to obtain a third prediction block; performs motion compensation on the third prediction block to obtain a fourth prediction block; uses the current prediction mode to perform inter-frame prediction on the fourth prediction block to obtain a prediction value.

[0185] Method 3:

[0186] Based on the current motion information, the decoder determines the initial prediction block corresponding to the current block; performs motion compensation on the initial prediction block to obtain a first prediction block; uses the index information of the target illumination compensation mode to determine the target illumination compensation mode; uses the target illumination compensation mode to perform illumination compensation processing on the first prediction block to obtain a second prediction block; uses the current prediction mode to perform inter-frame prediction on the second prediction block to obtain a prediction value.

[0187] The decoder can first determine the target illumination compensation mode, and then, based on the target illumination compensation mode, perform illumination compensation either before or after motion compensation. Method 1 is to perform illumination compensation after motion compensation, while method 2 is to perform illumination compensation before motion compensation. This embodiment of the present application does not limit this.

[0188] In the embodiment of the present application, when the decoder calculates the target illumination compensation mode, the reconstructed reference pixels of the reference block of the reference frame used can be pixels that have not been motion compensated or pixels that have been motion compensated. Among them, the third method is that the reconstructed reference pixels of the reference block of the reference frame are pixels that have been motion compensated, and the first method is that the reconstructed reference pixels of the reference block of the reference frame are pixels that have been motion compensated. This embodiment of the present application does not limit this.

[0189] It is understandable that when illumination compensation is performed before motion compensation, the difficulty of hardware implementation and the complexity of implementation can be reduced, while when illumination compensation is performed after motion compensation, the encoding and decoding effect is better.

[0190] In some embodiments of the present application, the index information of the target illumination compensation mode includes: first index information and second index information.

[0191] In an embodiment of the present application, the decoder can determine which adjacent edge the target illumination compensation mode uses for model calculation based on the index information of the target illumination compensation mode, and then directly solve the corresponding scaling factor and offset factor according to formulas (2)-(4), and finally substitute them into formula (1) to obtain the target illumination compensation mode.

[0192] In some embodiments of the present application, the decoder determines which illumination compensation mode to use only when the illumination compensation use flag of the current block is valid. If the illumination compensation use flag of the current block is invalid, illumination compensation processing is not used.

[0193] In the embodiment of the present application, the decoder uses the index information of the target illumination compensation mode to determine the target illumination compensation mode, which can be implemented in the following two ways:

[0194] Method 1: If the first index information is invalid, the target illumination compensation mode is determined to be the upper left illumination compensation mode;

[0195] If the first index information is valid and the second index information is invalid, determining that the target illumination compensation mode is the upper illumination compensation mode;

[0196] If the first index information is valid and the second index information is valid, it is determined that the target illumination compensation mode is the left illumination compensation mode.

[0197] Method 2: If the first index information is valid, the target illumination compensation mode is determined to be the upper left illumination compensation mode;

[0198] If the first index information is invalid and the second index information is invalid, determining that the target illumination compensation mode is the upper illumination compensation mode;

[0199] If the first index information is invalid and the second index information is valid, the target illumination compensation mode is determined to be the left illumination compensation mode.

[0200] It should be noted that the decoder uses different representations of the first index information and the second index information to reflect the three illumination compensation modes.

[0201] Exemplarily, the codeword transmission syntax during the illumination compensation process is shown in Table 1.

[0202] Table 1

[0203] IC_IndexIC_flagIC_index0IC_index1No_IC0--IC_TL10-IC_T110IC_L111

[0204] Wherein, IC_Index is the type of illumination compensation mode, including: IC_TL, IC_T and IC_L, IC_flag indicates the illumination compensation usage flag, IC_index0 is the first index information, and IC_index1 is the second index information. The syntax logic of Table 1 is described below with 1 being valid and 0 being invalid.

[0205] In the embodiment of the present application, with respect to Table 1, when the illumination compensation use flag is invalid, no illumination compensation mode is available; only when the illumination compensation use flag is valid, if IC_index0 is invalid, the target illumination compensation mode is determined to be IC_TL;

[0206] If IC_index0 is valid and IC_index1 is invalid, the target illumination compensation mode is determined to be IC_T;

[0207] If IC_index0 is valid and IC_index1 is valid, the target illumination compensation mode is determined to be IC_L.

[0208] It should be noted that, in addition to the above two types, the index information exchange of the target illumination compensation mode IC_L and the target illumination compensation mode IC_T can also be realized, or 0 is used to represent valid and 1 is used to represent invalid to implement the above judgment logic. This is not limited in the embodiments of the present application.

[0209] For example, an example is given in which 1 represents valid and 0 represents invalid for inter-frame prediction.

[0210] The decoder obtains the bitstream and parses the bitstream to obtain the illumination compensation permission flag of the current video sequence.

[0211] During the intra-frame prediction decoding process, if the illumination compensation allow flag is '1', the illumination compensation frame-level allow flag of the current frame is parsed. If the illumination compensation frame-level allow flag is '1', all of the following steps are performed; if the illumination compensation allow flag is '0' or the illumination compensation frame-level allow flag is '0', only steps a), b), d) and f) are performed:

[0212] a) Obtain the bitstream and decode it to obtain residual information, and then obtain the time domain residual information through inverse transformation and inverse quantization;

[0213] b) Analyze the code stream to obtain the inter-frame prediction mode and MV index of the current decoding block;

[0214] c) Parse the code stream to obtain the IC usage flag of the current decoding block. Taking the syntax of Table 1 as an example, if the IC usage flag of the current decoding block is true, continue to parse the code stream to obtain the first illumination compensation mode index flag of the current decoding block, otherwise the current decoding unit does not use illumination compensation technology; if the first illumination compensation mode index flag is true; continue to parse the second illumination compensation mode index flag, otherwise set the illumination compensation mode index to 1, indicating that the first illumination compensation linear model IC_TL is used (the upper and left reconstructed pixels can be used for illumination compensation linear model calculation); if the second illumination compensation mode index flag is true, set the illumination compensation mode index to 3, indicating that the third illumination compensation linear mode IC_L is used (only the left reconstructed pixels can be used for illumination compensation linear model calculation), otherwise set the illumination compensation mode index to 2, indicating that the second illumination compensation linear mode IC_T is used (only the upper reconstructed pixels can be used for illumination compensation linear model calculation). According to the illumination compensation mode index obtained by parsing, obtain the linear model parameters of the reconstructed pixels at the corresponding position to obtain the scaling factor a and the offset factor b;

[0215] d) Perform motion compensation on the current decoded block to obtain a predicted block;

[0216] e) If the illumination compensation flag is not 0, that is, illumination compensation is required for the current prediction block, all pixels in the current prediction block are linearly offset according to the scaling factor a and offset factor b obtained in c) to obtain the final prediction block;

[0217] The final prediction block;

[0218] f) Superimposing the residual information restored in step a) to obtain a reconstructed block of the current decoding unit, and outputting it after post-processing.

[0219] It can be understood that the index information of the target illumination compensation mode only occupies two code words, which can save code stream.

[0220] It should be noted that in the above process of calculating the linear model parameters, all pixel values ​​can be sorted, the maximum and minimum values ​​can be removed, and the average of the two maximum values ​​and the average of the two minimum values ​​can be calculated; or points can be taken at intervals, such as the first point and the second to last point on the upper side, and the first point and the second to last point on the left side, and the average of the two larger values ​​and the average of the two smaller values ​​can be calculated by sorting them, and the subsequent calculation steps are the same as above.

[0221] In some embodiments of the present application, the at least one first pixel is a pixel located at a preset pixel interval in an adjacent reconstructed block in a previous row of the current block;

[0222] The at least one second pixel is a pixel located at a preset pixel interval in an adjacent reconstructed block in a left column of the current block;

[0223] The at least one third pixel is a pixel located at a preset interval of pixels in an adjacent reconstructed reference block in a previous row of the initial prediction block;

[0224] The at least one fourth pixel is a pixel located at a preset interval of pixels in the Nth row of pixels in the initial prediction block;

[0225] The at least one fifth pixel is a pixel located at a preset pixel interval in a left column of an adjacent reconstructed reference block of the initial prediction block;

[0226] The at least one sixth pixel is a pixel located at a preset interval of pixels in the Mth column of pixels in the initial prediction block.

[0227] In the embodiment of the present application, the decoder calculates a and b of the target illumination compensation mode by sorting all pixel values, removing the maximum and minimum values, and finding the average of the two maximum values ​​and the average of the two minimum values, which is consistent with the process described in the above embodiment; or the decoder can take points at intervals on the pixel position, such as the first point and the second to last point on the upper side, the first point and the second to last point on the left side, and then sort them to find the average of the two larger values ​​and the average of the two smaller values, and then calculate a and b according to formulas (2)-(4), thereby obtaining the target illumination compensation mode of formula (1). As follows:

[0228] In some embodiments of the present application, the decoder determines that the target illumination compensation mode is the upper-left illumination compensation mode based on the upper reconstructed pixels and the left reconstructed pixels of the current frame and the upper reconstructed reference pixels and the left reconstructed reference pixels of the reference frame with the same corresponding pixel positions as follows:

[0229] sorting at least one first pixel value of at least one first pixel and at least one second pixel value of at least one second pixel to determine first n largest first pixel values ​​and first n smallest first pixel values, where n is a positive integer greater than or equal to 1 and less than the number of columns and rows of the current block;

[0230] sorting one of at least one third pixel and at least one fourth pixel and one of at least one fifth pixel and at least one sixth pixel to determine first n largest second pixel values ​​and first n smallest second pixel values;

[0231] determining a first maximum average of the first n largest first pixel values, a first minimum average of the first n smallest first pixel values, a second maximum average of the first n largest second pixel values, and a second minimum average of the first n smallest second pixel values;

[0232] determining a first scaling factor and a first offset factor according to the first maximum average value, the first minimum average value, the second maximum average value, and the second minimum average value;

[0233] Based on the first scaling factor, the first offset factor, and the initial upper-left illumination compensation mode, an upper-left illumination compensation mode is determined, and the upper-left illumination compensation mode is used as a target illumination compensation mode.

[0234] In some embodiments of the present application, the decoder determines that the target illumination compensation mode is the upper illumination compensation mode based on the upper reconstructed pixels of the current frame and the upper reconstructed reference pixels of the reference frame with the same corresponding pixel positions as follows:

[0235] sorting at least one first pixel value of at least one first pixel to determine first m largest third pixel values ​​and first m smallest third pixel values, where m is a positive integer greater than or equal to 1 and less than the number of columns and rows of the current block;

[0236] sorting the at least one third pixel or the at least one fourth pixel to determine first m maximum fourth pixel values ​​and first m minimum fourth pixel values;

[0237] determining a third maximum average of the first m largest third pixel values, a third minimum average of the first m smallest third pixel values, a fourth maximum average of the first m largest fourth pixel values, and a fourth minimum average of the first m smallest fourth pixel values;

[0238] determining a second scaling factor and a second offset factor according to the third maximum average value, the third minimum average value, the fourth maximum average value, and the fourth minimum average value;

[0239] An upper illumination compensation mode is determined based on the second scaling factor, the second offset factor, and the initial upper illumination compensation mode, where the upper illumination compensation mode is a target illumination compensation mode.

[0240] In some embodiments of the present application, the decoder determines that the target illumination compensation mode is the left illumination compensation mode based on the left reconstructed pixel of the current frame and the left reconstructed reference pixel of the reference frame with the same corresponding pixel position as follows:

[0241] Sort at least one second pixel value of at least one first pixel to determine first h largest fifth pixel values ​​and first h smallest fifth pixel values, where h is a positive integer greater than or equal to 1 and less than the number of columns and rows of the current block;

[0242] sorting the at least one fifth pixel or the at least one sixth pixel to determine first h largest sixth pixel values ​​and first h smallest sixth pixel values;

[0243] determining a fifth largest average of the first h largest fifth pixel values, a fifth smallest average of the first h smallest fifth pixel values, a sixth largest average of the first h largest sixth pixel values, and a sixth smallest average of the first h smallest sixth pixel values;

[0244] determining a third scaling factor and a third offset factor according to the fifth maximum average, the fifth minimum average, the sixth maximum average, and the sixth minimum average;

[0245] Based on the third scaling factor, the third offset factor, and the initial left illumination compensation mode, a left illumination compensation mode is determined, where the left illumination compensation mode is a target illumination compensation mode.

[0246] In some embodiments of the present application, the above-mentioned illumination compensation processing acts at any position in the implementation process of bidirectional optical flow technology, decoding-end motion vector correction, bidirectional prediction weighting, bidirectional gradient correction, inter-frame prediction filtering or inter-frame and intra-frame joint prediction.

[0247] Among them, the illumination compensation technology can be used in any of other technologies such as bidirectional optical flow (BDOF / BIO), decoder side motion vector correction (DMVR), bidirectional prediction weighting (BCW), bidirectional gradient correction (BGC), inter-frame prediction filtering (INTERPF) or combined inter and intra prediction (CIIP), such as illumination compensation technology acting before bidirectional optical flow technology and bidirectional weighted prediction technology, etc., which is not limited in the embodiments of the present application.

[0248] In some embodiments of the present application, at least one of the bidirectional optical flow technology, decoding-end motion vector correction, bidirectional prediction weighting, bidirectional gradient correction, inter-frame prediction filtering, or inter-frame and intra-frame joint prediction processing does not act on the same current block as the illumination compensation processing.

[0249] The above-mentioned illumination compensation technology may not work together with other technologies on the same current block. If the current block uses IC technology, the above-mentioned technologies such as BDOF / BIO correction compensation will no longer be used on the current block. This embodiment of the present application does not limit this.

[0250] In some embodiments of the present application, the upper left illumination compensation mode includes: a luminance upper left illumination compensation mode, a first chroma upper left illumination compensation mode, and a second chroma upper left illumination compensation mode;

[0251] The upper illumination compensation mode includes: a brightness upper illumination compensation mode, a first chromaticity upper illumination compensation mode, and a second chromaticity upper illumination compensation mode;

[0252] The left illumination compensation modes include: a brightness left illumination compensation mode, a first chromaticity left illumination compensation mode, and a second chromaticity left illumination compensation mode.

[0253] In the embodiment of the present application, the illumination compensation technology needs to recalculate the linear model between different color components. For example, in the YUV color space, the linear models corresponding to the three color components Y, U and V need to be calculated respectively.

[0254] It should be noted that the current CU can limit the minimum area and maximum area for IC technology.

[0255] In some embodiments of the present application, the size of the current block ranges from 64 pixels to 128×128 pixels.

[0256] The embodiment of the present application does not limit the specific area size. The illumination compensation technology is used as the area limit or width and height limit of the coding unit level, such as the minimum area is 64 and the maximum area is 128x128.

[0257] In some embodiments of the present application, the decoder can parse the code stream to obtain the frame-level illumination compensation permission flag of the current frame; use the frame-level illumination compensation permission flag as the illumination compensation permission flag, and then perform the decoding process.

[0258] The illumination compensation technology adds a frame-level switch. The decoding end parses the frame-level switch to determine whether to continue parsing the coding unit-level illumination compensation usage identifier and illumination compensation mode index information of the current frame.

[0259] In some embodiments of the present application, the decoder can parse the code stream to obtain the frame-level prediction mode illumination compensation permission flag of the current frame; use the frame-level prediction mode illumination compensation permission flag as the illumination compensation permission flag, and then perform the decoding process.

[0260] In an embodiment of the present application, the decoder adds a frame-level identification bit specific to the ordinary inter-frame prediction mode or SKIP, MERGE (DIRECT) to limit the use of illumination compensation technology in certain of the above-mentioned prediction modes, but is not limited to these. It can also be other specific prediction modes, which can be determined based on actual settings.

[0261] In some embodiments of the present application, the upper left illumination compensation mode includes: multiple sub-upper left illumination compensation modes; the multiple sub-upper left illumination compensation modes act on different pixel areas of the initial prediction block;

[0262] The upper illumination compensation mode includes: multiple sub-upper illumination compensation modes; the multiple sub-upper illumination compensation modes act on different pixel areas of the initial prediction block;

[0263] The left illumination compensation mode includes: multiple sub-left illumination compensation modes; the multiple sub-left illumination compensation modes act on different pixel areas of the initial prediction block.

[0264] It should be noted that multiple linear models can be solved within a coding unit block and act on different areas within the current block, so that the same block can select an illumination compensation mode based on its own characteristics, perform illumination compensation more in line with its own characteristics, and improve decoding accuracy.

[0265] For example, the decoding portion of inter-frame prediction provides options for inter-frame prediction, such as illumination compensation or local linear offset. For areas where there is a significant brightness difference between the current frame and the reference frame, illumination compensation technology can be used to calculate a linear offset model. After illumination compensation is applied to the current coding block, the predicted block will be closer to the original image block, resulting in smaller residuals and ultimately improved coding efficiency.

[0266] It is understood that the illumination compensation method of this application provides options for operations such as inter-frame prediction that require illumination compensation or local linear offset. For areas where there is a significant brightness difference between the current frame and the reference frame, illumination compensation technology can be used to calculate a linear offset model. After compensating the current coding block, the predicted block will be closer to the original image block, resulting in a smaller residual error and ultimately improving coding efficiency.

[0267] The illumination compensation method of this application was tested on AVS's official simulation platform, HPM10.0. After IC technology was integrated and enabled, the test results for 4K, 1080P, and 720P were shown in Table 2 under random access and low-latency test conditions. Table 2 shows the results under the random access test condition.

[0268] Table 2

[0269]

[0270] Under random access testing conditions, the luma component achieved a 0.38% BDBR reduction, while the UV components saw 0.27% and 0.14% BDBR reductions, respectively. This clearly demonstrates high performance and effectively improves decoder efficiency. Furthermore, under low-latency testing conditions, both the luma and UV components also achieved BDBR reductions, demonstrating high performance and effectively improving encoder coding efficiency.

[0271] From an application perspective, low-latency testing conditions are mainly aimed at applications such as the live broadcast industry and public service videos. In an era where the Internet is so developed, this technology will effectively reduce bit rate and bandwidth.

[0272] It should be noted that the illumination compensation of the embodiment of the present application can also be applied to the inter-frame non-rectangular block prediction technology, that is, the illumination compensation technology is used in the GPM (geometric partitioning mode) in VVC and the AWP (Angular Weighted Prediction) technology in AVS. After the decoding end combines the prediction blocks after motion compensation of the two reference frames in some way to obtain a new prediction block, the illumination compensation technology can be used for prediction compensation. The above-mentioned certain method can be to take the non-rectangular area prediction samples of different prediction blocks respectively and combine them. That is to say, the current block is a non-rectangular area of ​​the first sub-prediction block specified by one motion information and a complementary non-rectangular area of ​​the first prediction block specified by another motion information to obtain a second sub-prediction block. The inter-frame non-rectangular block prediction technology can be GPM in VVC, AWP in AVS, or inter-inter in AV, etc., which are not limited here.

[0273] The target illumination compensation mode is adopted to perform illumination compensation processing on the second sub-prediction block to obtain the target prediction block, which is the final prediction block of the current block.

[0274] In some embodiments of the present application, the illumination compensation method can add an identification bit to the image set (GOP, group of picture), namely, an image set-level illumination compensation permission flag, which is used to indicate whether the illumination compensation technology is turned on in the current GOP. The image set-level illumination compensation permission flag needs to be written into the bitstream, which is parsed at the decoding end. If the illumination compensation permission flag is valid, the image set-level illumination compensation permission flag in the bitstream is obtained; if the image set-level illumination compensation permission flag is valid, the illumination compensation frame-level permission flag in the bitstream is obtained. For example, under a specific configuration, the video has 500 frames, and 25 of the 500 frames form an image set. Only each frame with an image set-level permission flag of 1 has a frame-level image permission flag, namely, an illumination compensation frame-level permission flag. This can further save bitstream overhead.

[0275] In some embodiments of the present application, illumination compensation methods can be used in affine transformation prediction technologies, such as AFFINE and PROF (Prediction Refinement with Optical Flow for affine) in VVC, which is an improvement on AFFINE, and AFFINE, AFFINE_UMVE (Ultimate Motion Vector Expression), and ASP (Affine Secondary Prediction, similar to PROF in VVC) in AVS. Specifically, the decoder performs illumination compensation on the predicted block after affine motion compensation based on the affine transformation parameters of the reference block.

[0276] In an embodiment of the present application, when the decoder parses the bitstream, it obtains the current affine transformation parameters; based on the current affine transformation parameters, it performs affine motion compensation on the current block to obtain a third prediction block; uses the index information of the target illumination compensation mode to determine the model parameters; uses the model parameters to perform illumination compensation processing on the third prediction block to obtain a fourth prediction block; the fourth prediction block is the final prediction block of the current block.

[0277] In this embodiment of the present application, when parsing the bitstream, the decoder obtains corrected motion information; based on the corrected motion information, motion compensation is performed on the current block to obtain a fifth prediction block; model parameters are determined using index information of the target illumination compensation mode; illumination compensation is performed on the fifth prediction block using the model parameters to obtain a sixth prediction block. The sixth prediction block is the final prediction block for the current block.

[0278] The modified motion information is determined by any one of the following methods: advanced motion information expression, improvement of temporal motion information, motion vector angle prediction, and motion vector difference offset in merge mode.

[0279] In the embodiments of the present application, the illumination compensation method can be used in any technology that modifies the MV, such as the Merge mode with MVD (MMVD) in VVC, the Ultimate Motion Vector Expression (UMVE), the Enhanced Temporal Motion Vector Prediction (ETMVP), and the Motion Vector Angular Prediction (MVAP) in AVS. That is, the decoding end corrects / enhances the MV and performs illumination compensation on the prediction block after motion compensation of the corresponding reference block.

[0280] It is understandable that, since the decoder can directly obtain the frame-level illumination compensation frame-level permission flag from the bitstream when using illumination compensation technology during the decoding process of the current block, the decoder can first determine at the frame level whether it needs to continue parsing the block-level illumination compensation usage flag. Only when the illumination compensation frame-level permission flag is valid can the next level flag in the bitstream be obtained, and subsequent decoding can be performed to perform current block prediction (such as inter-frame prediction). Therefore, when the illumination compensation frame-level permission flag is invalid, the transmission bits of the bitstream will be greatly reduced. Therefore, when selecting illumination compensation for frame-level images based on different situations, adding the frame-level illumination compensation frame-level permission flag to indicate whether illumination compensation technology is used can save codec bit overhead and improve codec performance.

[0281] The present invention provides an illumination compensation method for a video encoding device, i.e., an encoder. The functions implemented by the method can be implemented by a second processor in the video encoding device calling program code. The program code can also be stored in a computer storage medium. Therefore, the video encoding device includes at least a second processor and a second storage medium.

[0282] FIG12 is a schematic diagram of a flow chart of an implementation method of an illumination compensation method according to an embodiment of the present application. As shown in FIG12 , the method includes:

[0283] S201: Determine an illumination compensation frame level permission flag based on brightness information of a current frame.

[0284] S202: If the illumination compensation frame level permission flag is valid, the illumination compensation function is enabled for the current frame, and illumination compensation is allowed for the current block of the current frame to obtain an illumination prediction value.

[0285] S203: Write the illumination compensation frame level permission flag into the bitstream.

[0286] The illumination compensation method provided in the embodiments of this application can be used in various application scenarios of prediction technologies, such as inter-frame prediction, inter-frame non-rectangular prediction, affine transformation prediction, etc., without limitation in the embodiments of this application. For example, illumination compensation technology can be superimposed on inter-frame prediction to achieve inter-frame prediction. That is, if the IC allows, the inter-frame prediction process on the encoder side is performed.

[0287] In an embodiment of the present application, a video image can be divided into multiple image blocks, and each image block currently to be encoded can be called a coding block, wherein each coding block may include a first image component, a second image component, and a third image component; and the current block is a coding block in the video image that is currently to be predicted for the first image component, the second image component, or the third image component.

[0288] Here, assuming that the current block performs a first image component prediction, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block can also be called a luminance block; or, assuming that the current block performs a second image component prediction, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block can also be called a chrominance block.

[0289] In some embodiments of the present application, the encoder determines a frame-level illumination compensation enable flag based on the brightness information of the current frame. If the illumination compensation frame-level enable flag is valid, the illumination compensation function is enabled for the current frame, illumination compensation is enabled or enabled for the current block of the current frame, and then, by traversing different illumination compensation modes, the corresponding illumination prediction value is obtained, and the illumination compensation frame-level enable flag is written into the bitstream.

[0290] In some embodiments of the present application, the illumination compensation frame-level permission flag includes at least one illumination compensation frame-level permission flag. That is, in embodiments of the present application, a single frame-level illumination compensation permission flag may be used to indicate that illumination compensation technology is enabled for the current image frame to be encoded (i.e., the current frame); or multiple frame-level illumination compensation permission flags may be used to indicate that illumination compensation technology is enabled for the current image frame to be encoded.

[0291] In some embodiments of the present application, if there is a brightness difference value greater than a preset brightness change threshold among at least one brightness difference value, the encoder determines a valid illumination compensation frame level permission flag corresponding to the brightness difference value; otherwise, it determines an invalid illumination compensation frame level permission flag.

[0292] It should be noted that the encoder can set a preset brightness change threshold. Then, by comparing at least one brightness difference value between the current frame and at least one reference frame with the preset brightness change threshold, the illumination compensation frame level permission flag of the current frame can be determined. Specifically, if at least one brightness difference value is greater than the preset brightness change threshold, the illumination compensation frame level permission flag corresponding to the brightness difference value is valid, i.e., the illumination compensation frame level permission flag of the current frame is valid; if at least one brightness difference value is less than or equal to the preset brightness change threshold, the illumination compensation frame level permission flag of the current frame is invalid.

[0293] In an embodiment of the present application, at least one illumination compensation frame level permission flag may correspond one-to-one to different prediction modes of the current frame; or, at least one illumination compensation frame level permission flag may correspond one-to-one to different regions of the current frame. That is, different illumination compensation frame level permission flags may be used for different prediction modes or for different image regions of a frame of image. In some embodiments of the present application, the illumination compensation frame level permission flag includes: N levels of illumination compensation frame level permission flags corresponding to the prediction modes; in this case, the process of the encoder determining the illumination compensation frame level permission flag based on the brightness information of the current frame may be: the encoder determines at least one brightness difference value between the current frame and at least one reference frame based on the brightness information of the current frame; and determines the illumination compensation frame level permission flag based on the at least one brightness difference value and a preset brightness change threshold.

[0294] In some embodiments of the present application, the preset brightness change threshold includes: a first preset brightness change threshold and a second preset brightness change threshold; the second brightness change threshold is greater than the first preset brightness change threshold; and the illumination compensation frame level permission flag includes: an N-level illumination compensation frame level permission flag corresponding to the prediction mode, where N is a positive integer greater than or equal to 1.

[0295] The process of the encoder determining the illumination compensation frame level permission flag based on at least one brightness difference value and a preset brightness change threshold can also be: if at least one brightness difference value is greater than the second preset brightness change threshold, then the N-level illumination compensation frame level permission flags are all valid; if at least one brightness difference value is less than the first preset brightness change threshold, then the N-level illumination compensation frame level permission flags are all invalid; if at least one brightness difference value is greater than the first preset brightness change threshold and less than the second preset brightness change threshold, then the first-level illumination compensation frame level permission flag is valid, and the other-level illumination compensation frame level permission flags are all invalid.

[0296] In some embodiments of the present application, the prediction mode may include: merge mode / direct derivation mode, skip mode and normal inter-frame mode, etc., which is not limited in the embodiments of the present application.

[0297] Exemplarily, when N=3, the first level illumination compensation frame level permission flag corresponds to the merge mode / direct export mode; the second level illumination compensation frame level permission flag corresponds to the skip mode; and the third level illumination compensation frame level permission flag corresponds to the normal inter-frame mode.

[0298] It should be noted that, for different prediction modes, the encoder can determine different threshold intervals according to the number of prediction modes to determine which prediction modes can turn on the illumination compensation function or technology when the threshold ranges are different. Taking three modes as an example, the embodiment of the present application can use the first preset brightness change threshold and the second preset brightness change threshold to divide into three threshold ranges. If at least one brightness difference value is greater than the maximum preset brightness change threshold, all prediction modes turn on the illumination compensation function. If at least one brightness difference value is less than the minimum preset brightness change threshold, all prediction modes do not turn on the illumination compensation function. If there are different threshold ranges between the minimum preset brightness change threshold and the maximum preset brightness change threshold, one or several prediction modes corresponding to each threshold range can be determined to turn on the illumination compensation function.

[0299] In an embodiment of the present application, if at least one brightness difference value is greater than the first preset brightness change threshold and less than the second preset brightness change threshold, the first level illumination compensation frame level permission flag is determined to be valid, and the other levels of illumination compensation frame level permission flags are all invalid.

[0300] It is understandable that the encoder can determine which prediction modes have the illumination compensation technology enabled and which have not enabled based on different prediction modes, and then can enable the illumination compensation technology at the frame level for part of the codewords of the current frame and transmit the code stream, and not transmit codewords in other cases, thereby saving codeword overhead and improving decompression efficiency.

[0301] In some embodiments of the present application, the encoder may further perform region division on the current frame to obtain M image regions and valid image region division identifiers; calculate brightness information of the M image regions to obtain corresponding M brightness information; and write the image region division identifiers into the bitstream. M is a positive integer greater than or equal to 1.

[0302] It should be noted that the decoder can set different illumination compensation frame level permission flags for different image regions divided into the current frame. In the embodiment of the present application, the current frame is divided into M image regions.

[0303] In some embodiments of the present application, the illumination compensation frame level permission flag includes: M illumination compensation frame level permission flags corresponding to the image regions. In this case, the brightness information of the current frame includes: brightness information of M image regions, M brightness information;

[0304] In some embodiments of the present application, the encoder may determine the illumination compensation frame level permission flag based on the brightness information of the current frame. The process may be as follows: the encoder determines the illumination compensation frame level permission flag based on M brightness information and a preset brightness information threshold. Or,

[0305] If there is brightness information greater than the preset brightness information threshold among the M brightness information, the encoder determines the valid illumination compensation frame level permission flag corresponding to the brightness information; otherwise, it determines the invalid illumination compensation frame level permission flag until the M brightness information is compared and M illumination compensation frame level permission flags are obtained.

[0306] It should be noted that each brightness information is compared with a preset brightness information threshold. If it is greater than the preset brightness information threshold, then the illumination compensation frame level permission flag of the image area corresponding to this brightness information is valid, otherwise it is invalid.

[0307] In some embodiments of the present application, the brightness information may be calculated by using a brightness histogram.

[0308] In some embodiments of the present application, the illumination compensation modes include: upper left illumination compensation mode, upper illumination compensation mode, and left illumination compensation mode. The illumination prediction values ​​include: first type prediction values, second type prediction values, and third type prediction values.

[0309] The encoder traverses multiple candidate motion information and determines the initial prediction value of the initial prediction block corresponding to the current block; the initial prediction block corresponds to multiple candidate motion information one by one, wherein the current block belongs to the current frame and the initial prediction block belongs to the reference frame; for the initial prediction block, the upper left illumination compensation mode, the upper illumination compensation mode and the left illumination compensation mode are traversed to perform illumination compensation processing to obtain the first type of prediction value, the second type of prediction value and the third type of prediction value corresponding to the multiple candidate motion information; wherein the upper left illumination compensation mode, the upper illumination compensation mode and the left illumination compensation mode are determined by the pixels of the adjacent edges at different positions. The initial prediction value, the first type of prediction value, the second type of prediction value and the third type of prediction value are used to calculate the rate-distortion cost with the original pixel value of the current block respectively, and the illumination compensation usage flag, as well as the current prediction mode, current motion information and target illumination compensation mode corresponding to the optimal rate-distortion cost are determined; the current prediction mode, current motion information and target illumination compensation mode are used to perform inter-frame prediction on the current coding block to obtain the current prediction value; the index information of the current prediction mode, the index information of the current motion information, the index information of the target illumination compensation mode and the illumination compensation usage flag are written into the bitstream.

[0310] For example, taking inter-frame prediction as an example, the encoder obtains encoding information, including the inter-frame prediction illumination compensation permission flag. After obtaining the image information, the image is divided into several CTUs, and further divided into several CUs. Each independent CU performs inter-frame prediction. The current CU can limit the minimum area and maximum area for IC technology.

[0311] During the inter-frame prediction process at the encoder, if the IC enable flag is '1', the sum of absolute differences (SAD) between the luminance histograms of the current image to be encoded and each reference image, i.e., at least one luminance difference value, is calculated. If the sum of absolute differences between the luminance histograms of the current image to be encoded and a reference image of a certain frame is greater than an illumination change threshold, illumination compensation technology is enabled for the current image to be encoded, or illumination compensation technology is enabled for certain prediction modes of the current image to be encoded, and the enable flag of the current image (i.e., illumination compensation frame-level enable flag) is written into the bitstream and transmitted to the decoder. The illumination change threshold can be pre-set or determined based on the empirical value of the current encoded image; the certain prediction modes for the current image to be encoded can be inter-frame prediction mode, merge mode, or skip mode, etc.

[0312] If illumination compensation technology is enabled for the current image to be encoded, all of the following steps are executed. If the IC enable flag is '0' or illumination compensation technology is not enabled for the current image to be encoded, only steps a), b), and f) are executed:

[0313] a) Inter-frame prediction first traverses all candidate MV motion information to perform motion compensation, calculates the predicted pixels after motion compensation under each MV, and calculates the rate-distortion cost based on the original pixels;

[0314] b) According to the principle of minimizing the rate-distortion cost of all MVs mentioned above, the optimal MV and prediction mode of the current coding unit, such as (SKIP, MERGE / DIRECT or INTER), are selected, and the optimal information and the corresponding rate-distortion cost information are recorded;

[0315] c) Traverse all candidate MVs again. This process starts the illumination compensation technology and traverses the three illumination compensation modes. First, match the reference block according to the prediction mode and MV, extract the left and / or top reconstructed pixels of the reference block and the left and / or top reconstructed pixels of the block to be encoded in the current frame according to the illumination compensation mode index, sort and average the reconstructed pixels (the specific operation is as mentioned above) and substitute them into the above formula to obtain the linear model parameters a and b. The illumination compensation mode index 1 is denoted as IC_TL, and the linear model can be calculated using the left and top reconstructed pixels of the reference block and the current block; illumination compensation mode index 2 is denoted as IC_T, and the linear model can only be calculated using the top reconstructed pixels of the reference block and the current block; illumination compensation mode index 3 is denoted as IC_L, and the linear model can only be calculated using the left reconstructed pixels of the reference block and the current block;

[0316] d) Perform motion compensation on the reference block to obtain a prediction block after ordinary motion compensation, and then perform illumination compensation on it. That is, linearly offset the pixels in each prediction block according to a linear model to obtain the final prediction block of the current coding unit;

[0317] e) Calculate the rate-distortion cost information of each MV based on the final predicted pixels and original pixels after the illumination compensation technology, and record the current illumination compensation mode index, the MV index of the minimum rate-distortion cost information, the corresponding prediction mode (such as SKIP, MERGE / DIRECT or INTER), and the corresponding cost value;

[0318] f) If the illumination compensation technology permission flag is '0', the MV index and prediction mode recorded in b) are transmitted to the decoding end via the code stream; if the illumination compensation technology permission flag is '1', the minimum cost value recorded in b) is compared with the minimum cost value recorded in e). If the rate-distortion cost in b) is smaller, the MV index and prediction mode recorded in b) are encoded as the optimal information of the current coding unit and transmitted to the decoding end via the code stream, and the illumination compensation technology current coding unit flag position is set to no, indicating that the illumination compensation technology is not used, and is also transmitted to the decoding end via the code stream; if the rate-distortion cost in e) is smaller, the MV index, illumination compensation mode index and prediction mode recorded in e) are encoded as the optimal information of the current coding unit and transmitted to the decoding end via the code stream, and the illumination compensation technology current coding unit flag position is set to true and the illumination compensation mode index of the current coding unit block is encoded, indicating that the illumination compensation technology is used, and is also transmitted to the decoding end via the code stream.

[0319] In some embodiments of the present application, the encoder may further determine the size of the current block during the encoding process; and determine the illumination compensation mode of the current block based on the size of the current block and a predicted size threshold.

[0320] In some embodiments of the present application, if the size of the current block is smaller than the prediction size threshold, the encoder determines that the illumination compensation mode of the current block is at least one of the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode; or,

[0321] If the size of the current block is greater than the predicted size threshold, the illumination compensation mode of the current block is determined to be at least one of the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode.

[0322] Exemplarily, the above-mentioned illumination compensation technology allows the use of IC_TL, IC_T and IC_L to perform prediction compensation on all prediction blocks, and the scope of use can also be restricted. For coding units with a small number of samples or coding units with a width / height smaller than a preset threshold, only IC_TL or IC_T or IC_L illumination compensation mode can be used. The above-mentioned illumination compensation technology allows the use of IC_TL, IC_T and IC_L to perform prediction compensation on all prediction blocks, and the scope of use can also be restricted. For coding units with a large number of samples or coding units with a width / height larger than a preset threshold, only IC_TL or IC_T or IC_L illumination compensation mode can be used.

[0323] It is understandable that for current blocks of different sizes, the encoder can use partial illumination modes to determine the illumination prediction value, which can reduce the need to traverse all illumination compensation modes, reduce calculation complexity, and save bits.

[0324] In some embodiments of the present application, during the process of encoding the current block, the encoder may divide the current block into regions to obtain at least two sub-blocks in at least two regions, and then encode the at least two sub-blocks.

[0325] In some embodiments of the present application, if the illumination compensation frame level permission flag is valid, the encoder turns on the illumination compensation function for the current frame and allows illumination compensation for the current block of the current frame. The process of obtaining the illumination prediction value is as follows:

[0326] Traverse multiple candidate motion information to determine the initial prediction value of the initial prediction block corresponding to each sub-block in at least two sub-blocks; the initial prediction block corresponds one-to-one to the multiple candidate motion information; if the illumination compensation frame level allows the identification as valid, then for the initial prediction block of each sub-block, traverse different illumination compensation modes to perform illumination compensation and obtain the corresponding illumination prediction value.

[0327] It should be noted that, when the encoder traverses different illumination compensation modes to perform illumination compensation for the initial prediction block of each sub-block and obtains the corresponding illumination prediction value, the encoder can determine the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame for each illumination compensation mode; determine the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame; h is a positive integer greater than or equal to 1 and less than the number of at least two sub-blocks; based on the h-th model parameter, perform illumination compensation on the h-th region to obtain the h-th illumination prediction value of the h-th region prediction block; continue to perform illumination compensation processing on the h+1-th region until the illumination compensation processing of at least two sub-blocks of at least two regions is completed.

[0328] In some embodiments of the present application, the encoder determines the hth model parameter corresponding to the hth region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame in at least one of the following ways:

[0329] Method 1: If there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the h-1-th region prediction block and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0330] Method 2:

[0331] If h is not 1, and there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel, the adjacent pixels in the 1st regional prediction block to the h-1th regional prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the 1st regional prediction block to the h-1th regional prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0332] Method 3:

[0333] After the encoder determines the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame, if h is not 1, it performs weighted processing on the h-th model parameter based on at least one of the obtained 1st model parameters to the h-1th new model parameters to obtain the h-th new model parameter; based on the h-th new model parameter, it performs illumination compensation on the h-th region to obtain the h-th region prediction block.

[0334] In some embodiments of the present application, the encoder may further generate a valid block region division identifier and write the block region division identifier into the bitstream.

[0335] It can be understood that, in the process of encoding and decoding the current block, the encoder can divide the current block into block areas and obtain at least two sub-blocks, on the premise that the illumination compensation function is performed on the current block (that is, when the illumination compensation usage flag of the current block is valid), and calculate the corresponding illumination compensation mode for each sub-block, and then perform illumination compensation based on the respective target illumination compensation modes. In this way, since the illumination compensation mode is determined in a targeted manner for the sub-block, different sub-blocks can be effectively targeted and processed more accurately, thereby improving the encoding precision and decoding performance.

[0336] In some embodiments of the present application, the illumination compensation modes in the embodiments of the present application may include: upper left illumination compensation mode, upper illumination compensation mode and left illumination compensation mode to perform illumination compensation processing to obtain illumination prediction values ​​corresponding to multiple candidate motion information.

[0337] It should be noted that the encoder determines the image component to be predicted of the current block; based on the parameters of the current block, the image component to be predicted is predictively encoded using multiple prediction modes to obtain initial prediction values, and then the rate-distortion cost corresponding to each prediction mode under the multiple prediction modes is calculated based on the initial prediction values; the minimum rate-distortion cost (i.e., the first minimum rate-distortion cost) is selected from the multiple rate-distortion costs calculated, and the minimum rate-distortion cost is determined as the optimal rate-distortion cost, and the prediction mode corresponding to the optimal rate-distortion cost is determined as the prediction mode of the current block.

[0338] However, in an embodiment of the present application, the encoder also needs to traverse the illumination compensation mode, determine the illumination prediction value, calculate the rate-distortion cost corresponding to each prediction mode under multiple prediction modes based on the illumination prediction value, and then compare it with the rate-distortion cost corresponding to each prediction mode under multiple prediction modes calculated based on the initial prediction value, and finally determine the optimal rate-distortion cost, so as to determine whether to perform illumination compensation processing on the current block.

[0339] In some embodiments of the present application, the encoder uses the initial prediction value and the illumination prediction value to perform rate-distortion cost calculations with the original pixel value of the current block, respectively, to determine the illumination compensation usage identifier, and the current prediction mode, current motion information, and target illumination compensation mode corresponding to the optimal rate-distortion cost; the current prediction mode, current motion information, and target illumination compensation mode are used to predict the current block to obtain a prediction value.

[0340] In some embodiments of the present application, the encoder determines an initial prediction block corresponding to the current block based on current motion information; performs motion compensation on the initial prediction block to obtain a first prediction block; uses a target illumination compensation mode to perform illumination compensation on the first prediction block to obtain a second prediction block; uses the current prediction mode to perform inter-frame prediction on the second prediction block to obtain a prediction value.

[0341] It is understood that the illumination compensation method of this application provides options for operations such as inter-frame prediction that require illumination compensation or local linear offset. For areas where there is a significant brightness difference between the current frame and the reference frame, illumination compensation technology can be used to calculate a linear offset model. After compensating the current coding block, the predicted block will be closer to the original image block, resulting in a smaller residual error and ultimately improving coding efficiency.

[0342] It should be noted that the illumination compensation of the embodiment of the present application can also be applied to the inter-frame non-rectangular block prediction technology, that is, the illumination compensation technology is used in the GPM in VVC and the AWP technology in AVS. After the encoding end combines the prediction blocks after motion compensation of the two reference frames in some way to obtain a new prediction block, the illumination compensation technology can be used for prediction compensation. The above-mentioned certain method can be to take the non-rectangular area prediction samples of different prediction blocks respectively and combine them. That is to say, the current block is a non-rectangular area of ​​the first sub-prediction block specified by one motion information and a complementary non-rectangular area of ​​the first prediction block specified by another motion information to obtain a second sub-prediction block. The inter-frame non-rectangular block prediction technology can be GPM in VVC, AWP in AVS, or inter-inter in AV, etc., which are not limited here.

[0343] The target illumination compensation mode is adopted to perform illumination compensation processing on the second sub-prediction block to obtain the target prediction block, which is the final prediction block of the current block.

[0344] In some embodiments of the present application, the illumination compensation method can add a flag to the image set GOP, namely, an image set-level illumination compensation enable flag, which indicates whether illumination compensation technology is enabled for the current GOP. This image set-level illumination compensation enable flag needs to be written into the bitstream. For example, in a specific configuration, a video has 500 frames, 25 of which constitute an image set. Only frames with an image set-level enable flag of 1 will have a frame-level image enable flag, namely, an illumination compensation frame-level enable flag. This can further reduce bitstream overhead.

[0345] In some embodiments of the present application, the illumination compensation method may be used in affine transformation prediction technologies, such as AFFINE and PROF in VVC (which is an improvement of AFFINE), and AFFINE, AFFINE_UMVE, and ASP in AVS. Specifically, the encoder performs illumination compensation on the predicted block after affine motion compensation based on the affine transformation parameters of the reference block.

[0346] In this embodiment of the present application, when parsing the bitstream, the encoder obtains current affine transformation parameters; based on the current affine transformation parameters, affine motion compensation is performed on the current block to obtain a third prediction block; model parameters are determined using index information of the target illumination compensation mode; illumination compensation is performed on the third prediction block using the model parameters to obtain a fourth prediction block. The fourth prediction block is the final prediction block of the current block.

[0347] In this embodiment of the present application, when parsing the bitstream, the encoder obtains corrected motion information; based on the corrected motion information, motion compensation is performed on the current block to obtain a fifth prediction block; model parameters are determined using index information of the target illumination compensation mode; illumination compensation is performed on the fifth prediction block using the model parameters to obtain a sixth prediction block. The sixth prediction block is the final prediction block for the current block.

[0348] The modified motion information is determined by any one of the following methods: advanced motion information expression, improvement of temporal motion information, motion vector angle prediction, and motion vector difference offset in merge mode.

[0349] In the embodiments of the present application, the illumination compensation method can be used in any technology that modifies MVs, such as MMVD in merge mode in VVC, UMVE (Ultimate Motion Vector Expression) in AVS, ETMVP, MVAP, and other technologies. Specifically, the decoder modifies / enhances the MV and performs illumination compensation on the prediction block after motion compensation of the corresponding reference block.

[0350] It is understandable that, since the encoder, in the process of encoding the current block, uses illumination compensation technology, it can determine whether illumination compensation technology is turned on in the current frame by traversing different illumination compensation modes, thereby generating a frame-level illumination compensation frame-level permission flag. In this way, the encoder can first determine from the frame level whether it is necessary to continue to use the block-level illumination compensation flag. Only when the illumination compensation frame-level permission flag is valid, subsequent encoding is performed to perform current block prediction (such as inter-frame prediction). Therefore, when the illumination compensation frame-level permission flag is invalid, the transmission bits of the code stream will be greatly reduced. In this way, when illumination compensation is selected for frame-level images based on different situations, adding a frame-level illumination compensation frame-level permission flag to indicate whether illumination compensation technology is used can save encoding and decoding bit overhead and improve encoding and coding performance.

[0351] It should be noted that the description of the encoder is consistent with the principle of the decoder and the implementation corresponds to that of the decoder, so they will not be repeated here.

[0352] Based on the implementation basis of the above embodiments, as shown in FIG13 , the embodiment of the present application provides a decoder 1, comprising:

[0353] The parsing part 10 is configured to obtain a code stream, parse the code stream, and obtain an illumination compensation permission flag;

[0354] The first acquisition part 11 is configured to, if the illumination compensation permission flag is valid, acquire an illumination compensation frame-level permission flag in the code stream; if the illumination compensation frame-level permission flag is valid, acquire an illumination compensation use flag in the code stream; if the illumination compensation use flag is valid, acquire index information of a target illumination compensation mode in the code stream;

[0355] The first illumination compensation part 12 is configured to perform illumination compensation on the current block based on the index information of the target illumination compensation mode.

[0356] In some embodiments of the present application, the illumination compensation frame level permission flag includes: at least one illumination compensation frame level permission flag;

[0357] The at least one illumination compensation frame level allows identification to correspond one-to-one with different prediction modes of the current frame; or the at least one illumination compensation frame level allows identification to correspond one-to-one with different regions of the current frame.

[0358] In some embodiments of the present application, the parsing part 10 is further configured to, if the i-th level illumination compensation frame level permission flag in the N-level illumination compensation frame level permission flag is valid, determine the i+1-th level illumination compensation frame level permission flag in the code stream, until the N-th level illumination compensation frame level permission flag in the code stream is determined; i is a positive integer greater than or equal to 1 and less than or equal to N, and N is a positive integer greater than or equal to 1; if the i+1-th level illumination compensation frame level permission flag is invalid, obtain the illumination compensation usage flag in the code stream.

[0359] In some embodiments of the present application, the illumination compensation frame level permission flag includes: M illumination compensation frame level permission flags corresponding to the image area;

[0360] The parsing part 10 is further configured to, if the j-th illumination compensation frame level permission flag among the M illumination compensation frame level permission flags is valid, determine the j+1-th illumination compensation frame level permission flag in the code stream, until the M-th illumination compensation frame level permission flag in the code stream is determined; j is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer greater than or equal to 1; if the j+1-th illumination compensation frame level permission flag is invalid, obtain the illumination compensation usage flag in the code stream.

[0361] In some embodiments of the present application, the illumination compensation frame level permission flag includes: M illumination compensation frame level permission flags corresponding to the image area;

[0362] The parsing part 10 is further configured to obtain the illumination compensation use flag in the code stream if any one of the M illumination compensation frame-level permission flags is valid.

[0363] In some embodiments of the present application, when N=3, the first level illumination compensation frame level permission identifier corresponds to the merge mode / direct export mode; the second level illumination compensation frame level permission identifier corresponds to the skip mode, and the third level illumination compensation frame level permission identifier corresponds to the ordinary inter-frame mode.

[0364] In some embodiments of the present application, the first illumination compensation part 12 is further configured to not perform illumination compensation if the first-level illumination compensation frame-level permission flag is invalid.

[0365] In some embodiments of the present application, the first illumination compensation part 12 is further configured to not perform illumination compensation if the first illumination compensation frame level permission flag is invalid.

[0366] In some embodiments of the present application, the parsing part 10 is further configured to obtain an image region division flag when parsing the code stream; if the image region division flag is valid, obtain M illumination compensation frame level permission flags;

[0367] The first illumination compensation part 12 is further configured to not perform illumination compensation if the image area division flag is invalid.

[0368] In some embodiments of the present application, the first acquisition part 11 is further configured to divide the current block to obtain at least two sub-blocks corresponding to at least two regions.

[0369] In some embodiments of the present application, the first illumination compensation part 12 is further configured to determine the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame based on the index information of the target illumination compensation mode; determine the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame; h is a positive integer greater than or equal to 1 and less than the number of at least two sub-blocks; based on the h-th model parameter, perform illumination compensation on the h-th region to obtain an h-th region prediction block; continue to perform illumination compensation processing on the h+1-th region until the illumination compensation processing of the at least two sub-blocks is completed.

[0370] In some embodiments of the present application, the first illumination compensation part 12 is further configured to calculate the h-th model parameter based on the h-th adjacent reconstructed reference pixel and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame if there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, calculate the h-th model parameter based on the adjacent pixels in the h-1-th region prediction block and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0371] In some embodiments of the present application, the first illumination compensation part 12 is further configured to, if h is not 1, there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then calculate the h-th model parameter based on the h-th adjacent reconstructed reference pixel, the adjacent pixels in the 1st regional prediction block to the h-1th regional prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then calculate the h-th model parameter based on the adjacent pixels in the 1st regional prediction block to the h-1th regional prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0372] In some embodiments of the present application, the first illumination compensation part 12 is further configured to determine the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame. If h is not 1, weighted processing is performed with the h-th model parameter based on at least one of the obtained 1st model parameter to the h-1th new model parameter to obtain the h-th new model parameter; and illumination compensation is performed on the h-th region based on the h-th new model parameter to obtain the h-th region prediction block.

[0373] In some embodiments of the present application, the parsing part 10 is further configured to obtain a block region division identifier when parsing the code stream;

[0374] The first acquisition part 11 is further configured to divide the current block to obtain the at least two sub-blocks if the block area division flag is valid;

[0375] The first illumination compensation part 12 is further configured to calculate block model parameters of the current block if the block area division flag is invalid.

[0376] In some embodiments of the present application, the decoder further comprises a decoding part 13;

[0377] The parsing part 10 is further configured to obtain current motion information and current prediction mode when parsing the code stream;

[0378] The decoding part 13 is configured to predict the current block according to the index information of the target illumination compensation mode, the current prediction mode and the current motion information to obtain a prediction value.

[0379] In some embodiments of the present application, the decoding part 13 is further configured to determine an initial prediction block corresponding to the current block based on the current motion information; perform motion compensation on the initial prediction block to obtain a first prediction block;

[0380] The first illumination compensation part 12 is further configured to use the index information of the target illumination compensation mode to determine model parameters; use the model parameters to perform illumination compensation processing on the first prediction block to obtain a second prediction block;

[0381] The decoding part 13 is configured to adopt the current prediction mode to perform inter-frame prediction on the second prediction block to obtain the prediction value.

[0382] In some embodiments of the present application, in the process of using the inter-frame non-rectangular block prediction technology, the current block is a second sub-prediction block obtained by splicing a non-rectangular area of ​​a first sub-prediction block specified by one motion information and a complementary non-rectangular area of ​​the first sub-prediction block specified by another motion information.

[0383] The first illumination compensation part 12 is further configured to adopt the target illumination compensation mode to perform illumination compensation processing on the second sub-prediction block to obtain a target prediction block.

[0384] In some embodiments of the present application, the parsing part 10 is further configured to obtain the image set level illumination compensation permission flag in the code stream if the illumination compensation permission flag is valid; and obtain the illumination compensation frame level permission flag in the code stream if the image set level illumination compensation permission flag is valid.

[0385] In some embodiments of the present application, the parsing part 10 is further configured to obtain current affine transformation parameters when parsing the code stream;

[0386] The decoding part 13 is further configured to perform affine motion compensation on the current block based on the current affine transformation parameters to obtain a third prediction block;

[0387] The first illumination compensation part 12 is further configured to use the index information of the target illumination compensation mode to determine model parameters; and use the model parameters to perform illumination compensation processing on the third prediction block to obtain a fourth prediction block.

[0388] In some embodiments of the present application, the parsing part 10 is further configured to obtain corrected motion information when parsing the code stream;

[0389] The decoding part 13 is further configured to perform motion compensation on the current block based on the modified motion information to obtain a fifth prediction block;

[0390] The first illumination compensation part 12 is further configured to use the index information of the target illumination compensation mode to determine model parameters; and use the model parameters to perform illumination compensation processing on the fifth prediction block to obtain a sixth prediction block.

[0391] In some embodiments of the present application, the modified motion information is determined by any one of advanced motion information expression, improvement of temporal motion information, motion vector angle prediction, and motion vector difference offset.

[0392] It is understandable that, since the decoder can directly obtain the frame-level illumination compensation frame-level permission flag from the bitstream when using illumination compensation technology during the decoding process of the current block, the encoder can first determine at the frame level whether it needs to continue parsing the block-level illumination compensation usage flag. Only when the illumination compensation frame-level permission flag is valid can the next level flag in the bitstream be obtained, and subsequent decoding can be performed to perform current block prediction (such as inter-frame prediction). Therefore, when the illumination compensation frame-level permission flag is invalid, the transmission bits of the bitstream will be greatly reduced. Therefore, when selecting illumination compensation for frame-level images based on different situations, adding the frame-level illumination compensation frame-level permission flag to indicate whether illumination compensation technology is used can save codec bit overhead and improve codec performance.

[0393] In the actual application of the present application, as shown in FIG14 , the embodiment of the present application further provides a decoder, including:

[0394] A first memory 14 and a first processor 15;

[0395] The first memory 14 stores a computer program that can be run on the first processor 15 . When the first processor 15 executes the program, the illumination compensation method corresponding to the decoder is implemented.

[0396] Among them, the first processor 15 can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (ASICs), single or multiple general integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor 15 can execute the corresponding steps of the illumination compensation method on the decoder side in the aforementioned embodiment.

[0397] The embodiment of the present application provides an encoder 2, as shown in FIG15 , including:

[0398] The second determining part 20 is configured to determine the illumination compensation frame level permission flag based on the brightness information of the current frame;

[0399] The second prediction part 21 is configured to enable the illumination compensation function of the current frame if the illumination compensation frame level permission flag is valid, perform illumination compensation on the current block of the current frame, and obtain an illumination prediction value;

[0400] The writing part 22 is configured to write the illumination compensation frame level permission flag into the bitstream.

[0401] In some embodiments of the present application, the second determination part 20 is further configured to determine at least one brightness difference value between the current frame and at least one reference frame based on the brightness information of the current frame; and determine the illumination compensation frame level permission flag based on the at least one brightness difference value and a preset brightness change threshold.

[0402] In some embodiments of the present application, the second determination part 20 is further configured to determine a valid illumination compensation frame level permission flag corresponding to the brightness difference value if there is a brightness difference value greater than the preset brightness change threshold among the at least one brightness difference value; otherwise, determine an invalid illumination compensation frame level permission flag.

[0403] In some embodiments of the present application, the preset brightness change threshold includes: a first preset brightness change threshold and a second preset brightness change threshold; the second brightness change threshold is greater than the first preset brightness change threshold; the illumination compensation frame level permission flag includes: an N-level illumination compensation frame level permission flag corresponding to the prediction mode;

[0404] The second determination part 20 is further configured such that if the at least one brightness difference value is greater than the second preset brightness change threshold, the N-level illumination compensation frame level permission flags are all valid; if the at least one brightness difference value is less than the first preset brightness change threshold, the N-level illumination compensation frame level permission flags are all invalid; if the at least one brightness difference value is greater than the first preset brightness change threshold and less than the second preset brightness change threshold, the first-level illumination compensation frame level permission flag is valid, and the other-level illumination compensation frame level permission flags are all invalid.

[0405] In some embodiments of the present application, when N=3, the first level illumination compensation frame level permission identifier corresponds to the merge mode / direct export mode; the second level illumination compensation frame level permission identifier corresponds to the skip mode, and the third level illumination compensation frame level permission identifier corresponds to the ordinary inter-frame mode.

[0406] In some embodiments of the present application, the second determining portion 20 is further configured to perform region division on the current frame to obtain M image regions and valid image region division identifiers; calculate brightness information of the M image regions to obtain corresponding M brightness information;

[0407] The writing part 22 is further configured to write the image area division identifier into the code stream.

[0408] In some embodiments of the present application, the brightness information of the current frame includes: brightness information of M image regions;

[0409] The second determining part 20 is further configured to determine an illumination compensation frame level permission flag based on the M brightness information and a preset brightness information threshold.

[0410] In some embodiments of the present application, the illumination compensation frame level permission flag includes: M illumination compensation frame level permission flags corresponding to image regions;

[0411] The second determination part 20 is further configured to determine a valid illumination compensation frame level permission flag corresponding to the brightness information if there is brightness information greater than the preset brightness information threshold among the M brightness information; otherwise, determine an invalid illumination compensation frame level permission flag, until the M brightness information are compared and M illumination compensation frame level permission flags are obtained.

[0412] In some embodiments of the present application, the second determining part 20 is further configured to divide the current block into regions to obtain at least two sub-blocks in at least two regions.

[0413] In some embodiments of the present application, the second prediction part 21 is further configured to traverse multiple candidate motion information and determine an initial prediction value of an initial prediction block corresponding to each sub-block of the at least two sub-blocks; the initial prediction block corresponds one-to-one to the multiple candidate motion information, wherein the at least two sub-blocks belong to the current frame and the initial prediction block belongs to the reference frame; if the illumination compensation frame level is allowed to be marked as valid, then for the initial prediction block of each sub-block, different illumination compensation modes are traversed to perform illumination compensation to obtain a corresponding illumination prediction value.

[0414] In some embodiments of the present application, the second prediction part 21 is further configured to determine the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame for each illumination compensation mode; determine the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame; h is a positive integer greater than or equal to 1 and less than the number of at least two sub-blocks; based on the h-th model parameter, perform illumination compensation on the h-th region to obtain the h-th illumination prediction value of the h-th region prediction block; continue to perform illumination compensation processing on the h+1-th region until the illumination compensation processing of at least two sub-blocks of the at least two regions is completed.

[0415] In some embodiments of the present application, the second prediction part 21 is further configured to calculate the h-th model parameter based on the h-th adjacent reconstructed reference pixel and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame if there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, calculate the h-th model parameter based on the adjacent pixels in the h-1-th region prediction block and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0416] In some embodiments of the present application, the second prediction part 21 is further configured to, if h is not 1, there is an h-th adjacent reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then calculate the h-th model parameter based on the h-th adjacent reconstructed reference pixel, the adjacent pixels in the 1st region prediction block to the h-1th region prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame; if h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then calculate the h-th model parameter based on the adjacent pixels in the 1st region prediction block to the h-1th region prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

[0417] In some embodiments of the present application, the second prediction part 21 is further configured to determine the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame. If h is not 1, weighted processing is performed on the h-th model parameter based on at least one of the obtained 1st model parameter to the h-1th new model parameter to obtain the h-th new model parameter; and illumination compensation is performed on the h-th region based on the h-th new model parameter to obtain the h-th illumination prediction value of the h-th region prediction block.

[0418] In some embodiments of the present application, the second determining portion 20 is further configured to generate a valid block area division identifier;

[0419] The writing part 22 is further configured to write the block area division identifier into the code stream.

[0420] In some embodiments of the present application, the second prediction part 21 is further configured to use the initial prediction value and the illumination prediction value to perform rate-distortion cost calculation with the original pixel value of the current block, respectively, to determine the illumination compensation usage identifier, and the current prediction mode, current motion information and target illumination compensation mode corresponding to the optimal rate-distortion cost; and use the current prediction mode, the current motion information and the target illumination compensation mode to predict the current block to obtain a prediction value.

[0421] In some embodiments of the present application, the second prediction part 21 is further configured to determine an initial prediction block corresponding to the current block based on the current motion information; perform motion compensation on the initial prediction block to obtain a first prediction block; use the target illumination compensation mode to perform illumination compensation processing on the first prediction block to obtain a second prediction block; use the current prediction mode to perform inter-frame prediction on the second prediction block to obtain the prediction value.

[0422] In some embodiments of the present application, in a process using an inter-frame non-rectangular block prediction technology, the current block is a second sub-prediction block obtained by splicing a non-rectangular region of a first sub-prediction block specified by one motion information and a complementary non-rectangular region of the first sub-prediction block specified by another motion information;

[0423] The second prediction part 21 is further configured to adopt the target illumination compensation mode to perform illumination compensation processing on the second sub-prediction block to obtain a target prediction block.

[0424] In some embodiments of the present application, the second determination part 20 is further configured to determine the image set level illumination compensation permission flag if the illumination compensation permission flag is valid; if the image set level illumination compensation permission flag is valid, determine the illumination compensation frame level permission flag for each frame in the image set.

[0425] In some embodiments of the present application, the second prediction part 21 is further configured to obtain current affine transformation parameters;

[0426] performing affine motion compensation on the current block based on the current affine transformation parameters to obtain a third prediction block;

[0427] The target illumination compensation mode is adopted to perform illumination compensation processing on the third prediction block to obtain a fourth prediction block.

[0428] In some embodiments of the present application, the second prediction part 21 is further configured to obtain corrected motion information;

[0429] Based on the modified motion information, motion compensation is performed on the current block to obtain a fifth prediction block;

[0430] The target illumination compensation mode is adopted to perform illumination compensation processing on the fifth prediction block to obtain a sixth prediction block.

[0431] In some embodiments of the present application, the modified motion information is determined by any one of advanced motion information expression, improvement of temporal motion information, motion vector angle prediction, and motion vector difference offset.

[0432] In some embodiments of the present application, the second determining part 20 is further configured to determine the size of the current block; and determine the illumination compensation mode of the current block based on the size of the current block and a predicted size threshold.

[0433] In some embodiments of the present application, the second determination part 20 is further configured to, if the size of the current block is smaller than the predicted size threshold, then the illumination compensation mode of the current block is at least one of the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode; or, if the size of the current block is larger than the predicted size threshold, then the illumination compensation mode of the current block is at least one of the upper left illumination compensation mode, the upper illumination compensation mode, and the left illumination compensation mode.

[0434] It is understandable that, since the encoder, in the process of encoding the current block, uses illumination compensation technology, it can determine whether illumination compensation technology is turned on in the current frame by traversing different illumination compensation modes, thereby generating a frame-level illumination compensation frame-level permission flag. In this way, the encoder can first determine from the frame level whether it is necessary to continue to use the block-level illumination compensation flag. Only when the illumination compensation frame-level permission flag is valid, subsequent encoding is performed to perform current block prediction (such as inter-frame prediction). Therefore, when the illumination compensation frame-level permission flag is invalid, the transmission bits of the code stream will be greatly reduced. In this way, when illumination compensation is selected for frame-level images based on different situations, adding a frame-level illumination compensation frame-level permission flag to indicate whether illumination compensation technology is used can save encoding and decoding bit overhead and improve encoding and coding performance.

[0435] In practical applications, as shown in FIG16 , an embodiment of the present application further provides an encoder, including:

[0436] a second memory 23 and a second processor 24;

[0437] The second memory 23 stores a computer program that can be run on the second processor 24. When the second processor 24 executes the program, the encoder corresponds to the illumination compensation method.

[0438] An embodiment of the present application provides a storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed by a first processor, it implements the illumination compensation method corresponding to the claim decoder; or when the computer program is executed by a second processor, it implements the illumination compensation method corresponding to the claim encoder.

[0439] The various components in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software functional modules.

[0440] If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage media include: ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM), etc. Various media that can store program codes are not limited in the embodiments of the present disclosure.

[0441] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0442] The present invention provides an illumination compensation method, encoder, decoder, and storage medium. The method comprises obtaining a bitstream and parsing the bitstream to obtain an illumination compensation permission flag, current motion information, and current prediction mode. If the illumination compensation permission flag is valid, an illumination compensation frame-level permission flag is obtained from the bitstream. If the illumination compensation frame-level permission flag is valid, an illumination compensation usage flag is obtained from the bitstream. If the illumination compensation usage flag is valid, an index of a target illumination compensation mode is obtained from the bitstream. Based on the index of the target illumination compensation mode, illumination compensation is performed on the current block. The above-mentioned implementation method allows the decoder to directly obtain the frame-level illumination compensation frame-level permission flag from the bitstream when illumination compensation technology is used during decoding of the current block. This allows the encoder to first determine at the frame level whether to continue parsing the block-level illumination compensation usage flag. Only when the illumination compensation frame-level permission flag is valid can the encoder obtain the next-level flag in the bitstream and then perform subsequent decoding to perform prediction (e.g., inter-frame prediction) on the current block. Therefore, when the illumination compensation frame-level permission flag is invalid, the transmission bits of the code stream will be greatly reduced. In this way, when selecting illumination compensation for frame-level images based on different situations, adding the frame-level illumination compensation frame-level permission flag to indicate whether illumination compensation technology is used can save the bit overhead of encoding and decoding and improve the performance of encoding and decoding.

Claims

1. A method for illumination compensation, characterized in that, Applied to the decoder, including: Acquire the bitstream and parse the bitstream to obtain the illumination compensation permission identifier; If the illumination compensation permission flag is valid, then obtain the illumination compensation frame-level permission flag in the bitstream; If the illumination compensation frame-level allow flag is valid, then obtain the illumination compensation usage flag in the bitstream; If the illumination compensation usage flag is valid, then obtain the index information of the target illumination compensation mode in the bitstream; Illumination compensation is performed on the current block based on the index information of the target illumination compensation mode.

2. The method according to claim 1, characterized in that, The illumination compensation frame-level permission identifier includes: at least one illumination compensation frame-level permission identifier; The at least one illumination compensation frame-level allowable identifier corresponds one-to-one with different prediction modes of the current frame; or, the at least one illumination compensation frame-level allowable identifier corresponds one-to-one with different regions of the current frame.

3. The method according to claim 1 or 2, characterized in that, The illumination compensation frame-level permission identifier includes: an N-level illumination compensation frame-level permission identifier corresponding to the prediction mode; If the illumination compensation frame-level allow flag is valid, then the illumination compensation usage flag in the bitstream is obtained, including: If the i-th level illumination compensation frame-level allowable identifier in the N-level illumination compensation frame-level allowable identifier is valid, then the (i+1)-th level illumination compensation frame-level allowable identifier in the bitstream is determined, until the N-th level illumination compensation frame-level allowable identifier in the bitstream is determined; i is a positive integer greater than or equal to 1 and less than or equal to N, and N is a positive integer greater than or equal to 1. If the (i+1)th level illumination compensation frame-level allow flag is invalid, then obtain the illumination compensation usage flag in the bitstream.

4. The method according to claim 1 or 2, characterized in that, The illumination compensation frame-level permission identifier includes: M illumination compensation frame-level permission identifiers corresponding to the image region; If the illumination compensation frame-level allow flag is valid, then the illumination compensation usage flag in the bitstream is obtained, including: If the j-th illumination compensation frame-level allowable identifier among the M illumination compensation frame-level allowable identifiers is valid, then the (j+1)-th illumination compensation frame-level allowable identifier in the bitstream is determined, until the M-th illumination compensation frame-level allowable identifier in the bitstream is determined; j is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer greater than or equal to 1. If the (j+1)th illumination compensation frame-level allow flag is invalid, then the illumination compensation usage flag in the bitstream is obtained.

5. The method according to claim 1 or 2, characterized in that, The illumination compensation frame-level permission identifier includes: M illumination compensation frame-level permission identifiers corresponding to the image region; If the illumination compensation frame-level allow flag is valid, then the illumination compensation usage flag in the bitstream is obtained, including: If any one of the M illumination compensation frame-level allow flags is valid, then the illumination compensation usage flag in the bitstream is obtained.

6. The method according to claim 3, characterized in that, When N=3, the first-level illumination compensation frame-level allow flag corresponds to the merge mode / direct export mode; the second-level illumination compensation frame-level allow flag corresponds to the skip mode; and the third-level illumination compensation frame-level allow flag corresponds to the normal inter-frame mode.

7. The method according to claim 3 or 6, characterized in that, The method further includes: If the first-level illumination compensation frame-level allow flag is invalid, then illumination compensation will not be performed.

8. The method according to claim 4, characterized in that, The method further includes: If the first illumination compensation frame-level allow flag is invalid, then illumination compensation will not be performed.

9. The method according to any one of claims 4, 5 or 8, characterized in that, The method further includes: When parsing the bitstream, obtain the image region segmentation identifier; If the image region division identifier is valid, then obtain M illumination compensation frame-level allow identifiers; If the image region segmentation identifier is invalid, then no illumination compensation will be performed.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: The current block is divided to obtain at least two sub-blocks corresponding to at least two regions.

11. The method according to claim 10, characterized in that, The illumination compensation for the current block based on the index information of the target illumination compensation mode includes: Based on the index information of the target illumination compensation mode, the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame are determined. Based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame, the h-th model parameter corresponding to the h-th region is determined; h is a positive integer greater than or equal to 1 and less than the number of at least two sub-blocks; Based on the h-th model parameter, illumination compensation is performed on the h-th region to obtain the h-th region prediction block; Continue processing the illumination compensation for the (h+1)th region until the illumination compensation for at least two sub-blocks has been completed.

12. The method according to claim 11, characterized in that, The step of determining the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame includes: If there is an h-th adjacent reconstructed reference pixel in the reconstructed reference pixels of the reference frame that is adjacent to the h-th region, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame. If h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the prediction block of the (h-1)-th region and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

13. The method according to claim 11, characterized in that, The step of determining the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame includes: If h is not 1, and there is an h-th adjacent reconstructed reference pixel in the reconstructed reference pixel of the reference frame that is adjacent to the h-th region, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel, the adjacent pixels in the first region prediction block to the (h-1)th region prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame. If h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the first region prediction block to the (h-1)-th region prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

14. The method according to any one of claims 11 to 13, characterized in that, After determining the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame, the method further includes: If h is not 1, then the h-th model parameter is weighted based on at least one of the obtained first to (h-1)th new model parameters and the h-th model parameter to obtain the h-th new model parameter. Based on the h-th new model parameters, illumination compensation is performed on the h-th region to obtain the prediction block of the h-th region.

15. The method according to any one of claims 11 to 14, characterized in that, The method further includes: When parsing the bitstream, obtain the block region division identifier; If the block region division identifier is valid, then the current block is divided to obtain at least two sub-blocks; If the block region partitioning identifier is invalid, then the block model parameters of the current block are calculated.

16. The method according to any one of claims 1 to 15, characterized in that, The method further includes: When parsing the bitstream, the current motion information and the current prediction mode are obtained; Based on the index information of the target illumination compensation mode, the current prediction mode, and the current motion information, the current block is predicted to obtain a predicted value.

17. The method according to claim 16, characterized in that, The step of predicting the current block based on the index information of the target illumination compensation mode, the current prediction mode, and the current motion information to obtain a predicted value includes: Based on the current motion information, determine the initial prediction block corresponding to the current block; Motion compensation is performed on the initial prediction block to obtain the first prediction block; The model parameters are determined using the index information of the target illumination compensation mode; Using the model parameters, the first prediction block is subjected to illumination compensation processing to obtain the second prediction block; Using the current prediction mode, inter-frame prediction is performed on the second prediction block to obtain the predicted value.

18. The method according to claim 17, characterized in that, The method further includes: In the process of using the inter-frame non-rectangular block prediction technique, the current block is a second sub-prediction block obtained by splicing the non-rectangular region of a first sub-prediction block specified by motion information and the complementary non-rectangular region of a first prediction block specified by another motion information. Using the target illumination compensation mode, the second sub-prediction block is subjected to illumination compensation processing to obtain the target prediction block.

19. The method according to any one of claims 1 to 15, characterized in that, The method further includes: If the illumination compensation permission flag is valid, then obtain the image set-level illumination compensation permission flag in the bitstream; If the image set-level illumination compensation permission flag is valid, then obtain the illumination compensation frame-level permission flag in the bitstream.

20. The method according to any one of claims 1 to 15, characterized in that, The method further includes: When parsing the bitstream, obtain the current affine transformation parameters; Based on the current affine transformation parameters, affine motion compensation is performed on the current block to obtain the third prediction block; The model parameters are determined using the index information of the target illumination compensation mode; Using the model parameters, the third prediction block is subjected to illumination compensation processing to obtain the fourth prediction block.

21. The method according to claim 16 or 17, characterized in that, The method further includes: When parsing the bitstream, the corrected motion information is obtained; Based on the corrected motion information, motion compensation is performed on the current block to obtain the fifth predicted block; The model parameters are determined using the index information of the target illumination compensation mode; Using the model parameters, the fifth prediction block is subjected to illumination compensation processing to obtain the sixth prediction block.

22. The method according to claim 21, characterized in that, The corrected motion information is determined by any one of the following methods: high-level motion information representation, improvement of temporal motion information, motion vector angle prediction, or motion vector difference offset.

23. A method for illumination compensation, characterized in that, Applied to encoders, including: Based on the brightness information of the current frame, determine the illumination compensation frame-level allow flag; If the illumination compensation frame-level allow flag is valid, then the illumination compensation function is enabled in the current frame, and illumination compensation is allowed in the current block of the current frame to obtain the illumination prediction value. Write the illumination compensation frame-level allow flag into the bitstream.

24. The method according to claim 23, characterized in that, The determination of the illumination compensation frame-level allow flag based on the brightness information of the current frame includes: Based on the brightness information of the current frame, determine at least one brightness difference value between the current frame and at least one reference frame; Based on the at least one brightness difference value and the preset brightness change threshold, the illumination compensation frame-level allow flag is determined.

25. The method according to claim 24, characterized in that, The step of determining the illumination compensation frame-level allow flag based on the at least one brightness difference value and a preset brightness change threshold includes: If any of the at least one brightness difference values ​​has a brightness difference value greater than the preset brightness change threshold, then a valid illumination compensation frame-level allow flag corresponding to that brightness difference value is determined. Otherwise, an invalid illumination compensation frame-level allow flag is determined.

26. The method according to claim 24, characterized in that, The preset brightness change threshold includes: a first preset brightness change threshold and a second preset brightness change threshold; the second brightness change threshold is greater than the first preset brightness change threshold; the illumination compensation frame-level allow flag includes: an N-level illumination compensation frame-level allow flag corresponding to the prediction mode; The step of determining the illumination compensation frame-level allow flag based on the at least one brightness difference value and a preset brightness change threshold includes: If at least one brightness difference value is greater than the second preset brightness change threshold, then all N-level illumination compensation frame-level allow flags are valid. If all of the at least one brightness difference value is less than the first preset brightness change threshold, then all of the N-level illumination compensation frame-level allow flags are invalid. If at least one brightness difference value is greater than the first preset brightness change threshold and less than the second preset brightness change threshold, then the first level illumination compensation frame-level allow flag is valid, and the other levels illumination compensation frame-level allow flags are invalid.

27. The method according to claim 26, characterized in that, When N=3, the first-level illumination compensation frame-level allow flag corresponds to the merge mode / direct export mode; the second-level illumination compensation frame-level allow flag corresponds to the skip mode; and the third-level illumination compensation frame-level allow flag corresponds to the normal inter-frame mode.

28. The method according to claim 23, characterized in that, The method further includes: The current frame is divided into regions to obtain M image regions and valid image region division identifiers; Calculate the brightness information of the M image regions to obtain the corresponding M brightness information; Write the image region division identifier into the bitstream.

29. The method according to claim 28, characterized in that, The brightness information of the current frame includes: brightness information of M image regions and M brightness information; The determination of the illumination compensation frame-level allow flag based on the brightness information of the current frame includes: Based on the M brightness information and the preset brightness information threshold, the illumination compensation frame-level allow flag is determined.

30. The method according to claim 29, characterized in that, The illumination compensation frame-level permission identifiers include: M illumination compensation frame-level permission identifiers corresponding to image regions; The step of determining the illumination compensation frame-level allowable identifier based on the M luminance information and a preset luminance information threshold includes: If any of the M brightness information is greater than the preset brightness information threshold, then a valid illumination compensation frame-level allow identifier corresponding to that brightness information is determined. Otherwise, invalid illumination compensation frame-level allow flags are determined until all M brightness information is compared, at which point M illumination compensation frame-level allow flags are obtained.

31. The method according to any one of claims 23 to 30, characterized in that, The method further includes: The current block is divided into regions to obtain at least two sub-blocks in at least two regions.

32. The method according to claim 31, characterized in that, If the illumination compensation frame-level permission flag is valid, then the current frame enables the illumination compensation function, and illumination compensation is performed on the current block of the current frame to obtain the illumination prediction value, including: By traversing multiple candidate motion information, the initial prediction value of the initial prediction block corresponding to each of the at least two sub-blocks is determined; the initial prediction block corresponds one-to-one with the multiple candidate motion information, wherein the at least two sub-blocks belong to the current frame, and the initial prediction block belongs to the reference frame; If the illumination compensation frame-level allow flag is valid, then for each sub-block's initial prediction block, different illumination compensation modes are traversed to perform illumination compensation, and the corresponding illumination prediction value is obtained.

33. The method according to claim 32, characterized in that, The initial prediction block for each sub-block is used to perform illumination compensation by traversing different illumination compensation modes to obtain the corresponding illumination prediction value, including: For each illumination compensation mode, determine the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame; Based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame, the h-th model parameter corresponding to the h-th region is determined; h is a positive integer greater than or equal to 1 and less than the number of at least two sub-blocks; Based on the h-th model parameter, illumination compensation is performed on the h-th region to obtain the h-th illumination prediction value of the prediction block of the h-th region; Continue processing the illumination compensation for the (h+1)th region until the illumination compensation for at least two sub-blocks of the at least two regions has been completed.

34. The method according to claim 33, characterized in that, The step of determining the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame includes: If there is an h-th adjacent reconstructed reference pixel in the reconstructed reference pixels of the reference frame that is adjacent to the h-th region, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame. If h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the prediction block of the (h-1)-th region and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

35. The method according to claim 33, characterized in that, The step of determining the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame includes: If h is not 1, and there is an h-th adjacent reconstructed reference pixel in the reconstructed reference pixel of the reference frame that is adjacent to the h-th region, then the h-th model parameter is calculated based on the h-th adjacent reconstructed reference pixel, the adjacent pixels in the first region prediction block to the (h-1)th region prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame. If h is not 1, and there is no reconstructed reference pixel adjacent to the h-th region in the reconstructed reference pixels of the reference frame, then the h-th model parameter is calculated based on the adjacent pixels in the first region prediction block to the (h-1)-th region prediction block, and the reconstructed pixels of the h-th region in the reconstructed pixels of the current frame.

36. The method according to any one of claims 33 to 35, characterized in that, After determining the h-th model parameter corresponding to the h-th region based on the reconstructed pixels of the current frame and the reconstructed reference pixels of the reference frame, the method further includes: If h is not 1, then the h-th model parameter is weighted based on at least one of the obtained first to (h-1)th new model parameters and the h-th model parameter to obtain the h-th new model parameter. Based on the h-th new model parameter, illumination compensation is performed on the h-th region to obtain the h-th illumination prediction value of the prediction block of the h-th region.

37. The method according to any one of claims 31 to 36, characterized in that, The method further includes: Generate a valid block region partition identifier and write the block region partition identifier into the bitstream.

38. The method according to claim 32, characterized in that, The method further includes: Using the initial prediction value and the illumination prediction value, the rate-distortion cost is calculated with the original pixel value of the current block to determine the illumination compensation usage identifier, as well as the current prediction mode, current motion information and target illumination compensation mode corresponding to the optimal rate-distortion cost; The current block is predicted using the current prediction mode, the current motion information, and the target illumination compensation mode to obtain a predicted value.

39. The method according to claim 38, characterized in that, The step of predicting the current coding block using the current prediction mode, the current motion information, and the target illumination compensation mode to obtain a predicted value includes: Based on the current motion information, determine the initial prediction block corresponding to the current block; Motion compensation is performed on the initial prediction block to obtain the first prediction block; Using the target illumination compensation mode, the first prediction block is subjected to illumination compensation processing to obtain the second prediction block; Using the current prediction mode, inter-frame prediction is performed on the second prediction block to obtain the predicted value.

40. The method according to claim 33, characterized in that, The method further includes: In the process of using the inter-frame non-rectangular block prediction technique, the current block is a second sub-prediction block obtained by splicing the non-rectangular region of a first sub-prediction block specified by motion information and the complementary non-rectangular region of a first prediction block specified by another motion information. Using the target illumination compensation mode, the second sub-prediction block is subjected to illumination compensation processing to obtain the target prediction block.

41. The method according to claims 23 to 38, characterized in that, The method further includes: If the illumination compensation permission flag is valid, then determine the image set level illumination compensation permission flag; If the image set-level illumination compensation allow flag is valid, then the illumination compensation frame-level allow flag for each frame in the image set is determined.

42. The method according to claim 38, characterized in that, The method further includes: Get the current affine transformation parameters; Based on the current affine transformation parameters, affine motion compensation is performed on the current block to obtain the third prediction block; Using the target illumination compensation mode, the third prediction block is subjected to illumination compensation processing to obtain the fourth prediction block.

43. The method according to claim 38, characterized in that, The method further includes: Obtain corrected motion information; Based on the corrected motion information, motion compensation is performed on the current block to obtain the fifth predicted block; Using the target illumination compensation mode, the fifth prediction block is subjected to illumination compensation processing to obtain the sixth prediction block.

44. The method according to claim 43, characterized in that, The corrected motion information is determined by any one of the following methods: high-level motion information representation, improvement of temporal motion information, motion vector angle prediction, or motion vector difference offset.

45. The method according to claim 23, characterized in that, The method further includes: Determine the size of the current block; Based on the current block size and the predicted size threshold, the illumination compensation mode of the current block is determined.

46. ​​The method according to claim 45, characterized in that, The step of determining the illumination compensation mode for the current block based on the current block size and the predicted size threshold includes: If the size of the current block is smaller than the predicted size threshold, then the illumination compensation mode of the current block is at least one of the following: top-left illumination compensation mode, top illumination compensation mode, and left illumination compensation mode; or, If the size of the current block is greater than the predicted size threshold, then the illumination compensation mode of the current block is at least one of the top-left illumination compensation mode, the top illumination compensation mode, and the left illumination compensation mode.

47. A decoder, characterized in that, include: The parsing section is configured to acquire the bitstream and parse the bitstream to obtain the illumination compensation permission identifier; The first acquisition part is configured to acquire the illumination compensation frame-level permission identifier in the bitstream if the illumination compensation permission identifier is valid. If the illumination compensation frame-level allow flag is valid, then the illumination compensation usage flag in the bitstream is obtained; if the illumination compensation usage flag is valid, then the index information of the target illumination compensation mode in the bitstream is obtained. The first illumination compensation section is configured to perform illumination compensation on the current block based on the index information of the target illumination compensation mode.

48. An encoder, characterized in that, include: The second determination part is configured to determine the illumination compensation frame-level allow flag based on the brightness information of the current frame; The second prediction part is configured to enable the illumination compensation function in the current frame if the illumination compensation frame-level allow flag is valid, and to perform illumination compensation on the current block of the current frame to obtain the illumination prediction value. The writing section is configured to write the illumination compensation frame-level enable flag into the bitstream.

49. A decoder, characterized in that, include: First memory and first processor; The first memory stores a computer program that can run on a first processor, which, when executing the program, implements the method according to any one of claims 1 to 22.

50. An encoder, characterized in that, include: Second memory and second processor; The second memory stores a computer program that can run on a second processor, which, when executing the program, implements the method according to any one of claims 23 to 46.

51. A storage medium, characterized in that, It stores a computer program that, when executed by a first processor, implements the method according to any one of claims 1 to 22; or, when executed by a second processor, implements the method according to any one of claims 23 to 46.

Images