ENCRYPTION METHODS, DECODATION METHODS, AND COMPUTER-READABLE INVARIABLE RECORDING MATERIALS

Abstract

The invention relates to a method of encoding, a method of decoding, and a computer-readable, invariant record. At the encoding / decoding end, multiple candidate mold areas are used for the illumination compensation mode for the current block to be determined. The target mold area corresponding to the minimum value is determined by performing value calculations on multiple candidate mold areas based on mold matching. The target illumination compensation model parameter is determined according to the target mold area. The second predictive block of the current block is determined by performing illumination compensation on the first predictive block of the current block according to the target illumination compensation model parameter. In this way, for the illumination compensation mode, mold areas are included, where different types of mold areas contain different patterns and different spatial information. Both the encoding and decoding ends together determine the optimal mold area for calculating the model parameter, where the optimal mold area contains the maximum useful information.Accordingly, the accuracy of model parameter calculations can be improved, thereby improving illumination compensation efficiency and encoding performance.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and specifically to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] Real-world videos often feature variations in illumination intensity, such as time-lapse lighting, cloud cover, or even changes in camera flash intensity. These variations differ primarily from the preceding and following frames in the strength of the image's DC component, leaving the texture information largely unchanged. However, due to the large DC component, inter-frame prediction techniques like motion estimation and compensation cannot effectively predict these variations, leading to the inclusion of significant residual information.

[0003] Local Illuminance Compensation (LIC) technology effectively removes these redundant DC components, accurately predicts brightness changes, and compensates accordingly, minimizing residual information and improving coding efficiency. Existing LIC technology uses the reconstructed samples of the upper and left adjacent neighbors of the current block and the upper and left adjacent neighbors of the reference block to establish an illumination model and calculate the model parameters.

[0004] Due to the hybrid video codec framework, codecs often divide images into fine-grained blocks during encoding and decoding, such as 128x128 pixel blocks and even smaller 4x4 pixel blocks. In this case, the reconstructed samples surrounding the blocks used to calculate model parameters may not provide useful information and may even introduce noise, leading to model inaccuracy. Therefore, existing LIC technology needs to be optimized.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which add a template area for the illumination compensation mode. By determining the optimal template area, the accuracy of the model parameter calculation can be improved, thereby improving the illumination compensation effect and the coding and decoding performance.

[0007] The technical solution of the embodiment of the present application can be implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0009] Determine illumination compensation mode indication information of the current block;

[0010] In a case where it is determined according to the illumination compensation mode indication information that the current block uses the illumination compensation mode, determining a plurality of candidate template regions used by the illumination compensation mode;

[0011] Performing cost calculation on the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value;

[0012] Determining target illumination compensation model parameters according to the target template area;

[0013] Perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine a second prediction block of the current block.

[0014] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0015] Determine multiple candidate template areas used in the illumination compensation mode;

[0016] Performing cost calculation on the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value;

[0017] Determining target illumination compensation model parameters according to the target template area;

[0018] Performing illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine a second prediction block of the current block;

[0019] Performing encoding decision on the illumination compensation mode according to the second prediction block to determine illumination compensation mode indication information of the current block;

[0020] The illumination compensation mode indication information is encoded, and the obtained encoding bits are written into a bit stream.

[0021] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on the information to be encoded; wherein the information to be encoded includes at least one of the following: illumination compensation mode indication information of the current block, prediction mode indication information, index value of motion information, and residual information.

[0022] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit, a second determining unit, and an encoding unit; wherein:

[0023] The first determining unit is configured to determine a plurality of candidate template regions used in the illumination compensation mode;

[0024] The first determining unit is configured to perform cost calculation on the candidate template region based on template matching to determine a target template region corresponding to a minimum cost value;

[0025] The first determining unit is configured to determine target illumination compensation model parameters according to the target template area;

[0026] The second determining unit is configured to perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block;

[0027] The second determining unit is configured to make an encoding decision on the illumination compensation mode according to the second prediction block, and determine illumination compensation mode indication information of the current block;

[0028] The encoding unit is configured to encode the illumination compensation mode indication information and write the obtained encoding bits into the bit stream.

[0029] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor; wherein,

[0030] a first memory for storing a computer program capable of running on the first processor;

[0031] The first processor is configured to execute the aforementioned encoding method when running a computer program.

[0032] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a decoding unit, a third determining unit, and a fourth determining unit; wherein:

[0033] The decoding unit is configured to determine illumination compensation mode indication information of the current block;

[0034] The third determining unit is configured to, when it is determined according to the illumination compensation mode indication information that the current block uses the illumination compensation mode, determine a plurality of candidate template regions used by the illumination compensation mode;

[0035] The third determining unit is configured to perform cost calculation on the candidate template region based on template matching to determine a target template region corresponding to a minimum cost value;

[0036] The fourth determining unit is configured to determine target illumination compensation model parameters according to the target template area;

[0037] The fourth determining unit is configured to perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block.

[0038] In a seventh aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor; wherein,

[0039] a second memory for storing a computer program capable of running on the second processor;

[0040] The second processor is configured to execute the decoding method as described above when running the computer program.

[0041] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a code stream generated by the encoding method as described above.

[0042] In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed, implements the method described in the first aspect or the method described in the second aspect.

[0043] The embodiments of the present application provide a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. A template area is added for the illumination compensation mode. Different types of template areas contain different samples and different spatial information. The encoder and decoder simultaneously decide the optimal template area for calculating the model parameters. This template area contains the most useful information, which can improve the accuracy of the model parameter calculation, thereby improving the illumination compensation effect and the encoding and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] FIG1A is a schematic diagram of an image captured in a low-light scenario;

[0046] FIG1B is a schematic diagram of an image captured in a bright light scenario;

[0047] FIG2 is a schematic diagram of the modeling relationship between the reference frame and the current frame;

[0048] FIG3 is a schematic diagram of reconstructed pixels of a reference frame and a current frame;

[0049] Figure 4 is a schematic diagram showing the comparison of three images in different colors;

[0050] FIG5A is a schematic block diagram of an encoder provided in an embodiment of the present application;

[0051] FIG5B is a schematic block diagram of a decoder according to an embodiment of the present application;

[0052] FIG6 is a schematic diagram of a network architecture of a coding and decoding system provided in an embodiment of the present application;

[0053] FIG7 is a schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application;

[0054] FIG8A is a schematic diagram of a first candidate template region provided in an embodiment of the present application;

[0055] FIG8B is a schematic diagram of a second candidate template region provided in an embodiment of the present application;

[0056] FIG8C is a schematic diagram of a third candidate template region provided in an embodiment of the present application;

[0057] FIG8D is a schematic diagram of a fourth candidate template region provided in an embodiment of the present application;

[0058] FIG8E is a schematic diagram of a fifth candidate template region provided in an embodiment of the present application;

[0059] FIG9 is a schematic diagram showing the size relationship between the current block and the template area provided in an embodiment of the present application;

[0060] FIG10 is a schematic diagram of template regions of a reference frame and a current frame according to an embodiment of the present application;

[0061] FIG11 is a schematic diagram of a flow chart of a decoding method provided in an embodiment of the present application;

[0062] FIG12 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0063] FIG13 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0064] FIG14 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0065] FIG15 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0066] FIG16 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The following will describe the technical solutions in the embodiments of this application in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0069] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0070] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0071] In video images, a coding block (CB) is generally represented by a first image component, a second image component, and a third image component. These three image components are a luminance component, a blue chrominance component, and a red chrominance component. Specifically, the luminance component is typically represented by the symbol Y, the blue chrominance component is typically represented by the symbols Cb or U, and the red chrominance component is typically represented by the symbols Cr or V. Thus, video images can be represented in either the YCbCr or YUV format.

[0072] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0073] Moving Picture Experts Group (MPEG)

[0074] International Standardization Organization (ISO)

[0075] International Electrotechnical Commission (IEC)

[0076] Joint Video Experts Team (JVET)

[0077] Alliance for Open Media (AOM)

[0078] Next-generation video coding standard H.266 / Versatile Video Coding (VVC)

[0079] VVC Test Model (VTM)

[0080] Audio Video Standard (AVS)

[0081] AVS High-Performance Model (HPM)

[0082] Transform coefficients

[0083] Quantization Parameter

[0084] Local Illuminance Compensation (LIC)

[0085] Skip technology

[0086] Merge prediction technology (Merge)

[0087] Intra block copy (IBC) mode

[0088] Motion Vector (MV)

[0089] Motion Vector Prediction (MVP)

[0090] Motion Vector Difference (MVD)

[0091] Block Vector (BV)

[0092] Digital video compression technology is understood to primarily compress massive amounts of digital video data for easier transmission and storage. With the surge in Internet video usage and increasing demand for higher-quality video, while existing digital video compression standards can conserve significant video data, there remains a need for improved digital video compression technologies to reduce bandwidth and traffic pressures associated with digital video transmission.

[0093] 1.1 Existing Codec Standards

[0094] Currently, common video codec standards (such as H.266 / VVC) all use a block-based hybrid coding framework. Each frame in the video is divided into square maximum coding units (LCUs) of the same size (such as 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) according to rules. Coding units may also be divided into prediction units (PUs) and transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and inter-frame prediction. Inter-frame prediction includes motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels in a video frame, intra-frame prediction is used in video coding technology to eliminate spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent frames in a video, the inter-frame prediction method is used in video coding and decoding technology to eliminate the temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0095] The basic process of a video codec is as follows. On the encoder side, a frame is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original image block to obtain a residual block. This residual block is transformed and quantized to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​entropy-encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The bitstream is then parsed to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoder side also performs similar operations to the decoder side to obtain a decoded image. The decoded image can serve as a reference frame for inter-frame prediction in subsequent frames. The block division information determined by the encoder, as well as information about the prediction, transform, quantization, entropy coding, loop filtering, and other modes or parameters, are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by parsing and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. During prediction, the current block can be divided into prediction units, and during transformation, the current block can be divided into transformation units. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or process may be optimized. The current block can be the current coding unit (CU) or the current prediction unit (PU), etc.

[0096] JVET, the international video coding standards organization, has established a group to research coding models that surpass H.266 / VVC. This model, or platform testing software, has been named ECM. Building on VTM10.0, ECM incorporates newer and more efficient compression algorithms, currently surpassing VVC's coding performance by approximately 13%. ECM not only increases the coding unit size for specific resolutions but also integrates numerous intra- and inter-frame prediction techniques.

[0097] 1.2 Light compensation technology

[0098] This proposed technology improves upon the ECM reference software to achieve higher coding efficiency.

[0099] Real-world natural video often contains variations in illumination intensity, such as a drop in light intensity over time, cloud cover, or changes in camera flash intensity. These variations differ primarily from the preceding and following frames in the strength of the image's DC component, with little change to the texture information within the content. However, due to the large DC component, inter-frame prediction techniques like motion search and motion compensation cannot effectively predict these variations, leading to the inclusion of a significant amount of residual information. LIC technology effectively removes this redundant DC information, accurately predicting brightness changes and compensating accordingly, minimizing residual information and improving coding efficiency. Hereinafter, local illumination compensation technology will be referred to as illumination compensation.

[0100] The latest video codec standard, H.266 / VVC, has been finalized. The Joint Video Coding Experts Group (JVET) has proposed exploring video codec standards based on VVC that surpass VVC's coding performance. They have established the Exploration Experiment EE2 with this goal in mind. The platform reference software used in the exploratory experiment is based on VTM11.0, into which new algorithms will be integrated and the branch will be renamed ECM. Several expert discussion groups have also been established specifically for ECM. The latest ECM reference software version 8.0 already boasts approximately 19% higher coding performance than VVC. VVC, the current state-of-the-art standard, only boasts approximately 27% higher coding performance than the previous-generation video codec standard, H.265 / HEVC. It's conceivable that in the near future, ECM could be the foundation for exploring and researching the next generation of video codec standards.

[0101] In the early days of ECM, the reference software already integrated coding tools that VVC did not have. These coding tools provided efficient coding performance and processing capabilities for different ECM coding scenarios, including LIC. The following is a brief introduction to LIC in ECM.

[0102] Illumination compensation is an inter-frame coding technique. During inter-frame coding, the current coding unit uses motion vectors (MVs) to obtain a corresponding reference block. This reference block often comes from a different coded frame, or in other words, the reference coding unit does not belong to the current image. In certain scenarios, images from different frames may vary significantly, and illumination compensation is very effective in addressing some of these variations.

[0103] Figure 1A is a schematic diagram of an image in a dark-light shooting scene, and Figure 1B is a schematic diagram of an image in a bright-light shooting scene. As can be seen from the figure, the texture information of the left and right parts is basically the same, and the difference lies in the brightness change between the two. The image on the right appears very bright because it is illuminated by the camera flash, while the image on the left is illuminated by normal natural light. There is a difference between the two in the figure, and the burden brought by this difference in video encoding is huge. Assuming that the left block uses the right block as the reference coding unit, the texture information of the two is the same, so the difference in texture details is very small, but the overall residual is very large. This is because the pixels of the right image are offset as a whole due to the effect of the flash, and the residual of the two includes this part of the offset. If this part of the residual is directly transformed and quantized and written into the bitstream, the overhead of this part will be huge.

[0104] The illumination compensation technology in existing ECM reference software uses linear fitting to eliminate the effects of flash or lighting changes, resulting in better overall prediction results. The main components of the illumination compensation technology are as follows:

[0105] The correlation between the reconstructed samples of the reference coding unit and the adjacent parts of the current coding unit is used to fit the changing correlation between the predicted samples and the reference samples within the coding unit. Because the adjacent reconstructed samples above and to the left of the current coding unit are available, if they exist, and the adjacent reconstructed samples above and to the left of the reference coding unit in the reference image are also available, the corresponding fitting model can be obtained by modeling the reconstructed samples of the current frame and the reconstructed samples of the reference image.

[0106] During the modeling process, ECM uses linear fitting to compensate for illumination. The model is simplified to consist of a scaling parameter a and a bias parameter b, which can fit the illumination changes between the current frame and the reference image. The change relationship is expressed by the model parameters as follows: Pred′(x,y)=a·Pred(x,y)+b

[0107] Where Pred(x,y) is the predicted block before illumination compensation, Pred`(x,y) is the predicted block after illumination compensation, a is the scaling parameter in the illumination compensation model, and b is the offset parameter in the illumination compensation model. Both a and b in the formula need to be calculated based on the image information of the current frame and the image information of the reference frame. They are obtained by modeling the adjacent reconstructed pixels of the current block space and the adjacent pixels of the reconstructed block in the corresponding reference frame. The derivation formula is as follows: Curr_Rec neigh =a·Ref_Rec neigh +b

[0108] In the above formula, Curr_Rec neigh is the reconstructed pixel of the current frame, Ref_Recneigh The reconstructed pixels of the reference frame are used. During the digital video encoding and decoding process, the coding block of the current frame is corrected for illumination differences through an illumination compensation model to obtain a compensated prediction block. Figure 2 is a schematic diagram of the modeling relationship between the reference frame and the current frame.

[0109] The calculation of the scaling parameter a and the offset parameter b requires the use of the adjacent reconstructed pixels of the corresponding reconstructed reference block in the reference frame and the adjacent reconstructed pixels of the coding unit in the current frame. The adjacent reconstructed pixels of the coding unit in the current frame are modeled and solved based on the correlation between the adjacent reconstructed pixels and the reconstructed pixels at the corresponding positions in the reference frame.

[0110] Figure 3 shows the reconstructed pixels of the reference and current frames. As shown in Figure 3, the reconstructed pixels are the most adjacent reconstructed sample pixels of the coding unit (CU). By modeling and solving the linear relationship between the reconstructed pixels at corresponding positions in the two frames, the scaling parameter a and the offset parameter b are obtained. This linear relationship is then applied to the coding unit (CU) of the reference frame to obtain the prediction block of the coding unit (CU) of the current frame.

[0111] The illumination compensation model in ECM is a linear model. The model parameters include the scaling factor a and the offset parameter b. The two are obtained by the least squares method (Least Square Error). The least squares method finds the best function match for the data by minimizing the sum of the squares of the errors. According to the width and height of the current block, the number of selected reconstruction samples is set. If the width and height of the current block are equal to 4, 4 reconstruction samples are taken from the upper and left adjacent reconstruction samples of the coding unit. For example, if the current block width is 16 and the height is 4, all 4 reconstruction samples are taken from the left adjacent reconstruction samples, and 4 are taken from the upper adjacent reconstruction samples with a step size of 3. If the width and height of the current block are not equal to 4, the pixel samples with the logarithm of 2 of the smaller side length are obtained from the upper and left adjacent reconstruction samples.

[0112] After obtaining the upper and left reconstruction samples, calculate the model parameters. Assume that the obtained reference frame reconstruction samples are recorded as x and the current frame reconstruction samples are recorded as y; then the sum of the reference frame reconstruction samples is recorded as sumX, and the sum of the current frame reconstruction samples is recorded as sumY; the sum of the squares of the reference frame reconstruction samples is recorded as sumXX, and the sum of the products of the reference frame reconstruction samples and the current frame reconstruction samples is recorded as sumXY. The parameter calculation is: a=(sumXY-sumXsumY) / (sumXX-sumXsumX) b=sumY-a*sumX

[0113] In the above, sumXsumY is sumX multiplied by sumY. The ECM reference software involves some shifting operations for simplification purposes, which are not detailed here. After obtaining the linear model parameters, a linear transformation is performed on the motion-compensated prediction block to obtain the final prediction block.

[0114] If the reconstructed samples used to calculate the linear model parameters are inter-frame reconstructed samples, an interpolation operation is required.

[0115] The illumination compensation technology in ECM can be used in ordinary inter-frame prediction mode, merge prediction mode and sub-block mode. Ordinary inter-frame prediction is inter mode, merge prediction mode is merge mode, and sub-block mode is affine mode. Affine mode is a new inter-frame prediction mode introduced by H.266, which has a good prediction effect for scenes with rotation and scaling.

[0116] The illumination compensation mode can also be used in the intra block copy mode. If the reconstructed samples used to calculate the linear model parameters are intra-frame reconstruction samples, they can directly participate in the linear model parameter calculation.

[0117] Furthermore, illumination compensation in ECM is coupled with existing technologies. In the current block, illumination compensation is not used simultaneously with bidirectional optical flow (BDOF) and symmetric motion vector difference (SMVD) prediction modes. Both of these prediction modes are bidirectional reference modes. It should be noted that illumination compensation is not restricted by coupling with existing technologies; that is, it can be used with bidirectional reference modes.

[0118] 1.3LIC technology to obtain transformation parameters

[0119] In the process of obtaining transformation parameters, the upper adjacent reconstructed samples and the left adjacent reconstructed samples of the current block and the upper adjacent reconstructed samples and the left adjacent reconstructed samples of the reference block can be used simultaneously to establish an illumination compensation model and calculate model parameters.

[0120] LIC technology offers high performance in scenarios where illumination changes significantly but texture remains largely unchanged, such as security video, outdoor video, and screen video. Since texture changes are largely absent, if a matching block can be found, the error between the current block and the reference block is primarily due to the transformed DC coefficient. Therefore, a model can be created by combining reconstructed samples around the current block with samples around the reference block. This DC coefficient difference can be compensated through linear or nonlinear transformations, significantly improving coding efficiency.

[0121] However, the modeling method of local illumination compensation that uses reference samples and reconstructed samples from the upper and left template areas as input may not be well suited to some application scenarios.

[0122] Due to the hybrid video codec framework, codecs often divide images into fine-grained blocks during encoding and decoding, such as 128x128 pixel blocks and even smaller 4x4 pixel blocks. In this case, the reconstructed samples around the blocks used to calculate model parameters may not provide useful information and may instead introduce noise, leading to model inaccuracy.

[0123] Figure 4 shows a comparison of three images in different colors. As shown in Figure 4, the center circle references the left circle. To the left of the center circle is a coding block 41. Coding block 41 references the same position as the left circle. Due to the color difference, a linear transformation using the LIC is required. As previously explained, if both the upper and left template regions are used for model parameter calculation, the left region will be entirely white and flat, significantly increasing the number of white samples. This will lead to a greater white content in the predicted block after the linear transformation of coding block 41. However, the actual non-white content within coding block 41 is gradually increasing. Using both upper and left samples would actually affect model accuracy. Similarly, the right circle references the center circle. To the left of the right circle is a coding block 42. Coding block 42 references the same position as the center circle. Due to the color difference, a linear transformation using the LIC is required. As previously explained, if both the upper and left template regions are used for model parameter calculation, the upper region will be entirely white and flat, significantly increasing the number of white samples. This will lead to a greater white content in the predicted block after the linear transformation of coding block 42. However, the actual situation in the coding block 42 is that the non-white portion is gradually increasing. If a solution using both the upper and left side samples is adopted, the accuracy of the model will be disturbed.

[0124] 1.4 The Position of LIC Technology in the Hybrid Video Coding Framework

[0125] Referring to FIG5A , which shows a schematic block diagram of a composition of an encoder provided in an embodiment of the present application. As shown in FIG5A , the encoder (specifically, a “video encoder”) 100 may include a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image cache unit 110, etc., wherein the filtering unit 108 may implement deblocking filtering and sample adaptive offset (SAO) filtering, and the encoding unit 109 may implement header information encoding and context-based adaptive binary arithmetic coding (CABAC).For the input original video signal, a video coding block can be obtained by dividing the coding tree unit (CTU). Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transformation and quantization unit 101, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block. Specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding on the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is the process of generating a motion vector, which can estimate the motion of the video coding block. The motion compensation unit 104 then calculates the motion vector based on the motion vector determined by the motion estimation unit 105. After determining the intra-frame prediction mode, the intra-frame prediction unit 103 is further configured to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109. In addition, the inverse transform and inverse quantization unit 106 is configured to reconstruct the video coding block and reconstruct a residual block in the pixel domain. The reconstructed residual block is subjected to the filter control analysis unit 107 and the filtering unit 108 to remove the block effect artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, and output the code stream of the video signal. The decoded image buffer unit 110 is configured to store the reconstructed video coding block for prediction reference. As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .

[0126] Referring to FIG5B , which shows a block diagram of a decoder provided in an embodiment of the present application, as shown in FIG5B , the decoder (specifically, a "video decoder") 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206. The decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal is encoded as shown in FIG5A, a code stream of the video signal is output; the code stream is input to the decoder 200 and first passes through the decoding unit 201 to obtain decoded transform coefficients; the transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate a residual block in the pixel domain; the intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from the previously decoded block of the current frame or picture; the motion compensation unit 204 determines the prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and uses The prediction information is used to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal passes through the filtering unit 205 to remove blocking artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image buffer unit 206, which stores reference images used for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal, thereby obtaining the restored original video signal.

[0127] Furthermore, an embodiment of the present application also provides a network architecture of a coding and decoding system including an encoder and a decoder, wherein FIG6 shows a schematic diagram of a network architecture of a coding and decoding system provided by an embodiment of the present application. As shown in FIG6 , the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During implementation, the electronic device can be various types of devices with video coding and decoding functions. For example, the electronic device can include a smart phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensing device, a server, etc., which are not specifically limited here. In addition, the decoder or encoder described in the embodiment of the present application can be the above-mentioned electronic device.

[0128] It should be noted that the method of the embodiment of the present application is primarily applied to the inter-frame coding portion shown in FIG5A , specifically the motion compensation unit 104 portion, and the inter-frame decoding portion shown in FIG5B , specifically the motion compensation unit 204 portion. The method of the embodiment of the present application can also be applied to the intra-frame prediction unit 103 portion shown in FIG5A and the intra-frame prediction unit 203 portion shown in FIG5B . In other words, the embodiment of the present application can be applied to both the encoder and the decoder, or even to both the encoder and the decoder simultaneously, but the embodiment of the present application is not specifically limited thereto.

[0129] It should also be noted that, when applied to the encoding end, the "current block" specifically refers to the encoding block currently to be predicted; when applied to the decoding end, the "current block" specifically refers to the decoding block currently to be predicted.

[0130] To facilitate understanding of the technical solutions of the embodiments of the present application, the technical solutions of the present application are described in detail below through specific embodiments. The above related technologies can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and all of them fall within the scope of protection of the embodiments of the present application. The embodiments of the present application include at least part of the following contents.

[0131] The embodiment of the present application adds a template area to the illumination compensation mode. Different types of template areas contain different samples and different spatial information. The codec end simultaneously decides the optimal template area for calculating the model parameters. This template area contains the most useful information and can improve the accuracy of the model parameter calculation, thereby improving the illumination compensation effect and the encoding and decoding performance.

[0132] In one embodiment of the present application, referring to FIG7 , a schematic flow chart of an encoding method provided by an embodiment of the present application is shown. As shown in FIG7 , the method may include:

[0133] S701: Determine multiple candidate template areas used in the illumination compensation mode;

[0134] In some embodiments, when the encoder determines that the illumination compensation technology is allowed for the current block, it determines multiple candidate template regions to be used in the illumination compensation mode. Specifically, the multiple candidate template regions are determined based on at least two reconstructed sub-template regions.

[0135] It should be noted that a sub-template region is a reconstructed region adjacent or non-adjacent to the current block and its reference block. In some embodiments, the sub-template region includes at least one of the following: an upper sub-template region, an upper-right sub-template region, a left sub-template region, a lower-left sub-template region, and an upper-left sub-template region. A candidate template region is obtained by combining one or more sub-template regions, and a target template region for the current block is further determined from the candidate template regions.

[0136] In some embodiments, the sub-template regions include: an upper sub-template region and a left sub-template region. Based on the at least two reconstructed sub-template regions, multiple candidate template regions are determined, including: determining a first candidate template region based on the upper sub-template region and the left sub-template region; determining a second candidate template region based on the left sub-template region; and determining a third candidate template region based on the upper sub-template region, as shown in Figures 8A to 8C.

[0137] In other embodiments, the sub-template region further includes: an upper right sub-template region and a lower left sub-template region. A fourth candidate template region is determined based on the upper and upper right sub-template regions; a fifth candidate template region is determined based on the left and lower left sub-template regions, as shown in Figures 8D and 8E.

[0138] It's important to note that a template region composed of two or more sub-template regions has more samples. In some cases, this allows for more sample selection when calculating model parameters than using only the upper or left template regions, potentially leading to higher model accuracy. Therefore, model parameter calculation accuracy can be improved by 1. increasing the number of samples used in model parameter calculation, or 2. expanding the template region to obtain more extensive spatial information.

[0139] Regarding point 1, we can use the sampling method and reduce the sampling step size to increase the number of samples; or even select all available samples in the template area to calculate the linear model parameters.

[0140] Regarding point 2, the template area can be directly expanded. For example, for the upper template area, the template height is kept unchanged and the width of the template area is increased; for the left template area, the template width is kept unchanged and the height of the template area is increased. Figure 8D shows the expansion of the upper template area; Figure 8E shows the expansion of the left template area.

[0141] It should also be noted that in other cases, when using a template area composed of more than two sub-template areas, too many samples will interfere with the accuracy of the model. At this time, the accuracy of model parameter calculation can be improved by limiting the spatial information by reducing the number of samples for model parameter calculation or reducing the area of ​​the template area.

[0142] In an embodiment of the present application, by setting different types of template areas to contain different samples and different spatial information, the codec simultaneously decides the optimal template area for calculating the model parameters. This template area contains the most useful information required by the current block, which can improve the accuracy of the model parameter calculation.

[0143] It should be noted that this is just an illustrative example of the combination of sub-template areas. In addition to the above five candidate template areas, different numbers and types of sub-template areas can be combined to form candidate template areas of more shapes, which will not be repeated here.

[0144] In some embodiments, the method further includes: determining the height of the candidate template region based on the height of the current block; and determining the width of the candidate template region based on the width of the current block. For example, as shown in FIG9 , the height H1 of the current block is correlated with the height H2 of the upper sub-template region, and the width W1 of the current block is correlated with the width W2 of the left sub-template region. Specifically, the correlation may be positive: the higher the height of the current block, the higher the height of the upper sub-template region, and the wider the width of the current block, the wider the width of the left sub-template region.

[0145] In some embodiments, the method further includes: determining the height of the candidate template area as a preset first value; determining the width of the candidate template area as a preset second value; wherein the first value is greater than or equal to 1, and the second value is greater than or equal to 1. It should be noted that the size of the candidate template area can also be a preset size, and the first value and the second value represent the number of sample points. Exemplarily, the first value and the second value are 1, indicating that the template area includes a row of samples adjacent to the upper side and a column of samples adjacent to the left side of the current block and the reference block. The first value and the second value are 2, indicating that the template area includes two rows of samples adjacent to the upper side and two columns of samples adjacent to the left side of the current block and the reference block.

[0146] In some embodiments, the method further includes: when the height of the current block is within a first height range, determining the height of the candidate template area as a first candidate height value; based on the height of the current block being within a second height range, determining the height of the candidate template area as a second candidate height value; when the width of the current block is within a first width range, determining the width of the candidate template area as a first candidate width value; based on the width of the current block being within a second width range, determining the width of the candidate template area as a second candidate width value.

[0147] In some embodiments, the method further includes: detecting whether the first sub-template area of ​​the current block and the first sub-template area of ​​the reference block are both available; if the first sub-template area of ​​the current block and the first sub-template area of ​​the reference block are both available, using the first sub-template area to determine the candidate template area; if the first sub-template area of ​​the current block and / or the first sub-template area of ​​the reference block are not available, ignoring the first sub-template area.

[0148] It should be noted that the selection of candidate template regions can include boundary checks. For sub-template regions where samples are not available, the sub-regions are not used to calculate the model parameters. For example, for the left boundary of the image, the left sub-template region, the lower left sub-template region, and the upper left sub-template region are all unavailable. For the upper boundary of the image, the upper sub-template region, the upper right sub-template region, and the upper left sub-template region are all unavailable.

[0149] S702: Calculate the cost of the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value;

[0150] In an embodiment of the present application, the cost function used for cost calculation can be the sum of absolute error SAD, the sum of absolute transformation difference SATD, the mean square error MSE, the sum of squared errors SSD, the mean absolute difference MAD, the mean sum of squared errors MSD, the absolute value DCT of the transformation coefficient, the normalized correlation coefficient NCC, etc., which are not specifically limited here.

[0151] In some embodiments, a cost calculation is performed on the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value, including: determining the current block template area and the reference block template area based on the candidate template area; determining the candidate illumination compensation model parameters corresponding to the candidate template area based on the first reconstructed sample of the current block template area and the second reconstructed sample of the reference block template area; determining the first prediction sample of the current block template area based on the candidate illumination compensation model parameters; performing a cost calculation based on the third reconstructed sample and the first prediction sample of the current block template area to determine the first generation value of the candidate template area; and determining the target template area corresponding to the minimum cost value based on the first generation value.

[0152] In some embodiments, the illumination model is a linear model constructed based on a linear function, and the illumination compensation model parameters include at least one of the following: a scaling parameter, an offset parameter, and a shift parameter. In other embodiments, the illumination model is a nonlinear model constructed based on a nonlinear function, and the model parameters include any parabolic equation parameter and a shift parameter.

[0153] During the modeling process, ECM uses linear fitting to compensate for illumination. The model is simplified to consist of a scaling parameter a and a bias parameter b, which can fit the illumination changes between the current frame and the reference image. The change relationship is expressed by the model parameters as follows: Pred′(x,y)=a·Pred(x,y)+b

[0154] Where Pred(x,y) is the first predicted block before illumination compensation, Pred`(x,y) is the second predicted block after illumination compensation, a is the scaling parameter in the illumination compensation model, and b is the offset parameter in the illumination compensation model. Both a and b in the formula need to be calculated using the reconstructed samples of the current frame and the reconstructed samples of the reference frame. They are obtained by modeling the reconstructed samples in the current block template area and the reconstructed samples in the reference block template area in the corresponding reference frame. The derivation formula is as follows: Curr_Rec neigh =a·Ref_Rec neigh +b

[0155] In the above formula, Curr_Rec neigh is the reconstructed sample of the current frame, Ref_Rec neigh It is a reconstructed sample of the reference frame. During the digital video encoding and decoding process, the current block is corrected for illumination differences through an illumination compensation model to obtain a compensated prediction block.

[0156] The calculation of the scaling parameter a and the offset parameter b requires the use of the reconstructed samples in the current block template area and the reconstructed samples in the reference block template area in the corresponding reference frame. Specifically, assuming that the obtained second reconstructed sample of the reference frame is recorded as x, and the first reconstructed sample of the current frame is recorded as y; then the sum of the second reconstructed samples of the reference frame is recorded as sumX, and the sum of the first reconstructed samples of the current frame is recorded as sumY; the sum of the squares of the second reconstructed samples of the reference frame is recorded as sumXX, and the sum of the products of the second reconstructed samples of the reference frame and the first reconstructed samples of the current frame is recorded as sumXY, then the parameter calculation is: a=(sumXY-sumXsumY) / (sumXX-sumXsumX) b=sumY-a*sumX

[0157] In the above, sumXsumY is sumX multiplied by sumY. Model parameters such as scaling parameter a and offset parameter b are solved using the least squares method. In a specific implementation, the shift parameter shift may also be included. These parameters are applied to the fourth reconstructed sample above and to the left of the reference block to obtain the compensated reconstructed samples above and to the left of the reference block (i.e., the first predicted sample above and to the left of the current block). The first generation value of these reconstructed samples and the third reconstructed sample above and to the left of the current block is calculated.

[0158] In some embodiments, the reconstructed samples of the reference block template area are motion-compensated reconstructed samples or samples that have not been motion-compensated. For example, the motion compensation may be interpolation filtering.

[0159] In some embodiments, determining a first prediction sample for the current block template region based on candidate illumination compensation model parameters includes applying the candidate illumination compensation model parameters to a fourth reconstructed sample of a reference block template region to obtain the first prediction sample for the current block template region. Specifically, an illumination compensation model is constructed based on the candidate illumination compensation model parameters, the fourth reconstructed sample is input into the illumination compensation model for illumination compensation, and the first prediction sample is output.

[0160] It should be noted that the first reconstructed sample is a reconstructed sample in the current block template region used to calculate the model parameters, the second reconstructed sample is a reconstructed sample in the reference block template region used to calculate the model parameters, the third reconstructed sample is a reconstructed sample in the current block template region used to calculate the first cost value, and the fourth reconstructed sample is a reconstructed sample in the reference block template region used to calculate the first prediction sample. The first reconstructed sample and the second reconstructed sample can be the same sample in the current block template region or different samples. The second reconstructed sample and the fourth reconstructed sample can be the same sample in the reference block template region or different samples.

[0161] It should also be noted that the reconstructed samples in the embodiment of the present application may be luminance component reconstructed samples, and the current block may be a luminance block. The reconstructed samples may also be chrominance component reconstructed samples, and the current block may be a chrominance block.

[0162] In some embodiments, the first template area of ​​the current block template area is sampled to determine a first reconstructed sample; the first template area of ​​the reference block template area is sampled to determine a second reconstructed sample; the second template area of ​​the current block template area is sampled to determine a third reconstructed sample; and the second template area of ​​the reference block template area is sampled to determine a fourth reconstructed sample.

[0163] In some embodiments, partial reconstructed samples are collected in the first template area and the second template area according to a preset sampling step size. The sampling step size may be related to the template size. For example, if the width and height of the current block are both equal to 4, 4 reconstructed samples are taken from the upper and left adjacent reconstructed samples of the coding unit. For example, if the current block width is 16 and the height is 4, all 4 reconstructed samples are taken from the left adjacent reconstructed samples, and 4 reconstructed samples are taken from the upper adjacent reconstructed samples with a step size of 3. If the width and height of the current block are not equal to 4, pixel samples with a number equal to the logarithm of 2 of the smaller side length are obtained from the upper and left adjacent reconstructed samples.

[0164] In some other embodiments, all reconstruction samples are collected in the first template area and the second template area.

[0165] In some embodiments, the first template area is not adjacent to the current block and its reference block, and the second template area is adjacent to the current block and its reference block; wherein the first template area and the second template area do not overlap with each other; or, the first template area and the second template area partially overlap.

[0166] It should be noted that the reconstructed samples used to calculate the model parameters are obtained from the first template region, and the reconstructed samples used to calculate the cost value are obtained from the second template region. When the first template region and the second template region are different regions, it can be understood that the template prediction method can be the same as the prediction method for the current block, that is, the model parameters are calculated using the second template region, and the predicted value is calculated using the first template region. The advantage of this is that the true predicted value of the model parameter can be better made on the template region.

[0167] Figure 10 is a schematic diagram of the template area of ​​the reference frame and the current frame in an embodiment of the present application. The template area includes two rows of reconstructed samples on the upper side and two columns of reconstructed samples on the left side of the current block and the reference block. The first template area is the first row and the first column, and the second template area is the second row and the second column. When making a decision, the reconstructed samples in the first row and the first column of the template are used to calculate the model parameters, and the model parameters are applied to the second row and the second column of the reference block template area to obtain the predicted samples in the second row and the second column of the current block template area. The cost is calculated based on the reconstructed samples and predicted samples in the second row and the second column of the current block template area, and the optimal template area is decided.

[0168] When applying, the second row and second column samples of the template area are used to calculate model parameters, and the model parameters are applied to the first prediction block to obtain a second prediction block after illumination compensation.

[0169] It should be noted that the template area used to calculate model parameters can be a row of samples adjacent to the top and a column of reconstructed samples adjacent to the left of the current block and its reference block. Alternatively, it can be multiple rows of samples adjacent to the top and multiple columns of reconstructed samples adjacent to the left of the current block and its reference block. Alternatively, it can be one or more rows of samples adjacent to the top and a row or more columns of reconstructed samples adjacent to the left of the current block and its reference block. These are not listed here.

[0170] In some embodiments, determining the first prediction sample of the current block template region based on candidate illumination compensation model parameters includes: applying the candidate illumination compensation model parameters to the prediction samples of the first prediction block to determine the first prediction sample of the current block template region. Specifically, an illumination compensation model is constructed based on the candidate illumination compensation model parameters, the prediction samples of the first prediction block are input into the illumination compensation model for illumination compensation, and the first prediction sample is output. In other words, the model parameters can also be applied to the prediction samples of the first prediction block to obtain the first prediction samples of the current block template region, and these prediction samples are used to calculate cost values ​​with the reconstructed samples of the current block template region. Specifically, the model parameters can be applied to the prediction samples of the first row and first column of the first prediction block, or to the prediction samples of the second row and second column of the first prediction block.

[0171] In some embodiments, the method further includes: determining a second prediction sample of the current block template area based on the reconstructed sample of the reference block template area; performing cost calculation based on the reconstructed sample and the second prediction sample of the current template area to determine a second generation value of the candidate template area; if the minimum cost value is the first generation value, determining that the current block uses an illumination compensation mode; if the minimum cost value is the second generation value, determining that the current block does not use an illumination compensation mode.

[0172] For example, the reconstructed samples of the reference block template area are used as the second prediction samples, or the reconstructed samples of the reference block template area are motion compensated to obtain the second prediction samples. The second generation value can be understood as the cost value corresponding to not using the illumination compensation mode for the current block. The minimum cost value is the first generation value, indicating that the illumination compensation mode is used, and the minimum cost value is the second generation value, indicating that the illumination compensation mode is not used.

[0173] In some embodiments, the method further includes: if the prediction mode of the current block is determined to be the merge prediction mode, determining a second prediction sample of the current block template region based on the reconstructed sample of the reference block template region. In other words, when the prediction mode of the current block is the merge prediction mode, the second generation value is calculated based on the reconstructed sample of the current block template region and the reconstructed sample of the reference block template region.

[0174] It should be noted that in actual applications, the second generation value corresponding to each candidate template region can be calculated, or only the second generation value corresponding to a specific template region can be calculated. For example, the specific template region can include the upper sub-template region and the left template region.

[0175] It should also be noted that the second generation value can be calculated based on all the reconstructed samples in the template area, or it can be calculated based on part of the reconstructed samples collected in the template area according to a preset sampling step.

[0176] S703: Determine target illumination compensation model parameters according to the target template area;

[0177] It should be noted that the target illumination compensation model parameters are determined by calculating the model parameters based on the reconstructed samples of the current block target template area and the reconstructed samples of the reference block target template area.

[0178] In some embodiments, the target illumination compensation model parameters are determined based on the target template area, including: determining the current block template area and the reference block template area based on the target template area; and determining the target illumination compensation model parameters corresponding to the target template area based on the third reconstructed sample of the current block template area and the fourth reconstructed sample of the reference block template area.

[0179] Specifically, the second template area of ​​the current block template area is sampled to determine the third reconstructed sample; and the second template area of ​​the reference block template area is sampled to determine the fourth reconstructed sample.

[0180] In some embodiments, the illumination model is a linear model constructed based on a linear function, and the illumination compensation model parameters include at least one of the following: a scaling parameter, an offset parameter, and a shift parameter. In other embodiments, the illumination model is a nonlinear model constructed based on a nonlinear function, and the model parameters include any parabolic equation parameter and a shift parameter.

[0181] In other embodiments, candidate illumination compensation model parameters obtained for the target template region during decision-making are obtained as target illumination compensation model parameters. It should be noted that by using the same reconstructed samples to calculate model parameters during decision-making and application, the candidate illumination parameters obtained for the target template region during decision-making can be directly obtained without recalculating the model parameters, thereby improving coding efficiency.

[0182] S704: Performing illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block;

[0183] In some embodiments, the method further includes: determining a reference block of the current block in a reference frame according to the target prediction mode; and determining a first prediction block according to the reference block.

[0184] It should be noted that the target prediction mode can be any prediction mode that uses the illumination compensation mode. For example, the target prediction mode can be an inter-frame prediction mode such as a normal inter-frame prediction mode, a merge prediction mode, and a sub-block mode. The target prediction mode can also be an intra-frame prediction mode such as an intra-frame block copy mode.

[0185] In some embodiments, determining a first prediction block based on a reference block includes: if the target prediction mode is an intra block copy mode, performing block copying on the reference block to determine the first prediction block; if the target prediction mode is an inter prediction mode, performing motion compensation on the reference block to determine the first prediction block.

[0186] Furthermore, when it is determined that the current block uses the illumination compensation mode, illumination compensation is performed on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block.

[0187] S705: Encoding and deciding the illumination compensation mode according to the second prediction block to determine illumination compensation mode indication information of the current block;

[0188] In some embodiments, a reconstructed block of the current block is determined based on the second prediction block, a cost is calculated based on the reconstructed block and the original block, whether to use the illumination compensation mode in the current prediction mode is determined based on the cost value, and illumination compensation mode indication information is generated.

[0189] In some embodiments, the illumination compensation mode indication information includes a first syntax element identifier; wherein the first syntax element identifier is used to indicate whether the current block uses the illumination compensation mode.

[0190] It should be noted that the current block can be an image block in any illumination compensation mode, and the first syntax element identifier can be a block-level syntax element identifier. The first syntax element identifier can also be a syntax element identifier at the image unit level where the current block is located, used to indicate whether the image unit where the current block is located uses the illumination compensation mode.

[0191] In some embodiments, the illumination compensation mode indication information also includes a second syntax element identifier; wherein the second syntax element identifier includes at least one of the following: a sequence-level syntax element identifier, used to indicate whether the image sequence where the current block is located allows the use of the illumination compensation mode; a frame-level syntax element identifier, used to indicate whether the frame where the current block is located allows the use of the illumination compensation mode; a coding tree unit-level syntax element identifier, used to indicate whether the coding tree unit where the current block is located allows the use of the illumination compensation mode; a slice-level syntax element identifier, used to indicate whether the slice where the current block is located allows the use of the illumination compensation mode.

[0192] It should be noted that the second syntax element identifier may be a syntax element identifier at the image unit level where the current block is located, which is used to indicate whether the image unit where the current block is located uses the illumination compensation mode. The image unit may be any image unit such as an image sequence, a frame, a slice, a coding tree unit, etc. Exemplarily, the second syntax element identifier is a sequence-level syntax element identifier (expressed as sps_lic_enable_flag). If it is determined according to the second syntax element identifier that the current image unit allows the use of the illumination compensation mode, the encoding end attempts to use the illumination compensation mode for prediction, and indicates whether the current block uses the illumination compensation mode through the first syntax element identifier. If it is determined according to the second syntax element identifier that the current image unit does not allow the use of the illumination compensation mode, the encoding end does not attempt to use the illumination compensation mode for prediction.

[0193] In some embodiments, the method further includes: determining the area of ​​the current block; when the area is within a preset area range, determining whether to use the illumination compensation mode based on the illumination compensation mode indication information of the current block; when the area is outside the preset area range, determining that the current block does not use the illumination compensation mode.

[0194] It should be noted that when the area of ​​the current block does not meet the preset area condition, it is determined that the current block does not use the illumination compensation mode. At this time, the encoding end does not attempt to use the illumination compensation mode for prediction; when the area of ​​the current block meets the preset area condition, the encoding end attempts to use the illumination compensation mode for prediction.

[0195] In some embodiments, the preset area range includes: the area of ​​the current block is greater than a first area threshold and less than a second area threshold; or, the preset area range includes: the area of ​​the current block is greater than the first area threshold. Exemplarily, if the area of ​​the current block is greater than threshold 1 (i.e., 32 sample points) and less than threshold 2 (i.e., 256 sample points), the encoder attempts the LIC prediction method. Alternatively, if the area of ​​the current block is greater than threshold 1 (i.e., 32 sample points), the encoder attempts the LIC prediction method.

[0196] It should also be noted that if the area of ​​the current block meets the preset area condition and it is determined according to the second syntax element identifier that the current image unit allows the use of the illumination compensation mode, the encoder attempts to use the illumination compensation mode for prediction.

[0197] In some embodiments, determining illumination compensation mode indication information for the current block includes determining the illumination compensation mode indication information for the current block based on whether the current block uses the illumination compensation mode. Specifically, determining whether the current block uses the illumination compensation mode based on an encoding decision result and encoding illumination compensation mode indication information for instructing a decoder whether to use the illumination compensation mode for the current block to perform illumination compensation on a predicted block.

[0198] In some embodiments, the method further includes: determining prediction mode indication information of the current block based on the prediction mode of the current block; when determining that the prediction mode of the current block is the first type of prediction mode based on the prediction mode indication information, determining illumination compensation mode indication information of the current block based on whether the current block uses an illumination compensation mode; encoding the prediction mode indication information and writing the obtained encoding bits into the bitstream.

[0199] It should be noted that when the prediction mode of the current block is the first type of prediction mode, the illumination compensation mode indication information of the current block is set according to the encoding decision result and encoded. Exemplarily, the first type of prediction mode includes the inter-frame AMVP mode, the normal inter-frame mode, etc.

[0200] In other embodiments, determining illumination compensation mode indication information for a current block includes: determining an index value of motion information for the current block; wherein the index value of the motion information is used to indicate illumination mode compensation information for the current block. It should be noted that the index value of the motion information is used to index the motion information of the current block, thereby determining a reference block for the current block. Since each piece of motion information contains all available technology usage flags, including an illumination compensation usage flag, whether the current block uses illumination compensation mode can be determined by inheriting surrounding information. In some embodiments, the motion information can be an inter-frame motion vector (MV) or an intra-frame block vector (BV).

[0201] In some embodiments, the method further includes: determining prediction mode indication information of the current block based on the prediction mode of the current block; determining an index value of motion information of the current block when the prediction mode of the current block is determined to be the second type of prediction mode based on the prediction mode indication information; using the index value of the motion information as illumination compensation mode indication information of the current block; encoding the prediction mode indication information, and writing the obtained coded bits into the bitstream.

[0202] It should be noted that when the prediction mode of the current block is the second type of prediction mode, the index value of the motion information is encoded as the illumination compensation mode indication information of the current block. The decoder decodes the index value of the motion information, and the current block inherits the illumination compensation mode indication information of the corresponding reference block. Exemplarily, the second type of prediction mode includes the inter-frame merge mode. It should also be noted that the inter-frame merge mode can also determine whether to use LIC rather than inheriting the surrounding information of the reference frame. Instead, the inter-frame merge mode can make an encoding decision, transmitting the LIC use flag and index value to the decoder. The decoder parses the LIC use flag in the merge mode to determine whether to use LIC technology for the current block and parses the index value to determine the motion information.

[0203] S706: Encode the illumination compensation mode indication information, and write the obtained coded bits into the bitstream.

[0204] In some embodiments, the method further includes: determining that the current block does not use an illumination compensation mode, and determining a reconstructed block of the current block based on the first prediction block; determining residual information of the current block based on the original block and the reconstructed block of the current block; encoding the residual information, and writing the obtained coded bits into the bitstream.

[0205] In some embodiments, the method further includes: determining that the current block uses an illumination compensation mode, and determining a reconstructed block of the current block based on the second prediction block; determining residual information of the current block based on the original block and the reconstructed block of the current block; encoding the residual information, and writing the obtained coded bits into the bitstream.

[0206] By adopting the above technical solution, a template area is added to the illumination compensation mode. Different types of template areas contain different samples and different spatial information. The encoding end decides the optimal template area for calculating the model parameters. This template area contains the most useful information and can improve the accuracy of the model parameter calculation, thereby improving the illumination compensation effect and encoding performance.

[0207] Based on the above embodiments, the encoding method provided in the embodiments of the present application is further illustrated.

[0208] In this embodiment, at the encoding end, the encoder traverses the prediction mode. If the current prediction mode type is a prediction mode that can use the illumination compensation mode, a flag that allows the use of the illumination compensation technology is obtained. This flag is a sequence-level flag, indicating that the current encoder allows the use of local illumination compensation technology, which can be in the form of sps_lic_enable_flag.

[0209] 1. If the LIC's use-permit flag is true, and the current block's area is greater than threshold 1, and the current block's area is less than threshold 2, the encoder attempts the LIC prediction method, i.e., executes step 2. If the LIC's use-permit flag or other conditions, such as area, are not met, the encoder does not attempt the LIC prediction method, i.e., skips step 2 and directly executes step 3.

[0210] It should be noted that the usage restrictions of the LIC technology can be modified according to the coupling between the various technologies. For example, in this embodiment, the LIC restriction cannot be used for coding units with a width and height product less than 32, while there is no restriction for large areas.

[0211] 2. Obtain the reference block in the reference frame according to the motion vector MV (or BV), and perform motion compensation on the reference block to obtain the initial prediction block;

[0212] Get the reconstructed samples on the upper and left sides of the current block;

[0213] Obtain reconstructed samples on the upper and left sides of the reference block in the reference frame and perform motion compensation.

[0214] Cost1: Calculate model parameters using the reconstructed samples above and to the left of the current block, as well as the reconstructed samples above and to the left of the reference block. The calculation process is similar to that described in 1.2 above, or the least squares method is used to solve the model parameter scaling parameter a and offset parameter b. In the specific implementation, the shift parameter shift may also be included. These parameters are applied to the reconstructed samples above and to the left of the reference block to obtain the compensated reconstructed samples above and to the left of the reference block. The SAD cost between these reconstructed samples and the reconstructed samples above and to the left of the current block is calculated, denoted as cost1.

[0215] Cost2: Only the reconstructed samples above the current block and the reconstructed samples above the reference block are used to calculate the model parameters. The calculation process is the same as in 1.2 above, or the least squares method is used to solve the model parameter scaling parameter a and offset parameter b. In the specific implementation, the shift parameter shift may also be included. These parameters are applied to the reconstructed samples above and to the left of the reference block to obtain the compensated reconstructed samples above and to the left of the reference block. The SAD cost of these reconstructed samples and the reconstructed samples above and to the left of the current block is calculated, recorded as cost2.

[0216] Cost3: Model parameter calculation is performed using only the reconstructed samples on the left side of the current block and the reconstructed samples on the left side of the reference block. The calculation process is the same as in 1.2 above, or the least squares method is used to solve the model parameter scaling parameter a and offset parameter b. In the specific implementation, the shift parameter shift may also be included. These parameters are applied to the reconstructed samples on the upper and left sides of the reference block to obtain the compensated reconstructed samples on the upper and left sides of the reference block. The SAD cost of these reconstructed samples and the reconstructed samples on the upper and left sides of the current block is calculated, which is recorded as cost3.

[0217] Decision: Compare the SAD cost values ​​for different template regions. The template region with the minimum cost is the region calculated using the illumination compensation model for the current block. The model parameters for the current block are calculated based on the samples in the determined template region. The calculation process is the same as above, resulting in the scaling parameter a, offset parameter b, and shift parameter shift. These parameters are applied to the initial prediction block to obtain the illumination-compensated prediction block. After obtaining the residual with the original block, transforming, and quantizing, the rate-distortion cost is calculated based on the codeword required for the current mode and the distortion between the current reconstructed image and the original image.

[0218] In some embodiments, the cost of not using the illumination compensation mode can also be calculated. If the current inter-frame prediction mode is Merge, the SAD cost value is calculated by reconstructing the upper and left samples of the current block and the upper and left reconstructed samples after motion compensation of the reference block, which is recorded as cost0. When making a decision, the SAD cost values ​​are compared. If cost0 is the minimum cost value, it means that the illumination compensation technology is not applicable to the current block, and the current block does not perform illumination compensation. Otherwise, the template area corresponding to the minimum cost is the area calculated using the illumination compensation model for the current block.

[0219] It should be noted that the template area used to calculate model parameters can be one row of reconstructed samples adjacent to the upper side and one column of reconstructed samples to the left of the current block and its reference block. Alternatively, it can be multiple rows of samples adjacent to the upper side and multiple columns of reconstructed samples to the left of the current block and its reference block. Alternatively, it can be multiple rows of samples non-adjacent to the upper side and multiple columns of reconstructed samples to the left of the current block and its reference block. These are not listed here.

[0220] It should also be noted that the model parameters are applied to the reference block template area to obtain illumination-compensated reconstructed samples, which are then used to calculate the SAD cost compared with the reconstructed samples above and to the left of the current block. The model parameters can also be applied to the first row and first column of the first prediction block after motion compensation to obtain compensated prediction samples in one row and one column, which are then used to calculate the SAD cost compared with the reconstructed samples above and to the left of the current block.

[0221] 3. The encoder continues to traverse other prediction technologies and calculates the rate-distortion cost corresponding to each prediction technology. The prediction mode corresponding to the minimum cost is selected as the optimal prediction mode for the current block.

[0222] If the rate-distortion cost of illumination compensation is the lowest, illumination compensation is used for the current block. If the current prediction mode is inter-frame AMVP, the illumination compensation technology's coding unit-level use flag needs to be set to true and written into the bitstream. If the current prediction mode is inter-frame merge, only the index of the current merge candidate needs to be written into the bitstream.

[0223] It should be noted that for the LIC technology in merge mode, the use of LIC can be determined not by inheriting surrounding information, but by the encoder. The use or non-use situation is transmitted to the decoder in the form of an identification bit and an index. The decoder parses the identification bit used by the technology in merge mode to determine whether to use the LIC technology for the current coding unit.

[0224] In yet another embodiment of the present application, referring to FIG11 , a flowchart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG11 , the method may include:

[0225] S1101: Determine illumination compensation mode indication information of the current block;

[0226] It should be noted that the illumination compensation mode indication information is used to indicate whether the current block uses the illumination compensation mode. In some embodiments, the illumination compensation mode indication information includes a first syntax element identifier; wherein the first syntax element identifier is used to indicate whether the current block uses the illumination compensation mode.

[0227] It should be noted that the current block can be an image block in any illumination compensation mode, and the first syntax element identifier can be a block-level syntax element identifier. The first syntax element identifier can also be a syntax element identifier at the image unit level where the current block is located, used to indicate whether the image unit where the current block is located uses the illumination compensation mode.

[0228] In some embodiments, the illumination compensation mode indication information also includes a second syntax element identifier; wherein the second syntax element identifier includes at least one of the following: a sequence-level syntax element identifier, used to indicate whether the image sequence where the current block is located allows the use of the illumination compensation mode; a frame-level syntax element identifier, used to indicate whether the frame where the current block is located allows the use of the illumination compensation mode; a coding tree unit-level syntax element identifier, used to indicate whether the coding tree unit where the current block is located allows the use of the illumination compensation mode; a slice-level syntax element identifier, used to indicate whether the slice where the current block is located allows the use of the illumination compensation mode.

[0229] It should be noted that the second syntax element identifier may be a syntax element identifier at the image unit level where the current block is located, which is used to indicate whether the image unit where the current block is located uses the illumination compensation mode. The image unit may be any image unit such as an image sequence, a frame, a slice, a coding tree unit, etc. Exemplarily, the second syntax element identifier is a sequence-level syntax element identifier (expressed as sps_lic_enable_flag). If it is determined according to the second syntax element identifier that the current image unit allows the use of the illumination compensation mode, the encoding end attempts to use the illumination compensation mode for prediction, and indicates whether the current block uses the illumination compensation mode through the first syntax element identifier. If it is determined according to the second syntax element identifier that the current image unit does not allow the use of the illumination compensation mode, the encoding end does not attempt to use the illumination compensation mode for prediction.

[0230] In some embodiments, the method further includes: determining the area of ​​the current block; when the area is within a preset area range, determining whether to use the illumination compensation mode based on the illumination compensation mode indication information of the current block; when the area is outside the preset area range, determining that the current block does not use the illumination compensation mode.

[0231] It should be noted that when the area of ​​the current block does not meet the preset area condition, it is determined that the current block does not use the illumination compensation mode. At this time, the encoding end does not attempt to use the illumination compensation mode for prediction; when the area of ​​the current block meets the preset area condition, the encoding end attempts to use the illumination compensation mode for prediction.

[0232] In some embodiments, the preset area range includes: the area of ​​the current block is greater than a first area threshold and less than a second area threshold; or, the preset area range includes: the area of ​​the current block is greater than the first area threshold. Exemplarily, if the area of ​​the current block is greater than threshold 1 (i.e., 32 sample points) and less than threshold 2 (i.e., 256 sample points), the encoder attempts the LIC prediction method. Alternatively, if the area of ​​the current block is greater than threshold 1 (i.e., 32 sample points), the encoder attempts the LIC prediction method.

[0233] It should also be noted that if the area of ​​the current block meets the preset area condition and it is determined according to the second syntax element identifier that the current image unit allows the use of the illumination compensation mode, the encoder attempts to use the illumination compensation mode for prediction.

[0234] In some embodiments, determining illumination compensation mode indication information for the current block includes: decoding a bitstream to determine the illumination compensation mode indication information for the current block. Specifically, the illumination compensation mode indication information for the current block is obtained by decoding the bitstream, thereby determining whether the current block uses the illumination compensation mode to perform illumination compensation on the predicted block.

[0235] In some embodiments, the method specifically includes: decoding a code stream to determine prediction mode indication information of a current block; when determining that the prediction mode of the current block is a first type of prediction mode according to the prediction mode indication information, decoding the code stream to determine illumination compensation mode indication information of the current block.

[0236] It should be noted that when the prediction mode of the current block is the first type of prediction mode, the illumination compensation mode indication information of the current block is obtained by decoding the bitstream. Exemplarily, the first type of prediction mode includes the inter-frame AMVP mode, the normal inter-frame mode, etc.

[0237] In other embodiments, determining the illumination compensation mode indication information of the current block includes: determining an index value of the motion information of the current block; determining the illumination mode indication information of a reference block of the current block based on the index value of the motion information; and using the illumination mode indication information of the reference block as the illumination compensation mode indication information of the current block. It should be noted that the index value of the motion information is used to index the motion information of the current block, thereby determining the reference block of the current block. Since each motion information contains all available technology usage flags, including an illumination compensation usage flag, it is possible to determine whether the current block uses the illumination compensation mode by inheriting surrounding information. In some embodiments, the motion information can be an inter-frame motion vector (MV) or an intra-frame block vector (BV).

[0238] In some embodiments, determining the index value of the motion information of the current block specifically includes: decoding the code stream to determine the prediction mode indication information of the current block; when determining that the prediction mode of the current block is the second type of prediction mode according to the prediction mode indication information, decoding the code stream to determine the index value of the motion information of the current block.

[0239] It should be noted that when the prediction mode of the current block is the second type of prediction mode, the decoder decodes the index value of the motion information, and the current block inherits the illumination compensation mode indication information of the corresponding reference block. Exemplarily, the second type of prediction mode includes the inter-frame merge mode. It should also be noted that the inter-frame merge mode can also determine whether to use LIC rather than inheriting surrounding information from the reference frame. Instead, the inter-frame merge mode can make an encoding decision, transmitting the LIC use flag and index value to the decoder. The decoder then parses the LIC use flag in the merge mode to determine whether to use LIC technology for the current block and parses the index value to determine the motion information.

[0240] In some other embodiments, determining the index value of the motion information of the current block specifically includes: constructing a motion information candidate list, and determining the index value of the first motion information in the motion information candidate list.

[0241] S1102: When it is determined according to the illumination compensation mode indication information that the current block uses the illumination compensation mode, determine a plurality of candidate template regions used by the illumination compensation mode;

[0242] In some embodiments, determining a plurality of candidate template regions used in the illumination compensation mode includes: determining a plurality of candidate template regions based on at least two reconstructed sub-template regions.

[0243] It should be noted that a sub-template region is a reconstructed region adjacent or non-adjacent to the current block and its reference block. In some embodiments, the sub-template region includes at least one of the following: an upper sub-template region, an upper-right sub-template region, a left sub-template region, a lower-left sub-template region, and an upper-left sub-template region. A candidate template region is obtained by combining one or more sub-template regions, and a target template region for the current block is further determined from the candidate template regions.

[0244] In some embodiments, the sub-template region includes an upper sub-template region and a left sub-template region. Based on the at least two reconstructed sub-template regions, multiple candidate template regions are determined, including: determining a first candidate template region based on the upper sub-template region and the left sub-template region; determining a second candidate template region based on the left sub-template region; and determining a third candidate template region based on the upper sub-template region, as shown in Figures 8A to 8C.

[0245] In other embodiments, the sub-template region further includes: an upper right sub-template region and a lower left sub-template region. A fourth candidate template region is determined based on the upper and upper right sub-template regions; a fifth candidate template region is determined based on the left and lower left sub-template regions, as shown in Figures 8D and 8E.

[0246] It's important to note that a template region composed of two or more sub-template regions has more samples. In some cases, this allows for more sample selection when calculating model parameters than using only the upper or left template regions, potentially leading to higher model accuracy. Therefore, model parameter calculation accuracy can be improved by 1. increasing the number of samples used in model parameter calculation, or 2. expanding the template region to obtain more extensive spatial information.

[0247] Regarding point 1, we can use the sampling method and reduce the sampling step size to increase the number of samples; or even select all available samples in the template area to calculate the linear model parameters.

[0248] Regarding point 2, the template area can be directly expanded. For example, for the upper template area, the template height is kept unchanged and the width of the template area is increased; for the left template area, the template width is kept unchanged and the height of the template area is increased. Figure 8D shows the expansion of the upper template area; Figure 8E shows the expansion of the left template area.

[0249] It should also be noted that in other cases, when using a template area composed of more than two sub-template areas, too many samples will interfere with the accuracy of the model. At this time, the accuracy of model parameter calculation can be improved by limiting the spatial information by reducing the number of samples for model parameter calculation or reducing the area of ​​the template area.

[0250] In an embodiment of the present application, by setting different types of template areas to contain different samples and different spatial information, the codec simultaneously decides the optimal template area for calculating the model parameters. This template area contains the most useful information required by the current block, which can improve the accuracy of the model parameter calculation.

[0251] It should be noted that this is just an illustrative example of the combination of sub-template areas. In addition to the above five candidate template areas, different numbers and types of sub-template areas can be combined to form candidate template areas of more shapes, which will not be repeated here.

[0252] In some embodiments, the method further includes: determining the height of the candidate template region based on the height of the current block; and determining the width of the candidate template region based on the width of the current block. For example, as shown in FIG9 , the height H1 of the current block is correlated with the height H2 of the upper sub-template region, and the width W1 of the current block is correlated with the width W2 of the left sub-template region. Specifically, the correlation may be positive: the higher the height of the current block, the higher the height of the upper sub-template region, and the wider the width of the current block, the wider the width of the left sub-template region.

[0253] In some embodiments, the method further includes: determining the height of the candidate template area as a preset first value; determining the width of the candidate template area as a preset second value; wherein the first value is greater than or equal to 1, and the second value is greater than or equal to 1. It should be noted that the size of the candidate template area can also be a preset size, and the first value and the second value represent the number of sample points. Exemplarily, the first value and the second value are 1, indicating that the template area includes a row of samples adjacent to the upper side and a column of samples adjacent to the left side of the current block and the reference block. The first value and the second value are 2, indicating that the template area includes two rows of samples adjacent to the upper side and two columns of samples adjacent to the left side of the current block and the reference block.

[0254] In some embodiments, the method further includes: when the height of the current block is within a first height range, determining the height of the candidate template area as a first candidate height value; based on the height of the current block being within a second height range, determining the height of the candidate template area as a second candidate height value; when the width of the current block is within a first width range, determining the width of the candidate template area as a first candidate width value; based on the width of the current block being within a second width range, determining the width of the candidate template area as a second candidate width value.

[0255] In some embodiments, the method further includes: detecting whether the first sub-template area of ​​the current block and the first sub-template area of ​​the reference block are both available; if the first sub-template area of ​​the current block and the first sub-template area of ​​the reference block are both available, using the first sub-template area to determine the candidate template area; if the first sub-template area of ​​the current block and / or the first sub-template area of ​​the reference block are not available, ignoring the first sub-template area.

[0256] It should be noted that the selection of candidate template regions can include boundary checks. For sub-template regions where samples are not available, the sub-regions are not used to calculate the model parameters. For example, for the left boundary of the image, the left sub-template region, the lower left sub-template region, and the upper left sub-template region are all unavailable. For the upper boundary of the image, the upper sub-template region, the upper right sub-template region, and the upper left sub-template region are all unavailable.

[0257] S1103: Calculate the cost of the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value;

[0258] In an embodiment of the present application, the cost function used for cost calculation can be the sum of absolute error SAD, the sum of absolute transformation difference SATD, the mean square error MSE, the sum of squared errors SSD, the mean absolute difference MAD, the mean sum of squared errors MSD, the absolute value DCT of the transformation coefficient, the normalized correlation coefficient NCC, etc., which are not specifically limited here.

[0259] In some embodiments, a cost calculation is performed on the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value, including: determining the current block template area and the reference block template area based on the candidate template area; determining the candidate illumination compensation model parameters corresponding to the candidate template area based on the first reconstructed sample of the current block template area and the second reconstructed sample of the reference block template area; determining the first prediction sample of the current block template area based on the candidate illumination compensation model parameters; performing a cost calculation based on the third reconstructed sample and the first prediction sample of the current block template area to determine the first generation value of the candidate template area; and determining the target template area corresponding to the minimum cost value based on the first generation value.

[0260] In some embodiments, the illumination model is a linear model constructed based on a linear function, and the illumination compensation model parameters include at least one of the following: a scaling parameter, an offset parameter, and a shift parameter. In other embodiments, the illumination model is a nonlinear model constructed based on a nonlinear function, and the model parameters include any parabolic equation parameter and a shift parameter.

[0261] In some embodiments, the reconstructed samples of the reference block template area are motion-compensated reconstructed samples or samples that have not been motion-compensated. For example, the motion compensation may be interpolation filtering.

[0262] In some embodiments, determining a first prediction sample for a current block template region based on candidate illumination compensation model parameters includes applying the candidate illumination compensation model parameters to a fourth reconstructed sample of a reference block template region to determine the first prediction sample for the current block template region. Specifically, an illumination compensation model is constructed based on the candidate illumination compensation model parameters, the fourth reconstructed sample is input into the illumination compensation model for illumination compensation, and the first prediction sample is output.

[0263] It should be noted that the first reconstructed sample is a reconstructed sample in the current block template region used to calculate the model parameters, the second reconstructed sample is a reconstructed sample in the reference block template region used to calculate the model parameters, the third reconstructed sample is a reconstructed sample in the current block template region used to calculate the first cost value, and the fourth reconstructed sample is a reconstructed sample in the reference block template region used to calculate the first prediction sample. The first reconstructed sample and the second reconstructed sample can be the same sample in the current block template region or different samples. The second reconstructed sample and the fourth reconstructed sample can be the same sample in the reference block template region or different samples.

[0264] In some embodiments, the first template area of ​​the current block template area is sampled to determine a first reconstructed sample; the first template area of ​​the reference block template area is sampled to determine a second reconstructed sample; the second template area of ​​the current block template area is sampled to determine a third reconstructed sample; and the second template area of ​​the reference block template area is sampled to determine a fourth reconstructed sample.

[0265] In some embodiments, partial reconstructed samples are collected in the first template area and the second template area according to a preset sampling step size. The sampling step size may be related to the template size. For example, if the width and height of the current block are both equal to 4, 4 reconstructed samples are taken from the upper and left adjacent reconstructed samples of the coding unit. For example, if the current block width is 16 and the height is 4, all 4 reconstructed samples are taken from the left adjacent reconstructed samples, and 4 reconstructed samples are taken from the upper adjacent reconstructed samples with a step size of 3. If the width and height of the current block are not equal to 4, pixel samples with a number equal to the logarithm of 2 of the smaller side length are obtained from the upper and left adjacent reconstructed samples.

[0266] In some other embodiments, all reconstruction samples are collected in the first template area and the second template area.

[0267] In some embodiments, the first template area is not adjacent to the current block and its reference block, and the second template area is adjacent to the current block and its reference block; wherein the first template area and the second template area do not overlap with each other; or, the first template area and the second template area partially overlap.

[0268] It should be noted that the reconstructed samples used to calculate the model parameters are obtained from the first template region, and the reconstructed samples used to calculate the cost value are obtained from the second template region. When the first template region and the second template region are different regions, it can be understood that the template prediction method can be the same as the prediction method for the current block, that is, the model parameters are calculated using the second template region, and the predicted value is calculated using the first template region. The advantage of this is that the true predicted value of the model parameter can be better made on the template region.

[0269] Figure 10 is a schematic diagram of the template area of ​​the reference frame and the current frame in an embodiment of the present application. The template area includes two rows of reconstructed samples on the upper side and two columns of reconstructed samples on the left side of the current block and the reference block. The first template area is the first row and the first column, and the second template area is the second row and the second column. When making a decision, the reconstructed samples in the first row and the first column of the template are used to calculate the model parameters, and the model parameters are applied to the second row and the second column of the reference block template area to obtain the predicted samples in the second row and the second column of the current block template area. The cost is calculated based on the reconstructed samples and predicted samples in the second row and the second column of the current block template area, and the optimal template area is decided.

[0270] When applying, the second row and second column samples of the template area are used to calculate model parameters, and the model parameters are applied to the first prediction block to obtain a second prediction block after illumination compensation.

[0271] It should be noted that the template area used to calculate model parameters can be a row of samples adjacent to the top and a column of reconstructed samples adjacent to the left of the current block and its reference block. Alternatively, it can be multiple rows of samples adjacent to the top and multiple columns of reconstructed samples adjacent to the left of the current block and its reference block. Alternatively, it can be one or more rows of samples adjacent to the top and a row or more columns of reconstructed samples adjacent to the left of the current block and its reference block. These are not listed here.

[0272] In some embodiments, determining the first prediction sample of the current block template region based on candidate illumination compensation model parameters includes: applying the candidate illumination compensation model parameters to the prediction samples of the first prediction block to determine the first prediction sample of the current block template region. Specifically, an illumination compensation model is constructed based on the candidate illumination compensation model parameters, the prediction samples of the first prediction block are input into the illumination compensation model for illumination compensation, and the first prediction sample is output. In other words, the model parameters can also be applied to the prediction samples of the first prediction block to obtain the first prediction samples of the current block template region, and these prediction samples are used to calculate cost values ​​with the reconstructed samples of the current block template region. Specifically, the model parameters can be applied to the prediction samples of the first row and first column of the first prediction block, or to the prediction samples of the second row and second column of the first prediction block.

[0273] In some embodiments, the method further includes: determining a second prediction sample of the current block template area based on the reconstructed sample of the reference block template area; performing cost calculation based on the reconstructed sample and the second prediction sample of the current template area to determine a second generation value of the candidate template area; if the minimum cost value is the first generation value, determining that the current block uses an illumination compensation mode; if the minimum cost value is the second generation value, determining that the current block does not use an illumination compensation mode.

[0274] For example, the reconstructed samples of the reference block template area are used as the second prediction samples, or the reconstructed samples of the reference block template area are motion compensated to obtain the second prediction samples. The second generation value can be understood as the cost value corresponding to not using the illumination compensation mode for the current block. The minimum cost value is the first generation value, indicating that the illumination compensation mode is used, and the minimum cost value is the second generation value, indicating that the illumination compensation mode is not used.

[0275] In some embodiments, the method further includes: if the prediction mode of the current block is determined to be the merge prediction mode, determining a second prediction sample of the current block template region based on the reconstructed sample of the reference block template region. In other words, when the prediction mode of the current block is the merge prediction mode, the second generation value is calculated based on the reconstructed sample of the current block template region and the reconstructed sample of the reference block template region.

[0276] It should be noted that in actual applications, the second generation value corresponding to each candidate template region can be calculated, or only the second generation value corresponding to a specific template region can be calculated. For example, the specific template region can include the upper sub-template region and the left template region.

[0277] It should also be noted that the second generation value can be calculated based on all the reconstructed samples in the template area, or it can be calculated based on part of the reconstructed samples collected in the template area according to a preset sampling step.

[0278] S1104: Determine target illumination compensation model parameters based on the target template area;

[0279] It should be noted that the target illumination compensation model parameters are determined by calculating the model parameters based on the reconstructed samples of the current block target template area and the reconstructed samples of the reference block target template area.

[0280] In some embodiments, the target illumination compensation model parameters are determined based on the target template area, including: determining the current block template area and the reference block template area based on the target template area; and determining the target illumination compensation model parameters corresponding to the target template area based on the third reconstructed sample of the current block template area and the fourth reconstructed sample of the reference block template area.

[0281] Specifically, the second template area of ​​the current block template area is sampled to determine the third reconstructed sample; and the second template area of ​​the reference block template area is sampled to determine the fourth reconstructed sample.

[0282] In some embodiments, the illumination model is a linear model constructed based on a linear function, and the illumination compensation model parameters include at least one of the following: a scaling parameter, an offset parameter, and a shift parameter. In other embodiments, the illumination model is a nonlinear model constructed based on a nonlinear function, and the model parameters include any parabolic equation parameter and a shift parameter.

[0283] In other embodiments, candidate illumination compensation model parameters obtained for the target template region during decision-making are obtained as target illumination compensation model parameters. It should be noted that by using the same reconstructed samples to calculate model parameters during decision-making and application, the candidate illumination parameters obtained for the target template region during decision-making can be directly obtained without recalculating the model parameters, thereby improving decoding efficiency.

[0284] S1105: Perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine a second prediction block of the current block.

[0285] In some embodiments, the method further includes: decoding the code stream to determine prediction mode indication information of the current block; determining a target prediction mode of the current block based on the prediction mode indication information; determining a reference block of the current block in a reference frame based on the target prediction mode; and determining a first prediction block based on the reference block.

[0286] In some embodiments, determining a first prediction block based on a reference block includes: if the target prediction mode is an intra block copy mode, performing block copying on the reference block to determine the first prediction block; if the target prediction mode is an inter prediction mode, performing motion compensation on the reference block to determine the first prediction block.

[0287] Furthermore, the method further includes: decoding the bitstream to determine residual information for the current block; and determining a reconstructed block for the current block based on the residual information and the second prediction block. It should be noted that if other techniques still need to be performed on the current block, the other techniques are continued to be performed on the second prediction block to obtain a final prediction block. Otherwise, the second prediction block after illumination compensation is the final prediction block, the bitstream is parsed to obtain residual information, and time-domain residual information is obtained through inverse quantization and inverse transformation. The final prediction block is superimposed with the time-domain residual information to obtain a reconstructed block.

[0288] In some embodiments, the method further includes: determining that the current block does not use an illumination compensation mode, and determining a reconstructed block for the current block based on the first prediction block and the residual information. It should be noted that if other techniques still need to be performed on the current block, the operations of the other techniques are continued on the first prediction block to obtain a final prediction block; otherwise, the first prediction block is the final prediction block, the bitstream is parsed to obtain residual information, and time domain residual information is obtained through inverse quantization and inverse transformation. The final prediction block is superimposed with the time domain residual information to obtain a reconstructed block.

[0289] By adopting the above technical solution, a template area is added to the illumination compensation mode. Different types of template areas contain different samples and different spatial information. The decoding end decides the optimal template area for calculating the model parameters. This template area contains the most useful information and can improve the accuracy of the model parameter calculation, thereby improving the illumination compensation effect and decoding performance.

[0290] Based on the above embodiments, the decoding method provided in the embodiments of the present application is further illustrated.

[0291] In this embodiment, at the decoding end, the decoding end parses or obtains a LIC use permission flag, which is a sequence level flag (sps_lic_enable_flag), indicating that the current decoder allows the use of the LIC technology.

[0292] 1. Parse the code stream to obtain the prediction type of the current block. If it is inter-frame prediction mode, parse the merge usage flag of the current block.

[0293] If the merge flag for the current block is true, the merge candidate list index corresponding to the current block is parsed. The merge candidate list is constructed, and motion vector information is obtained from adjacent, non-adjacent, historical, and temporal coding blocks. All available technology usage flags for each merge candidate are also obtained, including the illumination compensation usage flag.

[0294] If the merge usage flag of the current block is not true, and sps_lic_enable_flag is true, and the area of ​​the current block is greater than threshold 1, and the area of ​​the current block is less than threshold 2, then the code stream is parsed to obtain the usage flag (cu_lic_flag) of the LIC of the current block.

[0295] If the cu_lic_flag usage flag of the current block is no, go to step 3.

[0296] It should be noted that for the LIC technology in merge mode, the use of LIC can be determined not by inheriting surrounding information, but by the encoder. The use or non-use situation is transmitted to the decoder in the form of an identification bit and an index. The decoder parses the identification bit used by the technology in merge mode to determine whether to use the LIC technology for the current coding unit.

[0297] 2. Obtain the reference block in the reference frame based on the motion vector, and perform motion compensation to obtain the initial prediction block.

[0298] Get the reconstructed samples above and to the left of the current block.

[0299] Obtain reconstructed samples on the upper and left sides of the reference block in the reference frame and perform motion compensation.

[0300] Cost1: Model parameters are calculated using the reconstructed samples above and to the left of the current block and the reconstructed samples above and to the left of the reference block. The calculation process is similar to that described in 1.2 above, or the least squares method is used to solve the model parameters such as the scaling parameter a and the offset parameter b. In the specific implementation, the shift parameter shift may also be included. These parameters are applied to the reconstructed samples above and to the left of the reference block to obtain the compensated reconstructed samples above and to the left of the reference block. The SAD cost between these reconstructed samples and the reconstructed samples above and to the left of the current block is calculated, denoted as cost1.

[0301] Cost2: Model parameter calculation is performed using only the reconstructed samples above the current block and the reconstructed samples above the reference block. The calculation process is similar to that described in 1.2 above, or the least squares method is used to solve the model parameters such as the scaling parameter a and the offset parameter b. In the specific implementation, the shift parameter shift may also be included. These parameters are applied to the reconstructed samples above and to the left of the reference block to obtain the compensated reconstructed samples above and to the left of the reference block. The SAD cost between these reconstructed samples and the reconstructed samples above and to the left of the current block is calculated, denoted as cost2.

[0302] Cost3: Model parameter calculation is performed using only the reconstructed samples to the left of the current block and the reconstructed samples to the left of the reference block. The calculation process is similar to that described in 1.2 above, or the least squares method is used to solve the model parameters such as the scaling parameter a and the offset parameter b. In the specific implementation, the shift parameter shift may also be included. These parameters are applied to the reconstructed samples above and to the left of the reference block to obtain the compensated reconstructed samples above and to the left of the reference block. The SAD cost between these reconstructed samples and the reconstructed samples above and to the left of the current block is calculated, denoted as cost3.

[0303] Decision: Compare the SAD cost values ​​under different template regions. The template region corresponding to the minimum cost is the region of the current block calculated using the illumination compensation model. The model parameters of the current block are calculated based on the samples of the determined template region. The calculation process is the same as above to obtain the scaling parameter a, offset parameter b, and shift parameter shift. These parameters are applied to the initial prediction block to obtain the prediction block after illumination compensation.

[0304] In some embodiments, the cost of not using the illumination compensation mode can also be calculated. If the current inter-frame prediction mode is Merge, the SAD cost value is calculated by reconstructing the upper and left samples of the current block and the upper and left reconstructed samples after motion compensation of the reference block, which is recorded as cost0. When making a decision, the SAD cost values ​​are compared. If cost0 is the minimum cost value, it means that the illumination compensation technology is not applicable to the current block, and the current block does not perform illumination compensation. Otherwise, the template area corresponding to the minimum cost is the area calculated using the illumination compensation model for the current block.

[0305] It should be noted that the template area used to calculate model parameters can be one row of reconstructed samples adjacent to the upper side and one column of reconstructed samples to the left of the current block and its reference block. Alternatively, it can be multiple rows of samples adjacent to the upper side and multiple columns of reconstructed samples to the left of the current block and its reference block. Alternatively, it can be multiple rows of samples non-adjacent to the upper side and multiple columns of reconstructed samples to the left of the current block and its reference block. These are not listed here.

[0306] It should also be noted that the model parameters are applied to the reference block template area to obtain illumination-compensated reconstructed samples, which are then used to calculate the SAD cost compared with the reconstructed samples above and to the left of the current block. The model parameters can also be applied to the first row and first column of the first prediction block after motion compensation to obtain compensated prediction samples in one row and one column, which are then used to calculate the SAD cost compared with the reconstructed samples above and to the left of the current block.

[0307] 3. If there are other techniques to be performed on the current block, continue the operations of the other techniques and obtain the final prediction block; otherwise, the prediction block after illumination compensation is the final prediction block;

[0308] 4. Parse the bitstream and obtain the residual information, and obtain the time domain residual information through inverse quantization and inverse transformation. The final prediction block is superimposed with the time domain residual information to obtain the reconstructed sample block;

[0309] 5. After all reconstructed sample blocks are processed through loop filtering and other technologies, the final reconstructed image is obtained, which can be used as both video output and as a reference for subsequent decoding.

[0310] The coding and decoding method provided in the embodiment of the present application is applied to the inter-frame prediction part of the coding and decoding end. After integrating the above embodiment (excluding the calculation of cost0) into the latest ECM10.0, the test results under the test conditions of random access (RA) and low delay B (LDB) are as follows.

[0311] Table 1

[0312] Table 2

[0313] Negative numbers represent performance gains, meaning fewer bits are used for the same quality. Simulation results show that this technology achieves virtually no performance improvement under the random access (RA) test condition, but exhibits significant compression performance gains under the low-latency (LDB) test condition. Since only classes B, C, and E are included in the LDB test results, the current results indicate a compression efficiency improvement of approximately 0.1% for the luminance component.

[0314] In summary, the embodiments of the present application propose to simultaneously decide on the template area used to calculate the model parameters at the codec end; propose to expand the template area and add other template area modes to the LIC technology to calculate the model parameters. Different template areas can bring more samples and more spatial information to participate in the calculation of model parameters.

[0315] An embodiment of the present application also provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: illumination compensation mode indication information of the current block, prediction mode indication information, index value of motion information, and residual information.

[0316] In yet another embodiment of the present application, based on the same inventive concept as the previous embodiment, see Figure 12, which shows a schematic diagram of the composition structure of an encoder provided in an embodiment of the present application. As shown in Figure 12, the encoder 120 may include a first determination unit 1201, a second determination unit 1202, and an encoding unit 1203; wherein:

[0317] A first determining unit 1201 is configured to determine a plurality of candidate template regions used in an illumination compensation mode;

[0318] The first determining unit 1201 is configured to perform cost calculation on the candidate template region based on template matching, and determine the target template region corresponding to the minimum cost value;

[0319] The first determining unit 1201 is configured to determine target illumination compensation model parameters according to the target template area;

[0320] The second determining unit 1202 is configured to perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block;

[0321] The second determining unit 1202 is configured to make an encoding decision on the illumination compensation mode according to the second prediction block, and determine illumination compensation mode indication information of the current block;

[0322] The encoding unit 1203 is configured to encode the illumination compensation mode indication information and write the obtained encoding bits into the bit stream.

[0323] It can be understood that each functional unit of the encoder also executes the encoding method of any one of the aforementioned embodiments.

[0324] It is understood that in the embodiments of the present application, a "unit" may be a portion of a circuit, a portion of a processor, a portion of a program or software, and so forth. It may also be a module or a non-modular system. Furthermore, the various components in this embodiment may be integrated into a single processing unit, each unit may exist physically as a separate unit, or two or more units may be integrated into a single unit. These integrated units may be implemented in either hardware or software functional modules.

[0325] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0326] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 120. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method of any one of the aforementioned embodiments.

[0327] The present application also provides a computer-readable storage medium that stores a bitstream generated by the encoding method of any of the aforementioned embodiments. The bitstream is generated by bit-coding information to be encoded; the information to be encoded includes at least one of the following: illumination compensation mode indication information for the current block, prediction mode indication information, motion information index value, residual information, etc.

[0328] Based on the composition of the encoder 120 and the computer-readable storage medium, refer to Figure 13, which shows a specific hardware structure diagram of the encoder 120 provided in an embodiment of the present application. As shown in Figure 13, the encoder 120 may include: a first communication interface 1301, a first memory 1302 and a first processor 1303; each component is coupled together through a first bus system 1304. It can be understood that the first bus system 1304 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 1304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 1304 in Figure 13. Among them,

[0329] The first communication interface 1301 is used to receive and send signals when sending and receiving information with other external network elements;

[0330] A first memory 1302 is used to store computer programs that can be run on the first processor 1303;

[0331] The first processor 1303 is configured to, when running the computer program, execute:

[0332] Determine multiple candidate template areas used in the illumination compensation mode;

[0333] Calculate the cost of the candidate template area based on template matching and determine the target template area corresponding to the minimum cost value;

[0334] Determine target illumination compensation model parameters according to the target template area;

[0335] Performing illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block;

[0336] Performing encoding decision on the illumination compensation mode according to the second prediction block to determine illumination compensation mode indication information of the current block;

[0337] The illumination compensation mode indication information is encoded, and the obtained encoding bits are written into the bit stream.

[0338] It is understood that the first memory 1302 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 1302 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0339] The first processor 1303 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 1303. The above-mentioned first processor 1303 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 1302 , and the first processor 1303 reads the information in the first memory 1302 and completes the steps of the above method in combination with its hardware.

[0340] It is understood that the embodiments described herein can be implemented with hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of the present application or a combination thereof. For software implementation, the technology of the present application can be implemented by a module (such as a process, a function, etc.) that performs the functions of the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0341] Optionally, as another embodiment, the first processor 1303 is further configured to execute any one of the methods in the foregoing embodiments when running a computer program.

[0342] This embodiment provides an encoder in which a template area is added for the illumination compensation mode. Different types of template areas contain different samples and different spatial information. The codec simultaneously determines the optimal template area for calculating the model parameters. This template area contains the most useful information and can improve the accuracy of the model parameter calculation, thereby improving the illumination compensation effect and encoding performance.

[0343] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, FIG14 is a schematic diagram showing the structure of a decoder 140 provided in an embodiment of the present application. As shown in FIG14 , the decoder 140 may include: a decoding unit 1401, a third determining unit 1402, and a fourth determining unit 1403; wherein:

[0344] The decoding unit 1401 is configured to determine illumination compensation mode indication information of a current block;

[0345] The third determining unit 1402 is configured to determine a plurality of candidate template regions used by the illumination compensation mode when it is determined according to the illumination compensation mode indication information that the current block uses the illumination compensation mode;

[0346] The third determining unit 1402 is configured to perform cost calculation on the candidate template region based on template matching, and determine the target template region corresponding to the minimum cost value;

[0347] The fourth determining unit 1403 is configured to determine target illumination compensation model parameters according to the target template area;

[0348] The fourth determining unit 1403 is configured to perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block.

[0349] It can be understood that each functional unit of the decoder also executes the decoding method of any one of the aforementioned embodiments.

[0350] Based on the composition of the decoder 140 and the computer-readable storage medium, refer to Figure 15, which shows a specific hardware structure diagram of the decoder 140 provided in an embodiment of the present application. As shown in Figure 15, the decoder 140 may include: a second communication interface 1501, a second memory 1502 and a second processor 1503; each component is coupled together through a second bus system 1504. It can be understood that the second bus system 1504 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 1504 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 1504 in Figure 15. Among them,

[0351] The second communication interface 1501 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0352] The second memory 1502 is used to store computer programs that can be run on the second processor 1503;

[0353] The second processor 1503 is configured to, when running the computer program, execute:

[0354] Determine illumination compensation mode indication information of the current block;

[0355] In a case where it is determined according to the illumination compensation mode indication information that the current block uses the illumination compensation mode, determining a plurality of candidate template regions used by the illumination compensation mode;

[0356] Calculate the cost of the candidate template area based on template matching and determine the target template area corresponding to the minimum cost value;

[0357] Determine target illumination compensation model parameters according to the target template area;

[0358] Illumination compensation is performed on the first prediction block of the current block according to the target illumination compensation model parameters to determine a second prediction block of the current block.

[0359] Optionally, as another embodiment, the second processor 1503 is further configured to execute any one of the methods in the foregoing embodiments when running a computer program.

[0360] It can be understood that the hardware functions of the second memory 1502 are similar to those of the first memory 1302, and the hardware functions of the second processor 1503 are similar to those of the first processor 1303; they will not be described in detail here.

[0361] This embodiment provides a decoder in which a template area is added for the illumination compensation mode. Different types of template areas contain different samples and different spatial information. The codec simultaneously determines the optimal template area for calculating the model parameters. This template area contains the most useful information and can improve the accuracy of the model parameter calculation, thereby improving the illumination compensation effect and decoding performance.

[0362] In yet another embodiment of the present application, referring to FIG16 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG16 , the coding and decoding system 160 may include an encoder 1601 and a decoder 1602 .

[0363] In the embodiment of the present application, the encoder 1601 may be the encoder of any one of the aforementioned embodiments, and the decoder 1602 may be the decoder of any one of the aforementioned embodiments.

[0364] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0365] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0366] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined, if they do not conflict, to obtain new method embodiments. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined, if they do not conflict, to obtain new product embodiments. The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined, if they do not conflict, to obtain new method embodiments or device embodiments.

[0367] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0368] In an embodiment of the present application, a coding and decoding method, a bitstream, an encoder, a decoder, and a storage medium are provided. At the coding and decoding end, multiple candidate template regions are determined for use in the illumination compensation mode of the current block; a cost calculation is performed on the multiple candidate template regions based on template matching to determine a target template region corresponding to the minimum cost value; target illumination compensation model parameters are determined based on the target template region; illumination compensation is performed on the first prediction block of the current block based on the target illumination compensation model parameters to determine a second prediction block of the current block. In this way, template regions are added to the illumination compensation mode. Different types of template regions contain different samples and different spatial information. The coding and decoding end simultaneously determines the optimal template region for calculating the model parameters. This template region contains the most useful information, which can improve the accuracy of the model parameter calculation, thereby improving the illumination compensation effect and encoding and decoding performance.

Claims

1. A decoding method, applied to a decoder, the method comprising: Determine illumination compensation mode indication information of the current block; In a case where it is determined according to the illumination compensation mode indication information that the current block uses the illumination compensation mode, determining a plurality of candidate template regions used by the illumination compensation mode; Performing cost calculation on the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value; Determining target illumination compensation model parameters according to the target template area; Perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block.

2. The method according to claim 1, wherein: The determining of the plurality of candidate template areas used by the illumination compensation mode comprises: The plurality of candidate template regions are determined according to the at least two reconstructed sub-template regions.

3. The method according to claim 2, wherein: The method further comprises: Detecting whether the first sub-template area of ​​the current block and the first sub-template area of ​​the reference block are both available; If the first sub-template region of the current block and the first sub-template region of the reference block are both available, using the first sub-template region to determine the candidate template region; If the first sub-template region of the current block and / or the first sub-template region of the reference block is unavailable, the first sub-template region is ignored.

4. The method according to claim 2, wherein: The sub-template area includes at least one of the following: an upper sub-template area, an upper right sub-template area, a left sub-template area, a lower left sub-template area and an upper left sub-template area.

5. The method according to claim 4, wherein: The step of determining the plurality of candidate template regions according to the at least two reconstructed sub-template regions comprises: Determine a first candidate template region according to the upper sub-template region and the left sub-template region; Determine a second candidate template region according to the left sub-template region; A third candidate template region is determined according to the upper sub-template region.

6. The method according to claim 2, wherein: The method further comprises: Determine the height of the candidate template area according to the height of the current block; The width of the candidate template region is determined according to the width of the current block.

7. The method according to claim 2, wherein: The method further comprises: Determine that the height of the candidate template area is a preset first value; Determine the width of the candidate template area as a preset second value; The first value is greater than or equal to 1, and the second value is greater than or equal to 1.

8. The method according to claim 1, wherein: The cost calculation of the candidate template region based on template matching to determine the target template region corresponding to the minimum cost value includes: Determine a current block template region and a reference block template region according to the candidate template region; Determining candidate illumination compensation model parameters corresponding to the candidate template region according to the first reconstructed sample of the current block template region and the second reconstructed sample of the reference block template region; Determining a first prediction sample of the current block template area according to the candidate illumination compensation model parameters; Performing cost calculation based on the third reconstructed sample of the current block template area and the first predicted sample to determine the first generation value of the candidate template area; The target template region corresponding to the minimum cost value is determined according to the first cost value.

9. The method according to claim 8, wherein: The reconstructed samples of the reference block template area are reconstructed samples after motion compensation.

10. The method according to claim 8, wherein: The determining, according to the candidate illumination compensation model parameters, a first prediction sample of the current block template area comprises: Applying the candidate illumination compensation model parameters to the fourth reconstructed sample of the reference block template region to obtain the first predicted sample of the current block template region.

11. The method according to claim 10, wherein: The method further comprises: Sampling a first template area of ​​the current block template area to determine the first reconstructed sample; Sampling a first template area of ​​the reference block template area to determine the second reconstructed sample; Sampling a second template area of ​​the current block template area to determine the third reconstructed sample; The second template area of ​​the reference block template area is sampled to determine the fourth reconstructed sample.

12. The method according to claim 11, wherein: The first template area is not adjacent to the current block and its reference block, and the second template area is adjacent to the current block and its reference block; wherein, The first template area and the second template area do not overlap each other; or, the first template area and the second template area partially overlap each other.

13. The method according to claim 8, wherein: The determining, according to the candidate illumination compensation model parameters, a first prediction sample of the current block template area comprises: Applying the candidate illumination compensation model parameters to the prediction samples of the first prediction block to obtain the first prediction samples of the current block template area.

14. The method according to claim 8, wherein: The method further comprises: Determining a second predicted sample of the current block template region according to the reconstructed sample of the reference block template region; Performing cost calculation based on the reconstructed samples of the current template area and the second predicted samples to determine the second generation value of the candidate template area; If the minimum cost value is the first generation value, determining that the current block uses the illumination compensation mode; If the minimum cost value is the second generation value, it is determined that the current block does not use the illumination compensation mode.

15. The method according to claim 14, wherein: The method further comprises: If it is determined that the prediction mode of the current block is the merge prediction mode, a second prediction sample of the current block template area is determined according to the reconstructed sample of the reference block template area.

16. The method according to claim 14, wherein: The method further comprises: It is determined that the current block does not use the illumination compensation mode, and a reconstructed block of the current block is determined according to the first prediction block and the residual information.

17. The method according to claim 8, wherein: The step of determining target illumination compensation model parameters according to the target template area includes: Determine a current block template region and a reference block template region according to the target template region; According to the third reconstructed sample of the current block template area and the fourth reconstructed sample of the reference block template area, the target illumination compensation model parameters corresponding to the target template area are determined.

18. The method according to any one of claims 1 to 17, wherein: The illumination compensation model is a linear model constructed based on a linear function, and the illumination compensation model parameters include at least one of the following: a scaling parameter, an offset parameter, and a shift parameter.

19. The method according to claim 1, wherein: The determining of the illumination compensation mode indication information of the current block includes: The code stream is decoded to determine the illumination compensation mode indication information of the current block.

20. The method according to claim 19, wherein: The decoding bit stream determines the illumination compensation mode indication information of the current block, including: Decoding the bitstream to determine prediction mode indication information of the current block; When it is determined according to the prediction mode indication information that the prediction mode of the current block is the first type of prediction mode, the bitstream is decoded to determine the illumination compensation mode indication information of the current block.

21. The method according to claim 1, wherein: The determining of the illumination compensation mode indication information of the current block includes: Determine the index value of the motion information of the current block; Determining illumination mode indication information of a reference block of a current block according to an index value of the motion information; The illumination mode indication information of the reference block is used as the illumination compensation mode indication information of the current block.

22. The method according to claim 21, wherein: The determining the index value of the motion information of the current block includes: Decoding the bitstream to determine prediction mode indication information of the current block; When it is determined according to the prediction mode indication information that the prediction mode of the current block is the second type of prediction mode, the bitstream is decoded to determine the index value of the motion information of the current block.

23. The method according to any one of claims 19 to 22, wherein: The illumination compensation mode indication information includes a first syntax element identifier; wherein the first syntax element identifier is used to indicate whether the current block uses the illumination compensation mode.

24. The method according to claim 23, wherein: The illumination compensation mode indication information also includes a second syntax element identifier; wherein, The second syntax element identifier includes at least one of the following: A sequence-level syntax element identifier, used to indicate whether the image sequence where the current block is located allows the use of the illumination compensation mode; A frame-level syntax element identifier, used to indicate whether the frame where the current block is located allows the use of the illumination compensation mode; A coding tree unit level syntax element identifier, used to indicate whether the coding tree unit where the current block is located allows the use of the illumination compensation mode; The slice-level syntax element identifier is used to indicate whether the slice where the current block is located allows the use of the illumination compensation mode.

25. The method according to any one of claims 19 to 22, wherein: The method further comprises: Determine the area of ​​the current block; When the area is within a preset area range, determining whether to use the illumination compensation mode according to illumination compensation mode indication information of the current block; When the area is outside the preset area range, it is determined that the current block does not use the illumination compensation mode.

26. The method according to claim 25, wherein: The preset area range includes: the area of ​​the current block is greater than the first area threshold and less than the second area threshold; Alternatively, the preset area range includes: the area of ​​the current block is greater than the first area threshold.

27. The method of claim 1, wherein: The method further comprises: Decoding the bitstream to determine prediction mode indication information of the current block; Determining a target prediction mode of a current block according to the prediction mode indication information; Determining a reference block of a current block in a reference frame according to the target prediction mode; The first prediction block is determined according to the reference block.

28. The method according to claim 27, wherein: The determining the first prediction block according to the reference block comprises: If the target prediction mode is an intra block copy mode, performing block copying on the reference block to determine the first prediction block; If the target prediction mode is an inter-frame prediction mode, motion compensation is performed on the reference block to determine the first prediction block.

29. The method of claim 1, wherein: The method further comprises: Decode the bitstream and determine the residual information of the current block; A reconstructed block of the current block is determined according to the residual information and the second prediction block.

30. A coding method, applied to an encoder, the method comprising: Determine multiple candidate template areas used by the illumination compensation mode; Performing cost calculation on the candidate template area based on template matching to determine the target template area corresponding to the minimum cost value; Determining target illumination compensation model parameters according to the target template area; Performing illumination compensation on a first prediction block of a current block according to the target illumination compensation model parameters to determine a second prediction block of the current block; Performing encoding decision on the illumination compensation mode according to the second prediction block to determine illumination compensation mode indication information of the current block; The illumination compensation mode indication information is encoded, and the obtained encoding bits are written into a bit stream.

31. The method according to claim 30, wherein: The determining of the multiple candidate template areas used in the illumination compensation mode includes: The plurality of candidate template regions are determined according to the at least two reconstructed sub-template regions.

32. The method according to claim 31, wherein: The method further comprises: Detecting whether the first sub-template area of ​​the current block and the first sub-template area of ​​the reference block are both available; If the first sub-template region of the current block and the first sub-template region of the reference block are both available, using the first sub-template region to determine the candidate template region; If the first sub-template region of the current block and / or the first sub-template region of the reference block is unavailable, the first sub-template region is ignored.

33. The method according to claim 31, wherein: The sub-template area includes at least one of the following: an upper sub-template area, an upper right sub-template area, a left sub-template area, a lower left sub-template area and an upper left sub-template area.

34. The method of claim 33, wherein: The step of determining the plurality of candidate template regions according to the at least two reconstructed sub-template regions comprises: Determine a first candidate template region according to the upper sub-template region and the left sub-template region; Determine a second candidate template region according to the left sub-template region; A third candidate template region is determined according to the upper sub-template region.

35. The method of claim 31, wherein: The method further comprises: Determine the height of the candidate template area according to the height of the current block; The width of the candidate template region is determined according to the width of the current block.

36. The method of claim 31, wherein: The method further comprises: Determine that the height of the candidate template area is a preset first value; Determine the width of the candidate template area as a preset second value; The first value is greater than or equal to 1, and the second value is greater than or equal to 1.

37. The method of claim 30, wherein: The cost calculation of the candidate template region based on template matching to determine the target template region corresponding to the minimum cost value includes: Determine a current block template region and a reference block template region according to the candidate template region; Determining candidate illumination compensation model parameters corresponding to the candidate template region according to the first reconstructed sample of the current block template region and the second reconstructed sample of the reference block template region; Determining a first prediction sample of the current block template area according to the candidate illumination compensation model parameters; Performing cost calculation based on the third reconstructed sample of the current block template area and the first predicted sample to determine the first generation value of the candidate template area; The target template region corresponding to the minimum cost value is determined according to the first cost value.

38. The method of claim 37, wherein: The reconstructed samples of the reference block template area are reconstructed samples after motion compensation.

39. The method of claim 37, wherein: The determining, according to the candidate illumination compensation model parameters, a first prediction sample of the current block template area comprises: Applying the candidate illumination compensation model parameters to the fourth reconstructed sample of the reference block template region to obtain the first predicted sample of the current block template region.

40. The method of claim 39, wherein: The method further comprises: Sampling a first template area of ​​the current block template area to determine the first reconstructed sample; Sampling a first template area of ​​the reference block template area to determine the second reconstructed sample; Sampling a second template area of ​​the current block template area to determine the third reconstructed sample; The second template area of ​​the reference block template area is sampled to determine the fourth reconstructed sample.

41. The method of claim 40, wherein: The first template area is not adjacent to the current block and its reference block, and the second template area is adjacent to the current block and its reference block; wherein, The first template area and the second template area do not overlap each other; or, the first template area and the second template area partially overlap each other.

42. The method of claim 37, wherein: The determining, according to the candidate illumination compensation model parameters, a first prediction sample of the current block template area comprises: Applying the candidate illumination compensation model parameters to the prediction samples of the first prediction block to obtain the first prediction samples of the current block template area.

43. The method of claim 37, wherein: The method further comprises: Determining a second predicted sample of the current block template region according to the reconstructed sample of the reference block template region; Performing cost calculation based on the reconstructed samples of the current template area and the second predicted samples to determine the second generation value of the candidate template area; If the minimum cost value is the first generation value, determining that the current block uses the illumination compensation mode; If the minimum cost value is the second generation value, it is determined that the current block does not use the illumination compensation mode.

44. The method of claim 43, wherein: The method further comprises: If it is determined that the prediction mode of the current block is the merge prediction mode, a second prediction sample of the current block template area is determined according to the reconstructed sample of the reference block template area.

45. The method of claim 43, wherein: The method further comprises: Determine that the current block does not use the illumination compensation mode, and determine a reconstructed block of the current block according to the first prediction block; Determine residual information of the current block according to the original block of the current block and the reconstructed block; The residual information is encoded, and the obtained encoded bits are written into a bitstream.

46. ​​The method of claim 37, wherein: The step of determining target illumination compensation model parameters according to the target template area includes: Determine a current block template region and a reference block template region according to the target template region; Determine the Target illumination compensation model parameters corresponding to the target template area.

47. The method according to any one of claims 30 to 46, wherein: The illumination model is a linear model constructed based on a linear function, and the illumination compensation model parameters include at least one of the following: a scaling parameter, an offset parameter, and a shift parameter.

48. The method of claim 30, wherein: The determining of the illumination compensation mode indication information of the current block includes: According to whether the current block uses the illumination compensation mode, illumination compensation mode indication information of the current block is determined.

49. The method of claim 48, wherein: The method further comprises: Determining prediction mode indication information of the current block according to the prediction mode of the current block; When it is determined according to the prediction mode indication information that the prediction mode of the current block is the first type of prediction mode, determining the illumination compensation mode indication information of the current block according to whether the current block uses the illumination compensation mode; The prediction mode indication information is encoded, and the obtained encoding bits are written into a bit stream.

50. The method of claim 30, wherein: The determining of the illumination compensation mode indication information of the current block includes: An index value of motion information of a current block is determined; wherein the index value of the motion information is used to indicate illumination mode compensation information of the current block.

51. The method of claim 50, wherein: The method further comprises: Determining prediction mode indication information of the current block according to the prediction mode of the current block; When it is determined according to the prediction mode indication information that the prediction mode of the current block is the second type of prediction mode, determining an index value of motion information of the current block; Using the index value of the motion information as illumination compensation mode indication information of the current block; The prediction mode indication information is encoded, and the obtained encoding bits are written into a bit stream.

52. The method according to any one of claims 48 to 51, wherein: The illumination compensation mode indication information includes a first syntax element identifier; wherein the first syntax element identifier is used to indicate whether the current block uses the illumination compensation mode.

53. The method of claim 52, wherein: The illumination compensation mode indication information also includes a second syntax element identifier; wherein, The second syntax element identifier includes at least one of the following: A sequence-level syntax element identifier, used to indicate whether the image sequence where the current block is located allows the use of the illumination compensation mode; A frame-level syntax element identifier, used to indicate whether the frame where the current block is located allows the use of the illumination compensation mode; A coding tree unit level syntax element identifier, used to indicate whether the coding tree unit where the current block is located allows the use of the illumination compensation mode; The slice-level syntax element identifier is used to indicate whether the slice where the current block is located allows the use of the illumination compensation mode.

54. A method according to any one of claims 48 to 51, wherein: The method further comprises: Determine the area of ​​the current block; When the area is within a preset area range, determining whether to use the illumination compensation mode according to illumination compensation mode indication information of the current block; When the area is outside the preset area range, it is determined that the current block does not use the illumination compensation mode.

55. The method of claim 54, wherein: The preset area range includes: the area of ​​the current block is greater than the first area threshold and less than the second area threshold; Alternatively, the preset area range includes: the area of ​​the current block is greater than the first area threshold.

56. The method of claim 30, wherein: The method further comprises: Determine a reference block of the current block in the reference frame according to the target prediction mode; The first prediction block is determined according to the reference block.

57. The method of claim 56, wherein: The determining the first prediction block according to the reference block comprises: If the target prediction mode is an intra block copy mode, performing block copying on the reference block to determine the first prediction block; If the target prediction mode is an inter-frame prediction mode, motion compensation is performed on the reference block to determine the first prediction block.

58. The method of claim 30, wherein: The method further comprises: Determine that the current block uses the illumination compensation mode, and determine a reconstructed block of the current block according to the second prediction block; Determine residual information of the current block according to the original block of the current block and the reconstructed block; The residual information is encoded, and the obtained encoded bits are written into a bitstream.

59. A code stream, wherein The bit stream is generated by bit encoding according to the information to be encoded; wherein the information to be encoded includes at least one of the following: illumination compensation mode indication information of the current block, prediction mode indication information, index value of motion information, and residual information.

60. An encoder, comprising a first determining unit, a second determining unit and an encoding unit; wherein: The first determining unit is configured to determine a plurality of candidate template regions used by the illumination compensation mode; The first determining unit is configured to perform cost calculation on the candidate template region based on template matching to determine a target template region corresponding to a minimum cost value; The first determining unit is configured to determine target illumination compensation model parameters according to the target template area; The second determination unit is configured to perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block; The second determination unit is configured to make an encoding decision on the illumination compensation mode according to the second prediction block, and determine illumination compensation mode indication information of the current block; The encoding unit is configured to encode the illumination compensation mode indication information and write the obtained encoding bits into a bit stream.

61. An encoder comprising a first memory and a first processor; wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 30 to 58 when running the computer program.

62. A decoder comprising a decoding unit, a third determining unit and a fourth determining unit; wherein: The decoding unit is configured to determine illumination compensation mode indication information of a current block; The third determining unit is configured to determine a plurality of candidate template regions used by the illumination compensation mode when it is determined that the current block uses the illumination compensation mode according to the illumination compensation mode indication information; The third determining unit is configured to perform cost calculation on the candidate template region based on template matching to determine a target template region corresponding to a minimum cost value; The fourth determining unit is configured to determine target illumination compensation model parameters according to the target template area; The fourth determining unit is configured to perform illumination compensation on the first prediction block of the current block according to the target illumination compensation model parameters to determine the second prediction block of the current block.

63. A decoder comprising a second memory and a second processor; wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 29 when running the computer program.

64. A computer-readable storage medium, wherein: The computer-readable storage medium stores a code stream generated by the encoding method according to any one of claims 30 to 58.

65. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 29 is implemented, or the method according to any one of claims 30 to 58 is implemented.

Images