Image decoding method and device, image coding method and device, codec and storage medium

Abstract

The embodiment of the invention discloses an image decoding method and device, an image coding method and device, a codec and a storage medium, and belongs to the technical field of video coding and decoding. The image coding and decoding method comprises the following steps: performing chroma filtering on a to-be-filtered sample based on reconstruction brightness and reconstruction chroma corresponding to the to-be-filtered sample. By adopting the scheme provided by the embodiment of the invention, the chromaticity filtering quality can be improved, and the image coding and decoding performance and the coding and decoding quality can be further improved.

Description

Technical Field

[0001] This application relates to the field of video encoding and decoding technology, and in particular to an image decoding and encoding method, apparatus, codec, and storage medium. Background Technology

[0002] During the image encoding and decoding process of video, after the reconstruction of the current block in the image is completed, it is necessary to further perform in-loop filtering on the reconstruction result to improve the quality of the reconstructed image, thereby providing a better reference for subsequent encoding and decoding.

[0003] In related technologies, the loop filtering process includes Adaptive Loop Filter (ALF), specifically Luminance ALF and Chroma ALF. The Luminance ALF is used to perform ALF processing on the reconstructed luminance through a filter, while the Chroma ALF is used to perform ALF processing on the reconstructed chroma through a filter. Summary of the Invention

[0004] This application provides an image decoding and encoding method, apparatus, codec, and storage medium. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide an image decoding method applied to a decoder, the method comprising:

[0006] Based on the reconstructed luminance and reconstructed chrominance of the sample to be filtered, chrominance filtering is performed on the sample to be filtered.

[0007] On the other hand, embodiments of this application provide an image encoding method applied to an encoder, the method comprising:

[0008] Based on the reconstructed luminance and reconstructed chrominance of the sample to be filtered, chrominance filtering is performed on the sample to be filtered.

[0009] On the other hand, embodiments of this application provide a decoding device, the device comprising:

[0010] The decoding unit is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

[0011] On the other hand, embodiments of this application provide an encoding device, the device comprising:

[0012] The encoding unit is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

[0013] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the image decoding method or the image encoding method described above.

[0014] On the other hand, embodiments of this application provide a computer-readable storage medium storing a bitstream, the bitstream being generated by an image encoding method using an encoder, or the bitstream being decoded by an image decoding method using a decoder, wherein the image encoding method includes the image encoding method described above, and the image decoding method includes the image decoding method described above.

[0015] On the other hand, embodiments of this application provide a bitstream that is decoded using the image decoding method described above, or encoded using the image encoding method described above.

[0016] On the other hand, embodiments of this application provide a decoder, the decoder including a memory and a processor, the memory being used to store a computer program running on the processor; the processor being used to execute the image decoding method as described above when running the computer program.

[0017] On the other hand, embodiments of this application provide an encoder, the encoder including a memory and a processor, the memory for storing a computer program running on the processor; the processor for executing the image encoding method as described above when running the computer program.

[0018] In this embodiment, in addition to performing chromatic filtering on the sample to be filtered based on the reconstructed chromaticity corresponding to the sample to be filtered, the reconstructed luminance corresponding to the sample to be filtered is also used as the input for chromatic filtering, which helps to improve the quality of chromatic filtering, thereby improving image encoding and decoding performance and encoding and decoding quality. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating the encoding process at the encoding end in an exemplary embodiment of this application;

[0020] Figure 2 This is a schematic diagram illustrating the decoding process at the decoding end in an exemplary embodiment of this application;

[0021] Figure 3 This is a schematic diagram illustrating the adaptive loop filtering process in an AVS according to an exemplary embodiment of this application;

[0022] Figure 4This is a schematic diagram illustrating the adaptive loop filtering process in VVC according to an exemplary embodiment of this application;

[0023] Figure 5 This is a flowchart illustrating an exemplary embodiment of the image decoding method of this application;

[0024] Figure 6 This is a schematic diagram illustrating an adaptive loop filtering process in an exemplary embodiment of this application;

[0025] Figure 7 This is a schematic diagram illustrating a third filter in an exemplary embodiment of this application;

[0026] Figure 8 This is a schematic diagram illustrating the first and second filters in an exemplary embodiment of this application;

[0027] Figure 9 This is a schematic diagram illustrating the first and second filters in another exemplary embodiment of this application;

[0028] Figure 10 This is a schematic diagram illustrating the first and second filters in another exemplary embodiment of this application;

[0029] Figure 11 This is a schematic diagram illustrating the first and second filters in another exemplary embodiment of this application;

[0030] Figure 12 This is a flowchart illustrating an exemplary embodiment of the image encoding method of this application;

[0031] Figure 13 This invention provides a structural block diagram of a decoding apparatus according to an exemplary embodiment of the present application.

[0032] Figure 14 This invention provides a structural block diagram of an encoding apparatus according to an exemplary embodiment of the present application.

[0033] Figure 15 This invention illustrates a structural block diagram of a decoder provided in an exemplary embodiment of this application;

[0034] Figure 16 A structural block diagram of an encoder provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0036] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0037] Currently, most common video codec standards employ a block-based hybrid coding framework. A single frame in a video is divided into squares of equal size (e.g., 256×256, 128×128, 64×64) called Largest Coding Units (LCUs) or Coding Tree Units (CTUs). These LCUs or CTUs can be further subdivided into rectangular Coding Units (CUs) according to rules. Coding units may also be further divided into Prediction Units (PUs), Transform Units (TUs), and so on.

[0038] The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-prediction and inter-prediction. Inter-prediction includes motion estimation and motion compensation. Because there is a strong correlation between adjacent pixels in the same frame, intra-prediction can be used in video coding and decoding techniques to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, inter-prediction can be used in video coding and decoding techniques to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0039] At the encoding end, such as Figure 1 As shown, the encoder first divides the image into multiple coding blocks. Then, it uses intra-frame or inter-frame prediction (including motion estimation and motion compensation) algorithms to generate a prediction block for the current block. The encoder then subtracts the prediction block from the original block to obtain a residual block. This residual block is then transformed and quantized to obtain quantization coefficients. Finally, entropy coding is used to encode these quantization coefficients into the bitstream. Furthermore, the encoder reconstructs the image based on the inverse quantization / inverse transform results and the prediction block, obtaining a reconstructed block. This reconstructed block is then subjected to loop filtering to compensate for distortion information and provide a better reference for subsequent encoding.

[0040] On the decoding end, such as Figure 2 As shown, the decoder uses intra-frame or inter-frame prediction algorithms to predict the current block, and performs entropy decoding on the bitstream to obtain quantization coefficients. These coefficients are then dequantized and inverse transformed to obtain residual blocks. The predicted block and residual blocks are then added together to obtain the reconstructed block. For the reconstructed block, the decoder further performs loop filtering, and finally generates the decoded image based on the loop-filtered reconstructed block.

[0041] Under the AVS (Audio Video Coding Standard) standard, loop filtering includes adaptive loop filtering. For example... Figure 3 As shown, based on luminance and chrominance, adaptive loop filtering includes luminance ALF and chrominance ALF. The luminance ALF is used to filter the reconstructed luminance, while the chrominance ALF is used to filter the reconstructed chrominance.

[0042] Under the VVC (Versatile Video Coding) standard, such as Figure 4 As shown, the loop filtering process includes Luma Mapping with Chroma Scaling (LMCS), DeBlocking Filter (DBF), Sample Adaptive Offset (SAO), Adaptive Loop Filter (ALF), and Cross Component Adaptive Loop Filter (CCALF).

[0043] Among them, LMCS is used for luminance mapping based on an adaptive piecewise linear model (applied to the pixel level) to improve video coding efficiency, and luminance-based chrominance residual scaling (applied to the chrominance block level) to compensate for the impact of luminance signal mapping on chrominance signals.

[0044] DBF is used to reduce block artifacts in video coding. It works by applying it to the boundaries of coding blocks and reducing discontinuities between blocks through smoothing.

[0045] SAO is used to apply adaptive offsets to reduce coding artifacts (especially for high-frequency details).

[0046] ALF and CCALF are based on the Wiener filtering principle. They establish the Wiener-Hough equation based on the original image and the reconstructed image, and solve for the filter coefficients with the minimum mean square error, so as to reduce decoding error and improve coding performance.

[0047] The positions of ALF and CCALF in loop filtering are as follows: Figure 3 As shown. The ALF includes Luminance ALF (ALF Luma) and Chroma ALF (ALF Chroma). The Luminance ALF is designed to minimize the mean square error between the reconstructed luminance image and the original image using luminance information; its input is the output of SAO Luma. The Chroma ALF is designed to minimize the mean square error between the reconstructed chrominance image and the original image using chrominance information; its input is the output of SAO Cb and SAO Cr.

[0048] CCALF is designed to minimize the mean square error between the chroma reconstructed image and the original image by utilizing luminance information, and its output is the same as that of SAO Luma. It's important to note that CCALF filters the chroma reconstructed information output by SAO; therefore, the filtered result of CCALF (…) or The result needs to be superimposed with the chromaticity ALF filtering result to obtain the final chromaticity filtering result (including the two chromaticity components Cb and Cr).

[0049] To improve encoding and decoding performance, in the embodiments of this application, in the process of chroma adaptive filtering, in addition to taking the reconstructed chroma as input, the reconstructed luminance is also taken as part of the input, so as to perform chroma filtering on the sample to be filtered based on the reconstructed chroma and the reconstructed luminance, thereby improving the chroma filtering quality and thus improving the performance and quality of image encoding and decoding.

[0050] The method provided in this application can be applied to the chromaticity adaptive filtering process under the AVS standard and VVC standard, but this application does not limit it.

[0051] Please refer to Figure 5 The diagram illustrates a flowchart of an image decoding method provided in an exemplary embodiment of this application. The method is applied to a decoder and may include the following steps:

[0052] Step 501: Perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

[0053] In some embodiments, the reconstructed brightness corresponding to the sample to be filtered includes: the reconstructed brightness of at least one reference sample around the sample to be filtered, or the reconstructed brightness of at least one reference sample around the sample to be filtered and the sample to be filtered.

[0054] In some embodiments, the reconstructed chromaticity corresponding to the sample to be filtered includes: at least one reference sample around the sample to be filtered and the reconstructed chromaticity of the sample to be filtered.

[0055] In one possible implementation, before performing chroma filtering on the current sample to be filtered, the decoder determines the reference sample corresponding to the sample to be filtered, and determines the reconstructed luminance and reconstructed chroma of the reference sample. Then, based on the reconstructed luminance and reconstructed chroma of the reference sample, the decoder performs chroma filtering on the sample to be filtered to obtain the filtered reconstructed chroma.

[0056] Here, the chroma filtering refers to chroma adaptive filtering.

[0057] Optionally, the reconstructed luminance is the output of the previous stage of the chromaticity adaptive filtering process, and the reconstructed chromaticity is the output of the previous stage of the chromaticity adaptive filtering process. That is, before using the reconstructed luminance for luminance adaptive filtering, there is no need to perform other filtering processing on the reconstructed luminance, and before using the reconstructed chromaticity for chromaticity adaptive filtering, there is no need to perform other filtering processing on the reconstructed chromaticity.

[0058] Optionally, the reconstructed chromaticity may include the reconstructed chromaticity of different color components. Accordingly, for different color components, the decoder performs chromaticity filtering on the sample to be filtered based on the reconstructed luminance and the reconstructed chromaticity of different color components to obtain the filtered reconstructed chromaticity for different color components.

[0059] In an illustrative example, the reconstructed luminance is the luminance reconstruction value output by the SAO, and the reconstructed chrominance is the chrominance reconstruction value output by the SAO, including the chrominance reconstruction values ​​of the Cb component and the chrominance reconstruction values ​​of the Cr component. Accordingly, the decoder performs chrominance filtering on the sample to be filtered based on the luminance reconstruction value and the chrominance reconstruction value of the Cb component; and performs chrominance filtering on the sample to be filtered based on the luminance reconstruction value and the chrominance reconstruction value of the Cr component.

[0060] In one possible implementation, the decoder employs a filter corresponding to the chroma filtering, performing chroma filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chroma. The filtering coefficients included in the filter will be described in detail in the following embodiments.

[0061] Optionally, after performing chroma filtering on the sample to be filtered to obtain the reconstructed chroma, the decoder can generate a decoded image based on the reconstructed chroma, or it can further process the reconstructed chroma. For example, under the VVC standard, the decoder also needs to add the chroma adaptive filtering result to the CCALF filtering result.

[0062] In summary, in this embodiment, in addition to performing chromatic filtering on the sample to be filtered based on the reconstructed chromaticity corresponding to the sample to be filtered, the reconstructed luminance corresponding to the sample to be filtered is also used as the input for chromatic filtering, which helps to improve the quality of chromatic filtering, thereby improving image encoding and decoding performance and encoding and decoding quality.

[0063] How to reconstruct luminance during chroma filtering in the decoding stage

[0064] Optionally, when using the reconstructed brightness corresponding to the sample to be filtered as the input for chromatic adaptive filtering, the decoder can directly use the reconstructed brightness for chromatic adaptive filtering, or it can first further process the reconstructed brightness and then perform chromatic adaptive filtering based on the processing result of the reconstructed brightness.

[0065] In one possible implementation, the decoder determines the luminance information of the sample to be filtered based on the reconstructed luminance. Further, the decoder performs chromaticity filtering on the sample based on the luminance information and the reconstructed chromaticity.

[0066] Optionally, this brightness information is used to characterize the brightness features of the sample to be filtered and its surrounding samples. In other words, the process by which the decoder determines brightness information based on the reconstructed brightness is the process of extracting brightness features from the reconstructed brightness of the sample to be filtered and its surrounding samples.

[0067] In some embodiments, the brightness information is used to characterize the brightness variation features of the sample to be filtered and its surrounding samples, or the brightness mean feature, or the brightness uniformity feature, etc. Accordingly, the brightness information can be the brightness gradient, the brightness mean, or the brightness variance, etc. The embodiments of this application do not limit the specific form of the brightness information.

[0068] In some embodiments, the reconstructed chromaticity of the sample to be filtered is the reconstructed chromaticity of the first reference sample corresponding to the sample to be filtered, and the reconstructed luminance of the sample to be filtered is the reconstructed luminance of the second reference sample corresponding to the sample to be filtered.

[0069] Optionally, the decoder determines the reconstructed brightness of the second reference sample corresponding to the sample to be filtered, and determines the brightness information corresponding to the sample to be filtered based on the reconstructed brightness of the second reference sample.

[0070] In one possible design, the luminance information is a luminance gradient, which includes a horizontal luminance gradient (Gx) and a vertical luminance gradient (Gy).

[0071] The horizontal brightness gradient is used to characterize the rate of change of brightness of the sample in the horizontal direction around the sample to be filtered, and the vertical brightness gradient is used to characterize the rate of change of brightness of the sample in the vertical direction around the sample to be filtered.

[0072] like Figure 6 As shown, the decoder uses a gradient operator to determine the luminance gradient based on the reconstructed luminance of the second reference sample corresponding to the sample to be filtered, and then uses the luminance gradient as the input of the chromatic ALF.

[0073] In one possible implementation, the decoder determining the luminance gradient based on the reconstructed luminance may include the following steps:

[0074] 1. Determine the coordinates of the second reference sample based on the gradient operator and the coordinates of the sample to be filtered.

[0075] In one possible implementation, the decoder determines the corresponding luminance position of the sample to be filtered based on the coordinates of the sample to be filtered, and then determines the coordinates of the second reference sample based on the luminance position and the relative position between the second reference sample indicated by the gradient operator and the luminance position.

[0076] Optionally, the position of the same brightness is format-dependent. For example, if the coordinates of the sample to be filtered are (x, y), the coordinates of the corresponding brightness position are (2x, 2y) in YUV420 format; (x, y) in YUV444 format; and (2x, y) in YUV422 format.

[0077] Optionally, the gradient operator may include the Sobel operator, the Robert operator, other improved or custom operators for calculating the gradient. This application does not limit the specific type of operator.

[0078] Furthermore, the number and location of the second reference samples used to calculate the brightness gradient differ when different gradient operators are employed.

[0079] For example, the operators used to calculate the horizontal brightness gradient Gx and the vertical brightness gradient Gy are as follows:

[0080]

[0081] Based on this gradient operator, the sample to be filtered and its three surrounding reference samples can be determined.

[0082] For example, the operators used to calculate the horizontal brightness gradient Gx and the vertical brightness gradient Gy are as follows:

[0083]

[0084] Based on this gradient operator, the sample to be filtered and its three surrounding reference samples can be determined.

[0085] For example, the operators used to calculate the horizontal brightness gradient Gx and the vertical brightness gradient Gy are as follows:

[0086]

[0087] Based on this gradient operator, the sample to be filtered and its eight surrounding reference samples can be determined.

[0088] For example, the operators used to calculate the horizontal brightness gradient Gx and the vertical brightness gradient Gy are as follows:

[0089]

[0090] Based on this gradient operator, the sample to be filtered and its eight surrounding reference samples can be determined.

[0091] It should be noted that the gradient operators described above are for illustrative purposes only and do not constitute a limitation.

[0092] In an illustrative example, in YUV420 format, if the coordinates of the sample to be filtered are (x, y), and the gradient operators used are respectively At that time, the decoder determined the coordinates of the four second reference samples to be (2x, 2y), (2x+1, 2y), (2x, 2y+1), and (2x+1, 2y+1).

[0093] In another illustrative example, in the YUV420 format, if the coordinates of the sample to be filtered are (x, y), and the gradient operators used are respectively At that time, the decoder determined the coordinates of the nine second reference samples as (2x-1, 2y-1), (2x, 2y-1), (2x+1, 2y-1), (2x-1, 2y), (2x, 2y), (2x+1, 2y), (2x-1, 2y+1), (2x, 2y+1), (2x+1, 2y+1).

[0094] 2. Based on the coordinates of the second reference sample, obtain the reconstructed brightness of the second reference sample.

[0095] Based on the coordinates of the second reference sample, the decoder obtains the reconstructed brightness of each second reference sample.

[0096] Based on the example in the above steps, the decoder obtains the reconstructed brightness at the four positions (2x, 2y), (2x+1, 2y), (2x, 2y+1), and (2x+1, 2y+1), which are py0, py1, py2, and py3, respectively.

[0097] Combining with another example from the steps above, the decoder obtains the reconstructed luminance at nine positions: (2x-1, 2y-1), (2x, 2y-1), (2x+1, 2y-1), (2x-1, 2y), (2x, 2y), (2x+1, 2y), (2x-1, 2y+1), (2x, 2y+1), (2x+1, 2y+1), which are py0, py1, py2, py3, py4, py5, py6, py7, and py8, respectively.

[0098] 3. Based on the reconstructed brightness and gradient operator of the second reference sample, determine the brightness gradient corresponding to the sample to be filtered.

[0099] Furthermore, the decoder uses gradient operators to calculate the gradient of the reconstructed brightness to obtain the horizontal and vertical brightness gradients corresponding to the sample to be filtered.

[0100] Based on the examples in the steps above, the decoder utilizes the gradient operator. The horizontal brightness gradient corresponding to the sample to be filtered is calculated to be py0-py3, and the vertical brightness gradient is calculated to be py2-py1.

[0101] Combining another example from the steps above, the decoder utilizes the gradient operator. The horizontal brightness gradient corresponding to the sample to be filtered is calculated as py2+py5+py8-py0-py3-py6, and the vertical brightness gradient is py6+py7+py8-py0-py1-py2.

[0102] In this embodiment, the decoder determines the second reference sample corresponding to the sample to be filtered based on the gradient operator, and then determines the horizontal and vertical brightness gradients corresponding to the sample to be filtered based on the reconstructed brightness of the second reference sample. Since the brightness variation and chromaticity of the sample are related, the subsequent chromaticity filtering based on the brightness gradient and the reconstructed chromaticity of the sample to be filtered can further improve the chromaticity filtering quality, thereby improving the image decoding quality and performance.

[0103] After determining the luminance information corresponding to the sample to be filtered, the decoder further performs chromaticity filtering on the sample to be filtered based on the luminance information and the reconstructed chromaticity of the first reference sample corresponding to the sample to be filtered.

[0104] In some embodiments, the decoder employs a filter to perform chromatic filtering on the samples to be filtered based on luminance information and reconstructed chroma. The filter coefficients are encoded into the bitstream at the encoder.

[0105] Correspondingly, the decoder parses the bitstream, determines the filter coefficients, and then performs chroma filtering on the sample to be filtered based on the luminance information, the reconstructed chroma of the first reference sample, and the filter coefficients.

[0106] Optionally, the filter coefficients are written into the Adaptive Parameter Set (APS), and the APS ID used by the current image is written into the slice header or picture header. The decoder receives the bitstream, parses the APS ID, and then determines the filter coefficients from the APS based on the parsed APS ID.

[0107] Filter Structure Description

[0108] In one possible implementation, without introducing reconstructed luminance for chroma filtering, the decoder uses a third filter to perform chroma filtering on the sample to be filtered.

[0109] In an illustrative example, the third filter is as follows: Figure 7 As shown. This third filter includes 15 filter coefficients (coefficient indices from 0 to 14) and contains a total of 29 filter positions.

[0110] Optionally, the third filter is used to indicate the relative positional relationship between the sample to be filtered and the first reference sample. This relative positional relationship can be represented by coordinate offsets, which include horizontal and vertical coordinate offsets.

[0111] Optionally, after the decoder parses the filtering coefficients of the third filter from the bitstream, it obtains the coordinate offset of the corresponding first reference sample from the coordinate offset table based on the coefficient index of the filtering coefficients. Then, based on the coordinate offset and the coordinates of the sample to be filtered, the coordinates of the first reference sample are determined, and the reconstructed chromaticity of the first reference sample is obtained based on the coordinates of the first reference sample.

[0112] For each first reference sample, the decoder accumulates the product of the reconstructed chromaticity of the first reference sample and its filtering coefficients, thereby determining the reconstructed chromaticity of the filtered sample based on the accumulation result. When using... Figure 7 When using the third filter shown, the process can be represented as:

[0113] ptmp=AlfCoeffChroma[i]

[14] *p(x,y)

[0114] for(j=0;j<14;j++){

[0115] ptmp+=AlfCoeffChroma[i][j]*(p(x-Hor[j],y-Ver[j])+p(x+Hor[j],y+Ver[j]))

[0116] }

[0117] ptmp = (ptmp + 32) >> 6

[0118] p'(x,y)=Clip3(0,(1< <BitDepth)–1,ptmp)

[0119] Wherein, the coordinates of the sample to be filtered are (x,y), p(x,y) is the reconstructed chromaticity of the sample to be filtered, p'(x,y) is the reconstructed chromaticity after filtering, AlfCoeffChroma[i][j] is the obtained filtering coefficient, Hor[j] represents the horizontal coordinate offset between the reference sample corresponding to the filtering coefficient with coefficient index j and the sample to be filtered, and Ver[j] represents the vertical coordinate offset between the reference sample corresponding to the filtering coefficient with coefficient index j and the sample to be filtered.

[0120] Indicative Figure 7 The coordinate offset table corresponding to the third filter is shown in Table 1.

[0121] Table 1

[0122]

[0123]

[0124] Since luminance information is additionally referenced for chroma filtering, in one possible implementation, the filter used for chroma filtering includes a first filter and a second filter. The first filter is used for filtering based on the reconstructed chroma of the first reference sample, and the second filter is used for filtering based on the luminance information.

[0125] In some embodiments, the first filter is used to indicate the relative positional relationship between the sample to be filtered and the first reference sample. The relative positional relationship can be represented by coordinate offsets, which include horizontal coordinate offsets and vertical coordinate offsets.

[0126] Correspondingly, when the decoder determines the first reference sample corresponding to the sample to be filtered, it determines the first reference sample based on the coordinates of the sample to be filtered and the coordinate offset represented by the first filter, and then obtains the reconstructed chromaticity of the first reference sample.

[0127] In one possible implementation, the first filter is the same as the third filter, while the second filter is a newly added filter. Furthermore, since the filter coefficients contained in the newly added second filter need to be encoded into the bitstream, the bitstream is increased.

[0128] In some embodiments, when the second filter is used for filtering based on the brightness gradient, the filtering coefficients of the second filter include the filtering coefficients corresponding to the horizontal brightness gradient and the vertical brightness gradient, respectively.

[0129] In an illustrative example, such as Figure 8 As shown, the first filter and Figure 7 The third filter shown is the same as the second filter, which is a new filter containing two filter coefficients (coefficient indices 15 and 16).

[0130] Accordingly, adopt Figure 8 When the first and second filters shown perform chroma filtering, the process can be represented as follows:

[0131] ptmp=AlfCoeffChroma[i]

[14] *p(x,y)

[0132] for(j=0;j<14;j++){

[0133] ptmp+=AlfCoeffChroma[i][j]*(p(x-Hor[j],y-Ver[j])+p(x+Hor[j],y+Ver[j]))

[0134] }

[0135] ptmp+=AlfCoeffChroma[i]

[15] *Gx

[0136] ptmp+=AlfCoeffChroma[i]

[16] *Gy

[0137] ptmp = (ptmp + 32) >> 6

[0138] p'(x,y)=Clip3(0,(1< <BitDepth)–1,ptmp)

[0139] Wherein, the coordinates of the sample to be filtered are (x, y), p(x, y) is the reconstructed chromaticity of the sample to be filtered, p'(x, y) is the reconstructed chromaticity after filtering, AlfCoeffChroma[i][j] are the obtained filtering coefficients, Hor[j] represents the horizontal coordinate offset between the reference sample and the sample to be filtered corresponding to the filtering coefficient with coefficient index j, Ver[j] represents the vertical coordinate offset between the reference sample and the sample to be filtered corresponding to the filtering coefficient with coefficient index j, Gx is the horizontal luminance gradient, and Gy is the vertical luminance gradient. Furthermore, since the first filter and the third filter are the same, the coordinate offset table corresponding to the first filter is also shown in Table 1.

[0140] In another possible implementation, in order to reduce the impact of adding a second filter on the bitstream size, a first filter can be obtained by modifying the third filter so that the number of filter coefficients of the first filter is less than the number of filter coefficients of the third filter (i.e. the first filter is different from the third filter), thereby reducing the impact of adding filter coefficients of the second filter on the bitstream size.

[0141] Optionally, the first filter can be obtained based on the third filter in the following manner.

[0142] Method 1: Delete some filter coefficients in the third filter.

[0143] By changing the shape of the filter, a first filter with fewer filter coefficients can be obtained by deleting some of the filter coefficients in the third filter.

[0144] In one possible implementation, the first filter includes one or more filter coefficients selected from the third filter, i.e., the first filter contains only a portion of the filter coefficients from the third filter.

[0145] In one possible design, the removed filter coefficients can be located anywhere on the third filter.

[0146] In another possible design, in order to reduce the impact of changing the filter shape on the filtering performance, the deleted filter coefficients can be located in the third filter at a position where the filtering performance has less impact.

[0147] Since the impact on filtering performance decreases the further away from the current filtered sample, in one possible implementation, the deleted filter coefficients are located at the edge of the third filter. In other words, one or more filter coefficients selected from the third filter in the first filter are located at the non-edge of the third filter.

[0148] Optionally, the edge refers to a position where the distance between the edge and the middle position of the third filter is greater than a threshold. For example, the threshold can be 2, 3, etc., and this embodiment does not limit this.

[0149] In an illustrative example, if the third filter is as follows: Figure 7 As shown, the filter coefficients with coefficient indices 1 and 3 in the third filter can be deleted to obtain... Figure 9 The first filter is shown. This first filter contains 13 filter coefficients and 25 filter positions. Since the second filter contains 2 filter coefficients, the sum of the filter coefficients of the first and second filters is 15, consistent with the number of filter coefficients of the third filter.

[0150] Accordingly, adopt Figure 9 When the first and second filters shown perform chroma filtering, the process can be represented as follows:

[0151] ptmp=AlfCoeffChroma[i]

[12] *p(x,y)

[0152] for(j=0;j<12;j++){

[0153] ptmp+=AlfCoeffChroma[i][j]*(p(x-Hor[j],y-Ver[j])+p(x+Hor[j],y+Ver[j]))

[0154] }

[0155] ptmp+=AlfCoeffChroma[i]

[13] *Gx

[0156] ptmp+=AlfCoeffChroma[i]

[14] *Gy

[0157] ptmp = (ptmp + 32) >> 6

[0158] p'(x,y)=Clip3(0,(1< <BitDepth)–1,ptmp)

[0159] Wherein, the coordinates of the sample to be filtered are (x,y), p(x,y) is the reconstructed chromaticity of the sample to be filtered, p'(x,y) is the reconstructed chromaticity after filtering, AlfCoeffChroma[i][j] is the obtained filtering coefficient, Hor[j] represents the horizontal coordinate offset between the reference sample corresponding to the filtering coefficient with coefficient index j and the sample to be filtered, Ver[j] represents the vertical coordinate offset between the reference sample corresponding to the filtering coefficient with coefficient index j and the sample to be filtered, Gx is the horizontal luminance gradient, and Gy is the vertical luminance gradient.

[0160] Because the first filter differs from the third filter (the number of filter coefficients is reduced), the coordinate offset tables corresponding to the first filter and the third filter are also different.

[0161] Indicative Figure 9 The coordinate offset table corresponding to the first filter is shown in Table 2.

[0162] Table 2

[0163] j 0 1 2 3 4 5 6 7 8 9 10 11 Hor[j] 0 0 2 0 2 1 0 1 4 3 2 1 Ver[j] 4 3 2 2 -2 1 1 -1 0 0 0 0

[0164] In another illustrative example, if the third filter is as follows: Figure 7 As shown, the filter coefficients with coefficient indices of 0 and 8 in the third filter can be deleted to obtain... Figure 10 The first filter is shown. This first filter contains 13 filter coefficients and 25 filter positions. Since the second filter contains 2 filter coefficients, the sum of the filter coefficients of the first and second filters is 15, consistent with the number of filter coefficients of the third filter.

[0165] Accordingly, adopt Figure 10The process of chroma filtering performed by the first and second filters shown is similar to that of using... Figure 9 The process of chroma filtering for the first and second filters shown is similar, and will not be described in detail here.

[0166] Because the first filter differs from the third filter (the number of filter coefficients is reduced), the coordinate offset tables corresponding to the first filter and the third filter are also different.

[0167] Indicative Figure 10 The coordinate offset table corresponding to the first filter is shown in Table 3.

[0168] Table 3

[0169] j 0 1 2 3 4 5 6 7 8 9 10 11 Hor[j] 3 0 -3 2 0 -2 1 0 -1 3 2 1 Ver[j] 3 3 3 2 2 2 1 1 1 0 0 0

[0170] It should be noted that the number and position of the deleted filter coefficients in the above example are for illustrative purposes only. In other possible designs, the number of deleted filter coefficients may be one or more, and they may be located in other edge positions. This application embodiment does not constitute a limitation in this regard.

[0171] Method 2: Reuse some of the filter coefficients in the third filter.

[0172] Without changing the shape of the filter, a first filter with fewer filter coefficients can be obtained by increasing the number of multiplexed filter coefficients in the third filter.

[0173] In one possible implementation, the first filter adds one or more of the same filter coefficients to the third filter.

[0174] In one possible design, the reused filter coefficients are located at any position on the third filter.

[0175] In another possible design, in order to reduce the impact of multiplexed filter coefficients on filtering performance, the multiplexed filter coefficients can be located in the third filter at a position where they have less impact on filtering performance.

[0176] Since the impact on filtering performance decreases the further away from the current filtered sample, in one possible implementation, the reused filter coefficients are located at the edge of the third filter. In other words, one or more identical filter coefficients added in the first filter are located at the edge of the third filter.

[0177] Optionally, the edge refers to a position where the distance between the edge and the middle position of the third filter is greater than a threshold. For example, the threshold can be 2, 3, etc., and this embodiment does not limit this.

[0178] In an illustrative example, if the third filter is as follows: Figure 7 As shown, coefficient indices 0 and 2 in the third filter can reuse the same filter coefficient (after reuse, the coefficient index of the filter coefficient is 1), and coefficient indices 10 and 11 can reuse the same filter coefficient (after reuse, the coefficient index of the filter coefficient is 9), resulting in... Figure 11 The first filter is shown. This first filter contains 13 filter coefficients and 29 filter positions. Since the second filter contains 2 filter coefficients, the sum of the filter coefficients of the first and second filters is 15, consistent with the number of filter coefficients of the third filter.

[0179] Accordingly, adopt Figure 11 The process of chroma filtering performed by the first and second filters shown is similar to that of using... Figure 9 The process of chroma filtering for the first and second filters shown is similar, and will not be described in detail here.

[0180] Because the first filter differs from the third filter (the number of filter coefficients is reduced), the coordinate offset tables corresponding to the first filter and the third filter are also different.

[0181] Indicative Figure 11 The coordinate offset table corresponding to the first filter is shown in Table 4.

[0182] Table 4

[0183] j 0 1 2 3 4 5 6 7 8 9 10 11 Hor[j] 3 0 -3 0 0 -2 1 0 -1 3&4 2 1 Ver[j] 3 3&4 3 2 2 2 1 1 1 0 0 0

[0184] It should be noted that the number and position of the newly added multiplexing filter coefficients in the above example are only for illustrative purposes. In other possible designs, the number of newly added multiplexing filter coefficients can be one or two or more, and they can be located at other edge positions. This application embodiment does not constitute a limitation in this regard.

[0185] In this embodiment, the number of filter coefficients in the first filter (used for chroma filtering based on reconstructed chroma) is reduced by deleting or reusing filter coefficients at specific locations in the filter. This improves chroma filtering performance by introducing reconstructed luminance for chroma filtering, while minimizing the impact on the bitstream size caused by adding a second filter (for chroma filtering based on luminance information).

[0186] Regarding the process of parsing filter coefficients from the bitstream, without introducing reconstructed luminance for chroma filtering, the decoder can determine the filter coefficients at positions other than the filter center by parsing the bitstream, while the filter coefficients at the filter center are reconstructed based on the parsed filter coefficients.

[0187] For example, for Figure 7The prediction process for the filter coefficients at the center position of the third filter shown is as follows:

[0188] coeffChroma0=0

[0189] coeffChroma1=0

[0190] for(j=0;j<14;j++){

[0191] coeffChroma0+=2*AlfCoeffChroma[0][j]

[0192] coeffChroma1+=2*AlfCoeffChroma[1][j]

[0193] }

[0194] AlfCoeffChroma[0]

[14] +=64–coeffChroma0

[0195] AlfCoeffChroma[1]

[14] +=64–coeffChroma1

[0196] Where AlfCoeffChroma[0][j] is the filtering coefficient on the Cb component, and AlfCoeffChroma[1][j] is the filtering coefficient on the Cr component.

[0197] After predicting the filter coefficients at the center position, the decoder further determines the reconstructed values ​​of the filter coefficients at the center position based on the predicted filter coefficients and the residual values ​​parsed from the bitstream.

[0198] In one possible implementation, when introducing reconstructed luminance for chroma filtering, the decoder can continue the scheme of reconstructing the filter coefficients at the intermediate position described above. That is, the decoder parses the bitstream, determines the first filter coefficient and coefficient residual value in the first filter; based on the first filter coefficient, it determines the predicted coefficient value of the second filter coefficient in the first filter, where the second filter coefficient is located at the intermediate position of the first filter; based on the predicted coefficient value and coefficient residual value of the second filter coefficient, it determines the reconstructed coefficient of the second filter coefficient. Subsequently, the decoder performs chroma filtering based on the reconstructed coefficients of the first and second filter coefficients.

[0199] In another possible implementation, since the reconstruction of luminance is introduced, the sum of the filter coefficients at each position in the first filter no longer follows the principle of summing to 1 (the chroma filtering process does not amplify chroma). Therefore, for the filter coefficients corresponding to the reconstructed chroma, the process of reconstructing the filter coefficients at the middle position can be omitted. That is, the decoder parses the bit stream and determines the first filter coefficient and the second filter coefficient in the first filter, where the second filter coefficient is the filter coefficient located at the middle position of the first filter.

[0200] To indicate whether to introduce reconstructed luminance for chroma filtering, in one possible implementation, the encoder adds a high-level syntax element to the bitstream indicating whether to introduce reconstructed luminance for chroma filtering. Correspondingly, during decoding, the decoder first parses the element value of this high-level syntax element from the bitstream, and then determines whether to introduce reconstructed luminance during chroma filtering based on the element value.

[0201] In some embodiments, the high-level syntax element may include at least one of the following:

[0202] First syntax element

[0203] Optionally, the first syntax element is located in the sequence header and is used to indicate whether the current image sequence supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0204] Optionally, the element values ​​of the first syntax element include a first value and a second value. The first value indicates that the current image sequence supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance, and the second value indicates that the current image sequence does not support chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0205] Optionally, if the first syntax element does not exist in the bitstream, its element value defaults to the second value.

[0206] In some embodiments, the decoder parses the bitstream and determines the element value of the first syntax element in the sequence header. If the element value of the first syntax element is a first value, the decoder performs chroma filtering on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance; if the element value of the first syntax element is a second value, the decoder performs chroma filtering on the sample to be filtered based on the reconstructed chrominance. The process of the decoder performing chroma filtering on the sample to be filtered based on the reconstructed chrominance can be referred to as the process of performing chroma filtering on the reconstructed chrominance based on the third filter, and will not be elaborated upon in this embodiment.

[0207] In an illustrative example, the first syntax element added to the AVS sequence header is as follows:

[0208] Sequence Header Definition descriptor sequence_header(){ …… grad_chroma_alf_flag u(1) …… }

[0209] When the value of an element in grad_chroma_alf_flag is 1, it indicates that the current image sequence supports chroma filtering based on reconstructed luminance and reconstructed chrominance; when the value of an element in grad_chroma_alf_flag is 0, it indicates that the current image sequence does not support chroma filtering based on reconstructed luminance and reconstructed chrominance.

[0210] In some embodiments, since the process of performing chromaticity filtering based on reconstructed luminance and reconstructed chrominance is adaptive loop filtering, the preconditions for parsing the first syntax element may include at least one of the following: the current image sequence enables adaptive loop filtering, and the current image sequence is an image sequence containing chromaticity components.

[0211] Optionally, if adaptive loop filtering is enabled in the current image sequence, the decoder parses the first syntax element; if adaptive loop filtering is disabled in the current image sequence, the decoder does not parse the first syntax element.

[0212] Optionally, if the current image sequence contains both chroma and luminance components, the decoder parses the first syntax element; if the current image sequence contains only luminance components, the decoder does not parse the first syntax element.

[0213] Second syntax element

[0214] Optionally, the second syntax element is located in the image header and is used to indicate whether the current image supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0215] Optionally, the element values ​​of the second syntax element include a first value and a second value. The first value indicates that the current image supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance, and the second value indicates that the current image does not support chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0216] Optionally, if a second syntax element does not exist in the bitstream, its element value defaults to the second value.

[0217] In some embodiments, the decoder parses the bitstream and determines the element value of the second syntax element in the image header. If the element value of the second syntax element is a first value, the decoder performs chromatic filtering on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance; if the element value of the second syntax element is a second value, the decoder performs chromatic filtering on the sample to be filtered based on the reconstructed chrominance. The process of the decoder performing chromatic filtering on the sample to be filtered based on the reconstructed chrominance can be referred to as the process of performing chromatic filtering on the reconstructed chrominance based on the third filter, and will not be elaborated upon in this embodiment.

[0218] In an illustrative example, the second syntax element added to the image header of the intra-predicted image is as follows:

[0219] Intra-predictive image head definition descriptor intra_picture_header(){ …… picture_grad_chroma_alf_flag u(1) …… }

[0220] In another illustrative example, the second syntax element added to the image header of the inter-frame predicted image is as follows:

[0221] Inter-frame prediction image head definition descriptor inter_picture_header(){ …… picture_grad_chroma_alf_flag u(1) …… }

[0222] When the element value of picture_grad_chroma_alf_flag is 1, it indicates that the current image supports chroma filtering based on reconstructed luminance and reconstructed chroma; when the element value of picture_grad_chroma_alf_flag is 0, it indicates that the current image does not support chroma filtering based on reconstructed luminance and reconstructed chroma.

[0223] In some embodiments, since the process of performing chromaticity filtering based on reconstructed luminance and reconstructed chrominance is adaptive loop filtering, the preconditions for parsing the second syntax element may include at least one of the following: the current image enables adaptive loop filtering, and the current image is an image containing chromaticity components.

[0224] Optionally, if adaptive loop filtering is enabled in the current image, the decoder parses the second syntax element; if adaptive loop filtering is disabled in the current image, the decoder does not parse the second syntax element.

[0225] Optionally, if the current image contains both chroma and luminance components, the decoder parses the second syntax element; if the current image contains only the luminance component, the decoder does not parse the second syntax element.

[0226] In one possible implementation, when only a first syntax element is set and no second syntax element is set, the first syntax element is used to indicate the support of each image in the current image sequence for chroma filtering based on reconstructed luminance. Specifically, when the element value of the first syntax element is a first value, each image in the current image sequence supports chroma filtering based on reconstructed luminance and reconstructed chroma; when the element value of the first syntax element is a second value, none of the images in the current image sequence support chroma filtering based on reconstructed luminance and reconstructed chroma.

[0227] In another possible implementation, when a first syntax element and a second syntax element are provided, if the element value of the first syntax element is a first value, the decoder further parses the second syntax element. If the element value of the second syntax element is a first value, it is determined that the current image supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance. If the element value of the second syntax element is a second value, it is determined that the current image does not support chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0228] When the element value of the first syntax element is the second value, the decoder determines that none of the images in the current image sequence support chromatic filtering based on reconstructed luminance and reconstructed chrominance.

[0229] Of course, in addition to setting sequence-level or image-level syntax elements, slice-level or other-level syntax elements can also be set to indicate the support for chroma filtering based on reconstructed luminance at the slice level or other level. This embodiment does not limit this.

[0230] In this embodiment, by parsing sequence-level and / or image-level syntax elements in the bitstream, it is determined whether the image sequence and / or image supports chroma filtering based on reconstructed luminance and reconstructed chroma. This enables the application of different chroma filtering schemes to different image sequences and / or different images, thereby improving the flexibility of chroma filtering.

[0231] Please refer to Figure 12 The diagram illustrates a flowchart of an image encoding method provided in an exemplary embodiment of this application. The method is applied to an encoder and may include the following steps:

[0232] Step 1201: Perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

[0233] In some embodiments, the reconstructed brightness corresponding to the sample to be filtered includes: the reconstructed brightness of at least one reference sample around the sample to be filtered, or the reconstructed brightness of at least one reference sample around the sample to be filtered and the sample to be filtered.

[0234] In some embodiments, the reconstructed chromaticity corresponding to the sample to be filtered includes: at least one reference sample around the sample to be filtered and the reconstructed chromaticity of the sample to be filtered.

[0235] In one possible implementation, before performing chromatic filtering on the current sample to be filtered, the encoder determines the reference sample corresponding to the sample to be filtered, and determines the reconstructed luminance and reconstructed chrominance of the reference sample. Then, based on the reconstructed luminance and reconstructed chrominance of the reference sample, the encoder performs chromatic filtering on the sample to be filtered to obtain the filtered reconstructed chrominance.

[0236] Here, the chroma filtering refers to chroma adaptive filtering.

[0237] Optionally, the reconstructed luminance is the output of the previous stage of the chromaticity adaptive filtering process, and the reconstructed chromaticity is the output of the previous stage of the chromaticity adaptive filtering process. That is, before using the reconstructed luminance for luminance adaptive filtering, there is no need to perform other filtering processing on the reconstructed luminance, and before using the reconstructed chromaticity for chromaticity adaptive filtering, there is no need to perform other filtering processing on the reconstructed chromaticity.

[0238] Optionally, the reconstructed chromaticity may include reconstructed chromaticity for different color components. Accordingly, for different color components, the encoder performs chromaticity filtering on the sample to be filtered based on the reconstructed luminance and the reconstructed chromaticity for different color components to obtain the filtered reconstructed chromaticity for different color components.

[0239] In an illustrative example, the reconstructed luminance is the luminance reconstruction value output by the SAO, and the reconstructed chrominance is the chrominance reconstruction value output by the SAO, including the chrominance reconstruction values ​​of the Cb component and the chrominance reconstruction values ​​of the Cr component. Accordingly, the encoder performs chrominance filtering on the sample to be filtered based on the luminance reconstruction value and the chrominance reconstruction value of the Cb component; and performs chrominance filtering on the sample to be filtered based on the luminance reconstruction value and the chrominance reconstruction value of the Cr component.

[0240] In one possible implementation, the encoder employs a filter corresponding to chroma filtering, performing chroma filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chroma. The filtering coefficients included in the filter will be detailed in the following embodiments.

[0241] Optionally, after performing chroma filtering on the sample to be filtered to obtain the reconstructed chroma, the encoder can generate a decoded image based on the reconstructed chroma, or it can further process the reconstructed chroma. For example, under the VVC standard, the encoder also needs to add the chroma adaptive filtering result to the CCALF filtering result.

[0242] In summary, in this embodiment, in addition to performing chromatic filtering on the sample to be filtered based on the reconstructed chromaticity corresponding to the sample to be filtered, the reconstructed luminance corresponding to the sample to be filtered is also used as the input for chromatic filtering, which helps to improve the quality of chromatic filtering, thereby improving image encoding and decoding performance and encoding and decoding quality.

[0243] How to reconstruct luminance during chroma filtering in the encoding stage

[0244] Optionally, when using the reconstructed brightness corresponding to the sample to be filtered as the input for chromatic adaptive filtering, the encoder can directly use the reconstructed brightness for chromatic adaptive filtering, or it can first further process the reconstructed brightness and then perform chromatic adaptive filtering based on the reconstructed brightness processing result.

[0245] In one possible implementation, the encoder determines the luminance information of the sample to be filtered based on the reconstructed luminance. Further, the encoder performs chromaticity filtering on the sample based on the luminance information and the reconstructed chromaticity.

[0246] Optionally, this brightness information is used to characterize the brightness features of the sample to be filtered and its surrounding samples. In other words, the process by which the encoder determines the brightness information based on the reconstructed brightness is the process of extracting brightness features from the reconstructed brightness of the sample to be filtered and its surrounding samples.

[0247] In some embodiments, the brightness information is used to characterize the brightness variation features of the sample to be filtered and its surrounding samples, or the brightness mean feature, or the brightness uniformity feature, etc. Accordingly, the brightness information can be the brightness gradient, the brightness mean, or the brightness variance, etc. The embodiments of this application do not limit the specific form of the brightness information.

[0248] In some embodiments, the reconstructed chromaticity of the sample to be filtered is the reconstructed chromaticity of the first reference sample corresponding to the sample to be filtered, and the reconstructed luminance of the sample to be filtered is the reconstructed luminance of the second reference sample corresponding to the sample to be filtered.

[0249] Optionally, the encoder determines the reconstructed brightness of the second reference sample corresponding to the sample to be filtered, and determines the brightness information corresponding to the sample to be filtered based on the reconstructed brightness of the second reference sample.

[0250] In one possible design, the luminance information is a luminance gradient, which includes a horizontal luminance gradient (Gx) and a vertical luminance gradient (Gy).

[0251] The horizontal brightness gradient is used to characterize the rate of change of brightness of the sample in the horizontal direction around the sample to be filtered, and the vertical brightness gradient is used to characterize the rate of change of brightness of the sample in the vertical direction around the sample to be filtered.

[0252] In one possible implementation, the decoder uses a gradient operator to determine the luminance gradient based on the reconstructed luminance of the second reference sample corresponding to the sample to be filtered, and then uses the luminance gradient as the input to the chromatic ALF.

[0253] In one possible implementation, the encoder determining the luminance gradient based on the reconstructed luminance may include the following steps:

[0254] 1. Determine the coordinates of the second reference sample based on the gradient operator and the coordinates of the sample to be filtered.

[0255] In one possible implementation, the encoder determines the corresponding luminance position of the sample to be filtered based on the coordinates of the sample to be filtered, and then determines the coordinates of the second reference sample based on the luminance position and the relative position between the second reference sample indicated by the gradient operator and the luminance position.

[0256] Optionally, the position of the same brightness is format-dependent. For example, if the coordinates of the sample to be filtered are (x, y), the coordinates of the corresponding brightness position are (2x, 2y) in YUV420 format; (x, y) in YUV444 format; and (2x, y) in YUV422 format.

[0257] Optionally, the gradient operator may include the Sobel operator, the Robert operator, other improved or custom operators for calculating the gradient. This application does not limit the specific type of operator.

[0258] Furthermore, the number and location of the second reference samples used to calculate the brightness gradient differ when different gradient operators are employed.

[0259] 2. Based on the coordinates of the second reference sample, obtain the reconstructed brightness of the second reference sample.

[0260] Based on the coordinates of the second reference sample, the encoder obtains the reconstructed brightness of each second reference sample.

[0261] 3. Based on the reconstructed brightness and gradient operator of the second reference sample, determine the brightness gradient corresponding to the sample to be filtered.

[0262] Furthermore, the encoder uses a gradient operator to calculate the gradient of the reconstructed brightness to obtain the horizontal and vertical brightness gradients corresponding to the sample to be filtered.

[0263] The process of the encoder determining the coordinates of the second reference sample, obtaining the reconstructed brightness of the second reference sample, and determining the brightness gradient can be referred to the decoder side, and will not be elaborated here in this embodiment.

[0264] In this embodiment, the encoder determines the second reference sample corresponding to the sample to be filtered based on the gradient operator, and then determines the horizontal and vertical brightness gradients corresponding to the sample to be filtered based on the reconstructed brightness of the second reference sample. Since the brightness variation and chromaticity of the sample are related, the subsequent encoder performs chromaticity filtering based on the brightness gradient and the reconstructed chromaticity of the sample to be filtered, which can further improve the chromaticity filtering quality, thereby improving the image coding quality and performance.

[0265] After determining the luminance information corresponding to the sample to be filtered, the encoder further performs chromaticity filtering on the sample to be filtered based on the luminance information and the reconstructed chromaticity of the first reference sample corresponding to the sample to be filtered.

[0266] In some embodiments, the encoder employs a filter to perform chroma filtering on the samples to be filtered based on luminance information and reconstructed chroma. Optionally, the filter coefficients are obtained online during the actual encoding process based on the current video content.

[0267] Optionally, the encoder writes the filter coefficients of the filter into the Adaptive Parameter Set (APS) and writes the APS ID used by the current image into the slice header or picture header so that the decoder can parse the filter coefficients from the bitstream.

[0268] Filter Structure Description

[0269] In one possible implementation, the encoder uses a third filter to perform chromatic filtering on the sample to be filtered without introducing reconstructed luminance for chromatic filtering.

[0270] In this example, the third filter used for chroma filtering based on the reconstructed luminance is the same at both the encoding and decoding ends. As illustrated in the example... Figure 7 As shown, the third filter includes 15 filter coefficients (coefficient indices from 0 to 14) and contains a total of 29 filter positions.

[0271] Optionally, the third filter is used to indicate the relative positional relationship between the sample to be filtered and the first reference sample. This relative positional relationship can be represented by coordinate offsets, which include horizontal and vertical coordinate offsets.

[0272] Optionally, after the encoder parses the filtering coefficients of the third filter from the bitstream, it obtains the coordinate offset of the corresponding first reference sample from the coordinate offset table based on the coefficient index of the filtering coefficients. Then, based on the coordinate offset and the coordinates of the sample to be filtered, the coordinates of the first reference sample are determined, and the reconstructed chromaticity of the first reference sample is obtained based on the coordinates of the first reference sample.

[0273] For each first reference sample, the encoder accumulates the reconstructed chromaticity of the first reference sample and its filtering coefficients, thereby determining the reconstructed chromaticity of the filtered sample based on the accumulation result.

[0274] The process of the encoder using a third filter for chroma filtering is consistent with that on the decoder side, and will not be described in detail here.

[0275] Since luminance information is additionally referenced for chroma filtering, in one possible implementation, the filter used for chroma filtering includes a first filter and a second filter. The first filter is used for filtering based on the reconstructed chroma of the first reference sample, and the second filter is used for filtering based on the luminance information.

[0276] In some embodiments, the first filter is used to indicate the relative positional relationship between the sample to be filtered and the first reference sample. The relative positional relationship can be represented by coordinate offsets, which include horizontal coordinate offsets and vertical coordinate offsets.

[0277] Correspondingly, when the encoder determines the first reference sample corresponding to the sample to be filtered, it determines the first reference sample based on the coordinates of the sample to be filtered and the coordinate offset represented by the first filter, and then obtains the reconstructed chromaticity of the first reference sample.

[0278] In one possible implementation, the first filter is the same as the third filter, while the second filter is a newly added filter. Furthermore, since the filter coefficients contained in the newly added second filter need to be encoded into the bitstream, the bitstream is increased.

[0279] In some embodiments, when the second filter is used for filtering based on the brightness gradient, the filtering coefficients of the second filter include the filtering coefficients corresponding to the horizontal brightness gradient and the vertical brightness gradient, respectively.

[0280] In an illustrative example, such as Figure 8 As shown, the first filter and Figure 7 The third filter shown is the same as the second filter, which is a new filter containing two filter coefficients (coefficient indices 15 and 16).

[0281] In this embodiment, when the first filter and the third filter are the same, the encoder uses the first filter and the second filter to perform chroma filtering in the same way as the decoder. This will not be described in detail here.

[0282] In another possible implementation, in order to reduce the impact of adding a second filter on the bitstream size, a first filter can be obtained by modifying the third filter so that the number of filter coefficients of the first filter is less than the number of filter coefficients of the third filter (i.e. the first filter is different from the third filter), thereby reducing the impact of adding filter coefficients of the second filter on the bitstream size.

[0283] Optionally, the first filter can be obtained based on the third filter in the following manner.

[0284] Method 1: Delete some filter coefficients in the third filter.

[0285] By changing the shape of the filter, a first filter with fewer filter coefficients can be obtained by deleting some of the filter coefficients in the third filter.

[0286] In one possible implementation, the first filter includes one or more filter coefficients selected from the third filter, i.e., the first filter contains only a portion of the filter coefficients from the third filter.

[0287] In one possible design, the removed filter coefficients can be located anywhere on the third filter.

[0288] In another possible design, in order to reduce the impact of changing the filter shape on the filtering performance, the deleted filter coefficients can be located in the third filter at a position where the filtering performance has less impact.

[0289] Since the impact on filtering performance decreases the further away from the current filtered sample, in one possible implementation, the deleted filter coefficients are located at the edge of the third filter. In other words, one or more filter coefficients selected from the third filter in the first filter are located at the non-edge of the third filter.

[0290] Optionally, the edge refers to a position where the distance between the edge and the middle position of the third filter is greater than a threshold. For example, the threshold can be 2, 3, etc., and this embodiment does not limit this.

[0291] In the case where the first filter is obtained by deleting some of the filter coefficients from the third filter, the encoder's process of using the first filter and the second filter for chroma filtering can refer to Method 1 on the decoder side, which will not be elaborated here in this embodiment.

[0292] Method 2: Reuse some of the filter coefficients in the third filter.

[0293] Without changing the shape of the filter, a first filter with fewer filter coefficients can be obtained by increasing the number of multiplexed filter coefficients in the third filter.

[0294] In one possible implementation, the first filter adds one or more of the same filter coefficients to the third filter.

[0295] In one possible design, the reused filter coefficients are located at any position on the third filter.

[0296] In another possible design, in order to reduce the impact of multiplexed filter coefficients on filtering performance, the multiplexed filter coefficients can be located in the third filter at a position where they have less impact on filtering performance.

[0297] Since the impact on filtering performance decreases the further away from the current filtered sample, in one possible implementation, the reused filter coefficients are located at the edge of the third filter. In other words, one or more identical filter coefficients added in the first filter are located at the edge of the third filter.

[0298] Optionally, the edge refers to a position where the distance between the edge and the middle position of the third filter is greater than the threshold.

[0299] In the case where the first filter is obtained by adding multiplexed filtering coefficients to the third filter, the encoder's process of using the first filter and the second filter for chroma filtering can refer to Method 2 on the decoder side, which will not be elaborated here in this embodiment.

[0300] In this embodiment, the number of filter coefficients in the first filter (used for chroma filtering based on reconstructed chroma) is reduced by deleting or reusing filter coefficients at specific locations in the filter. This improves chroma filtering performance by introducing reconstructed luminance for chroma filtering, while minimizing the impact on the bitstream size caused by adding a second filter (for chroma filtering based on luminance information).

[0301] Regarding the process of writing filter coefficients into the bitstream, without introducing reconstructed luminance for chroma filtering, the decoder writes the filter coefficients from positions outside the filter center into the bitstream, and writes the residual between the predicted value and the original value of the filter coefficient at the center position into the bitstream. The predicted value of the filter coefficient at the center position is obtained based on the prediction of the filter coefficients from positions outside the center position.

[0302] For example, for Figure 7 The prediction process for the filter coefficients at the center position of the third filter shown is as follows:

[0303] coeffChroma0=0

[0304] coeffChroma1=0

[0305] for(j=0;j<14;j++){

[0306] coeffChroma0+=2*AlfCoeffChroma[0][j]

[0307] coeffChroma1+=2*AlfCoeffChroma[1][j]

[0308] }

[0309] AlfCoeffChroma[0]

[14] +=64–coeffChroma0

[0310] AlfCoeffChroma[1]

[14] +=64–coeffChroma1

[0311] Where AlfCoeffChroma[0][j] is the filtering coefficient on the Cb component, and AlfCoeffChroma[1][j] is the filtering coefficient on the Cr component.

[0312] After predicting the filter coefficients at the center position, the encoder further calculates the residual value between the original value and the predicted value of the filter coefficients at the center position, and then writes the residual value into the bitstream. Since the residual value is small, encoding the residual value of the filter coefficients at the center position can save codewords compared to directly encoding the original value.

[0313] In one possible implementation, when chroma filtering is performed using reconstructed luminance, the decoder can continue the scheme of reconstructing the filter coefficients at the intermediate position described above. That is, the encoder determines the predicted value of the coefficient of the second filter coefficient in the first filter based on the first filter coefficient in the first filter, the second filter coefficient being the filter coefficient located at the intermediate position of the first filter; determines the residual value between the predicted value of the coefficient and the original value of the second filter coefficient; and encodes the first filter coefficient in the first filter and the coefficient residual value into the bitstream.

[0314] In another possible implementation, since the reconstruction of luminance is introduced, the sum of the filter coefficients at each position in the first filter no longer follows the principle of summing to 1 (the chroma filtering process does not amplify chroma). Therefore, for the filter coefficients corresponding to the reconstructed chroma, the process of reconstructing the filter coefficients at the middle position can be omitted. That is, the encoder encodes the first filter coefficient and the second filter coefficient in the first filter into the bit stream. The second filter coefficient is the filter coefficient located at the middle position of the first filter.

[0315] To indicate whether to introduce reconstructed luminance for chroma filtering, in one possible implementation, the encoder adds a high-level syntax element to the bitstream indicating whether to introduce reconstructed luminance for chroma filtering.

[0316] In some embodiments, the high-level syntax elements set by the encoder may include at least one of the following:

[0317] First syntax element

[0318] Optionally, the first syntax element is located in the sequence header and is used to indicate whether the current image sequence supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0319] Optionally, the element values ​​of the first syntax element include a first value and a second value. The first value indicates that the current image sequence supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance, and the second value indicates that the current image sequence does not support chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0320] Optionally, if the first syntax element does not exist in the bitstream, its element value defaults to the second value.

[0321] In some embodiments, where the current image sequence supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance, the encoder sets the element value of the first syntax element in the sequence header to a first value.

[0322] If the current image sequence does not support chromaticity filtering based on reconstructed luminance and reconstructed chrominance, the encoder sets the element value of the first syntax element in the sequence header to the second value.

[0323] In some embodiments, since the process of performing chromaticity filtering based on reconstructed luminance and reconstructed chrominance is adaptive loop filtering, the preconditions for setting the first syntax element may include at least one of the following: the current image sequence enables adaptive loop filtering, and the current image sequence is an image sequence containing chromaticity components.

[0324] Optionally, if adaptive loop filtering is enabled in the current image sequence, the encoder sets the element value of the first syntax element based on the current image sequence's support for chromatic filtering based on reconstructed luminance and reconstructed chrominance; if adaptive loop filtering is disabled in the current image sequence, the decoder does not set the first syntax element.

[0325] Optionally, if the current image sequence contains both chroma and luminance components, the encoder sets the element value of the first syntax element based on the current image's support for chroma filtering based on reconstructed luminance and reconstructed chroma; if the current image sequence contains only luminance components, the decoder does not set the first syntax element.

[0326] Second syntax element

[0327] Optionally, the second syntax element is located in the image header and is used to indicate whether the current image supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0328] Optionally, the element values ​​of the second syntax element include a first value and a second value. The first value indicates that the current image supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance, and the second value indicates that the current image does not support chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0329] Optionally, if a second syntax element does not exist in the bitstream, its element value defaults to the second value.

[0330] In some embodiments, where the current image supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance, the encoder sets the element value of the second syntax element in the image header to the first value.

[0331] If the current image does not support chromaticity filtering based on reconstructed luminance and reconstructed chrominance, the encoder sets the element value of the second syntax element in the image header to the second value.

[0332] In some embodiments, since the process of performing chromaticity filtering based on reconstructed luminance and reconstructed chrominance is adaptive loop filtering, the preconditions for setting the second syntax element may include at least one of the following: the current image enables adaptive loop filtering, and the current image is an image containing chromaticity components.

[0333] Optionally, if adaptive loop filtering is enabled for the current image, the encoder sets the element value of the second syntax element based on the current image's support for chromatic filtering based on reconstructed luminance and reconstructed chrominance; if adaptive loop filtering is disabled for the current image, the decoder does not set the second syntax element.

[0334] Optionally, if the current image contains both chroma and luminance components, the encoder sets the element value of the second syntax element based on the current image's support for chroma filtering based on reconstructed luminance and reconstructed chroma; if the current image contains only the luminance component, the encoder does not set the second syntax element.

[0335] In some embodiments, when only the first syntax element is set, if the encoder sets the element value of the first syntax element to a first value, it indicates that each image in the current image sequence supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance; if the encoder sets the element value of the first syntax element to a second value, it indicates that none of the images in the current image sequence support chromaticity filtering based on reconstructed luminance and reconstructed chrominance.

[0336] In some embodiments, when both the first and second syntax elements are set simultaneously, the encoder sets the element value of the first syntax element to a first value, and then further sets the element value of the second syntax element based on the support for chroma filtering based on reconstructed luminance and reconstructed chroma in each image of the current image sequence. When the editor sets the element value of the first syntax element to a second value, it indicates that none of the images in the current image sequence support chroma filtering based on reconstructed luminance and reconstructed chroma, and there is no need to further set the second syntax element in the image header of each image.

[0337] Of course, in addition to setting sequence-level or image-level syntax elements, slice-level or other-level syntax elements can also be set to indicate the support for chroma filtering based on reconstructed luminance at the slice level or other level. This embodiment does not limit this.

[0338] In this embodiment, by adding sequence-level and / or image-level syntax elements, which indicate whether the image sequence and / or image supports chromaticity filtering based on reconstructed luminance and reconstructed chrominance, different chromaticity filtering schemes can be applied to different image sequences and / or different images, thus improving the flexibility of chromaticity filtering.

[0339] Please refer to Figure 13 This illustrates a structural block diagram of a decoding apparatus provided in an exemplary embodiment of this application. The decoding apparatus includes:

[0340] The decoding unit 1301 is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

[0341] Optionally, the decoding unit 1301 is used for:

[0342] The brightness information corresponding to the sample to be filtered is determined based on the reconstructed brightness corresponding to the sample to be filtered.

[0343] Determine the first reference sample corresponding to the sample to be filtered;

[0344] Based on the brightness information and the reconstructed chromaticity of the first reference sample, chromaticity filtering is performed on the sample to be filtered.

[0345] Optionally, the decoding unit 1301 is used for:

[0346] Determine the reconstructed brightness of the second reference sample corresponding to the sample to be filtered;

[0347] Based on the reconstructed brightness of the second reference sample, the brightness information corresponding to the sample to be filtered is determined.

[0348] Optionally, the brightness information is a brightness gradient, and the brightness gradient includes a horizontal brightness gradient and a vertical brightness gradient.

[0349] Optionally, the decoding unit 1301 is used for:

[0350] Based on the gradient operator and the coordinates of the sample to be filtered, the coordinates of the second reference sample are determined;

[0351] Based on the coordinates of the second reference sample, the reconstructed brightness of the second reference sample is obtained;

[0352] Based on the reconstructed brightness and the gradient operator of the second reference sample, the brightness gradient corresponding to the sample to be filtered is determined.

[0353] Optionally, the decoding unit 1301 is used for:

[0354] Analyze the bitstream to determine the filter coefficients;

[0355] Based on the brightness information, the reconstructed chromaticity of the first reference sample, and the filtering coefficients of the filter, chromaticity filtering is performed on the sample to be filtered.

[0356] Optionally, the filter includes a first filter and a second filter, wherein the first filter is used to filter based on the reconstructed chromaticity of the first reference sample, and the second filter is used to filter based on the luminance information.

[0357] Optionally, the decoding unit 1301 is used for:

[0358] The first reference sample is determined based on the coordinates of the sample to be filtered and the coordinate offset represented by the first filter.

[0359] Optionally, the first filter is the same as the third filter;

[0360] or,

[0361] The first filter is different from the third filter;

[0362] The third filter is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed chromaticity.

[0363] Optionally, if the first filter is different from the third filter, the first filter includes one or more filter coefficients selected from the third filter.

[0364] Optionally, the filter coefficients obtained by the screening are located on the non-edge of the third filter.

[0365] Optionally, if the first filter is different from the third filter, the first filter may add one or more identical filter coefficients to the third filter.

[0366] Optionally, the newly added one or more identical filter coefficients are located at the edge of the third filter.

[0367] Optionally, the decoding unit 1301 is used for:

[0368] The code stream is parsed to determine the first filter coefficient and the coefficient residual value in the first filter; the predicted value of the second filter coefficient in the first filter is determined based on the first filter coefficient, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter; the reconstructed coefficient of the second filter coefficient is determined based on the predicted value of the coefficient and the coefficient residual value of the second filter coefficient.

[0369] or,

[0370] The bitstream is parsed to determine the first filter coefficient and the second filter coefficient in the first filter, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter.

[0371] Optionally, the decoding unit 1301 is used for:

[0372] Parse the bitstream to determine the value of the first syntax element in the sequence header;

[0373] When the element value of the first syntax element is a first value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance.

[0374] Optionally, the decoding unit 1301 is used for:

[0375] When the element value of the first syntax element is the second value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed chromaticity.

[0376] Optionally, the decoding unit 1301 is used for:

[0377] Parse the bitstream to determine the element value of the second syntax element in the image header;

[0378] When the element value of the second syntax element is the first value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance.

[0379] Optionally, the decoding unit 1301 is used for:

[0380] When the element value of the second syntax element is the second value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed chromaticity.

[0381] Please refer to Figure 14 This illustrates a structural block diagram of an encoding apparatus provided in an exemplary embodiment of this application. The encoding apparatus includes:

[0382] The encoding unit 1401 is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

[0383] Optionally, encoding unit 1401 is used for:

[0384] The brightness information corresponding to the sample to be filtered is determined based on the reconstructed brightness corresponding to the sample to be filtered.

[0385] Determine the first reference sample corresponding to the sample to be filtered;

[0386] Based on the brightness information and the reconstructed chromaticity of the first reference sample, chromaticity filtering is performed on the sample to be filtered.

[0387] Optionally, encoding unit 1401 is used for:

[0388] Determine the reconstructed brightness of the second reference sample corresponding to the sample to be filtered;

[0389] Based on the reconstructed brightness of the second reference sample, the brightness information corresponding to the sample to be filtered is determined.

[0390] Optionally, the brightness information is a brightness gradient, and the brightness gradient includes a horizontal brightness gradient and a vertical brightness gradient.

[0391] Optionally, encoding unit 1401 is used for:

[0392] Based on the gradient operator and the coordinates of the sample to be filtered, the coordinates of the second reference sample are determined;

[0393] Based on the coordinates of the second reference sample, the reconstructed brightness of the second reference sample is obtained;

[0394] Based on the reconstructed brightness and the gradient operator of the second reference sample, the brightness gradient corresponding to the sample to be filtered is determined.

[0395] Optionally, encoding unit 1401 is used for:

[0396] The filter coefficients of the filter are encoded into the bitstream. The filter is used to perform chromaticity filtering on the sample to be filtered based on the luminance information and the reconstructed chromaticity of the first reference sample.

[0397] Optionally, the filter includes a first filter and a second filter, wherein the first filter is used to filter the reconstructed chromaticity of the first reference sample, and the second filter is used to filter the luminance information.

[0398] Optionally, encoding unit 1401 is used for:

[0399] The first reference sample is determined based on the coordinates of the sample to be filtered and the coordinate offset represented by the first filter.

[0400] Optionally, the first filter is the same as the third filter;

[0401] or,

[0402] The first filter is different from the third filter;

[0403] The third filter is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed chromaticity.

[0404] Optionally, if the first filter is different from the third filter, the first filter includes one or more filter coefficients selected from the third filter.

[0405] Optionally, the filter coefficients obtained by the screening are located on the non-edge of the third filter.

[0406] Optionally, if the first filter is different from the third filter, the first filter may add one or more identical filter coefficients to the third filter.

[0407] Optionally, the newly added one or more identical filter coefficients are located at the edge of the third filter.

[0408] Optionally, encoding unit 1401 is used for:

[0409] Based on the first filter coefficient in the first filter, determine the predicted value of the second filter coefficient in the first filter, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter; determine the residual value between the predicted value of the coefficient and the original value of the second filter coefficient; encode the first filter coefficient in the first filter and the coefficient residual value into the bitstream;

[0410] or,

[0411] The first filter coefficient and the second filter coefficient in the first filter are encoded into the bit stream, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter.

[0412] Optionally, encoding unit 1401 is used for:

[0413] If the current image sequence supports chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the first syntax element in the sequence header is set to the first value;

[0414] If the current image sequence does not support chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the first syntax element in the sequence header is set to the second value.

[0415] Optionally, encoding unit 1401 is used for:

[0416] If the current image supports chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the second syntax element in the image header is set to the first value;

[0417] If the current image does not support chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the second syntax element in the image header is set to the second value.

[0418] It should be noted that the image decoding process of the above-mentioned decoding unit can refer to the above-mentioned image decoding method embodiment, and the image encoding process of the above-mentioned encoding unit can refer to the above-mentioned image encoding method embodiment. This embodiment will not be described in detail here.

[0419] Please refer to Figure 15 This diagram illustrates a structural block diagram of a decoder provided in an exemplary embodiment of this application. The decoder may include one or more components such as a processor 1501 and a memory 1502. The components are coupled together via a bus system. It is understood that the bus system is used to enable communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0420] Memory 1502 is used to store computer programs that can run on processor 1501; processor 1501 is used to execute, when running the computer program:

[0421] Based on the reconstructed luminance and reconstructed chrominance of the sample to be filtered, chrominance filtering is performed on the sample to be filtered.

[0422] It is understood that the memory 1502 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0423] The processor 1501 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the aforementioned image decoding method can be completed by the integrated logic circuitry in the hardware of the processor 1501 or by software instructions. The processor 1501 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1502. Processor 1501 reads information from memory 1502 and, in conjunction with its hardware, completes the steps of the above method. It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by the processor. The memory can be implemented in the processor or external to the processor. Optionally, as another embodiment, processor 1501 is also configured to execute the image decoding method described in any of the foregoing embodiments when running the computer program.

[0424] Please refer to Figure 16This diagram illustrates a structural block diagram of an encoder provided in an exemplary embodiment of this application. The encoder may include one or more components such as a processor 1601 and a memory 1602. The components are coupled together via a bus system. It is understood that the bus system is used to enable communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0425] Memory 1602 is used to store computer programs that can run on processor 1601; processor 1601 is used to execute, when running the computer program:

[0426] Based on the reconstructed luminance and reconstructed chrominance of the sample to be filtered, chrominance filtering is performed on the sample to be filtered.

[0427] It is understood that the memory 1602 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1602 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0428] The processor 1601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the aforementioned image encoding method can be completed by the integrated logic circuitry in the hardware of the processor 1601 or by instructions in software form. The processor 1601 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1602. Processor 1601 reads information from memory 1602 and, in conjunction with its hardware, completes the steps of the above method. It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by the processor. The memory can be implemented in the processor or external to the processor. Optionally, as another embodiment, processor 1601 is also configured to execute the image encoding method described in any of the foregoing embodiments when running the computer program.

[0429] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement the image decoding method or the image encoding method described in the above embodiments.

[0430] This application also provides a computer-readable storage medium storing a bitstream, the bitstream being generated by an image encoding method using an encoder, or the bitstream being decoded by an image decoding method using a decoder, wherein the image encoding method includes the image encoding method described in any of the above embodiments, and the image decoding method includes the image decoding method described in any of the above embodiments.

[0431] This application also provides a bitstream, which is decoded using the image decoding method described in any of the above embodiments, or encoded using the image encoding method described in any of the above embodiments.

[0432] This application also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. A processor retrieves the computer instructions from the computer-readable storage medium and executes the computer instructions to implement the image decoding method or the image encoding method as described in the above embodiments.

[0433] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0434] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image decoding method, applied to a decoder, characterized in that, The method includes: Based on the reconstructed luminance and reconstructed chrominance of the sample to be filtered, chrominance filtering is performed on the sample to be filtered.

2. The method according to claim 1, characterized in that, The step of performing chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered includes: The brightness information corresponding to the sample to be filtered is determined based on the reconstructed brightness corresponding to the sample to be filtered. Determine the first reference sample corresponding to the sample to be filtered; Based on the brightness information and the reconstructed chromaticity of the first reference sample, chromaticity filtering is performed on the sample to be filtered.

3. The method according to claim 2, characterized in that, The step of determining the brightness information corresponding to the sample to be filtered based on the reconstructed brightness corresponding to the filtered sample includes: Determine the reconstructed brightness of the second reference sample corresponding to the sample to be filtered; Based on the reconstructed brightness of the second reference sample, the brightness information corresponding to the sample to be filtered is determined.

4. The method according to claim 3, characterized in that, The brightness information is a brightness gradient, and the brightness gradient includes a horizontal brightness gradient and a vertical brightness gradient.

5. The method according to claim 4, characterized in that, Determining the reconstructed brightness of the second reference sample corresponding to the sample to be filtered includes: Based on the gradient operator and the coordinates of the sample to be filtered, the coordinates of the second reference sample are determined; Based on the coordinates of the second reference sample, the reconstructed brightness of the second reference sample is obtained; The process of determining the brightness information corresponding to the sample to be filtered based on the reconstructed brightness of the second reference sample includes: Based on the reconstructed brightness and the gradient operator of the second reference sample, the brightness gradient corresponding to the sample to be filtered is determined.

6. The method according to claim 2, characterized in that, The method includes: Analyze the bitstream to determine the filter coefficients; The step of performing chromaticity filtering on the sample to be filtered based on the luminance information and the reconstructed chromaticity of the first reference sample includes: Based on the brightness information, the reconstructed chromaticity of the first reference sample, and the filtering coefficients of the filter, chromaticity filtering is performed on the sample to be filtered.

7. The method according to claim 6, characterized in that, The filter includes a first filter and a second filter. The first filter is used to filter based on the reconstructed chromaticity of the first reference sample, and the second filter is used to filter based on the luminance information.

8. The method according to claim 7, characterized in that, Determining the first reference sample corresponding to the sample to be filtered includes: The first reference sample is determined based on the coordinates of the sample to be filtered and the coordinate offset represented by the first filter.

9. The method according to claim 7, characterized in that, The first filter is the same as the third filter; or, The first filter is different from the third filter; The third filter is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed chromaticity.

10. The method according to claim 9, characterized in that, When the first filter and the third filter are different; The first filter includes one or more filter coefficients selected from the third filter.

11. The method according to claim 10, characterized in that, The filter coefficients obtained by the screening are located at the non-edge of the third filter.

12. The method according to claim 9, characterized in that, When the first filter and the third filter are different; The first filter adds one or more identical filter coefficients to the third filter.

13. The method according to claim 12, characterized in that, The newly added one or more identical filter coefficients are located at the edge of the third filter.

14. The method according to claim 7, characterized in that, The process of parsing the bitstream and determining the filter coefficients includes: The code stream is parsed to determine the first filter coefficient and the coefficient residual value in the first filter; the predicted value of the second filter coefficient in the first filter is determined based on the first filter coefficient, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter; the reconstructed coefficient of the second filter coefficient is determined based on the predicted value of the coefficient and the coefficient residual value of the second filter coefficient. or, The bitstream is parsed to determine the first filter coefficient and the second filter coefficient in the first filter, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter.

15. The method according to any one of claims 1 to 14, characterized in that, The method further includes: Parse the bitstream to determine the value of the first syntax element in the sequence header; The step of performing chromaticity filtering on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance includes: When the element value of the first syntax element is a first value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance.

16. The method according to claim 15, characterized in that, The method further includes: When the element value of the first syntax element is the second value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed chromaticity.

17. The method according to any one of claims 1 to 14, characterized in that, The method further includes: Parse the bitstream to determine the element value of the second syntax element in the image header; The step of performing chromaticity filtering on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance includes: When the element value of the second syntax element is the first value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed luminance and the reconstructed chrominance.

18. The method according to claim 17, characterized in that, The method further includes: When the element value of the second syntax element is the second value, chromaticity filtering is performed on the sample to be filtered based on the reconstructed chromaticity.

19. An image encoding method, applied to an encoder, characterized in that, The method includes: Based on the reconstructed luminance and reconstructed chrominance of the sample to be filtered, chrominance filtering is performed on the sample to be filtered.

20. The method according to claim 19, characterized in that, The step of performing chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered includes: The brightness information corresponding to the sample to be filtered is determined based on the reconstructed brightness corresponding to the sample to be filtered. Determine the first reference sample corresponding to the sample to be filtered; Based on the brightness information and the reconstructed chromaticity of the first reference sample, chromaticity filtering is performed on the sample to be filtered.

21. The method according to claim 20, characterized in that, The step of determining the brightness information corresponding to the sample to be filtered based on the reconstructed brightness includes: Determine the reconstructed brightness of the second reference sample corresponding to the sample to be filtered; Based on the reconstructed brightness of the second reference sample, the brightness information corresponding to the sample to be filtered is determined.

22. The method according to claim 21, characterized in that, The brightness information is a brightness gradient, and the brightness gradient includes a horizontal brightness gradient and a vertical brightness gradient.

23. The method according to claim 22, characterized in that, Determining the reconstructed brightness of the second reference sample corresponding to the sample to be filtered includes: Based on the gradient operator and the coordinates of the sample to be filtered, the coordinates of the second reference sample are determined; Based on the coordinates of the second reference sample, the reconstructed brightness of the second reference sample is obtained; The process of determining the brightness information corresponding to the sample to be filtered based on the reconstructed brightness of the second reference sample includes: Based on the reconstructed brightness and the gradient operator of the second reference sample, the brightness gradient corresponding to the sample to be filtered is determined.

24. The method according to claim 20, characterized in that, The method includes: The filter coefficients of the filter are encoded into the bitstream. The filter is used to perform chromaticity filtering on the sample to be filtered based on the luminance information and the reconstructed chromaticity of the first reference sample.

25. The method according to claim 24, characterized in that, The filter includes a first filter and a second filter. The first filter is used to filter the reconstructed chromaticity of the first reference sample, and the second filter is used to filter the luminance information.

26. The method according to claim 25, characterized in that, Determining the first reference sample corresponding to the sample to be filtered includes: The first reference sample is determined based on the coordinates of the sample to be filtered and the coordinate offset represented by the first filter.

27. The method according to claim 25, characterized in that, The first filter is the same as the third filter; or, The first filter is different from the third filter; The third filter is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed chromaticity.

28. The method according to claim 27, characterized in that, When the first filter and the third filter are different; The first filter includes one or more filter coefficients selected from the third filter.

29. The method according to claim 28, characterized in that, The filter coefficients obtained by the screening are located at the non-edge of the third filter.

30. The method according to claim 27, characterized in that, When the first filter and the third filter are different; The first filter adds one or more identical filter coefficients to the third filter.

31. The method according to claim 30, characterized in that, The newly added one or more identical filter coefficients are located at the edge of the third filter.

32. The method according to claim 25, characterized in that, Encoding the filter coefficients of the filter into the bitstream includes: Based on the first filter coefficient in the first filter, determine the predicted value of the second filter coefficient in the first filter, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter; determine the residual value between the predicted value of the coefficient and the original value of the second filter coefficient; encode the first filter coefficient in the first filter and the coefficient residual value into the bitstream; or, The first filter coefficient and the second filter coefficient in the first filter are encoded into the bit stream, wherein the second filter coefficient is the filter coefficient located in the middle position of the first filter.

33. The method according to any one of claims 19 to 32, characterized in that, The method further includes: If the current image sequence supports chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the first syntax element in the sequence header is set to the first value; If the current image sequence does not support chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the first syntax element in the sequence header is set to the second value.

34. The method according to any one of claims 19 to 32, characterized in that, The method further includes: If the current image supports chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the second syntax element in the image header is set to the first value; If the current image does not support chromaticity filtering based on the reconstructed luminance and the reconstructed chrominance, the element value of the second syntax element in the image header is set to the second value.

35. A decoding device, characterized in that, The device includes: The decoding unit is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

36. An encoding device, characterized in that, The device includes: The encoding unit is used to perform chromaticity filtering on the sample to be filtered based on the reconstructed luminance and reconstructed chrominance corresponding to the sample to be filtered.

37. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the image decoding method as described in any one of claims 1 to 18, or the image encoding method as described in any one of claims 19 to 34.

38. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a bitstream, which is generated by an image encoding method using an encoder, or the bitstream is decoded by an image decoding method using a decoder, wherein the image encoding method includes the method of any one of claims 19 to 34, and the image decoding method includes the method of any one of claims 1 to 18.

39. A bitstream, characterized in that, The bitstream is decoded using the image decoding method as described in any one of claims 1 to 18, or encoded using the image encoding method as described in any one of claims 19 to 34.

40. A decoder, characterized in that, The decoder includes a memory and a processor, the memory being used to store a computer program running on the processor; the processor being used to execute the image decoding method as described in any one of claims 1 to 18 when running the computer program.

41. An encoder, characterized in that, The encoder includes a memory and a processor, the memory being used to store a computer program running on the processor; the processor being used to execute the image encoding method as described in any one of claims 19 to 34 when running the computer program.

Images