A video coding loop filtering method, a video decoding method and apparatus

Abstract

The embodiment of the application provides a video coding loop filtering method, a video decoding method and device, the method comprises the following steps: filtering the reconstructed image of the current image by using a first filter to obtain a first reconstructed video unit; filtering the reconstructed image by using a second filter to obtain a second reconstructed video unit; determining filter coefficients according to the neighborhood information contained in the convolution template of the first reconstructed video unit and the second reconstructed video unit; filtering the first reconstructed video unit, the second reconstructed video unit and the filter coefficients based on the convolution template and the filter coefficients to obtain a third reconstructed video unit, which can solve the problem that the weighting fusion operation for a single pixel point in the related art is not good in filtering adaptation effect, and the convolution-based filtering mode can add more adjacent pixel information in the filtering correction process as much as possible, so that the reconstructed value is closer to the original value, and better output image quality is achieved.

Description

Technical Field

[0001] This application relates to the field of communications, and more specifically, to a video encoding / decoding loop filtering method, a video decoding method, and an apparatus. Background Technology

[0002] Neural Network In-Loop Filter (NNF) can be added as a new module to in-loop filtering, replacing the original Deblocking Filter (DBF) and Sample Adaptive Offset (SAO) modules, or running in parallel with traditional filtering modules. Generally, in codecs, after NNF filtering, an Adaptive Loop Filter (ALF) is still used to further improve image performance.

[0003] NNF filtering typically employs offline training, loading the weights obtained from offline training into a VTM (a reference software testing platform for VVC) for use, which yields good results. However, since existing networks are generally offline, there is no original image as a reference in practical applications, leading to issues of NNF filtering being either too strong or too weak. Therefore, when performing NNF filtering, it is necessary to adapt it to the characteristics of the current image.

[0004] One adaptation method is to introduce a scaling factor for intensity adjustment (Scale) operation. The Scale operation uses the least squares method to fit a straight line so that the reconstructed frame after NNF filtering approximates the original frame, achieving the effect of NNF filtering intensity adapting to the original image. In addition, strip-level or coding block-level switching operations are added to control the effective range of NNF filtering.

[0005] The second adaptation method is to add DBF filtering for weighting. Weighting schemes include using only the DBF output, using NN filtering, and DBF filtering weighting. The pixels involved in the weighting are the current pixels to be adapted. This scheme can be flexibly selected and controlled at the strip level or the coding block level.

[0006] However, both of the above adaptation methods are weighted fusion operations for individual pixels, resulting in poor filtering adaptation effects. There is still room for improvement in the correction of NNF filtering, and no effective solution has been proposed for this problem at present. Summary of the Invention

[0007] This application provides a video encoding / decoding loop filtering method, a video decoding method, and an apparatus to at least solve the problem of poor filtering adaptation effect in the weighted fusion operation for a single pixel in related technologies.

[0008] According to one embodiment of this application, a video encoding / decoding loop filtering method is provided, the method comprising:

[0009] The reconstructed image of the current image is filtered using the first filter to obtain the first reconstructed video unit;

[0010] The reconstructed image is filtered using a second filter to obtain a second reconstructed video unit;

[0011] The filter coefficients are determined based on the neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is the information of adjacent pixels centered on the pixel in the convolution template;

[0012] Based on the convolution template and the filter coefficients, the first reconstructed video unit and the second reconstructed video unit are filtered to obtain the third reconstructed video unit.

[0013] According to another embodiment of this application, a video decoding method is also provided, comprising:

[0014] Obtain a bitstream containing an encoded video sequence to obtain loop filter control information corresponding to the loop filter method, wherein the loop filter control information includes filter coefficients;

[0015] The reconstructed image of the current image is filtered using the first filter to obtain the first reconstructed video unit;

[0016] The reconstructed image of the current image is filtered using a second filter to obtain the second reconstructed video unit;

[0017] Based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit, as well as the filter coefficients, the first reconstructed video unit and the second reconstructed video unit are filtered to obtain the third reconstructed video unit.

[0018] According to another embodiment of this application, a video encoding / decoding loop filtering device is also provided, the device comprising:

[0019] The first filtering module is used to filter the reconstructed image of the current image using a first filter to obtain the first reconstructed video unit;

[0020] The second filtering module is used to filter the reconstructed image of the current image using a second filter to obtain the second reconstructed video unit;

[0021] The first determining module is used to determine filter coefficients based on the neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is the information of adjacent pixels centered on the pixel in the convolution template.

[0022] The third filtering module is used to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain the third reconstructed video unit.

[0023] According to another embodiment of this application, a video decoding apparatus is also provided, the apparatus comprising:

[0024] The acquisition module is used to acquire a bitstream containing the encoded video sequence and obtain loop filter control information corresponding to the loop filter method.

[0025] The first filtering module is used to filter the reconstructed image of the current image using a first filter to obtain the first reconstructed video unit;

[0026] The second filtering module is used to filter the reconstructed image of the current image using the second filter to obtain the second reconstructed video unit;

[0027] The third filtering module is used to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain the third reconstructed video unit.

[0028] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0029] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0030] In this embodiment, a first filter is used to filter the reconstructed image of the current image to obtain a first reconstructed video unit; a second filter is used to filter the reconstructed image to obtain a second reconstructed video unit; filter coefficients are determined based on the neighborhood information contained in the convolution templates of the first and second reconstructed video units, wherein the neighborhood information is the information of adjacent pixels centered on the pixel in the convolution template; based on the convolution template and the filter coefficients, the first and second reconstructed video units are filtered to obtain a third reconstructed video unit. This can solve the problem of poor filtering adaptation effect in the weighted fusion operation for a single pixel in related technologies. The convolution-based filtering method can add more neighboring pixel information as much as possible during the filtering correction process, so that the reconstructed value can better approximate the original value and achieve better output image quality. Attached Figure Description

[0031] Figure 1 This is a hardware structure block diagram of a mobile terminal for a video encoding / decoding loop filtering method according to an embodiment of this application;

[0032] Figure 2 This is a flowchart of a video encoding / decoding loop filtering method according to an embodiment of this application;

[0033] Figure 3 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 1 ;

[0034] Figure 4 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 1 ;

[0035] Figure 5 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 2 ;

[0036] Figure 6 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 3 ;

[0037] Figure 7 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 4 ;

[0038] Figure 8 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 5 ;

[0039] Figure 9 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 6 ;

[0040] Figure 10 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 2 ;

[0041] Figure 11 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 3 ;

[0042] Figure 12 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 4 ;

[0043] Figure 13 This is a flowchart of a video decoding method according to an embodiment of this application;

[0044] Figure 14 This is a schematic diagram of the convolution-based filtering adaptation operation at the decoding end according to this embodiment;

[0045] Figure 15 This is a block diagram of a video codec loop filter according to an embodiment of this application;

[0046] Figure 16 This is a block diagram of a video decoding apparatus according to an embodiment of this application. Detailed Implementation

[0047] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0049] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for the video encoding / decoding loop filtering method according to an embodiment of this application, as shown below. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or programmable logic device, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0050] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the video codec loop filtering method in this embodiment. The processor 102 executes various functional applications and video codec loop filtering by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0052] This embodiment provides a video codec loop filtering method operating on the aforementioned mobile terminal or network architecture. Figure 2 This is a flowchart of a video encoding / decoding loop filtering method according to an embodiment of this application, such as... Figure 2 As shown, this process, applied to the encoding / decoding end, includes the following steps:

[0053] Step S202: Filter the reconstructed image of the current image using the first filter to obtain the first reconstructed video unit;

[0054] In this embodiment, step S202 may specifically include: performing Deblocking Loop Filtering (DBF) on the reconstructed image; or performing Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or performing Deblocking Loop Filtering (DBF) and Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or performing Luminance Mapping Based on Chroma Scaling (LMCS) processing on the reconstructed image.

[0055] Step S204: Filter the reconstructed image using a second filter to obtain a second reconstructed video unit;

[0056] In this embodiment, step S204 may specifically include: performing neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image and weighted summing the neural network loop filtering result with the first reconstructed video unit to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image and adjusting the intensity of the neural network loop filtering result according to the original image to obtain the second reconstructed video unit; or performing other loop filtering on the reconstructed image besides the neural network loop filtering to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of the following: Deblocking Loop Filter (DBF), Sample Adaptive Compensation Loop Filter (SAO), and Luminance Mapping Based on Chroma Scaling (LMCS).

[0057] Step S206: Determine filter coefficients based on the neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is the information of adjacent pixels centered on the pixel in the convolution template;

[0058] Step S208: Based on the convolution template and the filter coefficients, filter the first reconstructed video unit and the second reconstructed video unit to obtain the third reconstructed video unit.

[0059] Through the above steps S202 to S208, the problem of poor filtering adaptation effect in the weighted fusion operation for a single pixel in related technologies can be solved. The convolution-based filtering method can add more neighboring pixel information as much as possible during the filtering correction process, so that the reconstructed value can better approximate the original value and achieve better output image quality.

[0060] In one embodiment, step S208 may specifically include: for each pixel in the video unit of the reconstructed image and the video unit of the original image, determining the difference between the pixel at each neighborhood position in the convolution template and each pixel, multiplying the differences to obtain a first pixel product, and determining the difference between the pixel in the video unit of the original image and the pixel in the video unit of the reconstructed image to obtain the covariance; and determining the filter coefficients based on the covariance using the Wiener filtering method.

[0061] In another embodiment, step S208 may further include: calculating the difference between the center pixel and the neighboring pixels of the convolution template based on the first reconstructed video unit, and multiplying it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a first result; simultaneously calculating the difference between the center pixel and the neighboring pixels of the convolution template based on the second reconstructed video unit, and multiplying it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a second result; adding the first result, the second result, and the second reconstructed video unit together, and then rounding down to obtain the third reconstructed video unit.

[0062] In S202 above, DBF filtering can be used; in S204 above, NN filtering can be used; and in S208 above, CCA filtering can be used. Correspondingly, the third reconstructed video unit is the CCAF filtering result. Specifically, the filtering process can be performed in the following way to obtain the third reconstructed video unit, which is the following CCAF filtering result:

[0063] NNout′=int(NNout C +w0*(DBFout t -NNout C )+w1*(DBFout l -NNout C )+w2*(DBFout c -NNout C )+w3*(DBFout r -NNout C )+w4*(DBFout b -NNout C )+w5*(NNout t -NNout C )+w6*(NNout l -NNout C )+w7*(NNout r -NNout C )+w8*(NNout b -NNoutC)>>BITS_NUM+0.5); where NNout′ is the CCAF filtering result, DBFout is the first reconstructed video unit, NNout is the second reconstructed video unit, and w i Let w0 be the filter coefficient for the i-th video unit, w0 to w4 be the filter coefficients for the first reconstructed video unit, and w5 to w8 be the filter coefficients for the second reconstructed video unit.

[0064] In this embodiment, the video unit of the reconstructed image mentioned above includes the reconstructed image, or slice, or coded block, or coded tree block.

[0065] In another embodiment, step S208 may further include at least one of the following: filtering the luminance components of the first reconstructed video unit and the second reconstructed video unit according to the filter coefficients based on the convolution template; filtering the first chroma components of the first reconstructed video unit and the second reconstructed video unit according to the filter coefficients based on the convolution template; and filtering the second chroma components of the first reconstructed video unit and the second reconstructed video unit according to the filter coefficients based on the convolution template.

[0066] In an optional embodiment, the method further includes: determining a first loss between pixels of a video unit in the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and pixels of the original image; determining a second loss between pixels of a video unit in the reconstructed image before filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and pixels of the original image; and determining whether to perform CCA filtering on the video units of the current image based on the first loss and the second loss.

[0067] In another optional embodiment, the method further includes: determining a first loss between pixels of a video unit in the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and pixels of the original image; filtering by preset filter coefficients, and determining a third loss between pixels of a video unit in the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and pixels of the original image; and determining whether to perform the filtering process by the filter coefficients or by the preset filter coefficients based on the first loss and the third loss.

[0068] Specifically, the process involves determining the first loss after filtering with the filter coefficients; determining the second loss before filtering with the filter coefficients, or determining the third loss after filtering with preset filter coefficients; determining whether to perform CCA filtering based on the first loss and the second loss, or the first loss and the third loss; determining to perform CCA filtering when the first loss is less than the second loss, or the first loss is less than the third loss, and setting the CCA filtering identifier to 1; and determining not to perform CCA filtering when the first loss is not less than the second loss, or the first loss is not less than the third loss, and setting the CCA filtering identifier to 0.

[0069] Furthermore, when the filter coefficients are the filter coefficients corresponding to each slice of the reconstructed image, if the first loss is less than the second loss, it is determined that each slice of the reconstructed image is subjected to CCA filtering based on the filter coefficients corresponding to the slice; if the first loss is not less than the second loss, it is determined that each slice of the reconstructed image is not subjected to CCA filtering based on the filter coefficients corresponding to the slice; or if the first loss is less than the second loss, it is determined that each block of the reconstructed image is subjected to CCA filtering based on the filter coefficients corresponding to the slice; if the first loss is not less than the second loss, it is determined that each block of the reconstructed image is not subjected to CCA filtering based on the filter coefficients corresponding to the slice, wherein the block is one of the following: one or more CTUs, one or more CUs, or a pixel block of a preset size.

[0070] In another optional embodiment, the method further includes: determining a first loss between pixels of a video unit in the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and pixels of the original image; determining a second loss between pixels of a video unit in the reconstructed image before filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and pixels of the original image; performing the filtering on the video unit of the current image according to the filter coefficients of neighboring video units of the current image, and determining a fourth loss between pixels of a video unit in the reconstructed image after filtering and pixels of the original image; determining the filter coefficients used by the video unit of the current image according to the first loss, the second loss, and the fourth loss, and performing the filtering according to the determined filter coefficients.

[0071] In another optional embodiment, the method further includes: determining a first loss between the pixels of the video of the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image; determining a second loss between the pixels of the video unit of the reconstructed image before filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image; determining whether to perform the filtering on the video unit of the current image based on the first loss and the second loss, and recalculating the filter coefficients for the video unit with the filtering enabled, and determining whether to perform filtering again based on the first loss and the second loss, repeating this process N times to confirm whether to perform the filtering on the video unit of the current image.

[0072] The convolution template in this embodiment includes one of the following: a rhombus template, a square template, or a cross-shaped template. Specifically, it can be a 3*3 rhombus template, a 5*5 rhombus template, a 7*7 rhombus template, a 3*3 square template, a 5*5 square template, a 7*7 square template, or a 5*5 cross-shaped template. These are just some commonly used ones and are not limited to the scope of protection of this embodiment.

[0073] In one embodiment, the method further includes: encoding the filter coefficients with the reconstructed image to obtain a video stream; wherein the pps_cca_enabled_flag of the video stream is an identifier for the CCA filtering, wherein the pps_cca_enabled_flag is 1 or 0, where 1 indicates that CCA filtering is performed on the current slice, and 0 indicates that CCA filtering is not performed on the current slice; or, provided that the pps_cca_enabled_flag is 1, the cca_block_flag of the video stream is used to identify whether CCA filtering is performed on the current block, wherein the cca_block_flag is 1 or 0, wherein 1 indicates that CCA filtering is performed on the current block, and 0 indicates that CCA filtering is not performed on the current block.

[0074] In this embodiment, the pps_cca_info_in_ph_flag of the video bitstream is used to indicate the position of the filter coefficient. pps_cca_info_in_ph_flag is 1 or 0. 1 indicates that the filter coefficient is in the picture header, and 0 indicates that the filter coefficient is in the slice header.

[0075] In another embodiment, step S206 may specifically include: performing CCA filtering on the slice or block with identifier 1 for the CCA filter, based on the first filtering result, the second filtering result, and the filter coefficients corresponding to each slice, to obtain the CCA filtering result. Specifically, obtaining the first sub-filtering result of the luminance component and / or chrominance component in the first filtering result, and obtaining the second sub-filtering result of the luminance component and / or chrominance component in the second filtering result; performing CCA filtering on each slice or block of the reconstructed image based on the first sub-filtering result, the second sub-filtering result, and the filter coefficients to obtain the CCA filtering result.

[0076] In one embodiment, the method further includes: encoding the filter coefficients with the reconstructed image to obtain a video stream; wherein, the `cca_luma_enabled_flag` of the video stream is used to identify whether the luminance component of the current slice undergoes CCA filtering, and the `cca_luma_enabled_flag` is 1 or 0, where 1 indicates that the luminance component of the current slice undergoes CCA filtering, and 0 indicates that the luminance component of the current slice does not undergo CCA filtering; the `cca_cb_enabled_flag` is used to identify whether the chrominance Cb component of the current slice undergoes CCA filtering, and the `cca_cb_enabled_flag` is 1 or 0, where 1 indicates that the chrominance Cb component of the current slice undergoes CCA filtering, and 0 indicates that the chrominance Cb component of the current slice does not undergo CCA filtering. The slice performs CCA filtering; cca_cr_enabled_flag is used to indicate whether the chroma Cr component of the current slice is CCA filtered. cca_cr_enabled_flag is 1 or 0, where 1 indicates that the chroma Cr component of the current slice is CCA filtered, and 0 indicates that the chroma Cr component of the current slice is not CCA filtered; or cca_block_flag[i] is used to indicate whether the luma or chroma component of the current block is CCA filtered. cca_block_flag[i] is 1 or 0, where 1 indicates that the luma or chroma component of the current block is CCA filtered, and 0 indicates that the luma or chroma component of the current block is not CCA filtered; i = 0 represents the luma component, i = 1 represents the chroma Cb component, and i = 2 represents the chroma Cr component.

[0077] In another embodiment, the method further includes: repeatedly iterating N times to perform the following steps to obtain the identifier of CCA filtering and the corresponding filter coefficients for each block: for the target block whose CCA filtering identifier is 1, determining the target filter coefficients of the target block; determining the first image cost after CCA filtering and the second image cost before CCA filtering for each block in the target block, and setting the identifier of CCA filtering for the target block whose first image cost is less than the second image cost to 1.

[0078] In another embodiment, the method further includes: when the filter coefficient is the filter coefficient corresponding to each block of the reconstructed image, performing the following steps on each block of the reconstructed image to update the filter coefficient of each block through a coefficient fusion mode, wherein the block being executed is called the current block: for the pixels of the current block, determining the first image cost after CCA filtering by the filter coefficient; if the upper adjacent block of the current block has a filter coefficient, determining the fourth image cost of the current block as the image cost of the upper adjacent block after CCA filtering by the filter coefficient; if the left adjacent block of the current block has a filter coefficient, determining the fifth image cost of the current block as the first image cost of the left adjacent block after CCA filtering by the filter coefficient; determining the sixth image cost of the current block that does not undergo CCA filtering by the filter coefficient; selecting the filter coefficient corresponding to the smallest image cost from the first image cost, the fourth image cost, the fifth image cost, and the sixth image cost as the filter coefficient of the current block, wherein the block is one of the following: one or more CTUs, one or more CUs, or a pixel block of a preset size.

[0079] In another embodiment, the method further includes: encoding the filter coefficients with the reconstructed image to obtain a video bitstream; assuming the CCA filter identifier is 1, the cca_block_flag of the video bitstream is used to identify whether the current block is subjected to CCA filtering, where cca_block_flag is 1 or 0, 1 indicating that CCA filtering is performed on the current block, and 0 indicating that CCA filtering is not performed on the current block; when cca_block_flag is 1, cca_block_merge_flag[i][BlkAddrX][BlkAddrY] is used to identify whether the current block uses the coefficient fusion mode, where cca_block_merge_flag[i][BlkAddrX][BlkAddrY] is 1 or 0, 1 indicating that the coefficient fusion mode is used; when cca_block_merge_flag[i][BlkAddrX][BlkAddrY] is 1, cca_block_merge_mode_flag[0][BlkAddrX][BlkAddrY] is 1. [AddrY] is used to identify the type of coefficient merging mode for the current block. The cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] is 1 or 0, where 1 indicates that the filter coefficients of the upper adjacent block are used, and 0 indicates that the filter coefficients of the left adjacent block are used; where i is 0 to represent the luminance component, i is 1 to represent the chrominance Cb component, and i is 2 to represent the chrominance Cr component.

[0080] In this embodiment, a multi-neighbor pixel-based convolutional filtering design adaptation algorithm (CCAF) is introduced during the NN filtering correction process. Since existing solutions perform weighted fusion operations on individual pixels, CCAF's convolutional filtering method can add as much neighboring pixel information as possible during the NN filtering correction process, allowing the reconstructed value to better approximate the original value and achieve better output image quality.

[0081] In this embodiment, the encoding end uses a convolution-based filtering adaptation algorithm. The process of the encoding end CCA filtering method includes:

[0082] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in recBuf.

[0083] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0084] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCAF stage, and the CCAF filter coefficients are calculated and filtered using the original image.

[0085] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCAF filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0086] In this embodiment, the neighborhood information refers to the neighborhood information centered on the pixel at the specified location in the CCF convolution template.

[0087] The neighborhood information involved in the calculation of CCA filter coefficients includes: the neighborhood information of the pixel at the specified location in the Rec image, the neighborhood information of the pixel at the specified location in the NNin image, and the neighborhood information of the pixel at the specified location in the NNout image.

[0088] As an optional implementation method, Figure 3 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 1 ,like Figure 3 As shown, the neighborhood information of CCA filtering includes: c represents the pixel to be filtered in the middle position, t represents the neighboring pixel above, b represents the neighboring pixel below, l represents the neighboring pixel to the left, and r represents the neighboring pixel to the right.

[0089] Figure 4 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 1 ,like Figure 4As shown in the diagram above, the process of applying a convolution-based filtering adaptation algorithm at the encoding end is as follows: The output (DBFout) of the traditional filtering module (e.g., DBF) and the output (NNout) of the NN filtering module are used as the inputs of the CCA module, and a multi-point weighted summation operation is performed on the outputs of the traditional module and the NN module.

[0090] This can be extended to adapting algorithms for any two filtering schemes, not limited to DBF filtering and NN filtering.

[0091] Step 1: DBF filtering. Perform DBF filtering on the reconstructed image Rec after LMCS and store the DBF filtering result in dbfoutBuf.

[0092] Step 2: NN filtering. After obtaining the reconstructed image of LMCS, NN filtering is performed.

[0093] First, configure the various data required to input the network, such as reconstructed image samples, QP information, CU partitioning information, deblocking filtering information, predicted samples, etc. Then, input this information into the NN filtering network module to obtain the reconstructed image samples after NN filtering. Store the NN filtering result in NNoutBuf.

[0094] Step 3: A convolution-based filtering adaptation algorithm is used. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering. Specific steps include:

[0095] The principle behind Wiener filtering is as follows:

[0096] The mean square error (MSE) after filtering is calculated as follows:

[0097]

[0098] Where n represents the number of coefficients, c is the coefficient to be found, and p is the offset of the center relative to the coordinates. The above equation can be decomposed as follows:

[0099]

[0100]

[0101]

[0102]

[0103]

[0104] By solving the coefficients based on the above equations, the optimal performance for a single pixel can be obtained.

[0105] For multiple pixels in a slice, the following equation can be written.

[0106]

[0107] Solve this equation to obtain the filter coefficients.

[0108] Step 3-a: Covariance calculation, including:

[0109] (1) For each pixel, calculate the difference between each position of the convolution template and the center point, and multiply the differences respectively. At the same time, calculate the org-current pixel value of the current point.

[0110] (2) Perform the calculation of (1) on all pixels of a slice.

[0111] In the main scheme, the filter template is set as follows: Figure 3 The template shown is not limited to this one; it can be expanded or reduced.

[0112] Step 3-b: Calculate the CCA filter coefficients based on the covariance, including:

[0113] Based on the covariance calculated in step 3-a, the equations are established and solved to obtain the filter coefficients (c0 to c8).

[0114]

[0115] Since the coefficients obtained directly from the formula are floating-point numbers, the coefficients are quantized into integers (w0 to w8).

[0116] w i =int((c i <<BITS_NUM+0.5)).

[0117] Here, BITS_NUM is set to 7. o(i) is the pixel value of the reconstructed image, r(i) is the pixel value of the image after NN filtering, and pi is the position offset of the pixel from its neighboring pixels. Shifting the coefficients to the left converts the coefficients to integers.

[0118] Step 3-c: Construct the CCA output signal, obtain the calculated filter coefficients, and then perform the filtering operation using the following formula:

[0119] NNout′=int((NNout C +w0*(DBFout t -NNout C )+w1*(DBFout l -NNout C )+w2*(DBFout c -NNoutC )+w3*(DBFout r -NNout C )+w4*(DBFout b -NNout C )+w5*(NNout t -NNout C )+w6*(NNout l -NNout C )+w7*(NNout r -NNout C )+w8*(NNout b -NNout C ))>>BITS_NUM+0.5).

[0120] Where DBFout refers to the output after passing through the DBF filter, NNout refers to the output after passing through the NN filter, and w i Refers to the i-th filter coefficient, where w0 to w4 are the filter coefficients for the DBF output, and w5 to w8 are the filter coefficients for the NN output.

[0121] (Since the coefficients were shifted to the left in step 3-b, they need to be shifted back to the right after use.)

[0122] Degenerates into a weighted scheme of DBF and NN:

[0123] After removing neighborhood information, all coefficients of w except w2 are 0;

[0124] NNout′=NNout C +w2*(DBFout c -NNout C ) = w2 * DBFout c +(1-w2)*NNout C ;

[0125] Degenerate to scale: Remove neighborhood information; all w coefficients except w2 are 0. Convert DBFout... c Turn into Rec c (i.e., NNin, which is also the reconstructed value before DBF).

[0126]

[0127] Optional implementation method 1: Change the CCA input.

[0128] Because there are now many ways to combine NN filtering modules with traditional filtering modules, the input of the CCA module is not fixed to the method in the main scheme. Output 1 and input 2 can be modified separately.

[0129] Method 1-1: Change CCA input 1, specifically change Rec. In the main Rec scheme, Rec represents the output of DBF. In actual operation, it can also be the reconstructed image after passing through traditional filtering modules such as DBF and SAO, or the output before DBF.

[0130] Rec represents the reconstructed image before DBF. Figure 5 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 2 ,like Figure 5 As shown, it includes the following steps:

[0131] Step 1: Store the reconstructed image after LMCS into recBuf.

[0132] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0133] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0134] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template; the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0135] Rec represents the reconstructed image after DBF and SAO. Figure 6 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 3 ,like Figure 6 As shown, the steps are as follows:

[0136] Step 1: Perform DBF and SAO filtering on the reconstructed image after LMCS, and store the SAO filtering result in saooutBuf.

[0137] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0138] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0139] For a single pixel in a slice, filter coefficients are calculated based on the convolution template, CCA filter coefficients are calculated based on the covariance, and filtering is performed based on the convolution template.

[0140] Methods 1-2: The CCA input 2 can also be changed to modify the NN-related parameter. In the main scheme, NN-related represents the output of the neural network (NN). However, due to the offline nature of NN, a weighted operation related to the reconstructed pixels is commonly performed after NN. Therefore, NN-related can also refer to the output related to NN. This includes:

[0141] NNrelated indicates that the output of NN is weighted by DBF. Figure 7 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 4 ,like Figure 7 As shown, it includes:

[0142] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0143] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0144] The pixel values ​​in recBuf are weighted together with the pixel values ​​in nnoutBuf, and the weighted result is placed in nnoutBuf.

[0145] nnout=dbfout*(1-k)+nnout*k.

[0146] k ranges from [0,1], for example, it can be 0.75, 0.5, etc.

[0147] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0148] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template; the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0149] NNrelated indicates the result after scaling the neural network. Figure 8 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 5 ,like Figure 8 As shown, it includes:

[0150] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in recBuf.

[0151] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0152] The pixel values ​​after LMCS (which are the inputs to the NN) are scaled together with the pixel values ​​in nnoutBuf, and the result is placed in nnoutBuf. The scale can be calculated using the least squares method or a preset value.

[0153] nnout=nnin+scale*(nnout-nnin).

[0154] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0155] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template; the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0156] Methods 1-3: There is also the broader CCA, which is not limited to use in NN-based encoding and decoding frameworks, but can also be used in traditional encoding and decoding frameworks.

[0157] For example, the reconstructed image without DBF and the reconstructed image with DBF can be used as two inputs to CCA to perform CCA operation. Figure 9 This is a schematic diagram of convolution-based filtering adaptation at the encoding end according to this embodiment. Figure 6 ,like Figure 9 As shown, the steps are as follows:

[0158] Step 1: Store the reconstructed image after LMCS into recBuf.

[0159] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in dbfoutBuf.

[0160] Step 3: Perform the CCA adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0161] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template; the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0162] Optional Implementation Method 2: CCA Decision-Making Mechanism.

[0163] Since the coefficients need to be calculated at the encoding end and then transmitted to the decoding end, this may increase the transmission overhead of the video encoded bitstream. Furthermore, when there are many pixels, calculating a set of coefficients for a slice may not provide performance gain for some pixels or may even degrade performance. Therefore, this solution proposes multiple decision-making mechanisms.

[0164] Optional implementation method 2-1: Switch control.

[0165] Slice-level switching: DBF filtering is applied to the reconstructed image after LMCS, and the DBF filtering result is stored in dbfoutBuf. NN filtering is applied to the reconstructed image after LMCS, and the NN filtering result is stored in nnoutBuf. A convolution-based filter adaptation algorithm is then performed. The results stored in the two buffers from the above steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0166] For a single pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template. The CCA filter coefficients are then calculated based on the covariance. Slice-level switching control includes:

[0167] (1) Calculate the cost of the adaptive coefficient obtained in 3-b (cost represents RDcost).

[0168] (2) Calculate the cost of reconstructing the image without using CCA.

[0169] (3) The current slice competes with and without using CCA, identifies the optimal solution and transmits it. If there is no performance gain or loss after CCA, the identifier is set to 0, indicating that the current slice will not perform CCA operation; otherwise, the identifier is set to 1, indicating that the current slice will perform CCA operation.

[0170] For slices with flag 1, perform CCA operation as described in step 2-b of Example 1 to obtain the reconstructed image. Classes with flag 0 are not processed.

[0171] Block-level switching: DBF filtering is applied to the reconstructed image after LMCS, and the DBF filtering result is stored in dbfoutBuf. NN filtering is applied to the reconstructed image after LMCS, and the NN filtering result is stored in nnoutBuf. A convolution-based filter adaptation algorithm is then performed. The results stored in the two buffers from the above steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0172] For a single pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolutional template. The CCA filter coefficients are then calculated based on the covariance. Block-level switching control is implemented, including:

[0173] (1) Calculate the cost of each block after using the adaptive coefficients obtained in 3-b (cost represents RDcost).

[0174] (2) Calculate the cost of reconstructing the image for each block without using CCA.

[0175] (3) For each block, compete for CCA and non-CCA, identify the optimal solution and transmit it. If there is no performance gain or deterioration after CCA, set the identifier to 0 to indicate that the block will not perform CCA operation; otherwise, set the identifier to 1 to indicate that the block will perform CCA operation.

[0176] For blocks with flag 1, perform CCA operation as described in step 2-b of Example 1 to obtain the reconstructed image. Blocks with flag 0 are not processed.

[0177] The coefficient optimization scheme in this embodiment includes: performing DBF filtering on the reconstructed image after LMCS and storing the DBF filtering result in dbfoutBuf; performing NN filtering on the reconstructed image after LMCS and storing the NN filtering result in nnoutBuf; performing a convolution-based filtering adaptation algorithm; and feeding the results stored in the two buffers from the above steps into the CCA stage to calculate coefficients and perform filtering operations.

[0178] For a single pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template. The CCA filter coefficients are then calculated based on the covariance, and coefficient optimization is performed, including:

[0179] (1) Calculate the cost before and after using CCA for each block (a block can refer to a CTU or an NN operation block). If there is no performance gain or loss after CCA, set the identifier to 0 to indicate that the block does not perform CCA operation; otherwise, set the identifier to 1 to indicate that the block will perform CCA operation.

[0180] (2) For the block with identifier 1, recalculate a set of coefficients and then repeat step (1). Iterate the coefficients N times according to this step (N is not limited and can be 3). Calculate the cost corresponding to the coefficient each time and compare the optimized coefficients with the final identifier of each block.

[0181] For blocks with flag 1, perform CCA operation as described in step 2-b of Example 1 to obtain the reconstructed image. Blocks with flag 0 are not processed.

[0182] Optional implementation method 2-2: competitive decision-making.

[0183] The selection process involves a competition between preset and adaptive coefficients, including:

[0184] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0185] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0186] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0187] For a single pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolutional template. The CCA filter coefficients are then calculated based on the covariance. A competitive decision-making process is then employed, including:

[0188] ① Calculate the cost of the adaptive coefficients obtained in 3-b (cost represents RDcost).

[0189] ②3-c Calculate the cost for the preset coefficients (the preset coefficients are set according to statistical information).

[0190] ③ The adaptive coefficient competes with the preset coefficient, identifies the optimal solution, and transmits it.

[0191] Filter using coefficients with low cost according to step 3-c of Example 1.

[0192] Optional Implementation Method 3: Block-level CCA.

[0193] Since the coefficients in the above schemes are all slice-level, the adaptability of using only one set of coefficients for a single slice of pixels may not be high. Therefore, this scheme proposes a block-level CCA technique. The block-level CCA operation steps include:

[0194] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0195] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0196] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage in blocks to calculate coefficients and perform filtering.

[0197] For each pixel in a block, the covariance is calculated based on the neighborhood information contained in the convolution template. CCA filter coefficients are then calculated based on the covariance, and filtering is performed on the current block using the convolution template. The specific operation is similar to that described above and will not be repeated here.

[0198] Add block-level switch control:

[0199] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0200] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0201] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage in blocks to calculate coefficients and perform filtering.

[0202] For each pixel in a block, the covariance is calculated based on the neighborhood information contained in the convolution template. The CCA filtering coefficients are then calculated based on the covariance. The cost before and after applying CCA is calculated for each block. If there is no performance gain or deterioration after CCA, the identifier is set to 0, indicating that CCA is not performed on this block; otherwise, the identifier is set to 1, indicating that CCA is performed on this block.

[0203] For blocks with flag 1, perform CCA operation as described in step 3-c of Example 1 to obtain the reconstructed image. Blocks with flag 0 are not processed.

[0204] Fusion Mode: Due to the significant overhead of block-level transmission coefficients, a coefficient fusion mode is introduced to save some transmission overhead. For the current block, since adjacent regions have a certain correlation, the current block can use the coefficients calculated from the upper or left block to complete the CCA filtering of the current block. The steps are as follows:

[0205] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0206] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0207] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage in blocks to calculate coefficients and perform filtering.

[0208] For a block of pixels, the covariance is calculated based on the neighborhood information contained in the convolution template, and the CCA filter coefficients are calculated based on the covariance. The main steps of the fusion process are as follows:

[0209] ① Calculate the cost after using the 3-b adaptive coefficient.

[0210] ② If the upper block of the current block has a CCA coefficient, then calculate the cost of directly using the upper block's coefficient for the current block.

[0211] ③ If the left block of the current block has a CCA coefficient, then calculate the cost of directly using the left block coefficient for the current block.

[0212] ④ Calculate the cost without performing CCA.

[0213] ⑤ Compare the above four costs. Select the optimal solution for the current block.

[0214] For blocks with flag 1, perform CCA operation as described in step 3-c of Example 1 to obtain the reconstructed image. Blocks with flag 0 are not processed.

[0215] Optional implementation method 4: Change the convolution template.

[0216] The main scheme uses a 3x3 diamond-shaped template. Expanding the module allows for the incorporation of more information during the CCA process, while shrinking the template saves bits during CCA transmission. Based on this, there are many other template options available.

[0217] The following solution only modifies the shape of the convolution template; the overall operation steps are as follows:

[0218] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0219] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0220] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0221] For a block of pixels, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0222] This embodiment is not limited to a 3x3 rhombus template. Figure 10 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 2 ,like Figure 10 As shown, it can be a 5x5 template or a 7x7 template, etc.

[0223] This embodiment is not limited to a rhombus-shaped template. Figure 11 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 3 ,like Figure 11As shown, it can also be a square or other shapes.

[0224] Figure 12 This is a schematic diagram of the convolution model of CCA filtering according to this embodiment. Figure 4 ,like Figure 12 As shown, the Rec and NN template shapes can be different.

[0225] In this embodiment, the center point of the main scheme is the center point of the NN template, or it can be the center point of Rec.

[0226] NNout′=int((DBFout c +w0*(DBFout t -DBFout c )+w1*(DBFout l -DBFout c )+w2*(DBFout r -DBFout c )+w3*(DBFout b -DBFout c )+w4*(NNout t -DBFout c )+w5*(NNout l -DBFout c )+w6*(NNout c -DBFout c )+w7*(NNout r -DBFout c )+w8*(NNout b -DBFout c ))>>BITS_NUM+0.5).

[0227] Optional implementation method 5: CCA of different components and CCA between components. The CCA method is not limited to the components. Luminance and chrominance can be used or turned off separately.

[0228] Both luminance and chrominance use CCA, and the steps are as follows:

[0229] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0230] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0231] Step 3: Perform a convolution-based filtering adaptation algorithm. The results of the luminance and chrominance components stored in the two buffers from the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0232] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0233] To use CCA for brightness only, follow these steps:

[0234] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0235] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0236] Step 3: Perform a convolution-based filtering adaptation algorithm. The results of the luminance components from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0237] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0238] To use CCA for chroma only, follow these steps:

[0239] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0240] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0241] Step 3: Perform a convolution-based filtering adaptation algorithm. The results of the chromaticity components stored in the two buffers from the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0242] For a pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filter coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0243] Cross Components CCA (CCA mode) considers that luminance generally contains more information than chrominance, so luminance can be used as the filter coefficient for calculating CCA, including:

[0244] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0245] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0246] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to calculate coefficients and perform filtering.

[0247] For a single pixel in a slice, the covariance is calculated based on the neighborhood information contained in the convolution template. The CCA filter coefficients are then calculated based on the covariance to construct the CCA output signal.

[0248] After obtaining the calculated filter coefficients, perform the filtering operation using the following formula:

[0249] NNout_chroma′=int((NNout_chroma C +w0*(DBFout_luma t -NNout_luma C )+w1*(DBFout_luma l -NNout_luma C )+w2*(DBFout_luma c -NNout_luma C )+w3*(DBFout_luma r -NNout_luma C )+w4*(DBFout_luma b -NNout_luma C )+w5*(NNout_luma t -NNout_luma C )+w6*(NNout_luma l -NNout_luma C )+w7*(NNout_luma r -NNout_luma C )+w8*(NNout_luma b -NNout_luma C ))>>BITS_NUM+0.5)

[0250] Where DBFout_luma refers to the output of the luminance component after passing through the DBF filter, NNout_luma refers to the output of the luminance component after passing through the NN filter, and NNout_chroma refers to the output of the luminance component after passing through the NN filter.C This refers to the output of the chrominance components after passing through an NN filter, w i Refers to the i-th filter coefficient, where w0 to w4 are the filter coefficients for the DBF output, and w5 to w8 are the filter coefficients for the NN output.

[0251] According to another embodiment of this application, a video decoding method is also provided. Figure 13 This is a flowchart of a video decoding method according to an embodiment of this application, such as... Figure 13 As shown, the process includes the following steps:

[0252] Step S1302: Obtain a bitstream containing the encoded video sequence and obtain loop filter control information corresponding to the loop filter method, wherein the loop filter control information includes filter coefficients;

[0253] Step S1304: Filter the reconstructed image of the current image using the first filter to obtain the first reconstructed video unit;

[0254] In this embodiment, step S1304 may specifically include: performing Deblocking Loop Filtering (DBF) on the reconstructed image; or performing Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or performing Deblocking Loop Filtering (DBF) and Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or performing Luminance Mapping Based on Chroma Scaling (LMCS) processing on the reconstructed image. Similar to step S202 above.

[0255] Step S1306: Filter the reconstructed image of the current image using the second filter to obtain the second reconstructed video unit;

[0256] In this embodiment, step S1306 may specifically include: performing neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image, and weighting and summing the neural network loop filtering result with the first reconstructed video unit to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image, and adjusting the intensity of the neural network loop filtering result according to the original image to obtain the second reconstructed video unit; or performing other loop filtering on the reconstructed image besides neural network loop filtering to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of the following: Deblocking Loop Filter (DBF), Sample Adaptive Compensation Loop Filter (SAO), and Luminance Mapping Based on Chroma Scaling (LMCS). Similar to step S204 above.

[0257] Step S1308: Based on the convolution templates and filter coefficients of the first reconstructed video unit and the second reconstructed video unit, filter the first reconstructed video unit and the second reconstructed video unit to obtain the third reconstructed video unit.

[0258] Through the above steps S1302 to S1308, the problem of poor filtering modification of weighted fusion operation for a single pixel in related technologies can be solved. The convolution-based filtering method can add more neighboring pixel information as much as possible during the filtering correction process, so that the reconstructed value can better approximate the original value and achieve better output image quality.

[0259] In this embodiment, step S1308 may specifically include: calculating the difference between the center pixel and neighboring pixels of the convolution template based on the first reconstructed video unit, and multiplying it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a first result; simultaneously calculating the difference between the center pixel and neighboring pixels of the convolution template based on the second reconstructed video unit, and multiplying it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a second result; adding the first result, the second result, and the second reconstructed video unit, and then rounding down to obtain the third reconstructed video unit. This is similar to step S208.

[0260] In S1304 above, DBF filtering can be used; in S1306 above, NN filtering can be used; and in S1308 above, CCA filtering can be used. Correspondingly, the third reconstructed video unit is the CCAF filtering result. Specifically, the filtering process can be performed in the following way to obtain the third reconstructed video unit, which is the following CCAF filtering result:

[0261] NNout′=int(NNout C +w0*(DBFout t -NNout C )+w1*(DBFout l -NNout C )+w2*(DBFout c -NNout C )+w3*(DBFout r -NNout C )+w4*(DBFout b -NNout C )+w5*(NNout t -NNout C )+w6*(NNout l -NNout C )+w7*(NNout r -NNoutC )+w8*(NNout b -NNoutC)>>BITS_NUM+0.5); where NNout′ is the CCAF filtering result, DBFout is the first reconstructed video unit, NNout is the second reconstructed video unit, and w i Let w0 be the filter coefficient for the i-th video unit, w0 to w4 be the filter coefficients for the first reconstructed video unit, and w5 to w8 be the filter coefficients for the second reconstructed video unit.

[0262] In this embodiment, the aforementioned loop filtering information includes at least one of the following: a sequence-level filter switch identifier, used to indicate whether the loop filtering method is turned on or off for the video sequence in which the current video frame is located; a filter image-level identifier, used to indicate whether the loop filtering model information appears in the image header information or in the title sequence information; and loop filtering model structure information, used to indicate whether the loop filtering method is applied to the video unit.

[0263] In one embodiment, the method further includes: determining the value of the sequence-level filter switch identifier; determining that the current encoded video sequence enables the loop filtering when the sequence-level filter switch identifier is a first value, and determining that the current encoded video sequence does not enable the loop filtering when the sequence-level filter switch identifier is a second value, wherein the first value and the second value are different.

[0264] In one embodiment, the method further includes: determining the value of the filtered image-level identifier; determining that the loop filter model information appears in the image header when the filtered image-level identifier is a third value, and determining that the loop filter model information appears in the video header when the filtered image-level identifier is a fourth value, wherein the third value and the fourth value are different.

[0265] In this embodiment, the video unit mentioned above includes images or video clips in the video.

[0266] In this embodiment, the aforementioned loop filtering model structure information includes at least one of the following: a video unit filter switch identifier, used to indicate whether the loop filtering mode is turned on or off in the video unit; a video unit component switch identifier, used to indicate whether the loop filtering mode is turned on or off in the video unit component; a video unit block-level filter switch identifier, used to indicate whether the loop filtering mode is turned on or off in each level of the video unit; an absolute value of the video unit filter coefficient, used to indicate the absolute value of the filter coefficient corresponding to each level in the video unit; and a video unit filter coefficient symbol, used to indicate the symbol of the filter coefficient corresponding to each level in the video unit; wherein, the video unit component includes at least one of the following: the luminance component of the video unit; the first chrominance component of the video unit; and the second chrominance component of the video unit.

[0267] In one embodiment, the method further includes: determining the value of the video unit filter switch identifier, and determining whether the current video unit enables or disables the loop filter based on the value.

[0268] In one embodiment, the method further includes: determining the value of the luminance component switch identifier of the video unit, and determining, based on the value, whether to enable the loop filter for the current video unit luminance component or not to enable the loop filter for the current video unit luminance component; determining the value of the first chrominance component switch identifier of the video unit, and determining, based on the value, whether to enable the loop filter for the current video unit first chrominance component or not; determining the value of the second chrominance component switch identifier of the video unit, and determining, based on the value, whether to enable the loop filter for the current video unit second chrominance component or not.

[0269] In one embodiment, the method further includes: determining the value of the video unit block-level filter switch identifier, and determining, based on the value, whether to enable the loop filter for the coding blocks in the video unit or not to enable the loop filter for the coding blocks in the video unit.

[0270] In one embodiment, the method further includes: filtering the video unit with the loop filtering method enabled by using the loop filtering method based on the absolute value of the video unit filter coefficient and the sign of the video unit filter coefficient.

[0271] In this embodiment, the aforementioned loop filtering model structure information further includes: a video unit block-level filter coefficient fusion mode switch identifier, used to indicate whether the coding block and its components in the video unit enable coefficient fusion mode; a video unit block-level filter coefficient fusion mode type identifier, used to indicate the location of the adjacent coding block of the current coding block in the video unit, wherein the current coding block uses the filter coefficients of the adjacent coding block to perform the loop filtering; wherein the coding block components include at least one of the following: a luminance component, a first chrominance component, and a second chrominance component.

[0272] In this embodiment, the aforementioned loop filter model structure information further includes: a cross-component filtering mode switch identifier, used to indicate whether the coding block and the components of the coding block in the video unit have cross-component filtering mode enabled.

[0273] On one hand, in step S1306 above, the identifier of the CCA filter can be obtained from the pps_cca_enabled_flag of the video stream; whether to perform CCA filtering is determined according to the identifier of the CCA filter, wherein the pps_cca_enabled_flag is 1 or 0, 1 indicates that CCA filtering is performed on the current slice, and 0 indicates that CCA filtering is not performed on the current slice; when the pps_cca_enabled_flag is 1, the filter coefficients are obtained from the video stream; or when the pps_cca_enabled_flag is 1, whether each block of the reconstructed image is subjected to CCA filtering is determined according to the cca_block_flag of the video stream, wherein the cca_block_flag is 1 or 0, wherein 1 indicates that CCA filtering is performed on the current block, and 0 indicates that CCA filtering is not performed on the current block; if the cca_block_flag is 1, the filter coefficients of each block are obtained, wherein the block is one of the following: one or more CTUs, one or more CUs, or a pixel block of a preset size.

[0274] On the other hand, in step S1306 above, the position of the filter coefficient can be determined by the pps_cca_info_in_ph_flag of the video bitstream. Here, pps_cca_info_in_ph_flag is 1 or 0, where 1 indicates that the filter coefficient is in the picture header and 0 indicates that the filter coefficient is in the slice header. When pps_cca_info_in_ph_flag is 1, the filter coefficient is obtained from the picture header; when pps_cca_info_in_ph_flag is 0, the filter coefficient is obtained from the slice header.

[0275] In another embodiment, step S1308 may specifically include: obtaining a first sub-filter result of the luminance component and / or chrominance component in the first filtering result for the slice or block with the identifier 1 of the CCA filter, and obtaining a second sub-filter result of the luminance component and / or chrominance component in the second filtering result; performing CCA filtering on each slice or block of the reconstructed image according to the first sub-filter result, the second sub-filter result and the filter coefficients to obtain the CCA filtering result.

[0276] In another embodiment, step S1308 may further include: determining whether the luminance component of the current slice is subject to CCA filtering based on the cca_luma_enabled_flag of the video bitstream; if cca_luma_enabled_flag is 1, performing CCA filtering on the luminance component of the current slice; if cca_luma_enabled_flag is 0, not performing CCA filtering on the luminance component of the current slice; and determining whether the chrominance Cb component of the current slice is subject to CCA filtering based on the cca_cb_enabled_flag; if cca_cb_enabled_flag is 1, performing CCA filtering on the chrominance Cb component of the current slice; if cca_cb_enabled_flag is 0, not performing CCA filtering on the chrominance Cb component of the current slice. The `cca_cr_enabled_flag` determines whether the chroma (Cr) component of the current slice should be CCA filtered. If `cca_cr_enabled_flag` is 1, the chroma (Cr) component of the current slice should be CCA filtered; if `cca_cr_enabled_flag` is 0, the chroma (Cr) component of the current slice should not be CCA filtered. Alternatively, the `cca_block_flag[i]` determines whether the luminance or chroma component of the current block should be CCA filtered. If `cca_block_flag[i]` is 1, the luminance or chroma component of the current block should be CCA filtered; if `cca_block_flag[i]` is 0, the luminance or chroma component of the current block should not be CCA filtered. Here, i = 0 represents the luminance component, i = 1 represents the chroma (Cb) component, and i = 2 represents the chroma (Cr) component.

[0277] In another embodiment, step S308 may specifically include: when the CCA filter identifier is 1, determining whether the current block is subject to CCA filtering using the cca_block_flag of the video bitstream; if the cca_block_flag is 1, performing CCA filtering on the current block; if the cca_block_flag is 0, not performing CCA filtering on the current block; when the cca_block_flag is 1, determining whether the current block uses the coefficient fusion mode using cca_block_merge_flag[i][Blk AddrX][Blk AddrY]; if the cca_block_merge_flag[i][Blk AddrX][Blk AddrY] is 1, determining whether the coefficient fusion mode is used; if the cca_block_merge_flag[i][Blk AddrX][Blk AddrY] is 0, determining whether the coefficient fusion mode is not used; when the cca_block_merge_flag[i][Blk AddrX][Blk AddrY] is 1, determining whether the current block uses the coefficient fusion mode using cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY]. [AddrX][Blk AddrY] determines the type of coefficient fusion mode for the current block. If cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] is 1, the filter coefficients of the upper adjacent block are used to perform CCA filtering on the current block. If cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] is 0, the filter coefficients of the left adjacent block are used to perform CCA filtering on the current block. Here, i is 0 to represent the luminance component, i is 1 to represent the chrominance Cb component, and i is 2 to represent the chrominance Cr component.

[0278] In this embodiment, the convolution template mentioned above is a 3*3 rhombus template, a 5*5 rhombus template, a 7*7 rhombus template, a 3*3 square template, a 5*5 square template, a 7*7 square template, and a 5*5 cross-shaped template. These are just some commonly used ones and are not limited to the scope of protection of this embodiment.

[0279] In this embodiment, the decoding end uses a convolution-based filtering adaptation algorithm. The overall process of the decoding end CCA filtering method is as follows:

[0280] Step 1: Perform DBF filtering on the reconstructed image after LMCS and store the DBF filtering result in dbfoutBuf.

[0281] Step 2: Perform NN filtering on the reconstructed image after LMCS and store the NN filtering result in nnoutBuf.

[0282] Step 3: Perform a convolution-based filtering adaptation algorithm. The results from the two buffers stored in the previous steps are fed into the CCA stage to parse the CCA coefficients from the bitstream and perform filtering operations.

[0283] 3-a, The decoder reads the filter coefficients based on the syntax elements in the bitstream;

[0284] 3-b, Reconstruct the output after CCA filtering, following the same procedure as step 3-c in Example 1.

[0285] Figure 14 This is a schematic diagram of the convolution-based filtering adaptation operation at the decoding end according to this embodiment, as shown below. Figure 14 As shown, at the decoding end, the convolution-based filtering adaptation algorithm and filter coefficients are obtained by parsing the bitstream. The decoding end CCAF includes:

[0286] Step 1: DBF filtering. Perform DBF filtering on the reconstructed image Rec after LMCS and store the DBF filtering result in dbfoutBuf.

[0287] Step 2: NN filtering. After obtaining the reconstructed image of LMCS, NN filtering is performed.

[0288] First, configure the various data required to input the network, such as reconstructed image samples, QP information, CU partitioning information, deblocking filtering information, predicted samples, etc. Then, input this information into the network module to obtain the reconstructed image samples after NN processing. Store the NN filtering results in nnoutBuf.

[0289] Step 3: A convolution-based filtering adaptation algorithm is used. The results from the two buffers stored in the previous steps are fed into the CCA stage to parse the CCA coefficients from the bitstream and perform filtering. Specific steps include:

[0290] Step 3-a: Calculate the filter coefficients and parse w0 to w8 from the bitstream.

[0291] Step 3-b: Construct the CCA output signal, obtain the filter coefficients, and then perform the filtering operation using the following formula:

[0292] NNout′=int((NNout C +w0*(DBFout t -NNout C )+w1*(DBFout l -NNout C )+w2*(DBFout c -NNout C )+w3*(DBFoutr -NNout C )+w4*(DBFout b -NNout C )+w5*(NNout t -NNout C )+w6*(NNout l -NNout C )+w7*(NNout r -NNout C )+w8*(NNout b -NNout C ))>>BITS_NUM+0.5)

[0293] Where DBFout refers to the output after passing through the DBF filter, NNout refers to the output after passing through the NN filter, and w i Refers to the i-th filter coefficient, where w0 to w4 are the filter coefficients for the DBF output, and w5 to w8 are the filter coefficients for the NN output.

[0294] In this embodiment, the transmission based on the convolutional filtering adaptation algorithm includes CCA-related syntax elements in the video encoded bitstream, including:

[0295] SPS layer:

[0296] The encoder determines whether the current sequence uses the CCA scheme based on the sps_cca_enabled_flag syntax element in SPS. The syntax semantics are shown in Table 1 below.

[0297] Table 1

[0298]

[0299] Here, sps_cca_enabled_flag equals 1 to indicate that CCA technology is enabled in the current sequence, and equals 0 to indicate that CCA technology is not enabled in the current sequence.

[0300] PPS layer:

[0301] The encoder determines the position of CCA scheme-related information in the syntax element based on the pps_cca_info_in_ph_flag syntax element in PPS. The syntax semantics are shown in Table 2 below.

[0302] Table 2

[0303]

[0304] A value of 1 for pps_cca_info_in_ph_flag indicates that CCA-related information appears in the picture header, while a value of 0 indicates that CCA-related information does not appear in the picture header but can appear in the slice header.

[0305] Picture header:

[0306] When pps_cca_info_in_ph_flag equals 1, it indicates that CCA-related information appears in the picture header. The decoder can read the CCA filter coefficients based on the cca_structure() structure in the picture header. The syntax and semantics are shown in Table 3 below.

[0307] Table 3

[0308]

[0309] Slice header:

[0310] When pps_cca_info_in_ph_flag equals 0, it indicates that CCA-related information does not appear in the picture header but can appear in the slice header. The decoder can read the CCA filter coefficients based on the cca_structure() structure in the slice header, as shown in Table 4 below.

[0311] Table 4

[0312]

[0313]

[0314] cca_structure():

[0315] The decoder determines whether the current image or slice uses CCA based on the syntax elements in the cca_structure() structure. If CCA is used, it further obtains the filtering coefficients of the luminance or chrominance components to complete the CCA operation, as shown in Table 5 below.

[0316] Table 5

[0317]

[0318]

[0319] A value of 1 for cca_enabled_flag indicates that CCA is enabled, while a value of 0 indicates that CCA is disabled.

[0320] A value of 1 for cca_luma_enabled_flag indicates that the current slice's luma component uses CCA, while a value of 0 indicates that the current slice's luma component does not use CCA.

[0321] num_cca_luma_coeff: The number of luminance filter coefficients. This is usually a preset value and does not need to be transmitted. Different implementations may use different numbers of filters.

[0322] cca_luma_coeff_abs[i]: Represents the absolute value of the i-th coefficient of the current slice brightness.

[0323] cca_luma_coeff_signed[i]: Represents the sign of the i-th coefficient of the current slice brightness.

[0324] cca_cb_enabled_flag: A value of 1 indicates that the Cb component of the current slice uses CCA, while a value of 0 indicates that the Cb component of the current slice does not use CCA.

[0325] cca_cr_enabled_flag: A value of 1 indicates that the Cr component of the current slice uses CCA, while a value of 0 indicates that the Cr component of the current slice does not use CCA.

[0326] cca_cb_coeff_abs[i]: Represents the absolute value of the i-th coefficient of the Cb component of the current slice.

[0327] cca_cb_coeff_signed[i]: Represents the sign of the i-th coefficient of the Cb component of the current slice.

[0328] cca_cr_coeff_abs[i]: Represents the absolute value of the i-th coefficient of the Cr component of the current slice.

[0329] cca_cr_coeff_signed[i]: Represents the sign of the i-th coefficient of the Cr component of the current slice.

[0330] num_cca_chroma_coeff: The number of chroma filter coefficients. This is usually a preset value and does not need to be transmitted. Different implementations may use different numbers of filters.

[0331] The syntax structure used only for the luminance component is shown in Table 6 below.

[0332] Table 6

[0333]

[0334] The syntax structure used only for the chroma component is shown in Table 7 below.

[0335] Table 7

[0336]

[0337]

[0338] Block-level switch:

[0339] If block-level switches are set for CCA operations to control the CCA switching operations of each block, then the decoder also needs to parse the block-level switches, as shown in Table 8 below.

[0340] Table 8

[0341]

[0342] `cca_block_flag[i]` indicates whether the block uses CCA; 1 indicates it is used, and 0 indicates it is not. 0 for i represents the luminance component, 1 for the chrominance Cb component, and 2 for the chrominance Cr component.

[0343] Block-level CCA coefficient transmission:

[0344] If CCA is performed at the block level, then the CCA coefficient brightness needs to be transmitted at the block level as shown in Table 9 below.

[0345] Table 9

[0346]

[0347] The chromaticity is shown in Table 10 below.

[0348] Table 10

[0349]

[0350]

[0351] Coefficient fusion mode:

[0352] If the block-level coefficients can be merged with the upper and left adjacent blocks, then a merging switch needs to be transmitted.

[0353] The brightness is shown in Table 11 below.

[0354] Table 11

[0355]

[0356] cca_block_merge_flag[i][Blk AddrX][Blk AddrY]: Indicates whether the block uses CCA's merge mode. 1 indicates use, 0 indicates no use. 0 for i represents the luminance component, 1 for the chrominance Cb component, and 0 for the chrominance Cr component.

[0357] cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY]: Identifies the merge mode type of the CCA of this block. A value of 1 indicates that the CCA coefficients of the upper adjacent block are used, and a value of 0 indicates that the CCA coefficients of the left adjacent block are used. i is 0 for the luminance component, 1 for the chrominance Cb component, and 0 for the chrominance Cr component.

[0358] The chromaticity is shown in Table 12 below.

[0359] Table 12

[0360]

[0361]

[0362] The Cross Components CCA pattern is shown in Table 13 below.

[0363] Table 13

[0364]

[0365]

[0366] cc_cca_structure()

[0367] The call locations of cc_cca_structure() and cca_structure() are the same, as shown in Table 14 below.

[0368] Table 14

[0369]

[0370]

[0371] cc_cca_enabled_flag: A value of 1 indicates that CC-CCA is used, and a value of 0 indicates that CC-CCA is not used. When this syntax element does not exist, it means that cca_enabled_flag is equal to 0.

[0372] cc_cca_cb_enabled_flag: A value of 1 indicates that the Cb component of the current slice uses CC-CCA, while a value of 0 indicates that the Cb component of the current slice does not use CC-CCA.

[0373] cc_cca_cr_enabled_flag: A value of 1 indicates that the Cr component of the current slice uses CC-CCA, while a value of 0 indicates that the Cr component of the current slice does not use CC-CCA.

[0374] cc_cca_cb_coeff_abs[i]: Represents the absolute value of the i-th coefficient of the Cb component of the current slice.

[0375] cc_cca_cb_coeff_signed[i]: Represents the sign of the i-th coefficient of the Cb component of the current slice.

[0376] cc_cca_cr_coeff_abs[i]: Represents the absolute value of the i-th coefficient of the Cr component of the current slice.

[0377] cc_cca_cr_coeff_signed[i]: Represents the sign of the i-th coefficient of the Cr component of the current slice.

[0378] num_cc_cca_chroma_coeff: The number of chroma filter coefficients. This is usually a preset value and does not need to be transmitted. Different implementations may use different numbers of filters.

[0379] According to another aspect of the embodiments of this application, a video encoding / decoding loop filtering device is also provided. Figure 15 This is a block diagram of a loop filter device according to an embodiment of this application, such as... Figure 15 As shown, the device includes:

[0380] The first filtering module 152 is used to filter the reconstructed image of the current image using the first filter to obtain the first reconstructed video unit;

[0381] The second filtering module 154 is used to filter the reconstructed image of the current image using the second filter to obtain the second reconstructed video unit.

[0382] The first determining module 156 is used to determine filter coefficients based on the neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is the information of adjacent pixels centered on the pixel in the convolution template.

[0383] The third filtering module 158 is used to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain the third reconstructed video unit.

[0384] In one embodiment, the first determining module 156 is further configured to, for each pixel in the video unit of the reconstructed image and the video unit of the original image, determine the difference between the pixel at each neighborhood position in the convolution template and each pixel, multiply the differences to obtain a first pixel product, and determine the difference between the pixel in the video unit of the original image and the pixel in the video unit of the reconstructed image to obtain the covariance; and determine the filter coefficients based on the Wiener filtering method according to the covariance.

[0385] In one embodiment, the video unit of the reconstructed image includes a reconstructed image, or slice, or coded block, or coded tree block.

[0386] In one embodiment, the third filtering module 158 is further configured to perform at least one of the following:

[0387] Based on the convolution template, the luminance components of the first reconstructed video unit and the second reconstructed video unit are filtered according to the filter coefficients;

[0388] Based on the convolution template, the first chroma component of the first reconstructed video unit and the second reconstructed video unit is filtered according to the filter coefficients;

[0389] Based on the convolution template, the second chroma components of the first reconstructed video unit and the second reconstructed video unit are filtered according to the filter coefficients.

[0390] In one embodiment, the first filtering module 152 is further configured to perform Deblocking Loop Filtering (DBF) on the reconstructed image; or perform Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or perform Deblocking Loop Filtering (DBF) and Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or perform Luminance Mapping Based on Chroma Scaling (LMCS) processing on the reconstructed image.

[0391] In one embodiment, the second filtering module 154 is further configured to perform neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or to perform neural network loop filtering on the reconstructed image and weightedly sum the neural network loop filtering result with the first reconstructed video unit to obtain the second reconstructed video unit; or to perform neural network loop filtering on the reconstructed image and adjust the intensity of the neural network loop filtering result according to the original image to obtain the second reconstructed video unit; or to perform other loop filtering on the reconstructed image besides the neural network loop filtering to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of the following: deblocking loop filtering DBF, sample adaptive compensation loop filtering SAO, and luminance mapping based on chroma scaling LMCS.

[0392] In one embodiment, the device further includes:

[0393] The second determining module is used to determine a first loss between the pixels of the video units of the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; and to determine a second loss between the pixels of the video units of the reconstructed image before filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image.

[0394] The CCA filtering module is used to determine whether to perform CCA filtering on the video units of the current image based on the first loss and the second loss.

[0395] In one embodiment, the device further includes:

[0396] The third determining module is used to determine a first loss between the pixels of the video units of the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image; and to determine a third loss between the pixels of the video units of the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image by filtering with preset filter coefficients.

[0397] The fourth filtering module is used to determine whether to perform the filtering process using the filter coefficients or to perform the filtering process using preset filter coefficients, based on the first loss and the third loss.

[0398] In one embodiment, the device further includes:

[0399] The fourth determining module is used to determine a first loss between the pixels of the video units of the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image; determine a second loss between the pixels of the video units of the reconstructed image before filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image; and perform the filtering on the video units of the current image according to the filter coefficients of the adjacent video units of the current image to determine a fourth loss between the pixels of the video units of the reconstructed image after filtering and the pixels of the original image.

[0400] The fifth filtering module is used to determine the filter coefficients used by the video unit of the current image based on the first loss, the second loss and the fourth loss, and to perform filtering based on the determined filter coefficients.

[0401] In one embodiment, the device further includes:

[0402] The fifth determining module is used to determine a first loss between the pixels of the video in the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image; and to determine a second loss between the pixels of the video unit in the reconstructed image before filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image.

[0403] The confirmation module is used to determine whether to perform the filtering on the video units of the current image based on the first loss and the second loss, and to recalculate the filter coefficients for the video units with the filtering enabled, and to determine whether to perform filtering again based on the first loss and the second loss, repeating this process N times to confirm whether to perform the filtering on the video units of the current image.

[0404] In one embodiment, the third filtering module 158 is further configured to calculate the difference between the center pixel and the neighboring pixels of the convolution template based on the first reconstructed video unit, and multiply it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a first result; simultaneously, it calculates the difference between the center pixel and the neighboring pixels of the convolution template based on the second reconstructed video unit, and multiplies it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a second result; and adds the first result, the second result, and the second reconstructed video unit together, and then performs rounding to obtain the third reconstructed video unit.

[0405] In one embodiment, the convolution template includes one of the following: a rhombus template, a square template, or a cross-shaped template.

[0406] According to another aspect of the embodiments of this application, a video decoding device is also provided. Figure 16 This is a block diagram of a video decoding apparatus according to an embodiment of this application, such as... Figure 16 As shown, the device includes:

[0407] The acquisition module 162 is used to acquire a bitstream containing the encoded video sequence and obtain loop filter control information corresponding to the loop filter method;

[0408] The first filtering module 152 is used to filter the reconstructed image of the current image using the first filter to obtain the first reconstructed video unit;

[0409] The second filtering module 154 is used to filter the reconstructed image of the current image using the second filter to obtain the second reconstructed video unit.

[0410] The third filtering module 158 is used to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain the third reconstructed video unit.

[0411] In one embodiment, the first filtering module 152 is further configured to perform Deblocking Loop Filtering (DBF) on the reconstructed image; or perform Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or perform Deblocking Loop Filtering (DBF) and Sample Adaptive Compensation Loop Filtering (SAO) on the reconstructed image; or perform Luminance Mapping Based on Chroma Scaling (LMCS) processing on the reconstructed image.

[0412] In one embodiment, the second filtering module 154 is further configured to perform neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or to perform neural network loop filtering on the reconstructed image and weightedly sum the neural network loop filtering result with the first reconstructed video unit to obtain the second reconstructed video unit; or to perform neural network loop filtering on the reconstructed image and adjust the intensity of the neural network loop filtering result according to the original image to obtain the second reconstructed video unit; or to perform other loop filtering on the reconstructed image besides neural network loop filtering to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of the following: deblocking loop filtering (DBF), sample adaptive compensation loop filtering (SAO), and luminance mapping based on chroma scaling (LMCS).

[0413] In one embodiment, the third filtering module 158 is further configured to calculate the difference between the center pixel and the neighboring pixels of the convolution template based on the first reconstructed video unit, and multiply it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a first result; simultaneously, it calculates the difference between the center pixel and the neighboring pixels of the convolution template based on the second reconstructed video unit, and multiplies it by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a second result; and adds the first result, the second result, and the second reconstructed video unit together, and then performs rounding to obtain the third reconstructed video unit.

[0414] In one embodiment, the loop filtering information includes at least one of the following:

[0415] A sequence-level filter switch identifier is used to indicate whether the loop filtering method is turned on or off for the video sequence containing the current video frame;

[0416] A filtered image-level identifier is used to indicate whether the loop filtering model information appears in the image header information or in the slice header information;

[0417] Loop filter model structure information is used to indicate the application of the loop filter method to the video unit.

[0418] In one embodiment, the device further includes:

[0419] The sixth determining module is used to determine the value of the sequence-level filter switch identifier; when the sequence-level filter switch identifier is a first value, it is determined that the current encoded video sequence enables the loop filter; when the sequence-level filter switch identifier is a second value, it is determined that the current encoded video sequence does not enable the loop filter, wherein the first value and the second value are different.

[0420] In one embodiment, the device further includes:

[0421] The seventh determining module is used to determine the value of the filtered image-level identifier; when the filtered image-level identifier is a third value, it is determined that the loop filtering model information appears in the image header; when the filtered image-level identifier is a fourth value, it is determined that the loop filtering model information appears in the chip header, wherein the third value and the fourth value are different.

[0422] In one embodiment, the video unit includes images or video clips from a video.

[0423] In one embodiment, the loop filter model structure information includes at least one of the following:

[0424] The video unit filter switch identifier is used to indicate whether the loop filtering mode is turned on or off in the video unit;

[0425] The video unit component switch identifier is used to indicate whether the loop filtering mode is turned on or off for the video unit component.

[0426] Video unit block-level filter switch identifier, used to indicate whether the loop filtering mode is turned on or off at each level in the video unit;

[0427] The absolute value of the filter coefficients of the video unit is used to indicate the absolute value of the filter coefficients corresponding to each level in the video unit.

[0428] Video unit filter coefficient symbol, used to indicate the filter coefficient symbol corresponding to each level in the video unit;

[0429] The video unit component includes at least one of the following:

[0430] The luminance component of the video unit;

[0431] The first chroma component of the video unit;

[0432] The second chroma component of the video unit.

[0433] In one embodiment, the device further includes:

[0434] The eighth determining module is used to determine the value of the video unit filter switch identifier, and determine whether the current video unit enables or disables the loop filter based on the value.

[0435] In one embodiment, the device further includes:

[0436] The ninth determining module is used to determine the value of the luminance component switch identifier of the video unit, and determine, based on the value, whether to enable the loop filter for the luminance component of the current video unit or not to enable the loop filter for the luminance component of the current video unit; determine the value of the first chrominance component switch identifier of the video unit, and determine, based on the value, whether to enable the loop filter for the first chrominance component of the current video unit or not; determine the value of the second chrominance component switch identifier of the video unit, and determine, based on the value, whether to enable the loop filter for the second chrominance component of the current video unit or not.

[0437] In one embodiment, the device further includes:

[0438] The tenth determining module is used to determine the value of the video unit block-level filter switch identifier, and determine whether to enable the loop filter for the coding block in the video unit or not to enable the loop filter for the coding block in the video unit based on the value.

[0439] In one embodiment, the device further includes:

[0440] The sixth filtering module is used to filter the video unit with the loop filtering method enabled, based on the absolute value and sign of the video unit filtering coefficients.

[0441] In one embodiment, the loop filter model structure information further includes:

[0442] The video unit block-level filter coefficient fusion mode switch identifier is used to indicate whether the coefficient fusion mode is enabled for the coding block and its components in the video unit.

[0443] A video unit block-level filter coefficient fusion mode type identifier is used to indicate the location of the adjacent coding blocks of the current coding block in the video unit, wherein the current coding block uses the filter coefficients of the adjacent coding blocks to perform the loop filtering;

[0444] The coded block components include at least one of the following: a luminance component, a first chrominance component, and a second chrominance component.

[0445] In one embodiment, the loop filtering model structure information further includes: a cross-component filtering mode switch identifier, used to indicate whether the coding block and the components of the coding block in the video unit have cross-component filtering mode enabled.

[0446] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0447] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0448] Embodiments of this application also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0449] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0450] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0451] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0452] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A video encoding / decoding loop filtering method, characterized in that, The method includes: The reconstructed image of the current image is filtered using the first filter to obtain the first reconstructed video unit; The reconstructed image is filtered using a second filter to obtain a second reconstructed video unit; The filter coefficients are determined based on the neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is the information of adjacent pixels centered on the pixel in the convolution template; Based on the convolution template and the filter coefficients, the first reconstructed video unit and the second reconstructed video unit are filtered to obtain the third reconstructed video unit.

2. The method according to claim 1, characterized in that, Determining the filter coefficients based on the neighborhood information contained in the convolution templates of the first and second reconstructed video units includes: For each pixel in the video unit of the reconstructed image and the video unit of the original image, the difference between the pixel at each neighborhood position in the convolution template and each pixel is determined. The differences are multiplied to obtain the first pixel product. The difference between the pixel in the video unit of the original image and the pixel in the video unit of the reconstructed image is then determined to obtain the covariance. The filter coefficients are determined based on the Wiener filtering method and the covariance.

3. The method according to claim 2, characterized in that, The video unit of the reconstructed image includes the reconstructed image, or slice, or coded block, or coded tree block.

4. The method according to claim 1, characterized in that, Based on the convolution template and the filter coefficients, the first reconstructed video unit and the second reconstructed video unit are filtered to obtain a third reconstructed video unit, which includes at least one of the following: Based on the convolution template, the luminance components of the first reconstructed video unit and the second reconstructed video unit are filtered according to the filter coefficients; Based on the convolution template, the first chroma component of the first reconstructed video unit and the second reconstructed video unit is filtered according to the filter coefficients; Based on the convolution template, the second chroma components of the first reconstructed video unit and the second reconstructed video unit are filtered according to the filter coefficients.

5. The method according to claim 1, characterized in that, Filtering the reconstructed image of the current image using the first filter includes: Perform a deblocking loop filter (DBF) on the reconstructed image; or Perform sample adaptive compensation loop filtering (SAO) on the reconstructed image; or The reconstructed image is subjected to Deblocking Loop Filtering (DBF) and Sample Adaptive Compensation Loop Filtering (SAO); or The reconstructed image is then subjected to Luminance Mapping Based on Chroma Scaling (LMCS) processing.

6. The method according to claim 1, characterized in that, The reconstructed image is filtered using a second filter to obtain a second reconstructed video unit, comprising: The reconstructed image is subjected to neural network loop filtering to obtain the second reconstructed video unit; or The reconstructed image is subjected to neural network loop filtering, and the result of the neural network loop filtering is weighted and summed with the first reconstructed video unit to obtain the second reconstructed video unit; or The reconstructed image is subjected to neural network loop filtering, and the intensity of the neural network loop filtering result is adjusted according to the original image to obtain the second reconstructed video unit, or The reconstructed image is subjected to loop filtering other than the neural network loop filtering to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of the following: deblocking loop filtering DBF, sample adaptive compensation loop filtering SAO, and luminance mapping based on chroma scaling LMCS.

7. The method according to claim 1, characterized in that, The method further includes: Determine the first loss between the pixels of the video units of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; Determine a second loss between the pixels of the video units of the reconstructed image before filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; According to the first loss lose Based on the second loss, determine whether to perform CCA filtering on the video units of the current image.

8. The method according to claim 1, characterized in that, The method further includes: Determine the first loss between the pixels of the video units of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; By filtering with preset filter coefficients, a third loss is determined between the pixels of the video unit of the reconstructed image after filtering according to the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and the pixels of the original image. Based on the first loss and the third loss lose The filter coefficients are used to determine whether to perform filtering or to perform filtering using preset filter coefficients.

9. The method according to claim 1, characterized in that, The method further includes: Determine the first loss between the pixels of the video units of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; Determine a second loss between the pixels of the video units of the reconstructed image before filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; The video unit of the current image is filtered based on the filter coefficients of the adjacent video units of the current image, and a fourth loss is determined between the pixels of the video unit of the reconstructed image after filtering and the pixels of the original image. Based on the first loss, the second loss and the fourth loss lose The filter coefficients used by the video unit of the current image are determined, and filtering is performed based on the determined filter coefficients.

10. The method according to claim 1, characterized in that, The method further includes: Determine the first loss between the pixels of the video units of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; Determine a second loss between the pixels of the video units of the reconstructed image before filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients, and the pixels of the original image; Based on the first loss and the second loss, it is determined whether to perform the filtering on the video units of the current image, and the filter coefficients are recalculated for the video units with the filtering enabled. The first loss and the second loss are used to determine whether to perform filtering again. This process is repeated N times to confirm whether to perform the filtering on the video units of the current image.

11. The method according to claim 1, characterized in that, Based on the convolution template and the filter coefficients, filtering is performed on the first reconstructed video unit and the second reconstructed video unit to obtain the third reconstructed video unit, which includes: The difference between the center pixel and the neighboring pixels of the convolution template is calculated based on the first video reconstruction unit, and multiplied by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a first result; at the same time, the difference between the center pixel and the neighboring pixels of the convolution template is calculated based on the second video reconstruction unit, and multiplied by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a second result; The first result, the second result, and the second reconstructed video unit are added together and then rounded down to obtain the third reconstructed video unit.

12. The method according to any one of claims 1 to 11, characterized in that, The convolution template includes one of the following: a diamond template, a square template, or a cross-shaped template.

13. A video decoding method, characterized in that, include: Obtain a bitstream containing an encoded video sequence to obtain loop filter control information corresponding to the loop filter method, wherein the loop filter control information includes filter coefficients; The reconstructed image of the current image is filtered using the first filter to obtain the first reconstructed video unit; The reconstructed image of the current image is filtered using a second filter to obtain the second reconstructed video unit; Based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit, as well as the filter coefficients, the first reconstructed video unit and the second reconstructed video unit are filtered to obtain the third reconstructed video unit.

14. The method according to claim 13, characterized in that, Filtering the reconstructed image of the current image using the first filter includes: Perform a deblocking loop filter (DBF) on the reconstructed image; or Perform sample adaptive compensation loop filtering (SAO) on the reconstructed image; or The reconstructed image is subjected to Deblocking Loop Filtering (DBF) and Sample Adaptive Compensation Loop Filtering (SAO); or The reconstructed image is then subjected to Luminance Mapping Based on Chroma Scaling (LMCS) processing.

15. The method according to claim 13, characterized in that, The reconstructed image of the current image is filtered using a second filter to obtain the second reconstructed video unit, which includes: The reconstructed image is subjected to neural network loop filtering to obtain the second reconstructed video unit; or The reconstructed image is subjected to neural network loop filtering, and the result of the neural network loop filtering is weighted and summed with the first reconstructed video unit to obtain the second reconstructed video unit, or... The reconstructed image is subjected to neural network loop filtering, and the intensity of the neural network loop filtering result is adjusted according to the original image to obtain the second reconstructed video unit; or The reconstructed image is subjected to loop filtering other than neural network loop filtering to obtain the second reconstructed video unit. The other loop filtering includes at least one of the following: deblocking loop filtering (DBF), sample adaptive compensation loop filtering (SAO), and luminance mapping based on chroma scaling (LMCS).

16. The method according to claim 13, characterized in that, Based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit, and the filter coefficients, filtering is performed on the first reconstructed video unit and the second reconstructed video unit to obtain the third reconstructed video unit, including: The difference between the center pixel and the neighboring pixels of the convolution template is calculated based on the first video reconstruction unit, and multiplied by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a first result; at the same time, the difference between the center pixel and the neighboring pixels of the convolution template is calculated based on the second video reconstruction unit, and multiplied by the filter coefficients at the corresponding positions of the neighboring pixels to obtain a second result; The first result, the second result, and the second reconstructed video unit are added together and then rounded down to obtain the third reconstructed video unit.

17. The method according to claim 13, characterized in that, The loop filtering information includes at least one of the following: A sequence-level filter switch identifier is used to indicate whether the loop filtering method is turned on or off for the video sequence containing the current video frame; A filtered image-level identifier is used to indicate whether the loop filtering model information appears in the image header information or in the slice header information; Loop filter model structure information is used to indicate the application of the loop filter method to the video unit.

18. The method according to claim 17, characterized in that, The method further includes: Determine the value of the sequence-level filter switch identifier; When the sequence-level filter switch identifier is set to a first value, it is determined that the current encoded video sequence enables the loop filter; when the sequence-level filter switch identifier is set to a second value, it is determined that the current encoded video sequence does not enable the loop filter, wherein the first value and the second value are different.

19. The method according to claim 17, characterized in that, The method further includes: Determine the value of the filtered image-level identifier; When the filtered image-level identifier takes a third value, it is determined that the loop filtering model information appears in the image header; when the filtered image-level identifier takes a fourth value, it is determined that the loop filtering model information appears in the video header. The third value and the fourth value are different.

20. The method according to claim 17, characterized in that, The video unit includes images or video clips from the video.

21. The method according to claim 17, characterized in that, The loop filter model structure information includes at least one of the following: The video unit filter switch identifier is used to indicate whether the loop filtering mode is turned on or off in the video unit; The video unit component switch identifier is used to indicate whether the loop filtering mode is turned on or off for the video unit component. Video unit block-level filter switch identifier, used to indicate whether the loop filtering mode is turned on or off at each level in the video unit; The absolute value of the filter coefficients of the video unit is used to indicate the absolute value of the filter coefficients corresponding to each level in the video unit. Video unit filter coefficient symbol, used to indicate the filter coefficient symbol corresponding to each level in the video unit; The video unit component includes at least one of the following: The luminance component of the video unit; The first chroma component of the video unit; The second chroma component of the video unit.

22. The method according to claim 21, characterized in that, The method further includes: The value of the video unit filter switch identifier is determined, and the loop filter is enabled or disabled for the current video unit based on the value.

23. The method according to claim 21, characterized in that, The method further includes: Determine the value of the luminance component switch identifier of the video unit, and determine whether to enable the loop filter for the luminance component of the current video unit or not to enable the loop filter for the luminance component of the current video unit based on the value. Determine the value of the first chroma component switch identifier of the video unit, and determine whether to enable or disable the loop filter for the first chroma component of the current video unit based on the value. The value of the second chroma component switch identifier of the video unit is determined, and the loop filter is enabled or disabled for the second chroma component of the current video unit based on the value.

24. The method according to claim 21, characterized in that, The method further includes: The value of the video unit block-level filter switch identifier is determined, and based on the value, it is determined whether to enable the loop filter for the coding block in the video unit or not to enable the loop filter for the coding block in the video unit.

25. The method according to claim 13, characterized in that, The method further includes: Based on the absolute value and sign of the video unit filter coefficients, the loop filtering method is used to filter the video unit with the loop filtering method enabled.

26. The method according to claim 17, characterized in that, The loop filter model structure information also includes: The video unit block-level filter coefficient fusion mode switch identifier is used to indicate whether the coefficient fusion mode is enabled for the coding block and its components in the video unit. A video unit block-level filter coefficient fusion mode type identifier is used to indicate the location of the adjacent coding blocks of the current coding block in the video unit, wherein the current coding block uses the filter coefficients of the adjacent coding blocks to perform the loop filtering; The coded block components include at least one of the following: a luminance component, a first chrominance component, and a second chrominance component.

27. The method according to claim 17, characterized in that, The loop filter model structure information also includes: Cross-component filtering mode switch identifier, used to indicate whether cross-component filtering mode is enabled for the coding block and its components in the video unit.

28. A video encoding / decoding loop filtering device, characterized in that, The device includes: The first filtering module is used to filter the reconstructed image of the current image using a first filter to obtain the first reconstructed video unit; The second filtering module is used to filter the reconstructed image of the current image using a second filter to obtain the second reconstructed video unit; The first determining module is used to determine filter coefficients based on the neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is the information of adjacent pixels centered on the pixel in the convolution template. The third filtering module is used to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain the third reconstructed video unit.

29. A video decoding device, characterized in that, The device includes: The acquisition module is used to acquire a bitstream containing an encoded video sequence and obtain loop filter control information corresponding to the loop filter method, wherein the loop filter control information includes filter coefficients; The first filtering module is used to filter the reconstructed image of the current image using a first filter to obtain the first reconstructed video unit; The second filtering module is used to filter the reconstructed image of the current image using the second filter to obtain the second reconstructed video unit; The third filtering module is used to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain the third reconstructed video unit.

30. A computer-readable storage medium storing a computer program, wherein, The computer program is configured to execute the method described in any one of claims 1 to 12, 13 to 27 when it is run.

31. An electronic device comprising a memory and a processor, the memory storing a computer program, the processor being configured to run the computer program to perform the method of any one of claims 1 to 12, 13 to 27.

Images