Filtering method, decoder, encoder and computer readable storage medium

Abstract

The embodiment of the invention provides a filtering method, a decoder, an encoder and a computer readable storage medium, which can improve the encoding and decoding accuracy. The method comprises the following steps: analyzing a code stream, and determining a frame-level scaling factor and an initial residual error of a current block; wherein the frame-level scaling factor is determined through a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame, the first frame-level scaling factor is obtained by screening block-level scaling factors in the current frame, and the first frame-level scaling factor does not comprise the block-level scaling factor corresponding to the difference block; the difference block is a block having a difference with other blocks in the current frame (S301); performing image block reconstruction and filtering based on the initial residual, and determining a first reconstructed image block and a first filtered image block (S302); and performing scaling processing by using the frame-level scaling factor, the first reconstructed image block and the first filtered image block to obtain a corrected image block corresponding to the current block (S303).

Description

A filtering method, decoder, encoder and computer-readable storage medium Technical Field

[0001] The embodiments of the present application relate to video coding technology, and relate to but are not limited to a filtering method, a decoder, an encoder, and a computer-readable storage medium. Background Art

[0002] Currently, loop filters are used in video codec standards such as Versatile Video Coding (VVC) to improve the subjective and objective quality of reconstructed images. Traditional loop filtering modules primarily include deblocking filters (DBF), sample adaptive offset (SAO), and adaptive loop filters (ALF). With the advancement of deep learning technology, research on neural network-based loop filters has also begun.

[0003] For a frame of coded image, the image-level scaling factor is usually calculated by traversing all pixels in the current image, and video encoding is performed based on the block level, such as the Coding Tree Unit (CTU), combined with the image-level scaling factor.

[0004] However, the picture-level scaling factor reflects the overall scalability of the image. If the block-level scaling factors of a few CTUs in the coded image differ significantly from those of the other CTUs, the calculated picture-level scaling factor will be affected by this difference and produce a significant error compared to the scaling factors of the majority of CTUs. Consequently, when the picture-level scaling factor is used to encode the majority of the remaining CTUs, the error in the picture-level scaling factor will reduce the accuracy of video encoding and decoding.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a filtering method, a decoder, an encoder, and a computer-readable storage medium, which can improve the accuracy of encoding and decoding.

[0007] In a first aspect, an embodiment of the present application provides a filtering method, applied to a decoder, the method comprising:

[0008] parsing a bitstream to determine a frame-level scaling factor and an initial residual of a current block; wherein the frame-level scaling factor is determined by using a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame, wherein the first frame-level scaling factor is obtained by screening block-level scaling factors in the current frame, wherein the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; the difference block is a block that is different from other blocks in the current frame;

[0009] Performing image block reconstruction and filtering based on the initial residual to determine a first reconstructed image block and a first filtered image block;

[0010] Scaling processing is performed using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain a corrected image block corresponding to the current block.

[0011] In a second aspect, an embodiment of the present application further provides a filtering method, applied to an encoder, the method comprising:

[0012] Performing image reconstruction and filtering based on the initial residual of each block in the current frame, and determining a second reconstructed image block and a second filtered image block corresponding to each block;

[0013] performing scaling factor calculation based on the second reconstructed image block and the second filtered image block to determine a block-level scaling factor corresponding to each block;

[0014] screening the block-level scaling factors corresponding to each block corresponding to the current frame to determine a first frame-level scaling factor, wherein the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block, wherein the difference block is a block that is different from other blocks in the current frame;

[0015] calculating a scaling factor based on a second reconstructed image and a second filtered image corresponding to the current frame to obtain a second frame-level scaling factor corresponding to the current frame; wherein the second reconstructed image is determined by each corresponding second reconstructed image block; and the second filtered image is determined by each corresponding second filtered image block;

[0016] The frame level scaling factor is determined based on the first frame level scaling factor and the second frame level scaling factor.

[0017] In a third aspect, an embodiment of the present application provides a decoder, including:

[0018] a parsing portion configured to parse the bitstream and determine a frame-level scaling factor and an initial residual of a current block; wherein the frame-level scaling factor is determined by a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame, the first frame-level scaling factor being obtained by screening block-level scaling factors in the current frame, wherein the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; the difference block being a block that is different from other blocks in the current frame;

[0019] A first reconstruction and filtering part is configured to perform image block reconstruction and filtering based on the initial residual to determine a first reconstructed image block and a first filtered image block;

[0020] The first determining part is configured to perform scaling processing using the frame-level scaling factor, the first reconstructed image block and the first filtered image block to obtain a corrected image block corresponding to the current block.

[0021] In a fourth aspect, an embodiment of the present application provides an encoder, including:

[0022] A second reconstruction and filtering part is configured to perform image reconstruction and filtering based on the initial residual of each block in the current frame, and determine a second reconstructed image block and a second filtered image block corresponding to each block;

[0023] The second determining portion is configured to calculate a scaling factor based on the second reconstructed image block and the second filtered image block to determine a block-level scaling factor corresponding to each block; screen the block-level scaling factors corresponding to each block corresponding to the current frame to determine a first frame-level scaling factor; the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; the difference block is a block that is different from other blocks in the current frame; calculate a scaling factor based on the second reconstructed image and the second filtered image corresponding to the current frame to obtain a second frame-level scaling factor corresponding to the current frame; the second reconstructed image is determined by the second reconstructed image block corresponding to each block; the second filtered image is determined by the second filtered image block corresponding to each block; and the frame-level scaling factor is determined based on the first frame-level scaling factor and the second frame-level scaling factor.

[0024] In a fifth aspect, an embodiment of the present application further provides a decoder, including:

[0025] a first memory and a first processor;

[0026] The first memory stores a computer program that can be run on the first processor, and the first processor implements the filtering method of the decoder when executing the program.

[0027] In a sixth aspect, an embodiment of the present application further provides an encoder, including:

[0028] a second memory and a second processor;

[0029] The second memory stores a computer program that can be run on the second processor, and the filtering method of the encoder is performed when the second processor executes the program.

[0030] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a first processor, the filtering method of the decoder of claim 1 is implemented; or, when the computer program is executed by a second processor, the filtering method of the encoder of claim 1 is implemented.

[0031] Embodiments of the present application provide a filtering method, decoder, encoder, and computer-readable storage medium. The frame-level scaling factor parsed from a bitstream by the decoder is determined by using a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame. The first frame-level scaling factor is obtained by filtering the block-level scaling factors of the current frame and does not include the block-level scaling factors corresponding to the difference blocks. This reduces the error caused by the block-level scaling factors corresponding to the difference blocks in calculating the overall frame-level scaling factor, enabling the calculated first frame-level scaling factor to more accurately represent the required correction magnitude for most blocks in the current frame, thereby improving the accuracy of the first frame-level scaling factor. Furthermore, because the frame-level scaling factor is determined from the first and second frame-level shrinkage factors by comparing the two frame-level scaling factors, the distortion correction performance of the frame-level shrinkage factor is further improved, thereby enhancing the encoding performance and accuracy of the encoder. In this way, the frame-level shrinkage factor is used at the decoder end to scale the first reconstructed image block and the first filtered image block obtained by reconstructing and filtering the image block through the initial residual, so as to realize decoding filtering of the current block in the current frame and obtain the corrected image block corresponding to the current block, which can improve the accuracy of the corrected image block and thus improve the decoding performance and decoding accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a schematic diagram of an application of a coding framework provided;

[0033] FIG2 is a schematic diagram of an application of another coding framework provided;

[0034] FIG3 is a network structure diagram of a loop filter of a multi-layer convolutional network according to an embodiment of the present application for filtering and optimizing an input reconstructed image;

[0035] FIG4 is a network structure diagram of a loop filter based on a multi-layer residual network provided in an embodiment of the present application;

[0036] FIG5 is a schematic diagram of an exemplary division of coding units provided in an embodiment of the present application;

[0037] FIG6A is a schematic diagram showing a distribution of block-level scaling factors according to an embodiment of the present application;

[0038] FIG6B is a schematic diagram of another distribution of block-level scaling factors provided in an embodiment of the present application;

[0039] FIG7A is a detailed schematic diagram of a video encoding system provided in an embodiment of the present application;

[0040] FIG7B is a detailed schematic diagram of a video decoding system provided in an embodiment of the present application;

[0041] FIG8 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0042] FIG9 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0043] FIG10 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0044] FIG11 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0045] FIG12 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0046] FIG13 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0047] FIG14 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0048] FIG15 is a schematic diagram of an exemplary coding framework provided in an embodiment of the present application;

[0049] FIG16 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application;

[0050] FIG17 is a structural diagram of a decoder provided in an embodiment of the present application;

[0051] FIG18 is a second structural diagram of a decoder provided in an embodiment of the present application;

[0052] FIG19 is a structural diagram 1 of an encoder provided in an embodiment of the present application;

[0053] Figure 20 is a second structural diagram of an encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the related applications and are not intended to limit the applications. It should also be noted that for ease of description, only the parts relevant to the related applications are shown in the drawings.

[0055] It should be noted that the terms “first”, “second”, “third”, etc. mentioned throughout the specification are only used to distinguish different features and do not have the function of limiting priority, sequence, size relationship, etc.

[0056] The nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0057] Next-generation video coding standard H.266 / Versatile Video Coding (VVC)

[0058] VVC Test Model (VTM)

[0059] Audio Video Coding Standard (AVS)

[0060] AVS High-Performance Model (HPM)

[0061] AVS's High Performance-Modular Artificial Intelligence Model (HPM-ModAI)

[0062] Convolutional Neural Network based in-Loop Filter (CNNLF)

[0063] Deblocking Filter (DBF)

[0064] Sample adaptive offset (SAO)

[0065] Adaptive loop filter (ALF)

[0066] Quantization Parameter (QP)

[0067] Coding Unit (CU)

[0068] Coding Tree Unit (CTU)

[0069] Mean Squared Error (MSE)

[0070] Mean Absolute Error (MAE)

[0071] Digital video compression technology is understood to primarily compress massive amounts of digital video data for easier transmission and storage. With the surge in Internet video usage and increasing demand for higher-quality video, while existing digital video compression standards can conserve significant video data, there remains a need for improved digital video compression technologies to reduce bandwidth and traffic pressures associated with digital video transmission.

[0072] During digital video encoding, the encoder reads unequal pixels from raw video sequences in different color formats, including luminance and chrominance components. This means the encoder reads a black-and-white or color image. The image is then divided into blocks, and the block data is passed to the encoder for encoding. Today, encoders typically use a hybrid frame coding scheme, which typically includes intra-frame prediction and inter-frame prediction, transform / quantization, inverse quantization / inverse transform, loop filtering, and entropy coding. The detailed process flow is shown in Figure 1. Intra-frame prediction refers only to information from the same frame, predicting pixel information within the current block to eliminate spatial redundancy. Inter-frame prediction includes motion estimation and motion compensation, which references image information from different frames and uses motion estimation to search for the motion vector that best matches the current block, eliminating temporal redundancy. Transformation converts the predicted image block to the frequency domain, redistributing the energy. Combined with quantization, it removes information that is insensitive to the human eye, eliminating visual redundancy. Entropy coding eliminates character redundancy based on the current context model and the probabilistic information of the binary bitstream. The loop filtering module primarily processes the inverse-transformed and inverse-quantized pixels, compensating for distortion and providing a better reference for subsequent pixel encoding.

[0073] For AVS3, in the loop filtering part, the traditional loop filtering module mainly includes the deblocking filter (DBF), the sample adaptive offset filter (SAO) and the adaptive loop filter (ALF). In the application of HPM-ModAI, the neural network based loop filter (CNNLF) is also used as the baseline solution of the intelligent loop filtering module, and is set between the SAO filter and the ALF filter. See Figure 2 for details. During the encoding test, according to the general test conditions of intelligent encoding, for the All Intra configuration, the ALF is turned on, and the DBF and SAO are turned off; for the Random Access and Low Delay configurations, the DBF of the I frame is turned on, the ALF is turned on, and the SAO is turned off.

[0074] In some embodiments, the scenario in which filtering processing can be performed can be a reference software test platform HPM based on AVS or a VVC reference software test platform (VVC TEST MODEL, VTM) based on Versatile Video Coding (VVC), which is not limited in the embodiments of the present application.

[0075] With the development of deep learning technology, the exploration of implementing the above-mentioned loop filtering module based on neural networks has gradually begun. The neural network loop filtering tool generally includes a loop filter of a multi-layer convolutional network and a loop filter of a multi-layer residual network. In some embodiments, the loop filter of the structure of the multi-layer convolutional network can be a network model as shown in Figure 3. The network model includes 12 hidden layers and 1 3x3 convolutional layer. Each hidden layer consists of a 3x3 convolutional layer and an activation layer (Leaky ReLU), and each convolutional layer contains 96 channels. The input of the network model is a reconstructed image 1 (NxN size) containing 4 luminance sub-blocks (Y) and 2 chrominance blocks (U, V). The network model performs filtering optimization on the input reconstructed image 1 through 12 stacked hidden network layers to obtain a filtered image. Then, according to the image-level scaling factor (SF), the residual between the input reconstructed image 1 and the filtered image is corrected, and the final output image 2 (NxN size) of the network is obtained based on the corrected residual and the reconstructed image, so that the final output image 2 is closer to the original image, bringing better coding performance.

[0076] In some embodiments, the loop filter of the multi-layer residual network can be a network model as shown in Figure 4. In addition to the reconstructed image rec_yuv, the input of the network model also includes block partition information par_yuv, prediction information pred_yuv, and QP information. Here, the QP information may include base QP information and slice QP information. The network model combines (cat) the above information and filters the combined information through a multi-layer residual network to obtain a filtered image output_yuv. The network model also corrects the residual between the input reconstructed image rec_yuv and the filtered image output_yuv according to the image-level scaling factor, and obtains the final output image of the network based on the corrected residual and the reconstructed image.

[0077] It can be seen from Figures 3 and 4 above that the loop filter of both the multi-layer convolutional network and the multi-layer residual network differs only in the network structure and processing method in the process of obtaining the filtered image. When calculating the scaling factor, the image-level scaling factor is calculated by directly traversing all corresponding pixels in the reconstructed image and the filtered image.

[0078] However, in the video encoding process of the current hybrid coding and decoding framework, encoding is performed on a block basis. The encoder first reads the image information and divides the image into several coding tree units (CTUs), and a coding tree unit can be further divided into several coding units (CUs). These coding units can be rectangular blocks or square blocks. The specific relationship can be seen in Figure 5. The encoder generally encodes based on the block level, such as CTU units or CU units. Based on the current block-level encoding process, if the block-level scaling factor can be used, the performance of correcting distortion is theoretically better. However, the use of more block-level scaling factors requires more bits to be encoded, thereby reducing the bit rate performance. Taking into account the overall performance of bit rate and distortion, most neural network-based loop filtering tools use image-level scaling factors.

[0079] Based on the above, the image-level scaling factor is derived by directly traversing all pixels within the image. As can be seen, when encoding each CTU or CU in the image, if the block-level scaling factors of some CTUs differ significantly from those of other CTUs, the calculation of the image-level scaling factor will result in significant errors.

[0080] For example, as shown in Figures 6A and 6B, an image is divided into a 4x2 layout with a total of 8 CTU block units. By traversing each CTU unit, the block-level scaling factor of each CTU unit can be obtained, as shown in Figures 6A and 6B. By traversing each pixel in the entire image, the image-level scaling factor can be obtained. Here, the image-level scaling factor in Figure 6A is 0.26, and the image-level scaling factor in Figure 6B is 0.18.

[0081] As can be seen in Figure 6A, the block-level scaling factor of 0.9 in the upper right corner differs significantly from the values ​​of the other block-level scaling factors in Figure 6A. Thus, although the block-level scaling factors of all CTUs except the upper right corner are identical in Figures 6A and 6B, the image-level scaling factor calculated in Figure 6A is still affected by the upper right corner CTU and is amplified by approximately 150% ((0.26 / 0.18)*100%) compared to Figure 6B. Therefore, when calculating the image-level scaling factor, the differences or uniqueness of a few blocks may lead to large errors in the calculated image-level scaling factor, thereby reducing the performance of distortion correction for most blocks in the entire image based on the image-level scaling factor and lowering codec accuracy.

[0082] The embodiments of the present application provide a filtering method, a decoder, an encoder, and a computer-readable storage medium, which can improve the accuracy of video encoding and decoding. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0083] Refer to Figure 7A, which shows a detailed framework diagram of a video coding system provided by an embodiment of the present application. As shown in Figure 7A, the video coding system 100 includes a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, a coding unit 109 and a decoded image cache unit 110, etc., wherein the filtering unit 108 can implement DBF filtering / SAO filtering / ALF filtering, and the coding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmatic Coding, CABAC). For the input original video signal, the coding tree unit (Coding Tree A video coding block can be obtained by dividing the video coding block into a plurality of frames (CTUs) and then the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transform and quantization unit 101, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block; specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector, which can estimate the motion of the video coding block, and then the motion compensation unit 104 calculates the motion vector based on the motion vector determined by the motion estimation unit 105. After determining the intra-frame prediction mode, the intra-frame prediction unit 103 is further configured to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109. In addition, the inverse transform and inverse quantization unit 106 is configured to reconstruct the video coding block and reconstruct a residual block in the pixel domain. The reconstructed residual block is subjected to the filter control analysis unit 107 and the filtering unit 108 to remove the block effect artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, and output the code stream of the video signal. The decoded image buffer unit 110 is configured to store the reconstructed video coding block for prediction reference.As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .

[0084] 7B shows a detailed schematic diagram of the framework of a video decoding system provided in an embodiment of the present application. As shown in FIG7B , the video decoding system 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206. The decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement DBF filtering, SAO filtering, and ALF filtering. After the input video signal is encoded as shown in FIG7A , a code stream of the video signal is output; the code stream is input to the video decoding system 200 and first passes through the decoding unit 201 to obtain decoded transform coefficients; the transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain; the intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from the previously decoded blocks of the current frame or picture; the motion compensation unit 204 determines prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and This prediction information is used to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal passes through the filtering unit 205 to remove blocking artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image buffer unit 206, which stores reference images used for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal, thereby obtaining the restored original video signal.

[0085] It should be noted that the method provided in the embodiment of the present application can be applied to the filtering unit 108 portion shown in Figure 7A (indicated by the black bold box). In other words, the method in the embodiment of the present application can be applied to a video encoding system (referred to as "encoder" for short). It should also be noted that when the embodiment of the present application is applied to an encoder, the "current block" specifically refers to the current block to be encoded in the video image (which can also be referred to as "encoding block" for short).

[0086] Based on this, when performing the filtering method, filtering processing is performed on the current block, which mainly acts on the filtering unit 108 of the video encoding system 1. The filtering method provided by the embodiment of the present application is implemented by the filtering unit 108.

[0087] The present application provides a filtering method for use in a video encoding device, i.e., an encoder. The functions implemented by this method can be implemented by a second processor in the video encoding device invoking program code. The program code can also be stored in a computer-readable storage medium. Therefore, the video encoding device includes at least a second processor and a second storage medium. The current decoding block and the current encoding block are both referred to as "current block" below.

[0088] FIG8 is a schematic diagram of an optional flow chart of a filtering method provided in an embodiment of the present application, which is applied to an encoder. The method includes:

[0089] S101 : Perform image reconstruction and filtering based on an initial residual of each block in a current frame, and determine a second reconstructed image block and a second filtered image block corresponding to each block.

[0090] In S101, for the encoder, the current frame is the original video frame. The encoder divides the input original video frame into at least one video coding block and encodes each block in the at least one video coding block through each block-level coding process. For the current block in the current frame, the encoder obtains residual pixel information of the current block through intra-frame or inter-frame prediction as the initial residual. Exemplarily, for intra-frame prediction, the encoder predicts the current block with reference to image information of adjacent blocks in the current frame to obtain a predicted block; the encoder then calculates the residual between the predicted block and the current block, i.e., the original image block, to obtain the initial residual.

[0091] The encoder performs image reconstruction and filtering on the initial residual of the current block in the current frame to determine the second reconstructed image block and the second filtered image block corresponding to the current block; and continues to reconstruct and filter the image blocks of the next block in the current frame until the current frame is processed, thereby obtaining the second reconstructed image block and the second filtered image block corresponding to each block.

[0092] In some embodiments, the encoder performs image reconstruction and filtering on the initial residual of the current block in the current frame to obtain a second reconstructed image block and a second filtered image block corresponding to the current block, which may include:

[0093] The encoder performs image reconstruction on the initial residual of the current block to obtain a second reconstructed image block; and filters the second reconstructed image block to obtain a second filtered image block.

[0094] In some embodiments, the encoder may perform image reconstruction on the initial residual of the current block, which may include: performing image reconstruction on the initial residual of the current block to obtain a second initial reconstructed image block; performing deblocking filtering on the second initial reconstructed image block to obtain a second deblocked image block; and performing sample adaptive compensation filtering on the second deblocked image block to obtain a second reconstructed image block. The encoder may filter the second reconstructed image block to obtain a second filtered image block corresponding to the current block.

[0095] Exemplarily, during the current block-level encoding process, the encoder reconstructs the initial residual of the current block to obtain a second initial reconstructed image block. The second initial reconstructed image block is used as the input of a filtering unit in the encoder, and is filtered through a DBF filtering module to obtain a second deblocked image block. The second deblocked image block is then subjected to SAO filtering to obtain a second reconstructed image block. The second reconstructed image block is used as the input of a loop filtering module in the filtering unit, and loop filtering is performed on the second reconstructed image block using a loop filtering module based on a multi-layer convolutional neural network or a loop filtering module based on a multi-layer residual neural network (CNNLF), obtaining a second filtered image block output by the loop filtering module.

[0096] It can be understood that the process of the encoder reconstructing and filtering the initial residual of each block in the current frame is consistent with the above description of the process of image block reconstruction and filtering of the current block, thereby obtaining the second reconstructed image block and the second filtered image block corresponding to each block in the current frame.

[0097] S102 : Calculate a scaling factor based on the second reconstructed image block and the second filtered image block to determine a block-level scaling factor corresponding to each block.

[0098] In S102 , the encoder calculates a scaling factor within a pixel range of each block based on the second reconstructed image block and the second filtered image block corresponding to each block in the current frame, and determines a block-level scaling factor corresponding to each block.

[0099] In some embodiments, the encoder's calculation process for the block-level scaling factor includes: a pixel residual calculation process, a block-level residual calculation process, and a block-level residual fitting process. During the pixel residual calculation process, for a current block in a current frame, the encoder may perform residual calculations for each pixel position in the block-level pixel range corresponding to the current block, based on the corresponding pixels in the current block (i.e., the original image block), the second reconstructed image block, and the second filtered image block, respectively, to obtain a raw pixel residual representing the difference between the original pixel and the reconstructed pixel, and a filtered pixel residual representing the difference between the filtered pixel and the reconstructed pixel, corresponding to each pixel position. During the block-level residual calculation process, at least one block-level residual is obtained by fusing the raw pixel residual and the filtered pixel residual at each pixel position in the pixel range of each block. Exemplarily, the raw pixel residual and the filtered pixel residual at each pixel position in the pixel range of each block are traversed and weighted to obtain a block-level raw pixel residual, a block-level filtered pixel residual, a first block-level weighted residual, and a second block-level weighted residual. Here, the first block-level weighted residual and the second block-level weighted residual are obtained by performing different weighting methods on the original pixel residual and the filtered pixel residual at each pixel position within the pixel range of the current block.

[0100] During the block-level residual fitting process, the encoder may fit at least one block-level residual calculated during the block-level residual calculation process to obtain a block-level scaling factor. Exemplarily, the block-level scaling factor is obtained by fitting using a least squares method within a preset value range (exemplarily, including a preset upper limit and a preset lower limit).

[0101] It should be noted that in S102, the encoder calculates the block-level scaling factor corresponding to each block during the block-level coding process of the block in the current frame; specifically, it is implemented during the filtering process of the block-level coding process. During the current block-level coding process, the block-level scaling factor corresponding to the current block in the current frame is calculated. The next block-level coding process continues, calculating the block-level scaling factor corresponding to the next block in the current frame, until the current frame is processed and the block-level scaling factor corresponding to each block is obtained.

[0102] In some embodiments, the encoder calculates the scaling factor based on the second reconstructed image block and the second filtered image block. The process of determining the block-level scaling factor corresponding to each block can be as shown in formulas (1) to (7), as follows:

[0103] orgResi(x i ,y i )=[org(x i ,y i )-rec(x i ,y i )] (1)

[0104] In formula (1), (x i ,y i ) represents the i-th pixel position in each block, that is, the horizontal coordinate value and vertical coordinate value of the i-th pixel position in the second reconstructed image block and the second filtered image block; org(x i ,y i ) represents the pixel corresponding to the i-th pixel position in each block, which is the original pixel; rec(x i ,y i ) represents the pixel corresponding to the i-th pixel position in the second reconstructed image block, which is the second reconstructed pixel. The encoder uses formula (1) to determine the original pixel residual orgResi(x i ,y i ). That is, the encoder calculates the pixel difference between the original pixel corresponding to each pixel position and the second reconstructed pixel corresponding to the same pixel position to obtain the original pixel residual.

[0105] cnnResi(x i ,yi )=[cnn(x i ,y i )-rec(x i ,y i )] (2)

[0106] In formula (2), cnn(x i ,y i ) represents the pixel corresponding to the i-th pixel position in the second filtered image block, which is the second filtered pixel. The encoder uses formula (2) to determine the pixel corresponding to the second reconstructed pixel rec(x i ,y i ) between the filtered pixel residual cnnResi(x i ,y i ).

[0107] In S102, the encoder may determine the block-level original pixel residual and the block-level filtered pixel residual corresponding to each block based on the original pixel residual and the filtered pixel residual obtained above, as shown in formula (3) and formula (4), as follows:

[0108]

[0109]

[0110] In formula (3) and formula (4), W k Indicates the width of each block (taking the kth block as an example), H k Represents the height value of each block. The encoder can traverse the original pixel residual orgResi(x i ,y i ), determine the block-level original pixel residual sum_orgResi corresponding to each block; and traverse the filtered pixel residual cnnResi(x i ,y i ), determine the block-level filtering pixel residual sum_cnnResi corresponding to each block.

[0111] In some embodiments, the encoder performs weighted processing based on the block-level original pixel residual and the block-level filtered pixel residual to obtain a block-level scaling factor corresponding to each block.

[0112] Here, the encoder can use a variety of optional weighted and statistical algorithms to process the block-level original pixel residuals and the block-level filtered pixel residuals to obtain the block-level scaling factor corresponding to each block. The specific selection is based on the actual situation and is not limited in the embodiment of this application. For example, weighted processing can be implemented through formulas (5) to (7) to obtain the block-level scaling factor as follows:

[0113]

[0114] The encoder uses formula (5) to calculate the filtered pixel residual cnnResi(x i ,y i ) are weighted and fused to determine the first block-level weighted residual sum_cnnMulti corresponding to each block.

[0115]

[0116] The encoder uses formula (6) to calculate the original pixel residual orgResi(x i ,y i ) and filtered pixel residual cnnResi(x i ,y i ) are weighted and fused to determine the second block-level weighted residual sum_crossMulti corresponding to each block.

[0117]

[0118] In formula (7), SF bottom is the preset lower limit value, SF up The encoder uses formula (7) to combine the preset upper limit and the preset lower limit to perform least squares processing on the first block-level weighted residual sum_cnnMulti, the second block-level weighted residual sum_crossMulti, the block-level original pixel residual sum_orgResi, and the block-level filtered pixel residual sum_cnnResi to obtain the block-level scaling factor BSF corresponding to each block.

[0119] It should be noted that each pixel in the embodiment of the present application may include a pixel value corresponding to at least one color component. For example, for an image whose color encoding method is YUV or YCbCr format, the at least one color component may include: a color component Y representing brightness (Luma), a blue chroma component and a red chroma component representing chroma (Chroma). Among them, the blue chroma component is usually represented by the symbol Cb or U, and the red chroma component is usually represented by the symbol Cr or V; U and V represent chroma and are used to describe color and saturation.

[0120] Thus, in some embodiments, the encoder calculates the scaling factor based on the second reconstructed image block and the second filtered image block. In the process of determining the block-level scaling factor corresponding to each block, the original pixel residual and the filtered pixel residual can be calculated separately for each color component in at least one color component to obtain the original pixel residual and the filtered pixel residual corresponding to each color component. Then, at each pixel position within the pixel range of each block, the original pixel residual and the filtered pixel residual corresponding to each color component are traversed and weighted statistically analyzed to obtain the block-level scaling factor corresponding to each color component.

[0121] S103: Screen the block-level scaling factors corresponding to each block corresponding to the current frame to determine a first frame-level scaling factor; the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; a difference block is a block that is different from other blocks in the current frame.

[0122] In S103, the encoder performs filtering on each block in the current frame through the filtering unit during each block-level encoding process until processing of the current frame is complete. Thus, after the encoder has processed all blocks in the current frame and obtained the block-level scaling factors corresponding to each block, the encoder filters the scaling factors corresponding to each block, that is, all the block-level scaling factors in the current frame. The encoder identifies blocks in the current frame that are different from other blocks based on the differences represented by the different block-level scaling factors, and then determines a first frame-level scaling factor based on the block-level scaling factors corresponding to each block, excluding the block-level scaling factors corresponding to the difference blocks.

[0123] In some embodiments, the encoder may determine the block-level scaling factor corresponding to the difference block from the block-level scaling factors corresponding to each block through methods such as boundary value screening, maximum and minimum value screening, and use the block-level scaling factors corresponding to each block, excluding the block-level scaling factor corresponding to the difference block, as the filtered block-level scaling factors. The filtered block-level scaling factors are then fused to obtain the first frame-level scaling factor.

[0124] In some embodiments, when the block-level scaling factor includes block-level scaling factors corresponding to each color component, the encoder may fuse the filtered block-level scaling factors on each color component to obtain a first frame-level scaling factor corresponding to each color component.

[0125] As can be seen, in S103, because the first frame-level scaling factor is calculated by filtering the block-level scaling factors corresponding to each block and using the filtered block-level scaling factors that do not include the block-level scaling factors corresponding to the difference blocks, the influence of the differences of a few blocks on the calculation of the first frame-level scaling factor is reduced, the accuracy of the first frame-level scaling factor is improved, and thus the coding accuracy is improved. Furthermore, because the first frame-level scaling factor is obtained by fusing the filtered block-level scaling factors, no unnecessary coding bits are consumed when the first frame-level scaling factor is encoded into the bitstream for transmission. This improves coding performance and accuracy while maintaining coding efficiency.

[0126] S104. Calculate a scaling factor based on the second reconstructed image and the second filtered image corresponding to the current frame to obtain a second frame-level scaling factor corresponding to the current frame; the second reconstructed image is determined by the second reconstructed image block corresponding to each block; and the second filtered image is determined by the second filtered image block corresponding to each block.

[0127] In S104, after the encoder completes filtering processing on each block in the current frame through the filtering unit, it can traverse the second reconstructed image blocks corresponding to each block obtained through each block-level encoding process to obtain a second reconstructed image corresponding to the current frame. Similarly, by traversing the second filtered image blocks corresponding to each block, a second filtered image corresponding to the current frame is obtained, i.e., the video image signal output by the filtering unit on the encoder side.

[0128] Based on the second reconstructed image and the second filtered image corresponding to the current frame, the encoder calculates the scaling factor by the pixels corresponding to each pixel position in the current frame, the second reconstructed image, and the second filtered image within the entire image (frame level) to obtain a second frame-level scaling factor.

[0129] In some embodiments, the encoder calculates the second frame-level scaling factor in a similar manner to the block-level scaling factor calculation process, including: a pixel residual calculation process, a block-level residual calculation process, and a block-level residual fitting process. In the pixel residual calculation process, the encoder can determine, based on the original pixel corresponding to each pixel position in the current frame, the original pixel residual between the second reconstructed pixel corresponding to the same pixel position in the second reconstructed image; and determine, based on the second filtered pixel corresponding to each pixel position in the second filtered image, the filtered pixel residual between the second reconstructed pixels corresponding to the same pixel position. In the block-level residual calculation process, the original pixel residual and the filtered pixel residual at each pixel position within the pixel range of the current frame are traversed and weighted to obtain a frame-level original pixel residual, a frame-level filtered pixel residual, a first frame-level weighted residual, and a second frame-level weighted residual. Here, the first frame-level weighted residual and the second frame-level weighted residual are obtained by applying different weighting methods to the original pixel residual and the filtered pixel residual at each pixel position within the current frame.

[0130] In some embodiments, since the encoder completes processing of each block in the current frame, obtains a second reconstructed image based on the second reconstructed image block corresponding to each block, obtains a second filtered image based on the second filtered image block corresponding to each block, and then calculates the second frame-level scaling factor based on the second reconstructed image and the second filtered image, the encoder may also directly use the filtered pixel residual and the original pixel residual corresponding to each pixel position in each block obtained during the pixel residual calculation process during the block-level scaling factor calculation process to determine the frame-level original pixel residual and the frame-level filtered pixel residual, thereby reducing the computational workload. Here, the original pixel residual corresponding to each pixel position in the second reconstructed image can be obtained by traversing the original pixel residual corresponding to each pixel position in the second reconstructed image block corresponding to each block; and the filtered pixel residual corresponding to each pixel position in the second filtered image can be obtained by traversing the filtered pixel residual corresponding to each pixel position in the second filtered image block. Furthermore, in the block-level residual fitting process, the first frame-level weighted residual can be obtained by weighting and fusing the filtered pixel residual corresponding to each pixel position in the second filtered image; the filtered pixel residual corresponding to each pixel position in the second filtered image and the original pixel residual corresponding to each pixel position in the second reconstructed image are weighted and fused to obtain the second frame-level weighted residual; combined with the preset upper limit value and the preset lower limit value, the first frame-level weighted residual, the second frame-level weighted residual, the frame-level original pixel residual and the frame-level filtered pixel residual are least squares processed to obtain the second frame-level scaling factor.

[0131] For example, the calculation formula of the second frame-level scaling factor may be similar to the above formulas (1) to (7). It should be noted that when calculating the second frame-level scaling factor, (x i ,y i ) represents the i-th pixel position within the pixel range of the current frame, W k Can be W m , represents the width value of the current frame (taking the mth frame in the current video stream sequence as an example); H k Can be H m , which represents the height value of the current frame (taking the mth frame in the current video stream sequence as an example).

[0132] In the embodiment of the present application, the above two methods for calculating the second frame-level scaling factor can be selected according to specific actual conditions, and are not limited in the embodiment of the present application.

[0133] It should be noted that, in some embodiments, similar to the process of calculating the block-level scaling factor on at least one color component in S102, the second frame-level scaling factor may also be obtained by respectively obtaining the original pixel residual and the filtered pixel residual corresponding to each color component of the at least one color component, and then traversing and weighted statistics of the original pixel residual and the filtered pixel residual corresponding to each color component at each pixel position within the pixel range of the current frame to obtain the second frame-level scaling factor corresponding to each color component.

[0134] S105 : Determine a frame-level scaling factor based on the first frame-level scaling factor and the second frame-level scaling factor.

[0135] In S105, the first frame-level scaling factor is obtained by screening the scaling factors corresponding to each block; the second frame-level scaling factor is obtained by calculating the scaling factors for pixels within the image range. The encoder may compare the distortion correction performance of the first and second frame-level scaling factors and determine the scaling factor with better distortion correction performance as the final frame-level scaling factor.

[0136] In some embodiments, the distortion correction performance can be compared by calculating the distortion costs of the first and second frame-level scaling factors, respectively. Alternatively, other relevant indicators can be used to measure and compare the distortion correction performance of the first and second frame-level scaling factors. The specific method is selected based on actual circumstances and is not limited in this embodiment.

[0137] In some embodiments, as shown in FIG9 , S105 may be implemented by executing S1051 - S1053 as follows:

[0138] S1051. Use a first frame-level scaling factor to scale the second reconstructed image and the second filtered image to obtain a first corrected image; use a second frame-level scaling factor to scale the second reconstructed image and the second filtered image to obtain a second corrected image.

[0139] S1051, the encoder may perform scaling processing on the second reconstructed image and the second filtered image using the first frame-level scaling factor and the second frame-level scaling factor, respectively, to obtain a first corrected image corresponding to the first frame-level scaling factor and a second corrected image corresponding to the second frame-level scaling factor.

[0140] In some embodiments, based on some embodiments of S103 above, the first frame-level scaling factor includes a first frame-level scaling factor corresponding to each color component. In the process of obtaining the first corrected image using the first frame-level scaling factor, the encoder may, for each pixel position in the second reconstructed image and the second filtered image, perform corresponding scaling and superposition processing on the first reconstructed pixel on each color component for each second reconstructed pixel in the second reconstructed image, and on the second filtered pixel on each color component for each second filtered pixel in the second filtered image, according to the first frame-level scaling factor corresponding to each color component, to obtain the first corrected image corresponding to each pixel position, wherein the second reconstructed image and the second filtered image are scaled using the first frame-level scaling factor.

[0141] In some embodiments, the encoder uses the first frame-level scaling factor to scale the second reconstructed image and the second filtered image to obtain the first corrected image, which can be implemented by formula (8) as follows:

[0142] output1(x i ,y i )=rec(x i ,y i )+RSF*[cnn(x i ,y i )-rec(x i ,y i )] (8)

[0143] In formula (8), RSF is the first frame level scaling factor; (x i ,y i ) represents the i-th pixel position within the pixel range of the current frame; cnn(x i ,y i ) represents the second filtered pixel corresponding to the i-th pixel position in the second filtered image; rec(x i ,y i ) represents the second reconstructed pixel corresponding to the i-th pixel position in the second reconstructed image. The encoder determines the second pixel residual cnn(x) between the second reconstructed pixel and the second filtered pixel for each pixel position in the pixel range of the current frame through formula (8). i ,y i )-rec(x i ,y i ); and using the first frame level scaling factor RSF, the second pixel residual is scaled and compared with the second reconstructed pixel rec(x i ,y i ) are superimposed to obtain the first corrected pixel output1(xi ,y i ), and then obtain the first corrected image by traversing the first corrected pixel corresponding to each pixel position.

[0144] In some embodiments, the encoder uses the first frame-level scaling factor to scale the second reconstructed image and the second filtered image to obtain the first corrected image, which can also be achieved by formula (9), as follows:

[0145] output1(x i ,y i )=(1-RSF)*rec(x i ,y i )+RSF*cnn(x i ,y i ) (9)

[0146] In formula (9), 1 is a preset coefficient. For example, when the scaling factor is greater than 0 and less than 1, the preset coefficient is 1. The encoder uses the difference (1-RSF) between the first frame-level scaling factor and the preset coefficient (1) through formula (9) to reconstruct the second reconstructed pixel rec(x i ,y i ) is scaled to obtain the second reconstructed corrected pixel (1-RSF)*rec(x i ,y i ); Using the first frame level scaling factor RSF, the second filtered pixel is scaled cnn(x i ,y i ), get the second filtered corrected pixel RSF*cnn(x i ,y i ); Combine the second reconstructed corrected pixel and the second filtered corrected pixel to obtain the first corrected pixel output1(x i ,y i ), and then obtain the first corrected image by traversing the first corrected pixel corresponding to each pixel position.

[0147] Here, the encoder uses the second frame-level scaling factor to scale the second reconstructed image and the second filtered image. The process of obtaining the second corrected image is consistent with the above-mentioned process of obtaining the first corrected image using the first frame-level scaling factor, and will not be repeated here.

[0148] S1052: Determine the distortion costs between the first corrected image and the second corrected image and the current frame respectively, and obtain a first distortion cost corresponding to the first frame-level scaling factor and a second distortion cost corresponding to the second frame-level scaling factor.

[0149] In S1052, the encoder determines the distortion cost between the first corrected image and the current frame, and obtains a first distortion cost corresponding to the first frame-level scaling factor; the encoder determines the distortion cost between the second corrected image and the current frame, and obtains a second distortion cost corresponding to the second frame-level scaling factor.

[0150] In some embodiments, the encoder determines the first and second distortion costs by calculating the MSE or MAE at each pixel position. Other error calculation methods may also be used to obtain a distortion cost that characterizes the error or distortion between the corrected image and the original image. The specific method is selected based on actual circumstances and is not limited in this embodiment.

[0151] S1053 : Determine a frame-level scaling factor from the first frame-level scaling factor and the second frame-level scaling factor by comparing the first distortion cost with the second distortion cost.

[0152] In S1053, the encoder compares the first distortion cost with the second distortion cost to evaluate which of the first frame-level scaling factor and the second frame-level scaling factor corresponds to higher coding performance, thereby determining the frame-level scaling factor from the first frame-level scaling factor and the second frame-level scaling factor.

[0153] In some embodiments, if the first distortion cost is less than or equal to the second distortion cost, it indicates that the distortion correction performance of the first frame-level scaling factor is higher than or equal to the second frame-level scaling factor, and the encoder determines the first frame-level scaling factor as the frame-level scaling factor. If the first distortion cost is greater than the second distortion cost, it indicates that the distortion correction performance of the second frame-level scaling factor is higher than the first frame-level scaling factor, and the encoder determines the second frame-level scaling factor as the frame-level scaling factor.

[0154] In this embodiment of the present application, after the encoder determines the frame-level scaling factor, it encodes the frame-level scaling factor into the bitstream for the decoder to read. This allows the decoder to parse the bitstream to determine the current frame and its corresponding frame-level scaling factor. During the decoding process of the current frame, the decoder's filtering unit performs residual correction on the reconstructed image during decoding based on the frame-level scaling factor.

[0155] In some embodiments, after performing image reconstruction and filtering based on the initial residual of each block in the current frame in S101 to obtain the second reconstructed image block and the second filtered image block corresponding to each block, the encoder can also obtain the second filtered image corresponding to the current frame based on the second filtered image block corresponding to each block, that is, by traversing the second filtered image block corresponding to each block.

[0156] Alternatively, in some embodiments, the filtering unit may further include an ALF filter. After performing image reconstruction and filtering based on the initial residual of each block in the current frame in S101 to obtain a second reconstructed image block and a second filtered image block corresponding to each block, the encoder may further perform adaptive filtering on the second filtered image block corresponding to each block during the filtering process of the encoding process corresponding to each block to obtain an adaptive filtered image block corresponding to each block. Adaptive filtering is performed on the second filtered image block corresponding to the current block to obtain an adaptive filtered image block corresponding to the current block, and adaptive filtering is continued on the second filtered image block corresponding to the next block in the current frame to obtain an adaptive filtered image block corresponding to the next block, until the current frame is processed and each adaptive filtered image block corresponding to each block is obtained. In this way, the encoder can obtain the second filtered image corresponding to the current frame by traversing the adaptive filtered image blocks corresponding to each block.

[0157] It will be appreciated that in the embodiments of the present application, image reconstruction and filtering are performed based on the initial residual of each block in the current frame to determine a second reconstructed image block and a second filtered image block corresponding to each block. Based on the second reconstructed image block and the second filtered image block, scaling factors are calculated within the image block to determine a block-level scaling factor corresponding to each block. Furthermore, the block-level scaling factors corresponding to each block in the current frame are screened to determine a first frame-level scaling factor, such that the first frame-level scaling factor does not include scaling factors corresponding to difference blocks, thereby reducing the impact of scaling factors corresponding to difference blocks in the current frame on the calculation of the frame-level scaling factor for the entire image and improving the accuracy of the first frame-level scaling factor. In the embodiments of the present application, the first frame-level scaling factor is compared with a second frame-level scaling factor calculated within the image-level range to determine the frame-level scaling factor, thereby improving the distortion correction performance of the frame-level scaling factor, thereby improving encoding performance and ultimately improving encoding accuracy.

[0158] In some embodiments, based on FIG. 8 or FIG. 9 , as shown in FIG. 10 , the above S103 may be implemented by executing S201 - S202 as follows:

[0159] S201 : Perform boundary value screening on the block-level scaling factor corresponding to each block to obtain N candidate block-level scaling factors.

[0160] In S201 , N is a positive integer greater than 0. The encoder determines and discards the block-level scaling factor corresponding to the difference block from the block-level scaling factors corresponding to each block by means of boundary value screening, thereby obtaining N candidate block-level scaling factors.

[0161] In some embodiments, based on FIG. 10 , as shown in FIG. 11 , S201 may be implemented through S2011 - S2013 as follows:

[0162] S2011 . Determine, from the block-level scaling factors corresponding to each block, a block-level scaling factor that is equal to a preset upper limit or a preset lower limit, as the block-level scaling factor corresponding to the difference block.

[0163] In S201, based on the above formula (7) for calculating the block-level scaling factor, the value of SF calculated by formula (7) is equal to the preset upper limit value up Or preset lower limit SF bottom The block-level scaling factors of the second reconstructed image block corresponding to these scaling factors have reached a critical correction threshold. The encoder determines, from the block-level scaling factors corresponding to each block, a block-level scaling factor that is equal to a preset upper limit or a preset lower limit as the block-level scaling factor corresponding to the difference block.

[0164] S2012: Determine a first ratio of the number of block-level scaling factors corresponding to the difference block to the total number of block-level scaling factors corresponding to each block.

[0165] S2013: If the first proportion does not exceed the preset proportion threshold, the block-level scaling factors corresponding to each block, excluding the block-level scaling factors corresponding to the difference block, are used as N candidate block-level scaling factors.

[0166] In S2012 and S2013, the encoder counts the number of block-level scaling factors corresponding to the difference blocks and calculates the ratio of the number of block-level scaling factors corresponding to the difference blocks to the total number of block-level scaling factors corresponding to each block, that is, to the total number of block-level scaling factors in the current frame. If the first ratio does not exceed a preset ratio threshold, indicating that the ratio of the number of difference blocks is not high, to reduce the impact of the difference blocks on the calculation of the global block scaling factor, the block-level scaling factors corresponding to the difference blocks are discarded from the block-level scaling factors corresponding to each block, and the block-level scaling factors other than the block-level scaling factors corresponding to the difference blocks are used as the N candidate block-level scaling factors.

[0167] In some embodiments, the preset proportion threshold may include a preset upper limit proportion threshold TH up and the first preset lower limit ratio threshold TH bottom , the specific selection is made according to the actual situation, and the embodiments of this application are not limited.

[0168] It can be understood that the block-level scaling factor corresponding to the difference block is determined by using a preset upper limit value and a preset lower limit value, and the block-level scaling factors corresponding to each block, excluding the block-level scaling factor corresponding to the difference block, are used as N candidate block-level scaling factors. The first frame-level scaling factor is calculated based on the N candidate block-level scaling factors. This reduces the influence of the differences of a small number of blocks on the calculation of the first frame-level scaling factor, improves the accuracy of the first frame-level scaling factor, and thereby improves encoding accuracy.

[0169] In some embodiments, based on FIG. 10 , as shown in FIG. 12 , S201 may be implemented through S2014 to S2017 as follows:

[0170] S2014: Determine at least one of a maximum block-level scaling factor and a minimum block-level scaling factor among the block-level scaling factors corresponding to each block.

[0171] In S2014, the encoder traverses the block-level scaling factors corresponding to each block, and determines at least one of a maximum block-level scaling factor corresponding to the maximum value and a minimum block-level scaling factor corresponding to the minimum value as at least one block-level scaling factor.

[0172] Illustratively, the encoder may determine a maximum block-level scaling factor corresponding to the maximum value among the block-level scaling factors corresponding to each block as the at least one block-level scaling factor; may also determine a minimum block-level scaling factor corresponding to the minimum value among the block-level scaling factors corresponding to each block as the at least one block-level scaling factor; may also determine the maximum block-level scaling factor and the minimum block-level scaling factor among the block-level scaling factors corresponding to each block as the at least one block-level scaling factor.

[0173] S2015 . Average the block-level scaling factors corresponding to each block except for the at least one block-level scaling factor to obtain a first average value.

[0174] In S2015 , the encoder performs mean calculation on the remaining block-level scaling factors except for at least one block-level scaling factor among the block-level scaling factors corresponding to each block to obtain a first mean.

[0175] S2016: If the difference between the at least one block-level scaling factor and the first mean is greater than a preset difference threshold, use the at least one block-level scaling factor as the block-level scaling factor corresponding to the difference block.

[0176] At S2016, the encoder calculates at least one block-level scaling factor, namely, the difference between at least one of the maximum block-level scaling factor and the minimum block-level scaling factor and the first mean. If the difference is greater than a preset difference threshold, indicating that the maximum block-level scaling factor or the minimum block-level scaling factor differs significantly from the remaining block-level scaling factors, the encoder uses the at least one block-level scaling factor as the block-level scaling factor corresponding to the difference block.

[0177] Illustratively, if the difference between the maximum block-level scaling factor and the first mean is greater than a preset difference threshold, the maximum block-level scaling factor is used as the block-level scaling factor corresponding to the difference block; if the difference between the minimum block-level scaling factor and the first mean is greater than the preset difference threshold, the minimum block-level scaling factor is used as the block-level scaling factor corresponding to the difference block; and if both the difference between the maximum block-level scaling factor and the first mean and the difference between the minimum block-level scaling factor and the first mean are greater than the preset difference threshold, both the maximum block-level scaling factor and the minimum block-level scaling factor are used as the block-level scaling factors corresponding to the difference block.

[0178] In some embodiments, the preset difference threshold may include a preset maximum difference threshold TH max and the preset minimum difference threshold TH min , the specific selection is made according to the actual situation, and the embodiments of this application are not limited.

[0179] S2017: Use the block-level scaling factors corresponding to each block, excluding the block-level scaling factor corresponding to the difference block, as N candidate block-level scaling factors.

[0180] In S2017, the encoder discards the block-level scaling factor corresponding to the difference block obtained by screening the maximum and minimum values ​​from the block-level scaling factors corresponding to each block, and obtains N candidate block-level scaling factors.

[0181] In some embodiments, there may be multiple maximum or minimum block-level scaling factors, for example, if there are multiple maximum block-level scaling factors among the block-level scaling factors corresponding to each block. Therefore, after selecting at least one block-level scaling factor as the block-level scaling factor corresponding to the difference block in S2016, a second ratio of the number of block-level scaling factors corresponding to the difference block, determined by at least one of the maximum or minimum block-level scaling factors, to the total number of block-level scaling factors corresponding to each block may be calculated. If the second ratio does not exceed a preset ratio threshold, the maximum or minimum block-level scaling factor is selected as the block-level scaling factor corresponding to the difference block, and the block-level scaling factors corresponding to each block, excluding the block-level scaling factor corresponding to the difference block, are selected as the N candidate block-level scaling factors.

[0182] It can be understood that the block-level scaling factor corresponding to the difference block is determined using the maximum scaling factor and the minimum scaling factor, and the block-level scaling factors corresponding to each block, excluding the block-level scaling factor corresponding to the difference block, are used as N candidate block-level scaling factors. The first frame-level scaling factor is calculated based on the N candidate block-level scaling factors. This reduces the influence of the differences of a small number of blocks on the calculation of the first frame-level scaling factor, improves the accuracy of the first frame-level scaling factor, and thereby improves encoding accuracy.

[0183] S202 : Perform averaging processing based on N candidate block-level scaling factors to obtain a first frame-level scaling factor.

[0184] In S202 , the encoder performs averaging processing based on the N candidate block-level scaling factors, and merges the N candidate block-level scaling factors into a first frame-level scaling factor.

[0185] In some embodiments, based on FIG. 11 and FIG. 12 , S202 may include S2021 as shown in FIG. 13 , as follows:

[0186] S2021: Average N candidate block-level scaling factors to obtain a first frame-level scaling factor.

[0187] In S2021, the encoder may average the N candidate block-level scaling factors obtained by screening according to the preset upper limit value and the preset lower limit value to obtain a first frame-level scaling factor; or average the N candidate block-level scaling factors obtained by screening according to the maximum block-level scaling factor and the minimum block-level scaling factor to obtain the first frame-level scaling factor.

[0188] In some embodiments, based on FIG. 11 , that is, based on the N candidate block-level scaling factors obtained by screening according to the preset upper limit value and the preset lower limit value, S202 may also be implemented by executing S2022 to S2025 as shown in FIG. 14 , as follows:

[0189] S2022: Determine at least one candidate block-level scaling factor among the N candidate block-level scaling factors, namely, a maximum candidate block-level scaling factor and a minimum candidate block-level scaling factor.

[0190] In S2022, based on S2013 in FIG. 11 , the encoder further screens the N candidate block-level scaling factors obtained by screening based on the preset upper limit value and the preset lower limit value, and determines, among the N candidate block-level scaling factors, at least one of a maximum candidate block-level scaling factor corresponding to the maximum value and a minimum candidate block-level scaling factor corresponding to the minimum value, as the at least one candidate block-level scaling factor.

[0191] Here, the process of the encoder determining at least one candidate block-level scaling factor from N candidate block-level scaling factors is consistent with the above-described process of determining at least one block-level scaling factor from the block-level scaling factors corresponding to each block, and is not repeated here.

[0192] S2023 . Among the N candidate block-level scaling factors, average the candidate block-level scaling factors except for the at least one candidate block-level scaling factor to obtain a second average value.

[0193] In S2023, the encoder averages the N candidate block-level scaling factors except for at least one candidate block-level scaling factor to obtain a second average value. The process of obtaining the second average value is consistent with the description of the process of calculating the first average value in S2015 above and is not repeated here.

[0194] S2024: If the difference between at least one candidate block-level scaling factor and the second mean is greater than a preset difference threshold, determine the candidate block-level scaling factors other than the at least one candidate block-level scaling factor among the N candidate block-level scaling factors as M updated block-level scaling factors.

[0195] In S2024, if the difference between at least one candidate block-level scaling factor and the second mean is greater than a preset difference threshold, the encoder uses the at least one candidate block-level scaling factor as the scaling factor corresponding to the difference block, discards at least one candidate block-level scaling factor from the N candidate block-level scaling factors, and obtains M updated block-level scaling factors. That is, the candidate block-level scaling factors other than the at least one candidate block-level scaling factor from the N candidate block-level scaling factors are determined as the M updated block-level scaling factors. Here, M is a positive integer greater than 0 and less than or equal to N.

[0196] S2025: Perform averaging processing on the M updated block-level scaling factors to obtain a first frame-level scaling factor.

[0197] In S2025 , the encoder averages the M updated block-level scaling factors obtained through the two screenings to obtain a first frame-level scaling factor.

[0198] Similarly, in some embodiments, based on FIG12 , that is, based on the N candidate block-level scaling factors obtained by screening based on the maximum block-level scaling factor and the minimum block-level scaling factor, the encoder may further screen the N candidate block-level scaling factors using a preset upper limit and a preset lower limit to obtain M updated block-level scaling factors, and then average the M updated block-level scaling factors to obtain the first frame-level scaling factor. This will not be further described here.

[0199] It should be noted that the above-mentioned Figures 13 and 14 are parallel method flows. In actual applications, one of the method flows can be selected to execute according to actual conditions, and the embodiments of this application are not limited thereto.

[0200] It can be understood that by performing a screening using the preset upper limit and the preset lower limit, the N candidate block-level scaling factors after the screening are further screened using the maximum scaling factor and the minimum scaling factor, thereby further reducing the impact of the differences of a few blocks on the calculation of the first frame-level scaling factor, improving the accuracy of the first frame-level scaling factor, and thereby improving encoding accuracy.

[0201] In some embodiments of the present application, an encoder framework is provided, as shown in FIG15 . In which, the neural network based in-loop filter (NNLF) can be implemented by a loop filter based on a multi-layer convolutional network or a loop filter based on a multi-layer residual network. In FIG15 , the scaling factor calculation is used to determine the second frame-level scaling factor SF and the first frame-level scaling factor RSF, and the frame-level scaling factor is determined based on SF and RSF. The enabling of the scaling factor calculation does not depend on the switches of DB, SAO, ALF, and NNLF, and is placed after NNLF and before ALF.

[0202] As shown in FIG15 , when the encoding end enters the loop filtering, it processes according to the established filter order. When entering the scaling factor calculation process, the scaling factor calculation process includes: through each block-level encoding process, for each pixel position in each block of the current frame of the current sequence, traversing each color component of each pixel corresponding to each pixel position in each block, the second reconstructed image block, and the second filtered image block, and calculating the block-level scaling factor corresponding to each block using formulas (1) to (7). When each block in the current frame is processed, L block-level scaling factors are obtained.

[0203] During the scaling factor calculation process, for the L block-level scaling factors corresponding to the current frame, a range of scaling factor derivation is limited by a preset upper and lower limit. The L block-level scaling factors are then screened using the derivation range. For the jth block-level scaling factor ctuSF(j) among the L block-level scaling factors, if ctuSF(j) equals the preset upper or lower limit, it indicates that the magnitude of correction required for the second reconstructed image block corresponding to ctuSF(j) has reached a critical value for correction, and the block is designated as a special block, or difference block. Each block-level scaling factor is compared with the preset upper and lower limits to determine and calculate the proportion of the special block in the L blocks, or a first proportion. If the proportion is less than a preset proportion threshold, the special block is discarded from the L block-level scaling factors for global image consideration.

[0204] During the scaling factor calculation process, after screening the preset upper and lower limits, the remaining block-level scaling factors, i.e., at least one candidate block-level scaling factor, are traversed to obtain a maximum block-level scaling factor ctuSF_max or a minimum block-level scaling factor ctuSF_min. A mean is calculated for the at least one candidate block-level scaling factor to obtain a second mean. If the difference between ctuSF_max or ctuSF_min and the second mean exceeds a preset difference threshold, ctuSF_max or ctuSF_min is considered a special block. To account for global image blocks, the special block is discarded from the at least one candidate block-level scaling factor, resulting in an updated block-level scaling factor set ctuSF_SET, i.e., M updated block-level scaling factors. The ctuSF_SET is averaged to obtain a first frame-level scaling factor (RSF).

[0205] During the scaling factor calculation process, based on each pixel position in the current frame, the second reconstructed image, and the second filtered image, each color component of each pixel in each block corresponding to each pixel position is traversed to calculate the second frame-level scaling factor SF for the current frame. The residuals of the second reconstructed image and the second filtered image are corrected using the SF and RSF, respectively, to obtain a first corrected image corresponding to the RSF and a second corrected image corresponding to the SF. The first and second corrected images are compared with the original image and the costs are calculated. For example, a first distortion cost D_RSF corresponding to the RSF and a second distortion cost D_SF corresponding to the SF are obtained. D_RSF and D_SF are compared. If D_RSF <= D_SF, RSF is used as the final scaling factor for the current color component of the current frame. Otherwise, if D_RSF > D_SF, SF is used as the final scaling factor and the selected scaling factor is encoded into the bitstream. If the scaling factor calculation for the current frame has been completed, the next frame is loaded for processing.

[0206] In some embodiments, the applicant tested a network model that integrates the scaling factor calculation method of the present application under the general test conditions (Random Access, RA) on some datasets in the general sequence Class D WQVGA specified by JVET. The test was based on the VTM11.0-nnvc reference platform, and the above-mentioned preset proportion threshold was configured as TH bottom =TH up =10%, and the preset difference threshold is configured as TH min =TH max=0.5. According to the test results, the network model that integrates the scaling factor calculation method of the present application has improved the RA performance of the color components Y, U, and V compared to the network model in the related art, as shown in Table 1 (the data in Table 1 is negatively correlated with the RA performance):

[0207] Table 1

[0208] Test data set Y component U component V component BasketballPass 0.00% -0.19% 0.03% BQSquare 0.01% -0.22% -0.29% BlowingBubbles 0.02% -0.22% 0.04% RaceHorese -0.01% -0.10% -0.15%

[0209] As shown in Table 1, compared to network models in related technologies, the network model integrating the scaling factor calculation module of this application achieves significant improvements in both the U and V components for different test datasets. The U component performance improved by -0.19%, -0.22%, -0.22%, and -0.10%, respectively; while the V component performance improved by -0.29% and -0.15%, respectively. This data demonstrates improved encoding and decoding performance and accuracy.

[0210] It can be understood that the embodiment of the present application reduces the influence of a few special blocks on the calculation of the frame-level scaling factor by considering the differences in each block-level scaling factor, so that the optimized frame-level scaling factor finally derived, that is, the first frame-level scaling factor, is more suitable for scaling processing of most image blocks. By using the first frame-level scaling factor to correct the residual of the reconstructed image and the neural network loop filter image, the corrected image can be closer to the original image to obtain better coding performance. At the same time, the scaling factor calculation method of the embodiment of the present application is only run on the encoding end, and the calculated first frame-level scaling factor is at the image level, so there is no need to consume the number of coding bits and there is no effect on the decoding complexity. By introducing the optimization method for the scaling factor of the embodiment of the present application, it is possible to significantly improve the coding performance on the basis of the neural network loop filter. In practical applications, the performance can be further improved by optimizing the algorithm and setting the threshold value.

[0211] The embodiment of the present application provides a filtering method, which is applied to a decoder. FIG16 is a schematic diagram of an optional flow chart of the filtering method provided in the embodiment of the present application, including S301-S303 as follows:

[0212] S301. Parse the bitstream to determine a frame-level scaling factor and an initial residual of the current block. The frame-level scaling factor is determined by using a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame. The first frame-level scaling factor is obtained by filtering block-level scaling factors in the current frame. The first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block. A difference block is a block that is different from other blocks in the current frame.

[0213] In S301, the decoder receives and parses the bitstream to obtain a frame-level scaling factor corresponding to the current frame, and performs image reconstruction on the current block in the current frame, such as through inverse transformation and inverse quantization, to obtain an initial residual of the current block in the current frame. Here, the frame-level scaling factor is determined by comparing the distortion costs corresponding to the first and second frame-level scaling factors corresponding to the current frame in the encoder filtering method described above. The first frame-level scaling factor is obtained by filtering the block-level scaling factors in the current frame and fusing them based on the filtered block-level scaling factors. Here, the filtered block-level scaling factors do not include the block-level scaling factors corresponding to the difference blocks; the difference blocks are blocks that differ from other blocks in the current frame. Therefore, the first frame-level scaling factor does not include the block-level scaling factors corresponding to the difference blocks.

[0214] S302, reconstructing and filtering an image block based on the initial residual to determine a first reconstructed image block and a first filtered image block;

[0215] In S302, the decoder reconstructs the image block based on the initial residual, and obtains a first reconstructed image block by superimposing the initial residual on the predicted image block. The decoder filters the first reconstructed image block to obtain a first filtered image block.

[0216] In some embodiments, the decoder performs image block reconstruction on the initial residual to obtain a first initial reconstructed image block; performs deblocking filtering on the first initial reconstructed image block to obtain a first deblocked image block; performs sample adaptive compensation filtering on the first deblocked image block to obtain a first reconstructed image block. The encoder performs loop filtering on the first reconstructed image block to obtain a first filtered image block.

[0217] S303 : Perform scaling processing using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain a corrected image block corresponding to the current block.

[0218] In S303, the decoder determines the first reconstructed pixel and the first filtered pixel corresponding to each pixel position based on the first reconstructed image block and the first filtered image block; scales the first reconstructed pixel and the first filtered pixel using a frame-level scaling factor to determine the corrected pixel corresponding to each pixel position; and determines the corrected image block based on the corrected pixel corresponding to each pixel position.

[0219] In some embodiments, the decoder can determine, for each pixel position, a first pixel residual between the first reconstructed pixel and the first filtered pixel; scale the first pixel residual using a frame-level scaling factor, and superimpose it with the first reconstructed pixel to obtain a corrected pixel corresponding to each pixel position.

[0220] In some embodiments, the decoder can use the difference between the frame-level scaling factor and the preset coefficient to scale the first reconstructed pixel to obtain a first reconstructed corrected pixel; use the frame-level scaling factor to scale the first filtered pixel to obtain a first filtered corrected pixel; and combine the first reconstructed corrected pixel and the first filtered corrected pixel to obtain a corrected pixel.

[0221] In some embodiments of the present application, the frame-level scaling factor includes a frame-level scaling factor corresponding to each color component. For each pixel position, the decoder performs corresponding scaling processing on the first pixel of the first reconstructed pixel on each color component and the second pixel of the first filtered pixel on each color component according to the frame-level scaling factor corresponding to each color component, to obtain a corrected pixel corresponding to each color component.

[0222] In an embodiment of the present application, the decoder can also reconstruct and filter the image block of the next block in the current frame, and use the frame-level scaling factor to scale the first reconstructed image block and the first filtered image block of the next block to obtain the corrected image block corresponding to the next block, until the processing of the current frame is completed. According to the corrected image block corresponding to each block in the current frame, the first filtered image of the current frame is obtained, that is, the video image signal output by the filtering unit on the decoder side.

[0223] Alternatively, after performing scaling processing using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain a corrected image block corresponding to the current block, the decoder may further perform adaptive filtering on the corrected image block to obtain a second filtered image block; continue filtering the next block in the current frame to obtain a second filtered image block corresponding to the next block; continue processing until the current frame is processed, and obtain the first filtered image of the current frame based on the second filtered image blocks corresponding to each block in the current frame.

[0224] It should be noted that the filtering method on the decoder side provided in the embodiment of the present application is applied to the filtering unit on the decoder side. The description of the relevant steps on the above-mentioned decoder side is consistent with the description when the filtering unit on the encoder side executes the corresponding steps, and will not be repeated here.

[0225] It can be understood that the frame-level scaling factor parsed from the bitstream by the decoder is determined by the first and second frame-level scaling factors corresponding to the current frame. The first frame-level scaling factor is obtained by filtering the block-level scaling factors of the current frame and does not include the block-level scaling factors corresponding to the difference blocks. This reduces the error caused by the block-level scaling factors corresponding to the difference blocks in the calculation of the entire frame-level scaling factor, allowing the calculated first frame-level scaling factor to more accurately represent the required correction magnitude for most blocks in the current frame, thereby improving the accuracy of the first frame-level scaling factor. Furthermore, since the frame-level scaling factor is determined by comparing the first and second frame-level scaling factors, the distortion correction performance of the frame-level scaling factor is further improved, as well as the accuracy of the corrected image block obtained by filtering the current block using the frame-level scaling factor, thereby improving decoding performance and accuracy.

[0226] Based on the implementation basis of the aforementioned embodiment, as shown in FIG17 , the embodiment of the present application provides a decoder 1, comprising:

[0227] The parsing portion 10 is configured to parse the bitstream and determine a frame-level scaling factor and an initial residual of a current block; wherein the frame-level scaling factor is determined by a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame, wherein the first frame-level scaling factor is obtained by screening block-level scaling factors in the current frame, wherein the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; wherein the difference block is a block that is different from other blocks in the current frame;

[0228] A first reconstruction and filtering part 11 is configured to perform image block reconstruction and filtering based on the initial residual to determine a first reconstructed image block and a first filtered image block;

[0229] The first determining part 12 is configured to perform scaling processing using the frame-level scaling factor, the first reconstructed image block and the first filtered image block to obtain a modified image block corresponding to the current block.

[0230] In some embodiments of the present application, the first determination part 12 is further configured to determine the first reconstructed pixel and the first filtered pixel corresponding to each pixel position based on the first reconstructed image block and the first filtered image block; use the frame-level scaling factor to scale the first reconstructed pixel and the first filtered pixel to determine the corrected pixel corresponding to each pixel position; and determine the corrected image block based on the corrected pixel corresponding to each pixel position.

[0231] In some embodiments of the present application, the first determination part 12 is further configured to determine, for each pixel position, a first pixel residual between the first reconstructed pixel and the first filtered pixel; scale the first pixel residual using the frame-level scaling factor, and superimpose it with the first reconstructed pixel to obtain a corrected pixel corresponding to each pixel position.

[0232] In some embodiments of the present application, the first determination part 12 is further configured to use the difference between the frame-level scaling factor and the preset coefficient to scale the first reconstructed pixel to obtain a first reconstructed corrected pixel; use the frame-level scaling factor to scale the first filtered pixel to obtain a first filtered corrected pixel; and combine the first reconstructed corrected pixel and the first filtered corrected pixel to obtain the corrected pixel.

[0233] In some embodiments of the present application, the frame-level scaling factor includes: a frame-level scaling factor corresponding to each color component; the first determining part 12 is further configured to, for each pixel position, perform corresponding scaling processing on the first pixel of the first reconstructed pixel on each color component and the second pixel of the first filtered pixel on each color component according to the frame-level scaling factor corresponding to each color component, to obtain the corrected pixel corresponding to each color component.

[0234] In some embodiments of the present application, the first reconstruction and filtering part 11 is further configured to reconstruct an image block based on the initial residual to obtain a first reconstructed image block; and filter the first reconstructed image block to obtain the first filtered image block.

[0235] In some embodiments of the present application, the first reconstruction and filtering part 11 is further configured to perform image block reconstruction on the initial residual to obtain a first initial reconstructed image block; perform deblocking filtering on the first initial reconstructed image block to obtain a first deblocked image block; and perform sample adaptive compensation filtering on the first deblocked image block to obtain the first reconstructed image block.

[0236] In some embodiments of the present application, the parsing part 10, the first reconstruction and filtering part 11, and the first determination part 12 are further configured to perform scaling processing using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain the corrected image block corresponding to the current block, and then perform image block reconstruction and filtering on the next block in the current frame, and perform scaling processing on the first reconstructed image block and the first filtered image block of the next block using the frame-level scaling factor to obtain the corrected image block corresponding to the next block, until the processing of the current frame is completed, and the first filtered image of the current frame is obtained according to the corrected image blocks corresponding to each block in the current frame.

[0237] In some embodiments of the present application, the decoder 1 also includes a first adaptive filtering part, which is configured to adaptively filter the corrected image block to obtain a second filtered image block; continue filtering the next block in the current frame to obtain the second filtered image block corresponding to the next block, continue processing until the processing of the current frame is completed, and obtain the first filtered image of the current frame according to the second filtered image block corresponding to each block in the current frame.

[0238] In the actual application of the present application, as shown in FIG18 , the embodiment of the present application further provides a decoder, including:

[0239] A first memory 14 and a first processor 15;

[0240] The first memory 14 stores a computer program that can be run on the first processor 15 , and the first processor 15 implements a filtering method corresponding to the decoder when executing the program.

[0241] Among them, the first processor 15 can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (ASICs), single or multiple general integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor 15 can execute the corresponding steps of the filtering method on the decoder side in the aforementioned embodiment.

[0242] It should be noted that the description of the above decoder embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the decoder embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0243] The embodiment of the present application provides an encoder 2, as shown in FIG19 , including:

[0244] The second reconstruction and filtering part 20 is configured to perform image reconstruction and filtering based on the initial residual of each block in the current frame, and determine a second reconstructed image block and a second filtered image block corresponding to each block;

[0245] The second determining portion 21 is configured to calculate a scaling factor based on the second reconstructed image block and the second filtered image block to determine a block-level scaling factor corresponding to each block; screen the block-level scaling factors corresponding to each block corresponding to the current frame to determine a first frame-level scaling factor; the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; the difference block is a block that is different from other blocks in the current frame; calculate a scaling factor based on the second reconstructed image and the second filtered image corresponding to the current frame to obtain a second frame-level scaling factor corresponding to the current frame; the second reconstructed image is determined by the second reconstructed image block corresponding to each block; the second filtered image is determined by the second filtered image block corresponding to each block; and the frame-level scaling factor is determined based on the first frame-level scaling factor and the second frame-level scaling factor.

[0246] In some embodiments of the present application, the second determining portion 21 is further configured to perform boundary value screening on the block-level scaling factor corresponding to each block to obtain N candidate block-level scaling factors, where N is a positive integer greater than 0; and perform averaging processing based on the N candidate block-level scaling factors to obtain the first frame-level scaling factor.

[0247] In some embodiments of the present application, the second determining portion 21 is further configured to determine, among the block-level scaling factors corresponding to each block, a block-level scaling factor that is equal to a preset upper limit or a preset lower limit, as the block-level scaling factor corresponding to the difference block; determine a first ratio of the number of block-level scaling factors corresponding to the difference block to the total number of block-level scaling factors corresponding to each block; and if the first ratio does not exceed a preset ratio threshold, use the block-level scaling factors, among the block-level scaling factors corresponding to each block, excluding the block-level scaling factor corresponding to the difference block, as the N candidate block-level scaling factors.

[0248] In some embodiments of the present application, the second determining unit 21 is further configured to determine at least one of a maximum block-level scaling factor and a minimum block-level scaling factor among the block-level scaling factors corresponding to each block; average the block-level scaling factors other than the at least one block-level scaling factor among the block-level scaling factors corresponding to each block to obtain a first average value; if a difference between the at least one block-level scaling factor and the first average value is greater than a preset difference threshold, use the at least one block-level scaling factor as the block-level scaling factor corresponding to the difference block; and use the block-level scaling factors other than the block-level scaling factor corresponding to the difference block among the block-level scaling factors corresponding to each block as the N candidate block-level scaling factors.

[0249] In some embodiments of the present application, the second determining part 21 is further configured to determine at least one candidate block-level scaling factor from among the N candidate block-level scaling factors: a maximum candidate block-level scaling factor and a minimum candidate block-level scaling factor;

[0250] Among the N candidate block-level scaling factors, candidate block-level scaling factors other than the at least one candidate block-level scaling factor are averaged to obtain a second average value; if a difference between the at least one candidate block-level scaling factor and the second average value is greater than a preset difference threshold, the candidate block-level scaling factors other than the at least one candidate block-level scaling factor among the N candidate block-level scaling factors are determined as M updated block-level scaling factors, where M is greater than 0 and less than or equal to N; and the M updated block-level scaling factors are averaged to obtain the first frame-level scaling factor.

[0251] In some embodiments of the present application, the second determining part 21 is further configured to scale the second reconstructed image and the second filtered image using the first frame-level scaling factor to obtain a first corrected image;

[0252] The second reconstructed image and the second filtered image are scaled using the second frame-level scaling factor to obtain a second corrected image; distortion costs between the first corrected image and the second corrected image and the current frame are determined respectively to obtain a first distortion cost corresponding to the first frame-level scaling factor and a second distortion cost corresponding to the second frame-level scaling factor; and the frame-level scaling factor is determined from the first frame-level scaling factor and the second frame-level scaling factor by comparing the first distortion cost with the second distortion cost.

[0253] In some embodiments of the present application, the second determining portion 21 is further configured to, if the first distortion cost is less than or equal to the second distortion cost, determine the first frame-level scaling factor as the frame-level scaling factor; and if the first distortion cost is greater than the second distortion cost, determine the second frame-level scaling factor as the frame-level scaling factor.

[0254] In some embodiments of the present application, the second determination part 21 is further configured to determine the original pixel residual between the second reconstructed pixel of the second reconstructed image block and the original pixel corresponding to each pixel position in each block; determine the filtered pixel residual between the second reconstructed pixel and the second filtered pixel corresponding to each pixel position in the second filtered image block; determine the block-level original pixel residual and the block-level filtered pixel residual corresponding to each block based on the original pixel residual and the filtered pixel residual; and perform weighted processing based on the block-level original pixel residual and the block-level filtered pixel residual to obtain the block-level scaling factor corresponding to each block.

[0255] In some embodiments of the present application, the second determination part 21 is further configured to weight and fuse the filtered pixel residuals corresponding to each pixel position in each block to determine the first block-level weighted residual corresponding to each block; weight and fuse the original pixel residuals and the filtered pixel residuals corresponding to each pixel position in each block to determine the second block-level weighted residual corresponding to each block; and perform least squares processing on the first block-level weighted residuals, the second block-level weighted residuals, the block-level original pixel residuals, and the block-level filtered pixel residuals in combination with a preset upper limit value and a preset lower limit value to obtain the block-level scaling factor corresponding to each block.

[0256] In some embodiments of the present application, the second determining part 21 is further configured to traverse the original pixel residual corresponding to each pixel position in the second reconstructed image to determine the frame-level original pixel residual; the original pixel residual corresponding to each pixel position in the second reconstructed image is obtained based on the original pixel residual corresponding to each pixel position in the second reconstructed image block corresponding to each block; traverse the filtered pixel residual corresponding to each pixel position in the second filtered image to determine the frame-level filtered pixel residual; the filtered pixel residual corresponding to each pixel position in the second filtered image is obtained based on the original pixel residual corresponding to each pixel position in the second filtered image block. The corresponding filtered pixel residual is obtained; the filtered pixel residual corresponding to each pixel position in the second filtered image is weighted and fused to obtain a first frame-level weighted residual; the filtered pixel residual corresponding to each pixel position in the second filtered image and the original pixel residual corresponding to each pixel position in the second reconstructed image are weighted and fused to obtain a second frame-level weighted residual; combined with a preset upper limit value and a preset lower limit value, the first frame-level weighted residual, the second frame-level weighted residual, the frame-level original pixel residual and the frame-level filtered pixel residual are least squares processed to obtain the second frame-level scaling factor.

[0257] In some embodiments of the present application, the first frame-level scaling factor includes a first frame-level scaling factor corresponding to each color component; the second determining part 21 is further configured to, for each pixel position in the second reconstructed image and the second filtered image, perform corresponding scaling and superposition processing on the first reconstructed pixel of each second reconstructed pixel in the second reconstructed image on the respective color component, and on the second filtered pixel of each second filtered pixel in the second filtered image on the respective color component, according to the first frame-level scaling factor corresponding to the respective color component, to obtain a first corrected pixel corresponding to each pixel position, containing the respective color components; and traverse the first corrected pixels corresponding to each pixel position to obtain a first corrected image.

[0258] In some embodiments of the present application, the second reconstruction and filtering part 22 is further configured to perform image reconstruction and filtering on the initial residual of the current block in the current frame to obtain a second reconstructed image block and a second filtered image block corresponding to the current block, and continue to perform image block reconstruction and filtering on the next block in the current frame until the current frame is processed, thereby obtaining a second reconstructed image block and a second filtered image block corresponding to each block.

[0259] In some embodiments of the present application, the second reconstruction and filtering part 22 is further configured to perform image reconstruction on the initial residual of the current block to obtain the second reconstructed image block; and filter the second reconstructed image block to obtain the second filtered image block.

[0260] In some embodiments of the present application, the encoder 2 also includes a filtered image output part, which is configured to perform image reconstruction and filtering based on the initial residual of each block in the current frame to obtain the second reconstructed image block and the second filtered image block corresponding to each block, and then obtain the second filtered image corresponding to the current frame based on the second filtered image block corresponding to each block.

[0261] In some embodiments of the present application, the filtered image output part is further configured to perform image reconstruction and filtering on the initial residual of each block in the current frame to obtain the second reconstructed image block and the second filtered image block corresponding to each block, and then perform adaptive filtering on the second filtered image block corresponding to each block to obtain the adaptive filtered image block corresponding to each block, and traverse the adaptive filtered image blocks corresponding to each block to obtain the second filtered image corresponding to the current frame.

[0262] In practical applications, as shown in FIG20 , an embodiment of the present application further provides an encoder, including:

[0263] a second memory 25 and a second processor 26;

[0264] The second memory 25 stores a computer program that can be run on the second processor 26. When the second processor 26 executes the program, the encoder performs a filtering method corresponding to the program.

[0265] It should be noted that the description of the above encoder embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the encoder embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0266] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a first processor, the filtering method corresponding to the decoder of the claim is implemented; or, when the computer program is executed by a second processor, the filtering method corresponding to the encoder of the claim is implemented.

[0267] The various components in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software functional modules.

[0268] If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned computer-readable storage medium includes: ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM), etc. Various media that can store program codes are not limited in the embodiments of the present disclosure.

[0269] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0270] Embodiments of the present application provide a filtering method, decoder, encoder, and computer-readable storage medium. The frame-level scaling factor parsed from a bitstream by the decoder is determined using a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame. The first frame-level scaling factor is obtained by filtering the block-level scaling factors of the current frame and does not include the block-level scaling factors corresponding to the difference blocks. This reduces the error caused by the block-level scaling factors corresponding to the difference blocks in calculating the overall frame-level scaling factor. This allows the calculated first frame-level scaling factor to more accurately represent the required correction magnitude for most blocks in the current frame, thereby improving the accuracy of the first frame-level scaling factor. Furthermore, because the frame-level scaling factor is determined from the first and second frame-level shrinkage factors by comparing the two frame-level scaling factors, the distortion correction performance of the frame-level shrinkage factor is further improved, as well as the accuracy of the corrected image block obtained by filtering the current block using the frame-level shrinkage factor, thereby improving decoding performance and accuracy.

[0271] Furthermore, the encoder performs image reconstruction and filtering based on the initial residual of each block in the current frame to determine a second reconstructed image block and a second filtered image block corresponding to each block. Based on the second reconstructed image block and the second filtered image block, the encoder calculates a scaling factor within the image block to determine a block-level scaling factor corresponding to each block. Furthermore, the encoder filters the block-level scaling factors corresponding to each block in the current frame to determine a first frame-level scaling factor. This first frame-level scaling factor excludes scaling factors corresponding to difference blocks in the current frame, thereby reducing the impact of scaling factors corresponding to difference blocks in the current frame on the calculation of the frame-level scaling factor for the entire image and improving the accuracy of the first frame-level scaling factor. Furthermore, the encoder compares the first frame-level scaling factor with a second frame-level scaling factor calculated within the image-level range to determine the frame-level scaling factor. This improves the distortion correction performance of the frame-level scaling factor, thereby improving encoding performance and ultimately improving encoding accuracy. Furthermore, because the filtered block-level scaling factors are combined to obtain the first frame-level scaling factor, no extra coding bits are consumed when the first frame-level scaling factor is encoded into the bitstream for transmission. This improves encoding performance and accuracy while maintaining coding efficiency. Furthermore, when screening the scaling factors corresponding to the difference blocks, a screening is performed using a preset upper limit and a preset lower limit. After the screening, the N candidate block-level scaling factors are screened again using the maximum scaling factor and the minimum scaling factor. This further reduces the impact of the differences of a few blocks on the calculation of the first frame-level scaling factor, improves the accuracy of the first frame-level scaling factor, and further improves encoding accuracy.

Claims

1. A filtering method, applied to a decoder, comprising: parsing a bitstream to determine a frame-level scaling factor and an initial residual of a current block; wherein the frame-level scaling factor is determined by using a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame, wherein the first frame-level scaling factor is obtained by screening block-level scaling factors in the current frame, wherein the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; the difference block is a block that is different from other blocks in the current frame; Performing image block reconstruction and filtering based on the initial residual to determine a first reconstructed image block and a first filtered image block; Scaling processing is performed using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain a corrected image block corresponding to the current block.

2. The method according to claim 1, wherein The performing scaling processing using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain a corrected image block corresponding to the current block includes: Determining a first reconstructed pixel and a first filtered pixel corresponding to each pixel position based on the first reconstructed image block and the first filtered image block; Scaling the first reconstructed pixel and the first filtered pixel using the frame-level scaling factor to determine a corrected pixel corresponding to each pixel position; The corrected image block is determined according to the corrected pixel corresponding to each pixel position.

3. The method according to claim 2, wherein: The scaling process of the first reconstructed pixel and the first filtered pixel using the frame-level scaling factor to determine the corrected pixel corresponding to each pixel position includes: For each pixel position, determining a first pixel residual between the first reconstructed pixel and the first filtered pixel; The first pixel residual is scaled using the frame-level scaling factor and superimposed with the first reconstructed pixel to obtain a corrected pixel corresponding to each pixel position.

4. The method according to claim 2, wherein: The scaling process of the first reconstructed pixel and the first filtered pixel using the frame-level scaling factor to determine the corrected pixel corresponding to each pixel position includes: performing scaling processing on the first reconstructed pixel using a difference between the frame-level scaling factor and a preset coefficient to obtain a first reconstructed corrected pixel; Scaling the first filtered pixels using the frame-level scaling factor to obtain first filtered corrected pixels; The first reconstructed corrected pixel and the first filtered corrected pixel are combined to obtain the corrected pixel.

5. The method according to claim 2, wherein: The frame-level scaling factor includes: a frame-level scaling factor corresponding to each color component; The scaling process of the first reconstructed pixel and the first filtered pixel using the frame-level scaling factor to determine the corrected pixel corresponding to each pixel position includes: For each pixel position, according to the frame-level scaling factor corresponding to each color component, the first pixel of the first reconstructed pixel on each color component and the second pixel of the first filtered pixel on each color component are scaled accordingly to obtain the corrected pixels corresponding to each color component.

6. The method according to any one of claims 1 to 5, wherein: The performing image block reconstruction and filtering based on the initial residual to determine a first reconstructed image block and a first filtered image block includes: Reconstructing an image block based on the initial residual to obtain a first reconstructed image block; Filter the first reconstructed image block to obtain the first filtered image block.

7. The method according to claim 6, wherein: The reconstructing the image block based on the initial residual to obtain a first reconstructed image block includes: Performing image block reconstruction on the initial residual to obtain a first initial reconstructed image block; performing deblocking filtering on the first initial reconstructed image block to obtain a first deblocked image block; Perform sample adaptive compensation filtering on the first deblocked image block to obtain the first reconstructed image block.

8. The method according to any one of claims 1 to 5 or 7, wherein: After performing scaling processing using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain a corrected image block corresponding to the current block, the method further includes: Performing image block reconstruction and filtering on a next block in the current frame, and scaling the first reconstructed image block and the first filtered image block of the next block using the frame-level scaling factor to obtain a corrected image block corresponding to the next block, until the current frame is processed. A first filtered image of the current frame is obtained based on the corrected image blocks corresponding to each block in the current frame.

9. The method according to any one of claims 1 to 5 or 7, wherein: After performing scaling processing using the frame-level scaling factor, the first reconstructed image block, and the first filtered image block to obtain a corrected image block corresponding to the current block, the method further includes: performing adaptive filtering on the corrected image block to obtain a second filtered image block; Continue filtering the next block in the current frame to obtain a second filtered image block corresponding to the next block, continue processing until the current frame is processed, and obtain a first filtered image of the current frame based on the second filtered image block corresponding to each block in the current frame.

10. A filtering method, applied to an encoder, comprising: Performing image reconstruction and filtering based on the initial residual of each block in the current frame, and determining a second reconstructed image block and a second filtered image block corresponding to each block; performing scaling factor calculation based on the second reconstructed image block and the second filtered image block to determine a block-level scaling factor corresponding to each block; screening the block-level scaling factors corresponding to each block corresponding to the current frame to determine a first frame-level scaling factor; the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; The difference block is a block in the current frame that is different from other blocks; Calculating a scaling factor based on a second reconstructed image and a second filtered image corresponding to the current frame to obtain a second frame-level scaling factor corresponding to the current frame; The second reconstructed image is determined by each corresponding second reconstructed image block; the second filtered image is determined by each corresponding second filtered image block; The frame level scaling factor is determined based on the first frame level scaling factor and the second frame level scaling factor.

11. The method according to claim 10, wherein: The screening of the block-level scaling factors corresponding to each block corresponding to the current frame to determine the first frame-level scaling factor includes: Perform boundary value screening on the block-level scaling factor corresponding to each block to obtain N candidate block-level scaling factors; N is a positive integer greater than 0; Averaging processing is performed based on the N candidate block-level scaling factors to obtain the first frame-level scaling factor.

12. The method according to claim 11, wherein The performing boundary value screening on the block-level scaling factor corresponding to each block to obtain N candidate block-level scaling factors includes: Determine, from the block-level scaling factors corresponding to each block, a block-level scaling factor that is equal to a preset upper limit value or a preset lower limit value as the block-level scaling factor corresponding to the difference block; Determine a first ratio of the number of block-level scaling factors corresponding to the difference block to the total number of block-level scaling factors corresponding to each block; If the first proportion does not exceed a preset proportion threshold, the block-level scaling factors corresponding to each block, excluding the block-level scaling factor corresponding to the difference block, are used as the N candidate block-level scaling factors.

13. The method according to claim 11, wherein The performing boundary value screening on the block-level scaling factor corresponding to each block to obtain N candidate block-level scaling factors includes: Determining at least one of a maximum block-level scaling factor and a minimum block-level scaling factor among the block-level scaling factors corresponding to each block; averaging the block-level scaling factors corresponding to each block except the at least one block-level scaling factor to obtain a first average value; If a difference between the at least one block-level scaling factor and the first mean value is greater than a preset difference threshold, using the at least one block-level scaling factor as a block-level scaling factor corresponding to the difference block; Among the block-level scaling factors corresponding to each block, except the block-level scaling factor corresponding to the difference block, are used as the N candidate block-level scaling factors.

14. The method according to claim 12, wherein: The averaging of the N candidate block-level scaling factors to obtain the first frame-level scaling factor includes: Determining at least one candidate block-level scaling factor among a maximum candidate block-level scaling factor and a minimum candidate block-level scaling factor among the N candidate block-level scaling factors; averaging the candidate block-level scaling factors other than the at least one candidate block-level scaling factor among the N candidate block-level scaling factors to obtain a second average value; If a difference between the at least one candidate block-level scaling factor and the second mean is greater than a preset difference threshold, determining the candidate block-level scaling factors other than the at least one candidate block-level scaling factor among the N candidate block-level scaling factors as M updated block-level scaling factors; where M is greater than 0 and less than or equal to N; The M updated block-level scaling factors are averaged to obtain the first frame-level scaling factor.

15. The method according to any one of claims 10 to 14, wherein: The determining the frame-level scaling factor based on the first frame-level scaling factor and the second frame-level scaling factor includes: Scaling the second reconstructed image and the second filtered image using the first frame-level scaling factor to obtain a first corrected image; Scaling the second reconstructed image and the second filtered image using the second frame-level scaling factor to obtain a second corrected image; Determining distortion costs between the first corrected image and the second corrected image and the current frame respectively, to obtain a first distortion cost corresponding to the first frame-level scaling factor and a second distortion cost corresponding to the second frame-level scaling factor; The frame-level scaling factor is determined from the first frame-level scaling factor and the second frame-level scaling factor by comparing the first distortion cost with the second distortion cost.

16. The method according to claim 15, wherein The determining the frame-level scaling factor from the first frame-level scaling factor and the second frame-level scaling factor by comparing the first distortion cost with the second distortion cost includes: If the first distortion cost is less than or equal to the second distortion cost, determining the first frame-level scaling factor as the frame-level scaling factor; If the first distortion cost is greater than the second distortion cost, the second frame-level scaling factor is determined as the frame-level scaling factor.

17. The method according to any one of claims 10 to 14 or 16, wherein: The calculating the scaling factor based on the second reconstructed image block and the second filtered image block to determine the block-level scaling factor corresponding to each block includes: Determine, according to the original pixel corresponding to each pixel position in each block, an original pixel residual between the original pixel and a second reconstructed pixel of the second reconstructed image block; Determining, according to the second filtered pixel corresponding to each pixel position in the second filtered image block, a filtered pixel residual between the pixel and the second reconstructed pixel; Determining a block-level original pixel residual and a block-level filtered pixel residual corresponding to each block based on the original pixel residual and the filtered pixel residual; A weighted process is performed based on the block-level original pixel residual and the block-level filtered pixel residual to obtain a block-level scaling factor corresponding to each block.

18. The method according to claim 17, wherein The performing weighted processing based on the block-level original pixel residual and the block-level filtered pixel residual to obtain the block-level scaling factor corresponding to each block includes: weighting and fusing the filtered pixel residuals corresponding to each pixel position in each block to determine a first block-level weighted residual corresponding to each block; weighting and fusing the original pixel residual and the filtered pixel residual corresponding to each pixel position in each block to determine a second block-level weighted residual corresponding to each block; In combination with a preset upper limit value and a preset lower limit value, least squares processing is performed on the first block-level weighted residual, the second block-level weighted residual, the block-level original pixel residual and the block-level filtered pixel residual to obtain a block-level scaling factor corresponding to each block.

19. The method according to any one of claims 10 to 14 or 16, wherein: The calculating the scaling factor based on the second reconstructed image and the second filtered image corresponding to the current frame to obtain the second frame-level scaling factor corresponding to the current frame includes: Traversing the original pixel residual corresponding to each pixel position in the second reconstructed image to determine a frame-level original pixel residual; the original pixel residual corresponding to each pixel position in the second reconstructed image is obtained based on the original pixel residual corresponding to each pixel position in the second reconstructed image block corresponding to each block; traversing the filtered pixel residual corresponding to each pixel position in the second filtered image to determine a frame-level filtered pixel residual; the filtered pixel residual corresponding to each pixel position in the second filtered image is obtained based on the filtered pixel residual corresponding to each pixel position in the second filtered image block; weighting and fusing the filtered pixel residuals corresponding to each pixel position in the second filtered image to obtain a first frame-level weighted residual; weighting and fusing the filtered pixel residual corresponding to each pixel position in the second filtered image and the original pixel residual corresponding to each pixel position in the second reconstructed image to obtain a second frame-level weighted residual; In combination with a preset upper limit value and a preset lower limit value, least squares processing is performed on the first frame-level weighted residual, the second frame-level weighted residual, the frame-level original pixel residual and the frame-level filtered pixel residual to obtain the second frame-level scaling factor.

20. The method according to claim 15, wherein The first frame-level scaling factor includes a first frame-level scaling factor corresponding to each color component; The scaling process of the second reconstructed image and the second filtered image using the first frame-level scaling factor to obtain a first corrected image includes: For each pixel position in the second reconstructed image and the second filtered image, scaling and superimposing the first reconstructed pixel of each second reconstructed pixel in the second reconstructed image on the respective color component and the second filtered pixel of each second filtered pixel in the second filtered image on the respective color component according to the first frame-level scaling factor corresponding to the respective color component, to obtain a first corrected pixel corresponding to each pixel position and containing the respective color component; The first corrected pixels corresponding to each pixel position are traversed to obtain a first corrected image.

21. The method according to any one of claims 10-14, 16, 18, and 20, wherein: The performing image reconstruction and filtering based on the initial residual of each block in the current frame, and determining a second reconstructed image block and a second filtered image block corresponding to each block, includes: Performing image reconstruction and filtering on an initial residual of a current block in the current frame to obtain a second reconstructed image block and a second filtered image block corresponding to the current block, and continuing to perform image block reconstruction and filtering on a next block in the current frame until processing of the current frame is completed, thereby obtaining a second reconstructed image block and a second filtered image block corresponding to each block.

22. The method according to claim 21, wherein The performing image reconstruction and filtering on the initial residual of the current block in the current frame to obtain a second reconstructed image block and a second filtered image block corresponding to the current block includes: Performing image reconstruction on the initial residual of the current block to obtain a second reconstructed image block; The second reconstructed image block is filtered to obtain the second filtered image block.

23. The method according to any one of claims 10-14, 16, 18, 20-22, wherein: After performing image reconstruction and filtering based on the initial residual of each block in the current frame and determining a second reconstructed image block and a second filtered image block corresponding to each block, the method further includes: Based on the second filtered image blocks corresponding to each block, a second filtered image corresponding to the current frame is obtained.

24. The method according to any one of claims 10-14, 16, 18, 20-22, wherein: After performing image reconstruction and filtering based on the initial residual of each block in the current frame and determining a second reconstructed image block and a second filtered image block corresponding to each block, the method further includes: Adaptively filter the second filtered image blocks corresponding to each block to obtain an adaptive filtered image block corresponding to each block, and traverse the adaptive filtered image blocks corresponding to each block to obtain a second filtered image corresponding to the current frame.

25. A decoder comprising: a parsing portion configured to parse the bitstream and determine a frame-level scaling factor and an initial residual of a current block; wherein the frame-level scaling factor is determined by a first frame-level scaling factor and a second frame-level scaling factor corresponding to the current frame, the first frame-level scaling factor being obtained by screening block-level scaling factors in the current frame, wherein the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block; the difference block being a block that is different from other blocks in the current frame; A first reconstruction and filtering part is configured to perform image block reconstruction and filtering based on the initial residual to determine a first reconstructed image block and a first filtered image block; The first determining part is configured to perform scaling processing using the frame-level scaling factor, the first reconstructed image block and the first filtered image block to obtain a corrected image block corresponding to the current block.

26. An encoder comprising: A second reconstruction and filtering part is configured to perform image reconstruction and filtering based on the initial residual of each block in the current frame, and determine a second reconstructed image block and a second filtered image block corresponding to each block; a second determining unit configured to calculate a scaling factor based on the second reconstructed image block and the second filtered image block to determine a block-level scaling factor corresponding to each block; and to filter the block-level scaling factors corresponding to each block corresponding to the current frame to determine a first frame-level scaling factor, wherein the first frame-level scaling factor does not include a block-level scaling factor corresponding to a difference block. The difference block is a block in the current frame that is different from other blocks; a scaling factor is calculated based on a second reconstructed image and a second filtered image corresponding to the current frame to obtain a second frame-level scaling factor corresponding to the current frame; The second reconstructed image is determined by each corresponding second reconstructed image block; the second filtered image is determined by each corresponding second filtered image block; and the frame-level scaling factor is determined based on the first frame-level scaling factor and the second frame-level scaling factor.

27. A decoder comprising: a first memory and a first processor; The first memory stores a computer program that can be run on the first processor, and the first processor implements the method according to any one of claims 1 to 9 when executing the program.

28. An encoder comprising: a second memory and a second processor; The second memory stores a computer program that can be run on the second processor, and the second processor implements the method according to any one of claims 10 to 24 when executing the program.

29. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a first processor, the computer program implements the method according to any one of claims 1 to 9; or, when the computer program is executed by a second processor, the computer program implements the method according to any one of claims 10 to 24.

Images