Decoding method, encoding method, decoder, and encoder

Abstract

The present application provides a decoding method, an encoding method, a decoder, and an encoder. The decoding method comprises: filtering a reconstructed block of a current block to obtain a first filtered block; determining a first residual block on the basis of the first filtered block and the reconstructed block; adjusting a direct-current component of the first residual block to obtain a second residual block; and determining a final reconstructed block on the basis of the second residual block and the reconstructed block. For the decoding method provided by the present application, the first residual block is determined on the basis of the first filtered block obtained by filtering the reconstructed block of the current block and on the basis of the reconstructed block of the current block, so that the second residual block is obtained by adjusting the direct-current component of the first residual block, and the final reconstructed block of the current block is determined on the basis of the second residual block and the reconstructed block, namely, determining the final reconstructed block on the basis of the corrected first filtered block, thus improving the decoding performance of the decoder.

Description

Decoding method, encoding method, decoder and encoder Technical Field

[0001] The present application relates to the field of coding and decoding technology, and more specifically, to a decoding method, an encoding method, a decoder, and an encoder. Background Art

[0002] Digital video compression technology mainly compresses huge digital image video data for easy transmission and storage.

[0003] With the surge in Internet videos and people's increasing demand for video clarity, although existing digital video compression standards can save a lot of video data, there is still a need to pursue better digital video compression technology to reduce the bandwidth and traffic pressure of digital video transmission.

[0004] In order to improve decoding performance, after the decoder determines the reconstructed image based on the bitstream, it needs to perform loop filtering on the reconstructed image to obtain the decoded image. The decoded image can be used as a reference frame for predicting subsequent images. Loop filtering mainly processes the pixels after inverse transformation and inverse quantization to compensate for the distortion information, thereby providing a better reference for subsequent images. Traditional loop filtering units used for loop filtering mainly include: DeBlocking Filter (DBF), Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF) and other tools.

[0005] However, with the development of technology, there is still a need to pursue better filtering technology to improve decoding performance.

[0006] Summary of the Invention

[0007] The present application provides a decoding method, an encoding method, a decoder and an encoder, which can improve decoding performance.

[0008] In a first aspect, the present application provides a decoding method, comprising:

[0009] filtering a reconstructed block of the current block to obtain a first filtered block;

[0010] Determining a first residual block based on the first filter block and the reconstructed block;

[0011] Adjusting a direct current component of the first residual block to obtain a second residual block;

[0012] A final reconstructed block of the current block is determined based on the second residual block and the reconstructed block.

[0013] In a second aspect, the present application provides an encoding method, comprising:

[0014] filtering a reconstructed block of the current block to obtain a first filtered block;

[0015] Determining a first residual block based on the first filter block and the reconstructed block;

[0016] Adjusting a direct current component of the first residual block to obtain a second residual block;

[0017] determining a second filter block based on the second residual block and the reconstructed block;

[0018] A filtered image with the minimum rate-distortion cost among a first filtered image to which the first filtering block belongs and a second filtered image to which the second filtering block belongs is determined as a reconstructed image of a current image to which the current block belongs.

[0019] In a third aspect, the present application provides a decoder, comprising:

[0020] a filtering unit, configured to filter a reconstructed block of a current block to obtain a first filtered block;

[0021] A first determining unit, configured to determine a first residual block based on the first filtering block and the reconstructed block;

[0022] an adjusting unit, configured to adjust a DC component of the first residual block to obtain a second residual block;

[0023] The second determining unit is configured to determine a final reconstructed block of the current block based on the second residual block and the reconstructed block.

[0024] In a fourth aspect, the present application provides an encoder, comprising:

[0025] a filtering unit, configured to filter a reconstructed block of a current block to obtain a first filtered block;

[0026] A first determining unit, configured to determine a first residual block based on the first filtering block and the reconstructed block;

[0027] an adjusting unit, configured to adjust a DC component of the first residual block to obtain a second residual block;

[0028] A second determining unit, configured to determine a second filter block based on the second residual block and the reconstructed block;

[0029] The third determining unit is configured to determine the filtered image with the minimum rate-distortion cost among the first filtered image to which the first filtering block belongs and the second filtered image to which the second filtering block belongs as the reconstructed image of the current image to which the current block belongs.

[0030] In a fifth aspect, the present application provides a decoder, comprising:

[0031] a processor adapted to implement computer instructions; and,

[0032] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the decoding method in the first aspect or its various implementations mentioned above.

[0033] In one implementation, there are one or more processors and one or more memories.

[0034] In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be provided separately from the processor.

[0035] In a sixth aspect, the present application provides an encoder, comprising:

[0036] a processor adapted to implement computer instructions; and,

[0037] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the encoding method in the second aspect or its various implementations mentioned above.

[0038] In one implementation, there are one or more processors and one or more memories.

[0039] In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be provided separately from the processor.

[0040] In the seventh aspect, the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are read and executed by a processor of a computer device, the computer device executes the decoding method involved in the first aspect mentioned above or the encoding method involved in the second aspect mentioned above.

[0041] In an eighth aspect, the present application provides a computer program product or computer program, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the decoding method of the first aspect or the encoding method of the second aspect.

[0042] In a ninth aspect, the present application provides a code stream, which is the code stream involved in the method described in the first aspect above or the code stream generated by the method described in the second aspect above.

[0043] Based on the above technical solution, for the decoding method provided in the present application, since the first residual block is determined based on the first filter block obtained by filtering the reconstructed block of the current block and the reconstructed block of the current block, the DC component of the first residual block is adjusted to obtain the second residual block, and the final reconstructed block of the current block is determined based on the second residual block and the reconstructed block, which is equivalent to determining the final reconstructed block based on the corrected first filter block, which can improve the decoding performance of the decoder. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] FIG1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application.

[0045] FIG2 is a schematic block diagram of a video encoder according to an embodiment of the present application.

[0046] FIG3 is a schematic structural diagram of the relationship between the coding tree unit and the coding unit provided in the present application.

[0047] FIG4 is a schematic block diagram of a video decoder according to an embodiment of the present application.

[0048] FIG5 is another example of a video encoder provided by the present application.

[0049] FIG6 is an example of the basic structure of the residual network provided in this application.

[0050] FIG7 is an example of the basic structure of the NNLF provided in this application.

[0051] FIG8 is an example of the basic structure of the LC NNLF provided in this application.

[0052] FIG9 is another example of a video encoder provided by the present application.

[0053] FIG10 is a schematic flowchart of the decoding method provided in this application.

[0054] FIG11 is an example of the principle of determining the first residual block provided by the present application.

[0055] FIG12 is an example of a residual image to be corrected provided by the present application.

[0056] FIG13 is an example of the first residual block provided in this application.

[0057] FIG14 is an example of the second residual block provided in this application.

[0058] FIG15 is an example of image block size adjustment by residual value provided in the present application.

[0059] FIG16 is an example of a sliding window provided in this application.

[0060] FIG17 is a schematic flowchart of the encoding method provided in this application.

[0061] FIG18 is a schematic block diagram of a decoder provided in the present application.

[0062] FIG19 is a schematic block diagram of the encoder provided in this application.

[0063] FIG20 is a schematic block diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0064] The solution provided by this application can be applied to the field of digital compression technology.

[0065] Among them, digital video compression technology mainly compresses huge digital image video data for easy transmission and storage.

[0066] The solution provided in this application can be applied to the field of digital video coding technology.

[0067] Among them, the field of digital video coding technology includes but is not limited to at least one of the following: image coding and decoding field, video coding and decoding field, hardware video coding and decoding field, dedicated circuit video coding and decoding field, and real-time video coding and decoding field. In addition, the solution provided in this application can be combined with the following standards: Audio Video Coding Standard (AVS), second-generation AVS standard (AVS2) or third-generation AVS standard (AVS3). For example, including but not limited to: H.264 / Audio Video Coding (AVC) standard, H.265 / High Efficiency Video Coding (HEVC) standard and H.266 / Versatile Video Coding (VVC) standard. In addition, the solution provided in this application can be used for lossy compression of images, and can also be used for lossless compression of images. Among them, the lossless compression can be visually lossless compression or mathematically lossless compression.

[0068] For ease of understanding, the video encoding and decoding system involved in the embodiment of the present application is first introduced with reference to FIG1 .

[0069] FIG1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application.

[0070] As shown in FIG. 1 , the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120 .

[0071] The encoding device 110 is used to encode (which can be understood as compressing) the video data to generate a code stream, and transmit the code stream to the decoding device 120. The decoding device 120 decodes the code stream generated by the encoding device 110 to obtain decoded video data.

[0072] The encoding device 110 can be understood as a device with a video encoding function, and the decoding device 120 can be understood as a device with a video decoding function, that is, the embodiments of the present application include a wider range of devices for the encoding device 110 and the decoding device 120, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, car computers, etc.

[0073] The encoding device 110 may transmit the encoded video data (eg, a code stream) to the decoding device 120 via the channel 130 .

[0074] Channel 130 may include one or more media and / or devices capable of transmitting encoded video data from encoding device 110 to decoding device 120 .

[0075] Channel 130 may include one or more communication media that enable encoding device 110 to transmit encoded video data directly to decoding device 120 in real time. Encoding device 110 may modulate the encoded video data according to a communication standard and transmit the modulated video data to decoding device 120. The communication media may include wireless communication media, such as radio frequency spectrum. The communication media may also include wired communication media, such as one or more physical transmission lines.

[0076] Channel 130 may include a storage medium that can store the video data encoded by encoding device 110. The storage medium includes various locally accessible data storage media, such as optical disks, DVDs, flash memories, etc. In this example, decoding device 120 may obtain the encoded video data from the storage medium.

[0077] Channel 130 may include a storage server that can store the video data encoded by encoding device 110. In this example, decoding device 120 can download the stored encoded video data from the storage server. Alternatively, the storage server can store the encoded video data and transmit the encoded video data to decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0078] The encoding device 110 includes a video encoder 112 and an output interface 113 .

[0079] The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter. The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may also be stored on a storage medium or storage server for subsequent reading by the decoding device 120.

[0080] In addition to the video encoder 112 and the input interface 113 , the encoding device 110 may also include a video source 111 .

[0081] Video source 111 may include at least one of a video capture device (e.g., a video camera), a video archive, a video input interface, and a computer graphics system. The video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data. Video encoder 112 encodes the video data from video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains encoding information for the picture or picture sequence in the form of a bitstream. The encoding information may include the encoded picture data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. An SPS may contain parameters applicable to one or more sequences. A PPS may contain parameters applicable to one or more pictures. A syntax structure is a set of zero or more syntax elements arranged in a specified order within the bitstream.

[0082] The decoding apparatus 120 includes an input interface 121 and a video decoder 122. The input interface 121 may include a receiver and / or a modem.

[0083] The decoding device 120 may include a display device 123 in addition to the input interface 121 and the video decoder 122 .

[0084] The input interface 121 can receive the encoded video data via the channel 130. The video decoder 122 is configured to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123. The display device 123 displays the decoded video data. The display device 123 can be integrated with the decoding device 120 or external to the decoding device 120. The display device 123 can include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0085] It should be understood that Figure 1 is only an example of the present application and should not be understood as a display of the present application. That is to say, the technical solution of the embodiment of the present application is not limited to the system framework shown in Figure 1. For example, the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.

[0086] The following is an introduction to the video encoding framework involved in the embodiments of the present application.

[0087] FIG2 is a schematic block diagram of a video encoder 200 according to an embodiment of the present application.

[0088] It should be understood that the video encoder 200 can be applied to image data in luminance and chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) used to describe color and saturation. For example, in terms of color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr), and 4:4:4 represents full pixel display (YYYYCbCrCbCrCbCrCbCr). Of course, it can also be applied to image data in red-green-blue (RGB) format, but this application does not specifically limit this.

[0089] After the video encoder 200 reads the video stream, it can divide each frame of the video stream into several coding tree units (CTUs). In some examples, CTUs can be called "tree blocks", "largest coding units" (LCU) or "coding tree blocks" (CTB). Each CTU can be associated with a pixel block of equal size within the image. Each pixel can correspond to one luminance (luminance or luma) sample and two chrominance (chroma) samples. Therefore, each CTU can be associated with one luminance sample block and two chrominance sample blocks. The size of a CTU can be, for example, 128×128, 64×64, 32×32, etc. Figure 3 is a schematic structural diagram of the relationship between the coding tree unit and the coding unit provided in this application. As shown in Figure 3, a CTU can be further divided into several coding units (CUs) for encoding, and the CU can be a rectangular block or a square block. A CU can be further divided into prediction units (PUs) and transform units (TUs), allowing for separation of coding, prediction, and transform, and greater flexibility in processing. In one example, a CTU is divided into CUs using a tree (e.g., a quadtree), and a CU is divided into TUs and PUs using a tree (e.g., a quadtree).

[0090] The video encoder and video decoder can support various PU sizes.

[0091] Assuming that the size of a particular CU is 2N×2N, the video encoder and video decoder may support PU sizes of 2N×2N or N×N for intra prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter prediction. The video encoder and video decoder may also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0092] As shown in Figure 2, the video encoder 200 may include: a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer, or different functional components. In this application, the current block may be referred to as the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block may also be referred to as a predicted image block or an image prediction block, and the reconstructed image block may also be referred to as a reconstructed block or an image reconstructed image block.

[0093] Prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame prediction unit 212. Because adjacent pixels in a video image have strong correlations, video encoding and decoding techniques use intra prediction to eliminate spatial redundancy between adjacent pixels. Because adjacent images in a video have strong similarities, inter prediction eliminates temporal redundancy between adjacent images, thereby improving coding efficiency.

[0094] The inter-prediction unit 211 can be used for inter-prediction, which includes motion estimation and motion compensation. It can refer to image information from different frames. Inter-prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block to eliminate temporal redundancy. The reference frame can be a P-frame and / or a B-frame. P-frames refer to forward-predicted frames, and B-frames refer to bidirectionally predicted frames. After using motion information to find a reference block, inter-prediction generates a prediction block based on the reference block. Motion information includes the frame list to which the reference frame belongs, the frame index, and the motion vector. The motion vector can be integer-pixel or fractional-pixel. If the motion vector is fractional-pixel, interpolation filtering is required to generate the required fractional-pixel block in the reference frame. The reference block is the integer-pixel or fractional-pixel block found based on the motion vector. Some technologies directly use the reference block as the prediction block, while others further process the reference block to generate a prediction block. Reprocessing a reference block to generate a prediction block can also be understood as using the reference block as the prediction block and then processing the prediction block to generate a new prediction block.

[0095] The intra prediction unit 212 only refers to the information of the same frame image to predict the pixel information in the current code image block to eliminate spatial redundancy. The reference frame used for intra prediction can be an I frame.

[0096] Intra prediction uses multiple prediction modes, including angular and non-angular prediction modes, to predict the image block to be coded. Based on the rate-distortion information calculated from the prediction block and the image block to be coded, the optimal prediction mode for the image block to be coded is selected and written into the bitstream for transmission to the decoder. The decoder parses the prediction mode, predicts the predicted block of the target decoding block, and superimposes it with the time-domain residual block obtained from the bitstream to obtain the reconstructed block.

[0097] Taking the H series of international digital video coding standards as an example, the H.264 / AVC standard has 8 angle prediction modes and 1 non-angle prediction mode, and H.265 / HEVC is expanded to 33 angle prediction modes and 2 non-angle prediction modes. The intra prediction modes used by HEVC are planar mode, direct current (DC) and 33 angle modes, a total of 35 prediction modes. The intra modes used by VVC are planar, DC and 65 angle modes, a total of 67 prediction modes, which include traditional prediction modes and non-traditional prediction modes. Non-traditional prediction modes may include matrix weighted intra-frame prediction (MIP) mode. Traditional prediction modes include: planar mode with mode number 0, DC mode with mode number 1, and angle prediction modes with mode numbers 2 to 66. It should be noted that with the increase of angle modes, the prediction results of intra prediction will be more accurate and more in line with the needs of the development of high-definition and ultra-high-definition digital video. The above intra prediction mode is only an example of this application and should not limit this application.

[0098] The residual unit 220 may generate a residual block for the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, the residual unit 220 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel blocks of the CU and the corresponding sample in the prediction blocks of the PUs of the CU.

[0099] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0100] The inverse transform / quantization unit 240 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.

[0101] Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 200 can reconstruct the pixel blocks of the CU.

[0102] The loop filter unit 260 processes the inverse-transformed and inverse-quantized pixels to compensate for distortion and provide a better reference for subsequent pixel encoding. For example, it can perform deblocking filtering to reduce the blocking artifacts of pixel blocks associated with the CU. In some embodiments, the loop filter unit 260 includes a deblocking filter (DBF) unit and a sample adaptive offset / adaptive loop filter (SAO / ALF) unit. The DBF unit is used to remove blocking artifacts, while the SAO / ALF unit is used to remove ringing artifacts.

[0103] The decoded image buffer 270 may store reconstructed pixel blocks.

[0104] The inter prediction unit 211 can use the reference image containing the reconstructed pixel blocks in the decoded image buffer 270 to perform inter prediction on PUs in other images. In addition, the intra prediction unit 212 can use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra prediction on other PUs in the same image as the CU.

[0105] The entropy coding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy coding unit 280 may perform one or more entropy coding operations on the quantized transform coefficients to generate entropy-coded data.

[0106] FIG4 is a schematic block diagram of a video decoder according to an embodiment of the present application.

[0107] 4 , the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded picture buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional components.

[0108] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the inverse quantization / transform unit 330, the reconstruction unit 340, and the loop filter unit 350 may decode the video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.

[0109] The prediction unit 320 includes an intra prediction unit 322 and an inter prediction unit 321 .

[0110] The intra prediction unit 322 may perform intra prediction to generate a prediction block for the PU. The intra prediction unit 322 may use an intra prediction mode to generate a prediction block for the PU based on pixel blocks of spatially neighboring PUs. The intra prediction unit 322 may also determine the intra prediction mode for the PU based on one or more syntax elements parsed from the codestream.

[0111] The inter prediction unit 321 may construct a first reference picture list (List 0) and a second reference picture list (List 1) based on syntax elements parsed from the codestream. Furthermore, if a PU is encoded using inter prediction, the entropy decoding unit 310 may parse the motion information of the PU. The inter prediction unit 321 may determine one or more reference blocks for the PU based on the motion information of the PU. The inter prediction unit 321 may generate a prediction block for the PU based on the one or more reference blocks of the PU.

[0112] The inverse quantization / transform unit 330 may inversely quantize (i.e., dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization. After inverse quantizing the transform coefficients, the inverse quantization / transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients to generate a residual block associated with the TU.

[0113] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0114] The loop filtering unit 350 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.

[0115] The video decoder 300 may store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0116] Combining Figures 2 and 4, the basic process of video encoding and decoding is as follows:

[0117] At the encoding end, a frame of an image is divided into image blocks. For the current block, the prediction unit 210 uses intra prediction or inter prediction to predict the prediction block of the current block (i.e., the block to be encoded). The residual unit 220 can calculate a residual block based on the prediction block and the original block of the current block (i.e., the block to be encoded), that is, the difference between the prediction block and the original block. This residual block can also be called residual information. This residual block is transformed and quantized by the transform / quantization unit 230, which can remove information that is insensitive to the human eye to eliminate visual redundancy. Optionally, the residual block before transformation and quantization by the transform / quantization unit 230 can be called a time domain residual block, and the time domain residual block after transformation and quantization by the transform / quantization unit 230 can be called a frequency residual block or a frequency domain residual block. The entropy coding unit 280 receives the quantized change coefficients output by the change quantization unit 230, performs entropy coding on the quantized change coefficients, and outputs a bitstream. For example, the entropy coding unit 280 can eliminate character redundancy based on the target context model and the probability information of the binary bitstream.

[0118] At the decoding end, the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block (i.e., the block to be decoded). The prediction unit 320 uses intra prediction or inter prediction based on the prediction information to predict the prediction block of the current block (i.e., the block to be decoded). The inverse quantization / transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. It is worth noting that the encoding end also needs to use operations similar to those of the decoder to obtain a decoded image. The decoded image can also be called a reconstructed image, and the reconstructed image can be a subsequent frame as a reference frame for inter prediction.

[0119] In addition, the block division information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are the same.

[0120] It should be noted that due to the need for parallel processing, images can be divided into slices, etc., and slices within the same image can be processed in parallel, that is, there is no data dependency between them. The term "frame" can be understood as an image or a slice, etc. In addition, the above is the basic process of a video codec under a block-based codec framework. As technology develops, some modules or steps of this framework or process may be optimized. That is, this application is not limited to this framework and process.

[0121] FIG5 is another example of a video encoder provided by the present application.

[0122] As shown in Figure 5, the encoder reads unequal pixels from video signals of different color formats (e.g., video sequences), including luminance and chrominance components. This means the encoder reads a black-and-white or color image. It then divides the image into blocks, which are then passed to the encoder for encoding. The encoder typically uses a hybrid frame coding mode and typically includes prediction units (including intra-frame prediction units and inter-frame prediction units), a transform and quantization unit, an inverse transform and inverse quantization unit, a loop filter unit, and an entropy coding unit. Optionally, it may also include a decoded image buffer unit. The intra prediction unit is used for intra prediction. This prediction only references information from the same frame and predicts pixel information within the current block, eliminating spatial redundancy. The inter prediction unit is used for inter prediction. Inter prediction can reference image information from different frames and uses motion estimation (ME) to search for the motion vector (MV) that best matches the current block, eliminating temporal redundancy. The transform and quantization unit performs both transformation and quantization. Transformation converts the predicted image blocks to the frequency domain, redistributing energy. Combined with quantization, this removes information that is insensitive to the human eye, eliminating visual redundancy. The entropy coding unit performs entropy coding, which eliminates character redundancy based on the current context model and the probability information of the binary bitstream. The loop filter unit performs loop filtering, which primarily processes inversely transformed and dequantized pixels to compensate for distortion and provide a better reference for subsequent pixel encoding.

[0123] Typically, a loop filter (LF) consists of tools such as a deblocking filter (DBF), sample adaptive offset (SAO), and an adaptive loop filter (ALF). In recent years, with the advancement of deep learning technology, research on neural network-based loop filters (NNLF) has also begun.

[0124] The following describes the contents related to loop filtering.

[0125] (1) Baseline NNLF.

[0126] After years of continuous research and optimization, a type of loop filter (NNLF) has been adopted as the baseline filtering tool for the neural network based common software (NCS) of neural network based video coding (NNVC), and is called a low complexity neural network based loop filter (LC NNLF).

[0127] (2) Residual Network.

[0128] In the field of deep learning, the residual network (ResNet) proposed the concept of residual learning.

[0129] FIG6 is an example of the basic structure of the residual network provided in this application.

[0130] As shown in Figure 6, the residual network consists of a neural network (NN) with a skip connection structure between the NN input and the NN output. This structure allows the NN to focus on learning the residual information of the image, improving its learning ability and prediction performance. Specifically, the NN outputs the predicted residual information, which is simply superimposed on the NN input to obtain the residual network output.

[0131] Due to the excellent performance of residual networks, the current baseline LC NNLF scheme borrows the concept of residual learning.

[0132] FIG7 is an example of the basic structure of the NNLF provided in this application.

[0133] As shown in Figure 7, the basic structure of NNLF includes a jump connection branch from the input reconstruction block to the output filter block. The output filter block can be simply expressed by the following formula:

[0134] cnn=rec+res.

[0135] Among them, cnn represents the output filter block, rec represents the input reconstruction block, and res represents the residual block output by the neural network.

[0136] For NNLF, it is essentially:

[0137] First, the residual block is predicted by the reconstructed block input by the NN, and finally the residual block is superimposed on the input reconstructed block through a simple addition operation, and the output is the filtered block, which makes its quality closer to the original block.

[0138] Therefore, NNLF has the function of predicting residual blocks.

[0139] FIG8 is an example of the basic structure of the LC NNLF provided in this application.

[0140] As shown in Figure 8, the LC NNLF can be a residual network, that is, it includes a NN and a jump connection structure. Assuming that the reconstructed block can be a block of size N1×N1, the luminance of the reconstructed block, the chrominance information of the reconstructed block, and various auxiliary information are used as the input of the NN to obtain the output of the NN, that is, the residual block. This auxiliary information includes but is not limited to: deblocking (DeBlocking) filter boundary strength information, quantization parameter (QP) information, etc. After obtaining the residual block, the residual block is superimposed on the reconstructed block using a jump connection structure to obtain a filter block. For example, the filter block can be a block of size N2×N2.

[0141] LC NNLF achieves a good balance between coding performance and complexity, and is therefore adopted as the current baseline filtering tool.

[0142] In video coding, when using NNLF to filter the reconstructed blocks, it is necessary to learn the prediction ability based on a large number of blocks and videos after millions of network trainings. However, since NNLF is largely regarded as a black box with a multi-layer network structure, the residual block lacks reasonable interpretability to a certain extent, which will cause the residual value in the predicted residual block to be smaller or larger than the true residual value, making it uncertain. Therefore, the residual value in the residual block needs to be corrected. However, if the residual value in the predicted residual block is increased or decreased alone, there will be certain limitations. In other words, for the residual block predicted by the network, this field urgently needs a solution that can reasonably correct it.

[0143] In view of this, the present application provides a decoding method, an encoding method, a decoder and an encoder, which can optimize the filtering performance of the neural network loop filter by correcting the residual block output by the neural network loop filter, thereby improving the decoding performance of the decoder.

[0144] FIG9 is another example of a video encoder provided by the present application.

[0145] As shown in FIG9 , the video encoder may include a loop filter unit, which may include a NNLF and a residual offset adjustment (ROA) unit. The use of the ROA unit may be independent of the switching of DB, SAO, and ALF. As an example, the ROA unit may be positioned after the NNLF and before the ALF. It is worth noting that this application does not specifically limit the specific position of the ROA unit and / or the order in which the filters are used.

[0146] The NNLF is any neural network-based loop filter, for example, the NNLF shown in FIG. 7 or FIG. 8 .

[0147] The ROA unit is used to adjust or correct the residual value in the residual block obtained by using the output and input of the NNLF. At the encoding end, by comparing the rate-distortion cost of the filter block output by the NNLF relative to the original block, and the rate-distortion cost of the filter block after the residual adjustment relative to the original block, it is determined whether the residual value in the residual block obtained by using the output and input of the NNLF should be adjusted or corrected. In other words, the filter block output by the NNLF is determined as the final reconstructed block, or the filter block output by the NNLF after the residual value adjustment is determined as the final reconstructed block. The selected filter block will be encoded into the bitstream for the decoder to read. At the decoder end, after parsing the actual filter block used, the reconstructed block is filtered.

[0148] It should be understood that the units other than the loop filtering unit in FIG9 may refer to the relevant illustrations and descriptions of FIG2 or FIG5 , and will not be described again here to avoid repetition.

[0149] The decoding method provided by this application is introduced below in conjunction with FIG10 .

[0150] FIG10 is a schematic flow chart of a decoding method 400 provided herein. It should be understood that the decoding method 400 may be performed by a decoder. For example, the decoding method 400 may be performed by the video decoder 122 shown in FIG1 or the video decoder 300 shown in FIG4 . For ease of description, the following description uses a decoder as an example.

[0151] As shown in FIG10 , the decoding method 400 may include:

[0152] S410 , filtering a reconstructed block of the current block to obtain a first filtered block.

[0153] Exemplarily, the decoder performs loop filtering on the reconstructed block of the current block to obtain the first filtered block.

[0154] Exemplarily, the current block may be a CTU, a block larger than a CTU, or a block smaller than a CTU.

[0155] Exemplarily, the current block may be an image block, a sub-image, a rectangular area, or a slice.

[0156] S420: Determine a first residual block based on the first filter block and the reconstructed block.

[0157] Exemplarily, the decoder subtracts the reconstructed block from the first filter block to obtain the first residual block.

[0158] S430: Adjust the direct current component of the first residual block to obtain a second residual block.

[0159] Exemplarily, the decoder adjusts the direct current component of the first residual block to 0 or approximately 0 to obtain the second residual block.

[0160] Exemplarily, the DC component of the first residual block includes an average value of pixel residual values ​​in the first residual block.

[0161] In this embodiment, the decoder adjusts the DC component of the first residual block, avoiding individually increasing or decreasing the pixel residual values ​​in the first residual block, thereby avoiding limitations in the decoder's adjustment of the first residual block, thereby improving the universality of the decoding method provided in this application.

[0162] S440: Determine a final reconstructed block of the current block based on the second residual block and the reconstructed block.

[0163] In this embodiment, since the first residual block is determined based on the first filter block obtained by filtering the reconstructed block of the current block and the reconstructed block of the current block, the DC component of the first residual block is adjusted to obtain the second residual block, and the final reconstructed block of the current block is determined based on the second residual block and the reconstructed block, which is equivalent to determining the final reconstructed block based on the corrected first filter block, which can improve the decoding performance of the decoder.

[0164] In addition, based on the baseline filtering tool LC NNLF of NNVC, on the basis of LC NNLF, the solution provided in this application is implemented based on all frame types (including I, P, B), and its performance is tested. Under the configurations of Random Access, Low Delay B, and All Intra, the general sequence specified by the Joint Video Experts Team (JVET) is tested, and the comparison anchor is LC NNLF. The test results of some sequences are shown in Tables 1, 2, and 3. The BD-rate in the table can be used to measure the performance of the algorithm, which includes changes in Peak Signal to Noise Ratio (PSNR) and Mean Structural Similarity Index Measure (MSIM). A negative value of BD-rate indicates that the performance has improved, and the larger the absolute value, the greater the performance improvement. EncT in the table represents the transformation of coding complexity, and DecT represents the transformation of complexity. Y represents brightness (Luma), Cb (U) represents blue chromaticity, and Cr (V) represents red chromaticity.

[0165] Table 1

[0166] As shown in Table 1, under the random access configuration, by introducing the optimization method of residual adjustment, the encoding and decoding performance can be improved on the basis of filtering (basically without increasing the encoding and decoding complexity), especially the encoding and decoding performance of the chrominance component.

[0167] Table 2

[0168] As shown in Table 2, under the Low Delay B configuration, by introducing the residual adjustment optimization method, the encoding and decoding performance can be improved on the basis of filtering (basically without increasing the encoding and decoding complexity), especially the encoding and decoding performance of the chrominance component.

[0169] Table 3

[0170] As shown in Table 3, under the All Intra configuration, by introducing the optimization method of residual adjustment, the encoding and decoding performance can be improved on the basis of filtering (basically without increasing the encoding and decoding complexity), especially the encoding and decoding performance of the chrominance component.

[0171] In some embodiments, the S420 may include:

[0172] Decoding the code stream and determining the first identifier;

[0173] If the first identifier indicates to adjust a filtered image corresponding to a current image to which the current block belongs, a first residual block is determined based on the first filtered block and the reconstructed block.

[0174] Exemplarily, the first identifier is an image-level identifier.

[0175] Exemplarily, the filtered image corresponding to the current image to which the current block belongs is: a filtered image obtained by filtering (eg, loop filtering) a reconstructed image of the current image.

[0176] Exemplarily, the first identifier indicates that the filtered image corresponding to the current image to which the current block belongs is adjusted, which can be understood as or equivalently replaced by: the first identifier indicates that the filtered image corresponding to the first filtering block belongs is adjusted, or the first identifier indicates that the current image to which the current block belongs allows the use of a residual offset adjustment (ROA) unit or module.

[0177] Exemplarily, if the value of the first identifier is 0 or 1, it indicates that the filtered image corresponding to the current image to which the current block belongs is adjusted.

[0178] Exemplarily, if the first flag is activated or enabled, it indicates that the filtered image corresponding to the current image to which the current block belongs is adjusted.

[0179] Exemplarily, the decoder may determine the first identifier by decoding the image header information in the code stream. In other words, the first identifier may be carried in the image header information in the code stream.

[0180] Exemplarily, the first identifier may be an identifier for a first component of the current block. In other words, the first identifier may indicate that an adjustment is to be made to a filtered image corresponding to the current image to which the current block belongs and corresponding to the first component. The first component may be a color component. For example, the first component may be a luminance component or a chrominance component. For another example, the first component may be a luminance component, a blue chrominance component, or a red chrominance component.

[0181] Exemplarily, the first identifier may be an identifier for a first channel of the current block. In other words, the first identifier may indicate that adjustment is to be made to a filtered image corresponding to the current image to which the current block belongs and corresponding to the first channel. The first channel may be a color channel. For example, the first channel may be a luma channel or a chroma channel.

[0182] In some embodiments, the decoder decodes the code stream and determines the second identifier; if the second identifier indicates adjusting the filtered image corresponding to the image in the image sequence to which the current image belongs, the decoder decodes the code stream and determines the first identifier.

[0183] Exemplarily, the second identifier is an identifier at the image sequence level.

[0184] Exemplarily, the filtered image corresponding to the image in the image sequence to which the current image belongs is: a filtered image obtained by filtering (eg, loop filtering) a reconstructed image of the image in the image sequence to which the current image belongs.

[0185] Exemplarily, the second identifier indicates that the filtered image corresponding to the image in the image sequence to which the current image belongs is adjusted, which can be understood as or equivalently replaced by: the second identifier indicates that the filtered image in the filtered image sequence to which the first filtering block belongs is adjusted, or the first identifier indicates that the image sequence to which the current image belongs allows the use of a residual offset adjustment (ROA) unit or module.

[0186] Exemplarily, if the value of the second identifier is 0 or 1, the filtered image corresponding to the image in the image sequence to which the current image belongs is adjusted.

[0187] Exemplarily, if the second flag is activated or enabled, the filtered image corresponding to the image in the image sequence to which the current image belongs is adjusted.

[0188] Exemplarily, the decoder may determine the second identifier by decoding sequence header information in the bitstream. In other words, the second identifier may be carried in the sequence header information in the bitstream.

[0189] Exemplarily, the decoder decodes the bitstream and determines the second identifier; if the second identifier indicates that a filtered image corresponding to an image in the image sequence to which the current image belongs is adjusted, the decoder decodes the bitstream and determines the first identifier; if the first identifier indicates that a filtered image corresponding to the current image to which the current block belongs is adjusted, the decoder determines a first residual block based on the first filtered block and the reconstructed block.

[0190] The first identifier and the second identifier are described below in conjunction with Table 4 and Table 5.

[0191] Table 4

[0192] As shown in Table 4, sequence_header represents sequence header information, which includes roa_enable_flag (ie, the second identifier), and roa_enable_flag is a sequence-level identifier.

[0193] Table 5

[0194] As shown in Table 5, picture_header represents picture header information, which may include picture_roa_enable_flag[Idx] (ie, the first identifier), where picture_roa_enable_flag[Idx] is a picture-level identifier.

[0195] When decoding a bitstream, the decoder first parses the sequence header information in the bitstream and obtains roa_enable_flag. When roa_enable_flag is 1, the decoder parses picture_roa_enable_flag[N] from the picture header information, where N can be 3, representing the three components Y, Cb, and Cr; N can also be 2, representing the two channels of luma and chroma. When picture_roa_enable_flag[N] is 1, it indicates that the filtered image of component N is adjusted; when picture_roa_enable_flag[N] is 0, it indicates that the filtered image of component N is not adjusted.

[0196] In some embodiments, the S430 may include:

[0197] determining an adjustment value based on at least one pixel residual value in the first residual block;

[0198] The second residual block is obtained by subtracting the adjustment value from a pixel residual value of the at least one pixel residual value.

[0199] Exemplarily, the decoder performs an arithmetic operation on the at least one pixel residual value to obtain the adjustment value; then, the decoder subtracts the adjustment value from each pixel residual value in the at least one pixel residual value to obtain the second residual block. Of course, in other alternative embodiments, the decoder may also adjust the pixel residual values ​​in the at least one pixel residual value using other adjustment methods based on the adjustment value, and this application does not specifically limit this.

[0200] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​in the first residual block.

[0201] Exemplarily, the decoder determines the adjustment value based on all pixel residual values ​​in the first residual block, and subtracts the adjustment value from each pixel residual value in the first residual block to obtain the second residual block.

[0202] In some embodiments, the at least one pixel residual value comprises a non-zero pixel residual value in the first residual block.

[0203] Exemplarily, the non-zero pixel residual values ​​in the first residual block include pixel residual values ​​in the first residual block that are not zero.

[0204] Exemplarily, the decoder determines the adjustment value based on all non-zero pixel residual values ​​in the first residual block, and subtracts the adjustment value from each non-zero pixel residual value in the first residual block to obtain the second residual block.

[0205] In some embodiments, the at least one pixel residual value comprises a pixel residual value in each group after dividing the pixel residual values ​​in the first residual block into at least one group according to at least one predefined numerical range.

[0206] Exemplarily, the decoder may divide the pixel residual values ​​in the first residual block into at least one group according to at least one predefined value range, and then determine the pixel residual values ​​in any one of the at least one group as the at least one pixel residual value. In other words, the decoder may determine the adjustment value based on the pixel residual values ​​in the any one group, and subtract the adjustment value from each pixel residual value in the any one group to obtain the second residual block.

[0207] Of course, the grouping involved in this application can also be understood or equivalently replaced by terms with similar meanings such as set and array. For example, the at least one pixel residual value includes the pixel residual value in each group after the pixel residual values ​​in the first residual block are divided into at least one group according to at least one predefined numerical range, which can also be understood or equivalently replaced by: the at least one pixel residual value includes the pixel residual value in each set after the pixel residual values ​​in the first residual block are divided into at least one set according to at least one predefined numerical range.

[0208] Exemplarily, the at least one predefined numerical range may be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in the decoder, or the at least one predefined numerical range may be agreed upon or defined by a standard protocol.

[0209] Exemplarily, the at least one numerical range may be at least one absolute value range. In this case, the decoder divides any pixel residual value in the first residual block into a group corresponding to the absolute value range to which the absolute value of the any pixel residual value belongs based on the absolute value of the any pixel residual value.

[0210] Exemplarily, the at least one numerical range may include a numerical range for grouping negative pixel residual values ​​in the first residual block, and a numerical range for grouping positive pixel residual values ​​in the first residual block. In this case, the decoder divides any pixel residual value in the first residual block into a group corresponding to the absolute value range to which the any pixel residual value belongs.

[0211] Exemplarily, the decoder may represent or determine the at least one numerical range using multiple numerical values. In other words, the at least one pixel residual value includes the pixel residual value within each group after the pixel residual values ​​in the first residual block are divided into at least one group according to at least one predefined numerical value. For example, the decoder may represent or determine N-1 numerical ranges using N numerical values. Specifically, the decoder may determine two adjacent numerical values ​​among the N numerical values ​​as one numerical range in the at least one numerical range.

[0212] Exemplarily, the at least one numerical range and the at least one group may be in a one-to-one correspondence, or the at least one numerical range and the at least one group may be in a many-to-one relationship. For example, if the at least one numerical range is at least one absolute value range, the at least one numerical range and the at least one group may be in a one-to-one correspondence. For another example, if the at least one numerical range may include a numerical range for grouping pixel residual values ​​having negative values ​​in the first residual block, and a numerical range for grouping pixel residual values ​​having positive values ​​in the first residual block, then the numerical ranges having the same absolute value of the upper limit value and the same absolute value of the lower limit value in the at least one numerical range may correspond to the same group in the at least one group.

[0213] For example, assuming that the pixel residual value in the first residual block is recorded as res, the decoder can determine the adjustment value corresponding to each group based on the following code:

[0214] Among them, {x1, x2, x3…, xN} is a positive integer used to divide the pixel residual values ​​in the first residual block into at least one absolute value range of at least one group, and {AVG_OFFSET_1, AVG_OFFSET_2, AVG_OFFSET_3,…, AVG_OFFSET_N} represents the average value of the pixel residual values ​​in the first group to the average value of the pixel residual values ​​in the Nth group.

[0215] In this embodiment, the decoder calculates the adjustment value based on the group in the first residual block, and adjusts the pixel residual value in the corresponding interval based on the adjustment value corresponding to the interval. This can avoid pixel distortion caused by excessive adjustment of some pixel residual values ​​when the pixel residual values ​​in the first residual block are quite different, thereby improving the quality of the final reconstructed image and improving the decoding performance of the decoder.

[0216] In some embodiments, the S430 may include:

[0217] A first residual image is determined based on the first residual block and at least one residual block; wherein the at least one residual block includes: a residual block determined based on a filtered block obtained after filtering a first block and a reconstructed block of the first block, wherein the first block is an image block in a current image to which the current block belongs; an adjustment value is determined based on at least one pixel residual value in a sliding window in the first residual image; and a residual image including the second residual block is obtained by subtracting the adjustment value from a pixel residual value in the at least one pixel residual value.

[0218] Exemplarily, the at least one residual block includes a residual block obtained by subtracting a reconstructed block of the first block from a filtered block obtained by filtering the first block. The at least one residual block includes the first residual block.

[0219] Exemplarily, the size of the sliding window is a non-integer multiple or an integer multiple of the size of the current block.

[0220] Exemplarily, the sliding window is a window of a predefined size. The predefined size window can be implemented by pre-storing a corresponding code, table, or other method for indicating relevant information in the decoder, or the predefined size window can be agreed upon or defined by a standard protocol.

[0221] In this embodiment, the decoder adjusts the pixel residual value based on the sliding window in the first residual image. Compared with the block-based pixel residual value adjustment scheme, it can achieve smooth adjustment of the pixel residual value between image blocks, and can avoid the problem of pixel distortion at the image block boundary when the block-based pixel residual value adjustment is performed, thereby improving the quality of the final reconstructed image and improving the decoding performance of the decoder.

[0222] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​within the sliding window.

[0223] Exemplarily, the decoder determines the adjustment value based on all pixel residual values ​​in the sliding window, and subtracts the adjustment value from each pixel residual value in the sliding window to obtain the second residual block.

[0224] In some embodiments, the at least one pixel residual value comprises a non-zero pixel residual value in the sliding window.

[0225] Exemplarily, the non-zero pixel residual values ​​in the sliding window include pixel residual values ​​in the sliding window that are not zero.

[0226] Exemplarily, the decoder determines the adjustment value based on all non-zero pixel residual values ​​in the sliding window, and subtracts the adjustment value from each non-zero pixel residual value in the sliding window to obtain the second residual block.

[0227] In some embodiments, the at least one pixel residual value comprises a pixel residual value in each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined value range.

[0228] Exemplarily, the decoder may divide the pixel residual values ​​in the sliding window into at least one group according to at least one predefined value range, and then determine the pixel residual values ​​within any one of the at least one group as the at least one pixel residual value. In other words, the decoder may determine the adjustment value based on the pixel residual values ​​within the any one group, and subtract the adjustment value from each pixel residual value within the any one group to obtain the second residual block.

[0229] Of course, the grouping involved in this application can also be understood or equivalently replaced by terms with similar meanings such as set, array, etc. For example, the at least one pixel residual value includes the pixel residual value in each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined numerical range, which can also be understood or equivalently replaced as: the at least one pixel residual value includes the pixel residual value in each set after the pixel residual values ​​in the sliding window are divided into at least one set according to at least one predefined numerical range.

[0230] Exemplarily, the at least one predefined numerical range may be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in the decoder, or the at least one predefined numerical range may be agreed upon or defined by a standard protocol.

[0231] Exemplarily, the at least one numerical range may be at least one absolute value range. In this case, the decoder divides any one pixel residual value in the sliding window into a group corresponding to the absolute value range to which the absolute value of the any one pixel residual value belongs based on the absolute value of the any one pixel residual value in the sliding window.

[0232] Exemplarily, the at least one numerical range may include a numerical range for grouping pixel residual values ​​having negative values ​​in the sliding window, and a numerical range for grouping pixel residual values ​​having positive values ​​in the sliding window. In this case, the decoder divides any pixel residual value in the sliding window into a group corresponding to the absolute value range to which the any pixel residual value belongs.

[0233] Exemplarily, the decoder may represent or determine the at least one numerical range using multiple numerical values. In other words, the at least one pixel residual value includes the pixel residual value within each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined numerical value. For example, the decoder may represent or determine N-1 numerical ranges using N numerical values. Specifically, the decoder may determine two adjacent numerical values ​​among the N numerical values ​​as one numerical range in the at least one numerical range.

[0234] Exemplarily, the at least one numerical range and the at least one group may be in a one-to-one correspondence, or the at least one numerical range and the at least one group may be in a many-to-one relationship. For example, if the at least one numerical range is at least one absolute value range, the at least one numerical range and the at least one group may be in a one-to-one correspondence. For another example, if the at least one numerical range may include a numerical range for grouping pixel residual values ​​that are negative in the sliding window, and a numerical range for grouping pixel residual values ​​that are positive in the sliding window, then the numerical ranges in the at least one numerical range that have the same absolute value of the upper limit value and the same absolute value of the lower limit value may correspond to the same group in the at least one group.

[0235] For example, assuming that the pixel residual value in the sliding window is recorded as res, the decoder can determine the adjustment value corresponding to each group based on the following code:

[0236] Among them, {x1, x2, x3…, xN} are positive integers used to divide the pixel residual values ​​in the sliding window into at least one absolute value range of at least one group, and {AVG_OFFSET_1, AVG_OFFSET_2, AVG_OFFSET_3,…, AVG_OFFSET_N} represents the average value of the pixel residual values ​​in the first group to the average value of the pixel residual values ​​in the Nth group.

[0237] In some embodiments, an average value of the at least one pixel residual value or a value obtained by rounding the average value of the at least one pixel residual value is determined as the adjustment value.

[0238] Exemplarily, if the average value of the at least one pixel residual value is an integer, the decoder determines the average value of the at least one pixel residual value as the adjustment value.

[0239] Exemplarily, if the average value of the at least one pixel residual value is not an integer, the decoder determines a value obtained by rounding the average value of the at least one pixel residual value as the adjustment value. For example, the decoder determines a value obtained by rounding up the average value of the at least one pixel residual value as the adjustment value. For another example, the decoder determines a value obtained by rounding down the average value of the at least one pixel residual value as the adjustment value.

[0240] In some embodiments, the adjustment value is determined based on an average value of the at least one pixel residual value and a preset first value range.

[0241] Exemplarily, the decoder determines whether to use the average value of the at least one pixel residual value as the adjustment value by comparing the average value with a preset first numerical range. For example, if the average value is within the first numerical range, the decoder determines the average value as the adjustment value. If the average value is outside the first numerical range, the decoder may determine the adjustment value as the upper limit or lower limit of the first numerical range.

[0242] In some embodiments, if the average value is within the first numerical range, the average value or a value obtained by rounding the average value is determined as the adjustment value. If the average value is greater than an upper limit of the first numerical range, the upper limit is determined as the adjustment value. If the average value is less than a lower limit of the first numerical range, the lower limit is determined as the adjustment value.

[0243] Of course, in other alternative embodiments, if the average value is outside the first numerical range, the decoder may determine the adjustment value using other methods. This application does not specifically limit this. For example, if the average value is outside the first numerical range, the decoder may determine the adjustment value using a predefined value.

[0244] In some embodiments, the S410 may include:

[0245] Filter the reconstructed block using any one of the following methods to obtain the first filtered block:

[0246] Deblocking filter, sample adaptive compensation filter, adaptive correction filter, neural network based loop filter.

[0247] Exemplarily, the decoder uses a neural network-based loop filter to filter the reconstructed block of the current block to obtain a first filtered block; then determines a first residual block based on the first filtered block and the reconstructed block; adjusts the DC component of the first residual block to obtain a second residual block; and then determines a final reconstructed block of the current block based on the second residual block and the reconstructed block.

[0248] The decoding method provided in this application is exemplarily described below with reference to Example 1.

[0249] Example 1:

[0250] In this embodiment, the loop filtering unit acts in the decoder, that is, the decoder first filters the reconstructed block of the current block through the neural network loop filter and outputs a first filtered block, then subtracts the reconstructed block from the first filtered block to obtain a first residual block, and corrects the first residual block to obtain a second reference block. Then, the decoder adds the second residual block and the reconstructed block to obtain a final reconstructed block. Thus, by correcting the first residual block, the filtering performance of the neural network loop filter is optimized, thereby improving the decoding performance of the decoder.

[0251] The specific process of the decoder is as follows:

[0252] a) When the decoder enters the loop filter unit, it is processed in accordance with the prescribed filter order. When entering the ROA unit, it first determines whether the ROA unit can be used in the current sequence based on the sequence-level identifier obtained from the decoded code stream (i.e., the second identifier mentioned above, which is recorded as roa_enable_flag). If roa_enable_flag is "0", the current sequence does not use the ROA unit. If roa_enable_flag is "1", the decoder uses the image-level identifier obtained from the decoded code stream (i.e., the first identifier mentioned above, which is recorded as picture_roa_enable_flag). If picture_roa_enable_flag is "0", the ROA unit is not used for the image blocks in the current image. If picture_roa_enable_flag is "1", the ROA unit processing is attempted for the image blocks in the current image, i.e., jump to b).

[0253] b) For the current block in the current image, first input the reconstructed block of the current block into the NNLF and output the first filter block; then jump to c).

[0254] c) After the decoder obtains the first filter block, it can subtract the reconstructed block from the first filter block to obtain a first residual block, and correct the first residual block to obtain a second reference block. Then, the decoder adds the second residual block and the reconstructed block to obtain a final reconstructed block.

[0255] The specific implementation method of residual adjustment is exemplified below.

[0256] FIG11 is an example of the principle of determining the first residual block provided by the present application.

[0257] As shown in FIG11 , the decoder can obtain a first residual block (res) by subtracting the input reconstructed block (rec) from the first filtered block (cnn) output by the filter.

[0258] It is worth noting that the present application does not specifically limit the filter used to filter the reconstructed block. The present application aims to correct the first residual block obtained by subtracting the filter input from the first filter block output by the filter. In theory, for any filtering tool, as long as the first residual block (res) of the first filter block (filtered) relative to the reconstructed block (rec) can be calculated, the scheme of the present application can be used to adjust the first residual block. For example, the first residual block obtained by subtracting the input from the first filter block output by DB, SAO, or ALF can be adjusted.

[0259] In one implementation, the decoder may directly correct the pixel residual values ​​in the first residual block at a block granularity.

[0260] For example, the decoder may correct the residual value block by block by traversing all residual blocks (eg, CTU level) of the residual image to be corrected, and finally obtain the corrected residual image.

[0261] For example, assuming that the residual image to be corrected includes CTU1 to CTU8 as shown in Figure 12, the decoder can correct the pixel residual value of each CTU in CTU1 to CTU8, that is, it can traverse CTU1 to CTU8, correct the pixel residual value in the CTU one by one, and finally obtain the corrected residual image.

[0262] Since the first residual block is uninterpretable, the present application corrects the first residual block by adjusting the direct current (DC) component of the first residual block to zero. For example, in order to adjust its direct current (DC) component to zero, it is necessary to adjust the average value of the pixel residual values ​​in the first residual block to zero. For example, all the pixel residual values ​​in the first residual block are summed and averaged to obtain the average value. Then, the average value is subtracted from each pixel residual value in the first residual block to obtain a corrected second residual block, whose direct current component is zero (or approximately zero). For example, for the first residual block shown in Figure 13, the average value of the pixel residual values ​​in the first residual block is calculated to be 2.375, which is rounded to 2; then, for the first residual block, the average value is subtracted from each pixel residual value in the first residual block to obtain the corrected second residual block shown in Figure 14.

[0263] Of course, in other alternative embodiments, the decoder may also perform residual adjustment using image blocks that are smaller or larger than the CTU. For example, as shown in FIG15 , the CTU is further divided into 2x2 blocks to obtain four sub-image blocks, and then the pixel residual values ​​in the sub-image blocks are corrected one by one.

[0264] In another implementation, the decoder may correct the pixel residual values ​​in the image at a sliding window granularity.

[0265] For example, the decoder can correct the pixel residual values ​​within the sliding window by moving the sliding window in the residual image to be corrected, and finally obtain a corrected residual image. The residual image to be corrected can be an image obtained by subtracting the reconstructed image of the current image from the filtered image of the current image.

[0266] For example, assuming that the residual image to be corrected is 8 CTUs as shown in Figure 16, and the sliding window is 1.5CTU×1.5CTU, the decoder can correct the pixel residual values ​​in the sliding window, that is, the pixel residual values ​​in the residual image to be corrected can be corrected by moving the sliding window according to a predefined sliding step size, and finally a corrected residual image is obtained.

[0267] Of course, regardless of whether the decoder adjusts the pixel residual value at a block granularity or at a sliding window granularity, in other alternative embodiments, the decoder may also sum and average the residual values ​​of some pixels in the first residual block or sliding window (e.g., the residual values ​​of some pixels that are not 0) to obtain the average value of the residual values ​​of some pixels. In this case, the decoder adjusts the residual values ​​of some pixels. Even after the decoder calculates the average value of all the residual values ​​of all pixels (or some of the residual values ​​of some pixels) in the first residual block or sliding window, it may determine whether to use the average value to adjust the residual value. For example, if the average value is within a predefined numerical range, the average value is used to adjust the residual value; if the average value is less than the lower limit of the predefined numerical range, the lower limit is used to adjust the residual value; if the average value is greater than the upper limit of the predefined numerical range, the upper limit is used to adjust the residual value.

[0268] FIG17 is a schematic flowchart of the encoding method 500 provided in this application.

[0269] It should be understood that the encoding method 500 may be performed by an encoder, for example, in the encoding framework 100 shown in FIG1 . For ease of description, the following description will be made using an encoder as an example.

[0270] As shown in FIG. 17 , the encoding method 500 may include:

[0271] S510, filtering a reconstructed block of the current block to obtain a first filtered block;

[0272] S520, determining a first residual block based on the first filtering block and the reconstructed block;

[0273] S530: Adjust the DC component of the first residual block to obtain a second residual block;

[0274] S540: Determine a second filter block based on the second residual block and the reconstructed block;

[0275] S550: Determine the filtered image with the minimum rate-distortion cost among the first filtered image to which the first filtering block belongs and the second filtered image to which the second filtering block belongs as a reconstructed image of the current image to which the current block belongs.

[0276] In some embodiments, the method 500 may further include:

[0277] Encode the first identifier;

[0278] Among them, when the rate-distortion cost of the first filtered image is less than or equal to the rate-distortion cost of the second filtered image, the first flag indicates that the filtered image corresponding to the current image to which the current block belongs is not adjusted; when the rate-distortion cost of the first filtered image is greater than the rate-distortion cost of the second filtered image, the first flag indicates that the filtered image corresponding to the current image is adjusted.

[0279] In some embodiments, the method 500 may further include:

[0280] Encode the second identifier;

[0281] The second flag indicates whether to adjust or not adjust the filtered image corresponding to the image in the image sequence to which the current image belongs.

[0282] In some embodiments, the S530 may include:

[0283] determining an adjustment value based on at least one pixel residual value in the first residual block;

[0284] The second residual block is obtained by subtracting the adjustment value from a pixel residual value of the at least one pixel residual value.

[0285] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​in the first residual block.

[0286] In some embodiments, the at least one pixel residual value includes a non-zero pixel residual value in the first residual block, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the first residual block are divided into at least one group according to at least one predefined numerical range.

[0287] In some embodiments, the S530 may include:

[0288] determining a first residual image based on the first residual block and at least one residual block;

[0289] The at least one residual block includes: a residual block determined based on a filtered block obtained by filtering a first block and a reconstructed block of the first block, wherein the first block is an image block in a current image to which the current block belongs;

[0290] determining an adjustment value based on at least one pixel residual value in a sliding window in the first residual image;

[0291] The adjustment value is subtracted from a pixel residual value in the at least one pixel residual value to obtain a residual image including the second residual block.

[0292] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​within the sliding window.

[0293] In some embodiments, the at least one pixel residual value includes a non-zero pixel residual value in the sliding window, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined numerical range.

[0294] In some embodiments, an average value of the at least one pixel residual value or a value obtained by rounding the average value of the at least one pixel residual value is determined as the adjustment value.

[0295] In some embodiments, the adjustment value is determined based on an average value of the at least one pixel residual value and a preset first value range.

[0296] In some embodiments, if the average value is within the first numerical range, the average value or the value obtained by rounding the average value is determined as the adjustment value; if the average value is greater than the upper limit value of the first numerical range, the upper limit value is determined as the adjustment value; if the average value is less than the lower limit value of the first numerical range, the lower limit value is determined as the adjustment value.

[0297] In some embodiments, the S510 may include:

[0298] Filter the reconstructed block using any one of the following methods to obtain the first filtered block:

[0299] Deblocking filter, sample adaptive compensation filter, adaptive correction filter, neural network based loop filter.

[0300] It should be understood that the encoding method can be understood as the inverse process of the decoding method. Therefore, the specific scheme of the encoding method 500 can refer to the relevant content of the decoding method 400. For the sake of ease of description, this application will not go into details.

[0301] The encoding method provided in this application is exemplified below with reference to Example 2.

[0302] Example 2:

[0303] In this embodiment, the loop filtering unit acts in the encoder, that is, the encoder first filters the reconstructed block of the current block through the neural network loop filter and outputs a first filtered block, then subtracts the reconstructed block from the first filtered block to obtain a first residual block, and corrects the first residual block to obtain a second reference block. Then, the encoder adds the second residual block and the reconstructed block to obtain a final reconstructed block. Thus, by correcting the first residual block, the filtering performance of the neural network loop filter is optimized, thereby improving the decoding performance of the encoder.

[0304] The specific process of the encoder is as follows:

[0305] a) When the encoder enters the loop filter unit, it processes according to the specified filter order. When entering the ROA unit, it first determines whether the ROA unit can be used in the current sequence based on the sequence-level identifier (i.e., the second identifier mentioned above, which is recorded as roa_enable_flag). If roa_enable_flag is "0", the ROA unit is not used in the current sequence. If roa_enable_flag is "1", ROA unit processing is attempted for the current sequence, that is, jumping to b).

[0306] b) For the current block in the current image, first input the reconstructed block of the current block into the NNLF and output the first filter block; then jump to c).

[0307] c) After the encoder obtains the first filter block, it can use the first filter block to subtract the reconstructed block to obtain a first residual block, and correct the first residual block to obtain a second reference block. Then, the second residual block and the reconstructed block are combined to determine the second filter block; then jump to d).

[0308] d) The encoder compares the first filtered image of the first filter block with the original image of the current image and calculates the rate-distortion cost (denoted as C NNLF ), compare the second filtered image of the second filter block with the original image of the current image and calculate the rate-distortion cost (denoted as C ROA ). Then, the encoder compares C ROA and C NNLF , determine the reconstructed image of the current image.

[0309] For example, if C ROA <C NNLF , the encoder uses the second filtered image output by the ROA unit as the final reconstructed image; if C ROA ≥C NNLF , the first filtered image output by NNLF is used as the final reconstructed image. Then jump to e).

[0310] e) Encode the picture-level identifier of the current picture (i.e., the first identifier mentioned above, which is recorded as picture_roa_enable_flag) into the bitstream, and jump to f).

[0311] f) For the current image, if the processing is complete, load the next frame for processing and jump to step b).

[0312] The specific implementation method of residual adjustment is exemplified below.

[0313] FIG11 is an example of the first residual block provided in this application.

[0314] As shown in FIG11 , the encoder may obtain a first residual block (res) by subtracting the input reconstructed block (rec) from the first filtered block (cnn) output by the filter.

[0315] It is worth noting that the present application does not specifically limit the filter used to filter the reconstructed block. The present application aims to correct the first residual block obtained by subtracting the filter input from the first filter block output by the filter. In theory, for any filtering tool, as long as the first residual block (res) of the first filter block (filtered) relative to the reconstructed block (rec) can be calculated, the scheme of the present application can be used to adjust the first residual block. For example, the first residual block obtained by subtracting the input from the first filter block output by DB, SAO, or ALF can be adjusted.

[0316] In one implementation, the encoder may directly correct the pixel residual values ​​in the first residual block at a block granularity.

[0317] For example, the encoder may correct the residual value block by block by traversing all residual blocks (eg, CTU level) of the residual image to be corrected, and finally obtain the corrected residual image.

[0318] For example, assuming that the residual image to be corrected includes CTU1 to CTU8 as shown in Figure 12, the encoder can correct the pixel residual value of each CTU in CTU1 to CTU8, that is, it can traverse CTU1 to CTU8, correct the pixel residual value in the CTU one by one, and finally obtain the corrected residual image.

[0319] Since the first residual block is uninterpretable, the present application corrects the first residual block by adjusting the direct current (DC) component of the first residual block to zero. For example, in order to adjust its direct current (DC) component to zero, it is necessary to adjust the average value of the pixel residual values ​​in the first residual block to zero. For example, all the pixel residual values ​​in the first residual block are summed and averaged to obtain the average value. Then, the average value is subtracted from each pixel residual value in the first residual block to obtain a corrected second residual block, whose direct current component is zero (or approximately zero). For example, for the first residual block shown in Figure 13, the average value of the pixel residual values ​​in the first residual block is calculated to be 2.375, which is rounded to 2; then, for the first residual block, the average value is subtracted from each pixel residual value in the first residual block to obtain the corrected second residual block shown in Figure 14.

[0320] Of course, in other alternative embodiments, the encoder may also perform residual adjustment using image blocks that are smaller or larger than the CTU. For example, as shown in FIG15 , the CTU is further divided into 2x2 blocks to obtain four sub-image blocks, and then the pixel residual values ​​in the sub-image blocks are corrected one by one.

[0321] In another implementation, the encoder may correct the pixel residual values ​​in the image at a sliding window granularity.

[0322] For example, the encoder can correct the pixel residual values ​​within the sliding window by moving the sliding window in the residual image to be corrected, and finally obtain a corrected residual image. The residual image to be corrected can be an image obtained by subtracting the reconstructed image of the current image from the filtered image of the current image.

[0323] For example, assuming that the residual image to be corrected is 8 CTUs as shown in Figure 16, and the sliding window is 1.5CTU×1.5CTU, the encoder can correct the pixel residual values ​​in the sliding window, that is, the pixel residual values ​​in the residual image to be corrected can be corrected by moving the sliding window according to a predefined sliding step size, and finally a corrected residual image is obtained.

[0324] Of course, regardless of whether the encoder adjusts the pixel residual value at a block granularity or at a sliding window granularity, in other alternative embodiments, the encoder may also sum and average the residual values ​​of some pixels in the first residual block or sliding window (for example, the residual values ​​of some pixels that are not 0) to obtain the average value of the residual values ​​of some pixels. In this case, the encoder adjusts the residual values ​​of some pixels. Even after the encoder calculates the average value of all the residual values ​​of all pixels (or some of the residual values ​​of some pixels) in the first residual block or sliding window, it may determine whether to use the average value to adjust the residual value. For example, if the average value is within a predefined numerical range, the average value is used to adjust the residual value; if the average value is less than the lower limit of the predefined numerical range, the lower limit is used to adjust the residual value; if the average value is greater than the upper limit of the predefined numerical range, the upper limit is used to adjust the residual value.

[0325] The preferred embodiments of the present application have been described in detail above with reference to the accompanying drawings. However, the present application is not limited to the specific details of the embodiments described above. Within the scope of the technical concept of the present application, various simple variations of the technical solutions of the present application may be made, and these simple variations all fall within the scope of protection of the present application. For example, the specific technical features described in the specific embodiments described above may be combined in any suitable manner, unless there is any contradiction. To avoid unnecessary repetition, the present application will not further describe the various possible combinations. For another example, the various different embodiments of the present application may also be combined in any manner, as long as they do not violate the principles of the present application and should also be considered as the content disclosed in the present application. It should also be understood that in the various method embodiments of the present application, the order of execution of the processes described above does not necessarily indicate the order of execution. The order of execution of the processes should be determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in the embodiments of the present application, the term "and / or" is merely a description of the association relationship between related objects, indicating that three relationships can exist. Specifically, A and / or B can represent: A exists alone, A and B exists simultaneously, or B exists alone. In addition, the character “ / ” in this application generally indicates that the preceding and following related objects are in an “or” relationship.

[0326] The method embodiment of the present application is described in detail above. The device embodiment of the present application is described in detail below in conjunction with Figures 18 to 20.

[0327] FIG18 is a schematic block diagram of a decoder 600 according to an embodiment of the present application.

[0328] As shown in FIG18 , the decoder 600 may include:

[0329] A filtering unit 610 is configured to filter a reconstructed block of a current block to obtain a first filtered block;

[0330] A first determining unit 620 is configured to determine a first residual block based on the first filter block and the reconstructed block;

[0331] an adjusting unit 630, configured to adjust a DC component of the first residual block to obtain a second residual block;

[0332] The second determining unit 640 is configured to determine a final reconstructed block of the current block based on the second residual block and the reconstructed block.

[0333] In some embodiments, the first determining unit 620 is specifically configured to:

[0334] Decoding the code stream and determining the first identifier;

[0335] If the first identifier indicates to adjust a filtered image corresponding to a current image to which the current block belongs, a first residual block is determined based on the first filtered block and the reconstructed block.

[0336] In some embodiments, the first determining unit 620 is specifically configured to:

[0337] Decoding the code stream to determine a second identifier;

[0338] If the second identifier indicates to adjust the filtered image corresponding to the image in the image sequence to which the current image belongs, the code stream is decoded to determine the first identifier.

[0339] In some embodiments, the adjusting unit 630 is specifically configured to:

[0340] determining an adjustment value based on at least one pixel residual value in the first residual block;

[0341] The second residual block is obtained by subtracting the adjustment value from a pixel residual value of the at least one pixel residual value.

[0342] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​in the first residual block.

[0343] In some embodiments, the at least one pixel residual value includes a non-zero pixel residual value in the first residual block, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the first residual block are divided into at least one group according to at least one predefined numerical range.

[0344] In some embodiments, the adjusting unit 630 is specifically configured to:

[0345] determining a first residual image based on the first residual block and at least one residual block;

[0346] The at least one residual block includes: a residual block determined based on a filtered block obtained by filtering a first block and a reconstructed block of the first block, wherein the first block is an image block in a current image to which the current block belongs;

[0347] determining an adjustment value based on at least one pixel residual value in a sliding window in the first residual image;

[0348] The adjustment value is subtracted from a pixel residual value in the at least one pixel residual value to obtain a residual image including the second residual block.

[0349] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​within the sliding window.

[0350] In some embodiments, the at least one pixel residual value includes a non-zero pixel residual value in the sliding window, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined numerical range.

[0351] In some embodiments, the adjusting unit 630 is specifically configured to:

[0352] An average value of the at least one pixel residual value or a value obtained by rounding the average value of the at least one pixel residual value is determined as the adjustment value.

[0353] In some embodiments, the adjusting unit 630 is specifically configured to:

[0354] The adjustment value is determined based on an average value of the at least one pixel residual value and a preset first value range.

[0355] In some embodiments, the adjusting unit 630 is specifically configured to:

[0356] If the average value is within the first value range, determining the average value or a value obtained by rounding the average value as the adjustment value;

[0357] If the average value is greater than the upper limit of the first value range, determining the upper limit as the adjustment value;

[0358] If the average value is smaller than a lower limit value of the first numerical range, the lower limit value is determined as the adjustment value.

[0359] In some embodiments, the filtering unit 610 is specifically configured to:

[0360] Filter the reconstructed block using any one of the following methods to obtain the first filtered block:

[0361] Deblocking filter, sample adaptive compensation filter, adaptive correction filter, neural network based loop filter.

[0362] FIG19 is a schematic block diagram of an encoder 700 according to an embodiment of the present application.

[0363] As shown in FIG19 , the encoder 700 may include:

[0364] A filtering unit 710 is configured to filter a reconstructed block of a current block to obtain a first filtered block;

[0365] A first determining unit 720 is configured to determine a first residual block based on the first filtering block and the reconstructed block;

[0366] an adjusting unit 730, configured to adjust a DC component of the first residual block to obtain a second residual block;

[0367] A second determining unit 740 is configured to determine a second filter block based on the second residual block and the reconstructed block;

[0368] The third determining unit 750 is configured to determine the filtered image with the minimum rate-distortion cost among the first filtered image to which the first filtering block belongs and the second filtered image to which the second filtering block belongs as the reconstructed image of the current image to which the current block belongs.

[0369] In some embodiments, the third determining unit 750 is further configured to:

[0370] Encode the first identifier;

[0371] Among them, when the rate-distortion cost of the first filtered image is less than or equal to the rate-distortion cost of the second filtered image, the first flag indicates that the filtered image corresponding to the current image to which the current block belongs is not adjusted; when the rate-distortion cost of the first filtered image is greater than the rate-distortion cost of the second filtered image, the first flag indicates that the filtered image corresponding to the current image is adjusted.

[0372] In some embodiments, the third determining unit 750 is further configured to:

[0373] Encode the second identifier;

[0374] The second flag indicates whether to adjust or not adjust the filtered image corresponding to the image in the image sequence to which the current image belongs.

[0375] In some embodiments, the adjusting unit 730 is specifically configured to:

[0376] determining an adjustment value based on at least one pixel residual value in the first residual block;

[0377] The second residual block is obtained by subtracting the adjustment value from a pixel residual value of the at least one pixel residual value.

[0378] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​in the first residual block.

[0379] In some embodiments, the at least one pixel residual value includes a non-zero pixel residual value in the first residual block, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the first residual block are divided into at least one group according to at least one predefined numerical range.

[0380] In some embodiments, the adjusting unit 730 is specifically configured to:

[0381] determining a first residual image based on the first residual block and at least one residual block;

[0382] The at least one residual block includes: a residual block determined based on a filtered block obtained by filtering a first block and a reconstructed block of the first block, wherein the first block is an image block in a current image to which the current block belongs;

[0383] determining an adjustment value based on at least one pixel residual value in a sliding window in the first residual image;

[0384] The adjustment value is subtracted from a pixel residual value in the at least one pixel residual value to obtain a residual image including the second residual block.

[0385] In some embodiments, the at least one pixel residual value comprises all pixel residual values ​​within the sliding window.

[0386] In some embodiments, the at least one pixel residual value includes a non-zero pixel residual value in the sliding window, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined numerical range.

[0387] In some embodiments, the adjusting unit 730 is specifically configured to:

[0388] An average value of the at least one pixel residual value or a value obtained by rounding the average value of the at least one pixel residual value is determined as the adjustment value.

[0389] In some embodiments, the adjusting unit 730 is specifically configured to:

[0390] The adjustment value is determined based on an average value of the at least one pixel residual value and a preset first value range.

[0391] In some embodiments, the adjusting unit 730 is specifically configured to:

[0392] If the average value is within the first value range, determining the average value or a value obtained by rounding the average value as the adjustment value;

[0393] If the average value is greater than the upper limit of the first value range, determining the upper limit as the adjustment value;

[0394] If the average value is smaller than a lower limit value of the first numerical range, the lower limit value is determined as the adjustment value.

[0395] In some embodiments, the filtering unit 710 is specifically configured to:

[0396] Filter the reconstructed block using any one of the following methods to obtain the first filtered block:

[0397] Deblocking filter, sample adaptive compensation filter, adaptive correction filter, neural network based loop filter.

[0398] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, the decoder 600 shown in Figure 18 may correspond to the corresponding subject in the decoding method 400 for executing the embodiment of the present application, and the aforementioned and other operations and / or functions of the various units in the decoder 600 are respectively for implementing the corresponding processes in the various methods such as the decoding method 400. The encoder 700 shown in Figure 19 may correspond to the corresponding subject in the encoding method 500 for executing the embodiment of the present application, that is, the aforementioned and other operations and / or functions of the various units in the encoder 700 are respectively for implementing the corresponding processes in the various methods such as the encoding method 500.

[0399] It should also be understood that each unit in the decoder 600 or encoder 700 that the embodiment of the application is related to is divided based on logical function, and in actual applications, the function of a unit can also be realized by multiple units, or the function of multiple units is realized by one unit, or even, these functions can also be assisted by one or more other units to realize.For example, part or all of the decoder 600 or encoder 700 are merged into one or several other units.For another example, a certain (some) unit in the decoder 600 or encoder 700 can also be split into multiple units smaller in function to constitute, and this can realize same operation, without affecting the realization of the technical effect of the embodiment of the application.For another example, this decoder 600 or encoder 700 can also include other units, and in actual applications, these functions can also be assisted by other units to realize, and can be realized by the collaboration of multiple units.

[0400] According to another embodiment of the present application, a computer program (including program code) capable of executing each step involved in the corresponding method can be run on a general-purpose computing device of a general-purpose computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), etc., to construct the decoder 600 or encoder 700 involved in the embodiment of the present application, and to implement the encoding method or decoding method of the embodiment of the present application. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into an electronic device via a computer-readable storage medium, and run therein to implement the corresponding method of the embodiment of the present application. In other words, the units involved above can be implemented in hardware form, can be implemented in software form, or can be implemented in the form of a combination of hardware and software. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the integrated logic circuit of the hardware in the processor and / or the instruction in software form, and the steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as the hardware decoding processor executing the execution, or the hardware and software combination in the decoding processor executing the execution. Alternatively, the software may be located in a mature storage medium in the art, such as a random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or register. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps in the method embodiments described above.

[0401] FIG20 is a schematic structural diagram of an electronic device 800 provided in this application.

[0402] As shown in Figure 20, the electronic device 800 includes at least a processor 810 and a computer-readable storage medium 820. The processor 810 and the computer-readable storage medium 820 may be connected via a bus or other means. The computer-readable storage medium 820 is used to store a computer program 821, which includes computer instructions. The processor 810 is used to execute the computer instructions stored in the computer-readable storage medium 820. The processor 810 is the computing core and control core of the electronic device 800. It is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0403] For example, the processor 810 may also be referred to as a central processing unit (CPU). The processor 810 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, discrete hardware components, and the like.

[0404] Exemplarily, the computer-readable storage medium 820 may be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it may be at least one computer-readable storage medium located away from the aforementioned processor 810. Specifically, the computer-readable storage medium 820 includes, but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0405] Exemplarily, the electronic device 800 may be an encoder or encoding framework involved in an embodiment of the present application; a first computer instruction is stored in the computer-readable storage medium 820; the processor 810 loads and executes the first computer instruction stored in the computer-readable storage medium 820 to implement the corresponding steps in the encoding method provided in the present application; in other words, the first computer instruction in the computer-readable storage medium 820 is loaded by the processor 810 and the corresponding steps are executed. To avoid repetition, it will not be repeated here.

[0406] Exemplarily, the electronic device 800 may be a decoder or decoding framework involved in an embodiment of the present application; a second computer instruction is stored in the computer-readable storage medium 820; the processor 810 loads and executes the second computer instruction stored in the computer-readable storage medium 820 to implement the corresponding steps in the decoding method provided in the present application; in other words, the second computer instruction in the computer-readable storage medium 820 is loaded by the processor 810 and the corresponding steps are executed. To avoid repetition, it will not be repeated here.

[0407] According to another aspect of the present application, the present application also provides a coding and decoding system, including the encoder and decoder mentioned above.

[0408] According to another aspect of the present application, the present application also provides a computer-readable storage medium (Memory), which is a memory device in the electronic device 800 for storing programs and data. For example, a computer-readable storage medium 820. It can be understood that the computer-readable storage medium 820 here can include both the built-in storage medium in the electronic device 800 and, of course, the extended storage medium supported by the electronic device 800. The computer-readable storage medium provides a storage space, which stores the operating system of the electronic device 800. In addition, one or more computer instructions suitable for being loaded and executed by the processor 810 are also stored in the storage space. These computer instructions can be one or more computer programs 821 (including program codes).

[0409] According to another aspect of the present application, the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. For example, computer program 821. In this case, the data processing device 800 can be a computer, the processor 810 reads the computer instructions from the computer-readable storage medium 820, and the processor 810 executes the computer instructions so that the computer performs the encoding method or decoding method provided in the various optional embodiments mentioned above. In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process of the embodiment of the present application is run in whole or in part or the functions of the embodiment of the present application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0410] According to another aspect of the present application, the present application further provides a code stream, which may be a code stream decoded using the decoding method provided by the present application or a code stream generated using the encoding method provided by the present application.

[0411] Those skilled in the art will appreciate that the units and process steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0412] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A decoding method, characterized in that: include: Filtering a reconstructed block of the current block to obtain a first filtered block; Determine a first residual block based on the first filtering block and the reconstructed block; Adjusting a direct current component of the first residual block to obtain a second residual block; Based on the second residual block and the reconstructed block, a final reconstructed block of the current block is determined.

2. The method according to claim 1, characterized in that The determining a first residual block based on the first filtering block and the reconstructed block comprises: Decoding the code stream and determining the first identifier; If the first identifier indicates to adjust the filtered image corresponding to the current image to which the current block belongs, a first residual block is determined based on the first filtered block and the reconstructed block.

3. The method according to claim 2, characterized in that The decoding of the code stream to determine the first identifier includes: Decoding the code stream to determine a second identifier; If the second identifier indicates to adjust the filtered image corresponding to the image in the image sequence to which the current image belongs, the code stream is decoded to determine the first identifier.

4. The method according to any one of claims 1 to 3, characterized in that The step of adjusting the DC component of the first residual block to obtain a second residual block includes: Determining an adjustment value based on at least one pixel residual value in the first residual block; The second residual block is obtained by subtracting the adjustment value from a pixel residual value in the at least one pixel residual value.

5. The method according to claim 4, characterized in that The at least one pixel residual value includes all pixel residual values ​​in the first residual block.

6. The method according to claim 4, characterized in that The at least one pixel residual value includes a non-zero pixel residual value in the first residual block, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the first residual block are divided into at least one group according to at least one predefined numerical range.

7. The method according to any one of claims 1 to 3, characterized in that The step of adjusting the DC component of the first residual block to obtain a second residual block includes: Determine a first residual image based on the first residual block and at least one residual block; The at least one residual block includes: a residual block determined based on a filter block obtained after filtering a first block and a reconstructed block of the first block, wherein the first block is an image block in a current image to which the current block belongs; determining an adjustment value based on at least one pixel residual value in a sliding window in the first residual image; The adjustment value is subtracted from a pixel residual value in the at least one pixel residual value to obtain a residual image including the second residual block.

8. The method according to claim 7, characterized in that The at least one pixel residual value includes all pixel residual values ​​within the sliding window.

9. The method according to claim 7, characterized in that: The at least one pixel residual value includes a non-zero pixel residual value in the sliding window, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined numerical range.

10. The method according to any one of claims 4 to 9, characterized in that The determining of the adjustment value comprises: An average value of the at least one pixel residual value or a value obtained by rounding the average value of the at least one pixel residual value is determined as the adjustment value.

11. The method according to any one of claims 4 to 9, characterized in that The determining of the adjustment value comprises: The adjustment value is determined based on an average value of the at least one pixel residual value and a preset first value range.

12. The method according to claim 11, characterized in that The determining the adjustment value based on the average value of the at least one pixel residual value and a preset first value range includes: If the average value is within the first value range, the average value or a value obtained by rounding the average value is determined as the adjustment value; If the average value is greater than the upper limit value of the first value range, determining the upper limit value as the adjustment value; If the average value is smaller than a lower limit value of the first numerical range, the lower limit value is determined as the adjustment value.

13. The method according to any one of claims 1 to 12, characterized in that The filtering of the reconstructed block of the current block to obtain a first filtered block includes: The reconstructed block is filtered by any one of the following methods to obtain the first filtered block: Deblocking filter, sample adaptive compensation filter, adaptive correction filter, neural network based loop filter.

14. A coding method, characterized in that: include: Filtering a reconstructed block of the current block to obtain a first filtered block; Determine a first residual block based on the first filtering block and the reconstructed block; Adjusting a direct current component of the first residual block to obtain a second residual block; Determine a second filter block based on the second residual block and the reconstructed block; A filtered image with the minimum rate-distortion cost among a first filtered image to which the first filtering block belongs and a second filtered image to which the second filtering block belongs is determined as a reconstructed image of a current image to which the current block belongs.

15. The method according to claim 14, characterized in that The method further comprises: Encode the first identifier; Among them, when the rate-distortion cost of the first filtered image is less than or equal to the rate-distortion cost of the second filtered image, the first identifier indicates that the filtered image corresponding to the current image to which the current block belongs is not adjusted; when the rate-distortion cost of the first filtered image is greater than the rate-distortion cost of the second filtered image, the first identifier indicates that the filtered image corresponding to the current image is adjusted.

16. The method according to claim 14 or 15, characterized in that The method further comprises: encoding a second identifier; The second identifier indicates whether to adjust or not adjust the filtered image corresponding to the image in the image sequence to which the current image belongs.

17. The method according to any one of claims 14 to 16, characterized in that The step of adjusting the DC component of the first residual block to obtain a second residual block includes: Determining an adjustment value based on at least one pixel residual value in the first residual block; The second residual block is obtained by subtracting the adjustment value from a pixel residual value in the at least one pixel residual value.

18. The method according to claim 17, characterized in that The at least one pixel residual value includes all pixel residual values ​​in the first residual block.

19. The method according to claim 17, characterized in that The at least one pixel residual value includes a non-zero pixel residual value in the first residual block, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the first residual block are divided into at least one group according to at least one predefined numerical range.

20. The method according to any one of claims 14 to 16, characterized in that The step of adjusting the DC component of the first residual block to obtain a second residual block includes: Determine a first residual image based on the first residual block and at least one residual block; The at least one residual block includes: a residual block determined based on a filter block obtained after filtering a first block and a reconstructed block of the first block, wherein the first block is an image block in a current image to which the current block belongs; determining an adjustment value based on at least one pixel residual value in a sliding window in the first residual image; The adjustment value is subtracted from a pixel residual value in the at least one pixel residual value to obtain a residual image including the second residual block.

21. The method according to claim 20, characterized in that The at least one pixel residual value includes all pixel residual values ​​within the sliding window.

22. The method according to claim 20, characterized in that The at least one pixel residual value includes a non-zero pixel residual value in the sliding window, or the at least one pixel residual value includes a pixel residual value in each group after the pixel residual values ​​in the sliding window are divided into at least one group according to at least one predefined numerical range.

23. The method according to any one of claims 17 to 22, characterized in that The determining of the adjustment value comprises: An average value of the at least one pixel residual value or a value obtained by rounding the average value of the at least one pixel residual value is determined as the adjustment value.

24. The method according to any one of claims 17 to 22, characterized in that The determining of the adjustment value comprises: The adjustment value is determined based on an average value of the at least one pixel residual value and a preset first value range.

25. The method according to claim 24, characterized in that The determining the adjustment value based on the average value of the at least one pixel residual value and a preset first value range includes: If the average value is within the first value range, the average value or a value obtained by rounding the average value is determined as the adjustment value; If the average value is greater than the upper limit value of the first value range, determining the upper limit value as the adjustment value; If the average value is smaller than a lower limit value of the first numerical range, the lower limit value is determined as the adjustment value.

26. The method according to any one of claims 14 to 25, characterized in that The filtering of the reconstructed block of the current block to obtain a first filtered block includes: The reconstructed block is filtered by any one of the following methods to obtain the first filtered block: Deblocking filter, sample adaptive compensation filter, adaptive correction filter, neural network based loop filter.

27. A decoder, characterized in that: include: A filtering unit, configured to filter a reconstructed block of a current block to obtain a first filtered block; A first determining unit, configured to determine a first residual block based on the first filtering block and the reconstructed block; an adjusting unit, configured to adjust a direct current component of the first residual block to obtain a second residual block; The second determining unit is configured to determine a final reconstructed block of the current block based on the second residual block and the reconstructed block.

28. An encoder, characterized in that include: A filtering unit, configured to filter a reconstructed block of a current block to obtain a first filtered block; A first determining unit, configured to determine a first residual block based on the first filtering block and the reconstructed block; an adjusting unit, configured to adjust a direct current component of the first residual block to obtain a second residual block; A second determining unit, configured to determine a second filter block based on the second residual block and the reconstructed block; The third determining unit is used to determine the filtered image with the smallest rate-distortion cost in the first filtered image to which the first filtering block belongs and the second filtered image to which the second filtering block belongs as the reconstructed image of the current image to which the current block belongs.

29. An electronic device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the method according to any one of claims 1 to 13 or the method according to any one of claims 14 to 26.

30. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 13 or the method according to any one of claims 14 to 26.

31. A computer program product comprising a computer program / instructions, characterized in that The computer program / instructions, when executed by a processor, implement the method according to any one of claims 1 to 13 or the method according to any one of claims 14 to 26.

32. A code stream, characterized in that The code stream is a code stream decoded according to the method according to any one of claims 1 to 13, or the code stream is a code stream generated by the method according to any one of claims 14 to 26.

Images